DSNY golden age

This is a full style-conversion world-morph lora WIP, a project over 1.5 years in the making. It's intended to convert any prompt into the 2d hand-animated style of Disney's golden age ('95-'05). It is NOT made to make specific characters or scenes from these movies (the character names and movie titles were not captioned). While the art styles range somewhat between these movies, outputs stay within this range and follow it's universal rules (thick colored linework, hand painted backgrounds, flat character colors/shading, and the general tone/composition/anatomy of Disney). It is extremely flexible and capable, with near perfect anatomy/coherence at a 90% hit rate. I've been perfecting my dataset across 120+ training runs on Flux, Wan, Klein, ZIT, and ZIB. I plan to extend this project to the other current models (and could release a ready Flux version if interest is shown), with the ultimate goal being video generation in this style, which I'm working on (please pray for my 3090).

The lora was trained on base but intended for inference on Z-image Turbo (ZIT). It also works with Base with a stronger style but worse anatomy, like a missing finger in 1 out of 4 gens.

trigger: dsny cartoon artstyle

Recommended settings (Turbo): weight 0.9, euler simple, 10-12 steps, cfg 1

Weight over 1.0 can help on certain longer prompts but usually damages anatomy.

Weight below 0.8 can make a cleaner image but can introduce 3d shading and black linework (turbo's bias).

Refrain from stylistic keywords or any description that could interfere with the style (example: ultra realistic, reflective, shiny, depth of field, textured, 3d, photography, etc.).

The biggest challenge was overcoming Turbo's biases from distillation. Loras that worked perfectly on base were too weak on turbo. I fixed this balance after 20+ trainings so that a strength of 1.0 can be used on both Turbo and Base (although I prefer a 0.9 strength for my prompting style).

Variety and creativity is improved over base ZIT so prompts can be shorter and concise with excellent results.

This project is only halfway done, with only 11 of 21 planned movies in the dataset. I'm releasing as a WIP due to popular demand, and will upload new versions soon now that params are solved. Dataset images come directly from the source without any upscaling or synthetic data, ensuring a style as true as possible to the movies.

training settings:

type: lokr
linear: 64
linear_alpha: 32
conv: 64
conv_alpha: 8
lokr_full_rank: true
lokr_factor: 4
caption_dropout_rate: 0.05
resolution:
- 512
batch_size: 3
steps: 3500
optimizer: prodigy
timestep_type: weighted
lr: 0.7
dtype: bf16
loss_type: mse
differential_guidance_scale: 2

trained on 63 images at a batch size of 3 for 3500 steps, so effectively 166 steps per image. this "overkill" was necessary for full convergence and a deep bake.