Name: Photo Background - 2d Compositing｜写真背景・二次元合成
Rating: 5 (828 reviews)
Author: motimalu

<h1 id="photo-background-2d-compositingyuan">Photo Background - 2d Compositing｜写真背景・二次元合成</h1>Trained on 2d illustrations composited on a photo background.This is a small LoRA I thought would be interesting to see how models trained on illustrations or real world images/video can produce the composite, mixed reality effect.ℹ️ LoRA work best when applied to the base models on which they are trained. Please read the About This Version on the appropriate base models and workflow/training information.Metadata is included in all uploaded files, you can drag the generated videos into ComfyUI to use the embedded workflows.

Trained on Anima Base 1Updated dataset with a mix of natural language and tag captionsPartitioned dataset and trained at multi-res 512, 768, 1024, 1280, 1536Training config:<pre><code># trained using diffusion-pipe commit b0aa4f1e03169f3280c8518d37570a448420f8be
# NCCL_P2P_DISABLE="1" NCCL_IB_DISABLE="1" NCCL_CUMEM_ENABLE="0" deepspeed --num_gpus=1 train.py --deepspeed --config anima-lora.toml --i_know_what_i_am_doing

output_dir = '/mnt/d/anima/training_output/anima-base-1-niji-v4'

dataset = 'dataset-anima-niji.toml'

# training settings
epochs = 2
# Per-resolution batch sizes
micro_batch_size_per_gpu = [[512, 64], [768, 48], [1024, 32], [1280, 24], [1536, 16]]
pipeline_stages = 1
gradient_accumulation_steps = 1
gradient_clipping = 1
warmup_steps = 30
lr_scheduler = 'cosine'

# misc settings
save_every_n_epochs = 1
activation_checkpointing = true

partition_method = 'parameters'

save_dtype = 'bfloat16'
caching_batch_size = 1
map_num_proc = 8
steps_per_print = 1
compile = true

[model]
type = 'anima'
transformer_path = '/mnt/c/workspace/models/diffusion_models/anima-base-v1.0.safetensors'
vae_path = '/mnt/c/workspace/models/vae/qwen_image_vae.safetensors'
llm_path = '/mnt/c/workspace/models/text_encoders/qwen_3_06b_base.safetensors'
dtype = 'bfloat16'
#cache_text_embeddings = false
llm_adapter_lr = 0
#timestep_sample_method = 'uniform'
flux_shift = true
multiscale_loss_weight = 0.5
sigmoid_scale = 1.3

[adapter]
type = 'lora'
rank = 32
dtype = 'bfloat16'

[optimizer]
type = 'adamw_optimi'
lr = 4e-5
betas = [0.9, 0.99]
weight_decay = 0.01
eps = 1e-8</code></pre><pre><code>resolutions = [512, 768, 1024, 1280, 1536]

enable_ar_bucket = true
min_ar = 0.5
max_ar = 2.0
num_ar_buckets = 9

# micro_batch_size_per_gpu = [[512, 64], [768, 768, 48], [1024, 32], [1280, 24], [1536, 16]]

# images_niji\1536x1536\captions.json with 124 entries.
[[directory]]
path = '/mnt/d/training_data/images_niji/1536x1536'
resolutions = [512, 1024, 1280, 1536]

# images_niji\1280x1280\captions.json with 28 entries.
[[directory]]
path = '/mnt/d/training_data/images_niji/1280x1280'
resolutions = [512, 1024, 1280]

# images_niji\1024x1024\captions.json with 137 entries.
[[directory]]
path = '/mnt/d/training_data/images_niji/1024x1024'
resolutions = [512, 768, 1024]

# images_niji\768x768\captions.json with 43 entries.
[[directory]]
path = '/mnt/d/training_data/images_niji/768x768'
resolutions = [512, 768]

# images_niji\512x512\captions.json with 5 entries.
[[directory]]
path = '/mnt/d/training_data/images_niji/512x512'
resolutions = [512]
</code></pre>

v4.0 [anima-base-1]

Trained on <a target="_blank" rel="ugc" href="https://civitai.com/models/2458426/anima-official?modelVersionId=2836417">Anima Preview 3 Base</a>Training config:<pre><code># trained using diffusion-pipe commit 6e95020cad0b3cd7dcb93ce42b358669051bf6d2

output_dir = '/mnt/d/anima/training_output/nijigen'

dataset = 'dataset-anima.toml'

# training settings
epochs = 1000
# Per-resolution batch sizes
# micro_batch_size_per_gpu = [[512, 32], [1024, 32], [1536, 16]]
micro_batch_size_per_gpu = 16
pipeline_stages = 1
gradient_accumulation_steps = 1
gradient_clipping = 1
warmup_steps = 100

# misc settings
save_every_n_epochs = 1
#save_every_n_steps = 1000
#save_every_n_examples = 4096000
#checkpoint_every_n_epochs = 1
#checkpoint_every_n_minutes = 120
activation_checkpointing = true
#reentrant_activation_checkpointing = true

partition_method = 'parameters'
# partition_method = 'manual'
# partition_split = [10]

save_dtype = 'bfloat16'
caching_batch_size = 1
map_num_proc = 8
steps_per_print = 1
compile = true

[model]
type = 'anima'
transformer_path = '/ComfyUI/models/diffusion_models/anima-preview3-base.safetensors'
vae_path = '/ComfyUI/models/vae/qwen_image_vae.safetensors'
llm_path = '/ComfyUI_windows_portable/ComfyUI/models/text_encoders/qwen_3_06b_base.safetensors'
dtype = 'bfloat16'
#cache_text_embeddings = false
llm_adapter_lr = 0
#timestep_sample_method = 'uniform'
#flux_shift = true
#multiscale_loss_weight = 0.5
sigmoid_scale = 1.3

[adapter]
type = 'lora'
rank = 32
dtype = 'bfloat16'

[optimizer]
type = 'adamw_optimi'
lr = 2e-5
betas = [0.9, 0.99]
weight_decay = 0.01
eps = 1e-8</code></pre><pre><code>resolutions = [512, 1024, 1536]

enable_ar_bucket = true
min_ar = 0.5
max_ar = 2.0
num_ar_buckets = 9

[[directory]]
path = '/mnt/d/training_data/images_niji_captions'</code></pre>

v3.0 [anima-preview-3]

v2.0 [anima-preview]

Trained on <a target="_blank" rel="ugc" href="https://civitai.com/models/833294/noobai-xl-nai-xl?modelVersionId=1190596">NoobAI-XL (NAI-XL) V-Pred 1.0-Version</a>Previews are generated in Forge with DynamicThresholding (CFG-Fix) Integrated enabled, settings:<pre><code>dynthres_enabled: True, dynthres_mimic_scale: 5, dynthres_threshold_percentile: 1, dynthres_mimic_mode: Half Cosine Down, dynthres_mimic_scale_min: 1, dynthres_cfg_mode: Half Cosine Down, dynthres_cfg_scale_min: 3, dynthres_sched_val: 1, dynthres_separate_feature_channels: enable, dynthres_scaling_startpoint: ZERO, dynthres_variability_measure: STD, dynthres_interpolate_phi: 1</code></pre>A ComfyUI workflow reference with DynamicThresholding, Upscaling, and FaceDetailer can be found here: <a target="_blank" rel="ugc" href="https://civitai.com/posts/11457095">https://civitai.com/posts/11457095</a>

v1.0 [noobai-v-pred-1]

Trained on <a target="_blank" rel="ugc" href="https://civitai.com/models/795765/illustrious-xl">Illustrious XL 0.1</a>

v1.0 [Illustrious]

v1.0 [wan-t2v-14b]

Trained with <a target="_blank" rel="ugc" href="https://github.com/tdrussell/diffusion-pipe">https://github.com/tdrussell/diffusion-pipe</a>Training data consists of:<ul><li>37 images as a combination of<ul><li>Images used from other versions this model card</li><li>Images extracted as keyframes from several videos</li></ul></li><li>23 video clips ~70 frames each<ul><li>70 frames was too long for the 368 resolution for videos (exceeded 24gb vram)</li></ul></li></ul>Training configs:dataset.toml<pre><code># Aspect ratio bucketing settings
enable_ar_bucket = true
min_ar = 0.5
max_ar = 2.0
num_ar_buckets = 7

# Frame buckets (1 is for images)
frame_buckets = [1]

[[directory]]
# Set this to where your dataset is
path = '/mnt/d/huanvideo/training_data/images'
# Reduce as necessary
num_repeats = 5

[[directory]] # IMAGES
# Path to the directory containing images and their corresponding caption files.
path = '/mnt/d/huanvideo/training_data/images'
num_repeats = 5
resolutions = [1024]
frame_buckets = [1] # Use 1 frame for images.


[[directory]] # VIDEOS
# Path to the directory containing videos and their corresponding caption files.
path = '/mnt/d/huanvideo/training_data/videos'
num_repeats = 5
resolutions = [368] 
frame_buckets = [33, 49, 81] # Define frame buckets for videos.</code></pre>config.toml<pre><code># Dataset config file.
output_dir = '/mnt/d/huanvideo/training_output'
dataset = 'dataset.toml'

# Training settings
epochs = 50
micro_batch_size_per_gpu = 1
pipeline_stages = 1
gradient_accumulation_steps = 4
gradient_clipping = 1.0
warmup_steps = 100

# eval settings
eval_every_n_epochs = 5
eval_before_first_step = true
eval_micro_batch_size_per_gpu = 1
eval_gradient_accumulation_steps = 1

# misc settings
save_every_n_epochs = 15
checkpoint_every_n_minutes = 30
activation_checkpointing = true
partition_method = 'parameters'
save_dtype = 'bfloat16'
caching_batch_size = 1
steps_per_print = 1
video_clip_mode = 'single_middle'

[model]
type = 'hunyuan-video'

transformer_path = '/mnt/d/huanvideo/models/diffusion_models/hunyuan_video_720_cfgdistill_fp8_e4m3fn.safetensors'
vae_path = '/mnt/d/huanvideo/models/vae/hunyuan_video_vae_bf16.safetensors'
llm_path = '/mnt/d/huanvideo/models/llm'
clip_path = '/mnt/d/huanvideo/models/clip'

dtype = 'bfloat16'
transformer_dtype = 'float8'
timestep_sample_method = 'logit_normal'

[adapter]
type = 'lora'
rank = 32
dtype = 'bfloat16'

[optimizer]
type = 'adamw_optimi'
lr = 5e-5
betas = [0.9, 0.99]
weight_decay = 0.02
eps = 1e-8</code></pre>