MiniMax H3 Ref2V Version:
Suggested strength: 0.8-1.2
Example prompt:
<Video 1> is the source video for the target video edit.
0-4s: <Subject 1> is an Optical Illusion video of a bustling forest with sunlight peeking through, attribute_transfer
Audio: calming ambient bird chirping outdoor forest
Shoutouts to Ostris tutorial. I might be the worst MiniMax H3 Prompter with 8GB VRAM so I would love to see what you all are capable of with decent hardware.
This LoRa works but can be finicky and require prompting that points the direction in a specific way, like prompting for fisheye video for the spiral video, or prompting for two lettuce heads for the lettuce bra video.
It is trained on tags like turn <Video 1> into a visually striking, high-contrast video of an ocean sunset or sunrise, capturing a dramatic, moody atmosphere while preserving the outlines and shapes of the original video
Most of the videos should have their workflow. Tried many things including providing a <Picture> for the <Video> to be based off of but it did not go too well, had better luck describing environments in prompt form.
Klein 9B Version:
*UPDATE: You can now use this on-site generator - still recommend doing a first pass-through of your image with the turn the image into a black and white stencil against a white background then run that image through this model
Suggested strength: 1.0-1.5
Suggested prompt: turn the image into <your prompt> while preserving the outlines and shapes of the original image
Basically an Optical Illusion generator using your provided image.
You can try it out online here.
Usage: I highly suggest editing your input image with the prompt turn the image into a black and white stencil against a white background then run that image through this model. If your image is too human like it will likely just try to generate a human, even if you stencil it. It is very picky about prompt and picture choice as this is not a controlnet but a LoRa. I just checked and my dataset actually contains zero QR codes, so that will be a priority for V2. For tricker images/prompts I suggest fewer steps, higher CFG, higher strength.
