Ideogram V4 Visual Guidance/Prompting with Reference Images

Why this workflow?

Ideogram V4 is an excellent text-to-image model. But what happens if we also feed it visual information from one or two reference images?

This workflow adds experimental visual guidance to Ideogram V4 by reusing the Qwen3-VL-based visual-conditioning path of the enhanced Krea2 text encoder by ethanfel. The result is often surprisingly strong: composition, appearance, objects, colours, and even quite detailed structural information can survive the visual-language pathway and meaningfully influence a completely new generation.

The goal is not pixel-perfect image editing. Instead, the reference images act as visual prompts that are combined with normal text prompting.

Core idea

The workflow uses two conditioning branches:

• Text-only conditioning

• Visual guidance conditioning, containing the same text prompt plus visual tokens extracted from the reference image(s)

Sampling is split into two stages:

• At higher sigmas, only the text-only branch is used

• At lower sigmas, the visual guidance branch is added

This helps Ideogram V4 first establish the coarse scene structure from the text prompt and only afterwards incorporate the visual guidance.

A second control, Visual Guidance Branch Weight, determines how strongly the visual guidance branch contributes once it becomes active.

Workflow handling

Up to two reference images can be used.

Each reference image can be:

• used directly,

• masked,

• cropped to the masked region,

• or have irrelevant regions blanked or blurred before visual encoding.

This is useful when only one person, object, clothing item, landscape feature, etc. should influence the result.

A reference image can be disabled completely by disconnecting its link between:

• Ref. Image Interface and Text Encode (Krea2)

Automatic person masking is included only as an optional convenience feature. The corresponding nodes can simply be deleted if the package is not installed.

Most operating details are already documented directly inside the workflow via note boxes, including:

• reference image handling,

• masking options,

• sigma split,

• visual guidance branch weight,

• and recommended starting settings.

The workflow can also be run with an empty or nearly empty text prompt. This can be useful for pure image reinterpretation or image fusion, although the result is naturally less controllable.

Three key visual-guidance parameters

Visual guidance is mainly controlled by three parameters. They affect different aspects of the process and should not be regarded as interchangeable “strength” controls.

1. Sigma Threshold

Controls when visual guidance starts.

• Higher values mean visual guidance enters earlier

• Lower values give the text-only branch more time to stabilise the coarse scene first

So this parameter mainly affects the stability of the base scene defined by the text prompt.

2. Visual Guidance Branch Weight

Controls how strongly the visual-guidance branch influences the result once it is active.

Important: this branch contains text + visual tokens, not visual information alone.

Rough interpretation:

• 0.0 -> text only

• >= 1.0 -> predominantly guided by the reference image(s)

So this parameter mainly affects the overall influence strength of the visual tokens relative to the text-only branch.

3. vision_megapixels

Controls the image resolution used inside Text Encode (Krea2) for visual encoding.

Higher values generally preserve more structural and fine-grained visual information. In practice, this affects the detail level contained in the visual tokens.

Rule of thumb

These three controls interact. A useful rule of thumb is: The higher vision_megapixels, the lower the Visual Guidance Branch Weight will usually need to be to maintain a good balance with the text prompt.

In short:

• Sigma Threshold = when visual guidance starts

• Visual Guidance Branch Weight = how strongly it acts

• vision_megapixels= how much visual detail it can provide

Recommended starting points

Useful starting values are often around:

• Sigma Threshold: 0.92 ... 0.96

• Visual Guidance Branch Weight: 0.65 ... 0.85

However, these are only starting points, not strict operating limits.

Depending on:

• the reference images,

• the requested scene,

• the amount of visual detail,

• and Ideogram V4’s own model biases,

much more extreme settings can still be useful.

Examples:

• Sigma Threshold = 1.0 may work better when Ideogram otherwise commits too early to an unwanted default scene structure

• Visual Guidance Branch Weight > 1.0 can sometimes be useful if important reference information is otherwise too weak

A concrete example is a riverside scene: Ideogram had a strong bias toward showing only a narrow strip of the opposite riverbank. In that case, only Sigma Threshold = 1.0 worked well, because visual guidance had to be active from the very beginning in order to steer the global composition.

So the practical message is: Use the recommended values as a starting point, but do not hesitate to test substantially stronger settings when the model’s own priors dominate too much. Also note that the parameter effects are not always linear. Even small changes can sometimes lead to noticeably different interpretations.

Use cases

The workflow supports several interesting usage modes. The example panels on the page illustrate the three most important ones.

1. Guided scene generation

Use two reference images plus a text prompt.

Typical pattern:

• one image provides a person, object, or appearance cue,

• the other provides environment, atmosphere, or scene context,

• the text prompt defines the actual event or scenario.

This is illustrated by the panel with the firefighter / woman / mountain fire scene and also by the dog / harbour riverside example.

2. Image reinterpretation

Use one reference image plus a text prompt that specifies the output domain or style.

Examples:

• "Real photograph."

• "Impressionist painting."

This allows a single reference image to be reinterpreted into different output styles or domains while preserving substantial scene structure. This is illustrated by the Image Reinterpretation panel.

3. Image fusion

Use two reference images with only a very short or weak text prompt.

In this mode, Ideogram can synthesise a coherent new scene from structural elements contributed by both images. This is more than simple blending: often, the model builds a new consistent world out of both references. This is illustrated by the Image Fusion panel.

Required custom nodes

Main dependencies:

• ComfyUI-Krea2TextEncoder (comfyui-krea2-text-encoder)

• rgthree-comfy

• ComfyUI-KJNodes

• WAS-NS

• ComfyUI-Crystools

Optional:

• A Person Mask Generator (a-person-mask-generator)

The optional Person Mask Generator nodes can simply be removed without affecting the core workflow.

Limitations and observations

This is an experimental technique, not a dedicated image-editing workflow.

Some practical limitations:

• exact geometry is not guaranteed,

• details may drift,

• text inside reference images is usually not reproduced faithfully,

• multiple reference images can interact in unexpected ways.

A few important observations:

• Strong model priors matter. Famous or highly recognisable images may trigger strong internal biases.

• Two-reference behaviour is not always simple blending. The model may invent its own semantic relationship between the references.

• Parameter effects are not fully linear. Small changes can sometimes noticeably alter the interpretation.

• High visual detail can require lower vision branch weight. Otherwise the text prompt may lose too much influence.

In short: the workflow is powerful, but it remains exploratory and benefits from experimentation.

Acknowledgements

Special thanks to ethanfel for the Krea2 text encoder implementation that provides the Qwen3-VL-based visual-conditioning path used by this workflow.

Thanks also to:

• the ComfyUI developers for Ideogram V4 support,

• rgthree for rgthree-comfy and especially the Power Puter used for the sigma logic,

• the developers of KJNodes, WAS-NS, Crystools, and "A Person Mask Generator" for the utility nodes used in the workflow.

The actual workflow idea explored here includes:

• visual guidance for Ideogram V4,

• selective reference preprocessing,

• sigma-dependent delayed visual guidance,

• and weighted combination of text-only conditioning with text+visual conditioning.

Final note

This workflow is meant as a practical and compact experiment in visual prompting / visual guidance for Ideogram V4.

If you try it, I recommend:

1. start with the notes already embedded in the workflow,

2. begin with moderate parameter settings,

3. and then experiment freely when the model’s own priors dominate too much.

Have fun!