Compact Illustrious / SDXL / Pony Utility Workflow

A compact, practical workflow for SDXL, Illustrious, and Pony models, balancing useful features with simplicity without becoming a giant node jungle.

It's Features :

⚡ First Block Cache : up to ~2× faster generation
✨ Face Detailer : fixes common face issues in Illustrious, SDXL & Pony
📝 Metadata Saver : easily preserve resources for CivitAI uploads
🎮 ControlNet Support : pose, depth, edges & more
🎛️ Simple utility controls : everything you need, without unnecessary complexity

Small, fast, and packed with useful tools.

ControlNet Installation Guide :

This workflow uses ControlNet with an SDXL-based model (Illustrious). If you don't already have the required ControlNet model installed, follow the instructions below.

⚠️ Important: Illustrious is based on the SDXL architecture, so you need an SDXL-compatible ControlNet. Do not use SD 1.5 ControlNet models.

1. Download the ControlNet Model

You can find SDXL ControlNet models on Hugging Face:

SDXL ControlNet Collection:
Hugging Face - SDXL ControlNet Models

Common ControlNet types include:

• Canny : edges and overall structure

• Depth : depth and spatial structure

• OpenPose : human pose

• Lineart : line and illustration structure

• SoftEdge / HED : softer structural guidance

For example, the official Diffusers SDXL ControlNet models include Canny and Depth variants.

2. Download Using Git

If you have Git installed, you can clone a Hugging Face model repository directly:

git clone https://huggingface.co/<MODEL_OWNER>/<MODEL_NAME>

For example:

git clone https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0

After downloading, copy the required model files into:

ComfyUI/models/controlnet/

3. Download Using Hugging Face CLI

Alternatively, you can download the model directly with the Hugging Face CLI:

hf download <MODEL_OWNER>/<MODEL_NAME> --local-dir ComfyUI/models/controlnet/<MODEL_NAME>

Example:

hf download diffusers/controlnet-canny-sdxl-1.0 --local-dir ComfyUI/models/controlnet/controlnet-canny-sdxl-1.0

If you don't have the Hugging Face CLI installed:

pip install -U huggingface_hub

4. Install the Model Manually

You can also download the .safetensors file directly from the model's Hugging Face page and place it here:

ComfyUI/
└── models/
    └── controlnet/
        └── your_controlnet_model.safetensors

ComfyUI automatically detects ControlNet models placed in ComfyUI/models/controlnet/. You may need to refresh or restart ComfyUI before the model appears in the ControlNet Loader dropdown.

5. Install ControlNet Preprocessors

If this workflow also requires a ControlNet preprocessor such as OpenPose, Depth, Canny, Lineart, or HED, install the ComfyUI ControlNet Auxiliary Preprocessors:

ComfyUI ControlNet Auxiliary Preprocessors - GitHub

From your ComfyUI custom_nodes folder:

cd ComfyUI/custom_nodes
git clone https://github.com/Fannovel16/comfyui_controlnet_aux.git

Then restart ComfyUI.

Note: The ControlNet model and the preprocessor are separate components. For example, an OpenPose ControlNet requires an appropriate OpenPose preprocessor to generate the pose conditioning image. ControlNet itself does not automatically convert a normal image into a Canny map, depth map, pose map, etc.

Quick Setup

If you're using the same ControlNet as this workflow, the basic setup is:

ComfyUI/
├── models/
│   └── controlnet/
│       └── YOUR_CONTROLNET_MODEL.safetensors
│
└── custom_nodes/
    └── comfyui_controlnet_aux/

After installation, restart ComfyUI and select the ControlNet model from the Load ControlNet Model node.