FreeFuse

FreeFuse: Multi-Subject LoRA Fusion via Adaptive Token-Level Routing at Test Time

📌 TODO

• Release basic inference code
• Release inference code for controlnet, ipadapter and redux
• Supports ComfyUI
• Supports Z-Image-turbo in diffusers
• Supports Z-Image-turbo in ComfyUI
• Supports FLUX.2-klein-4B/9B in diffusers
• Supports FLUX.2-klein-4B/9B in ComfyUI
• Supports Krea2 in ComfyUI
• Experimental MOSS Image Base/SFT support in Diffusers and ComfyUI
• Anima Base v1.0 support in ComfyUI
• Support ControlNet+FreeFuse in ComfyUI
• Native MiniMax H3 text-to-audio-video support in ComfyUI

📖 Abstract

This paper proposes FreeFuse, a training-free framework for multi-subject text-to-image generation through automatic fusion of multiple subject LoRAs. In contrast to prior studies that focus on retraining LoRA to alleviate feature conflicts, our analysis reveals that simply spatially confining the subject LoRA's output to its target region and preventing other LoRAs from directly intruding into this area is sufficient for effective mitigation. Accordingly, we implement Adaptive Token-Level Routing during the inference phase. We introduce FreeFuseAttn, a mechanism that exploits the flow matching model's intrinsic semantic alignment to dynamically match subject-specific tokens to their corresponding spatial regions at early denoising timesteps, thereby bypassing the need for external segmentors. FreeFuse distinguishes itself through high practicality: it necessitates no additional training, model modifications, or user-defined masks spatial conditions. Users need only provide subject activation words to achieve seamless integration into standard workflows. Extensive experiments validate that FreeFuse outperforms existing approaches in both identity preservation and compositional fidelity.

🎨 Results

Results on Flux.dev

Results on SDXL

🏗️ Architecture

🚀 Quick Start

Installation with ComfyUI

# use comfy-cli
pip install comfy-cli
comfy node install freefuse

# or manual installation
cd <your ComfyUI path>/custom_nodes
git clone --depth 1 --branch comfyui https://github.com/yaoliliu/FreeFuse.git freefuse_comfyui

You can find example workflows in the freefuse_comfyui/workflows folder, including the MiniMax H3 video comparison workflow.

Installation with diffusers

You just need clone this repo and use uv to install dependencies.

apt-get update && apt-get install -y git-lfs && git lfs install
git clone https://github.com/yaoliliu/FreeFuse.git
git lfs pull
# or using huggingface if lfs budget is used up
hf download lsmpp/freefuse_example_loras --local-dir=./loras

uv venv
source .venv/bin/activate
uv sync

Running on Flux

without background identification

python main_freefuse_flux.py

with background identification and style LoRA

python main_freefuse_flux_bg_except.py

with redux

python redux_freefuse_flux_bg_except.py

with ipadapter

python ipadapter_freefuse_flux_bg_except.py

with controlnet

python controlnet_freefuse_flux_bg_except.py

If you running out of your vram, you can set the quantization variable in the script to 'True' to enable quantization, which will reduce the vram usage.

Running on SDXL

python main_freefuse_sdxl.py

Running on Z-Image-turbo

python main_freefuse_z_image.py

Acknowledgement

Thanks for Tekemo and jrewingwannabe947's ZIT LoRAs, which are used in our Z-Image-turbo examples.

Thanks to gpl1967 and merermero for the Krea2 support request and discussion in issue #26.

Thanks to Rudolf (GPL1967) and Agent Cow (monomatic, Hamburg) for contributing the Krea2 AutoPrompt and depth-control workflow in PR #32.

Thanks to mertkasar for identifying the bypass LoRA token-masking issue and sharing a proposed fix in issue #34.

More model comparisons

Single-LoRA character references are shown at the left of each row. See reference generation settings.

Results on MiniMax H3 video (ComfyUI)

Single-LoRA references, generated separately with single-character prompts (seed 42):

Starfire · Teen Titans LoRA only	Albedo · Albedo LoRA only

https://github.com/user-attachments/assets/401615de-1685-4e76-8975-03529ce28443

Global LoRAs on the left; automatic FreeFuse on the right, with the same prompt, seed and sampling settings. This preview is silent; the native workflow generates 768 × 768 videos with 124 frames and audio. See the H3 workflow and model setup.