Poor color transfer on custom target sketch even with official test2 reference and full reference canvas

[honsoncn-stack]

Hi, thank you for releasing MagicColor.

I am trying to reproduce and diagnose MagicColor inference behavior. The official test1 / test2 examples work on my remote GPU setup, but when I replace only the target sketch with a custom sketch, the result becomes almost grayscale or only weakly colored.

Environment

• MagicColor commit: c774dce8d5074fee4f5e34b45ab30a86c22fa0fb
• GPU: NVIDIA RTX 4090 24GB
• Python: 3.10
• PyTorch: 2.1.2 + CUDA 11.8
• Inference precision: FP32
• denoising_steps: 20
• guidance_scale: 3.5
• processing_res: 512
• seed: 123
• setting_config: dafault

The official test1 and test2 samples produce colorful outputs in my setup, so I do not think the checkpoint or runner is completely broken.

Problem

I used the official test2 Doraemon reference image and mask, but replaced the target sketch with my own Doraemon lineart.

The output does not correctly follow the reference colors. It is mostly grayscale or only weakly blue-gray, even when I make the used reference canvas almost fully occupied by the Doraemon reference.

What I tested

I tested the following variants:

1. Official test2 control sample.
2. Custom Doraemon target sketch + official test2 Doraemon reference 1 only.
3. Same as 2, but with the reference enlarged and centered on the used reference canvas.
4. Same as 2, but with the Doraemon reference cropped by its mask bbox and stretched to fill the whole 512x512 used reference canvas, using a full-canvas mask.

The dataloader log shows load 1 instances ...., so the custom experiments use only one Doraemon reference instance, not the other official test2 references.

Approximate color diagnostics

setting	mean_saturation	saturated_ratio	hue_bin_count	observation
official test2 control	0.2025	0.3125	7	successful colorization
custom target + official ref1	0.0641	0.0277	2	very weak color activation
custom target + ref enlarged to 448	0.0396	0.0192	0	mostly grayscale
custom target + ref stretched to full canvas	0.0948	0.0619	2	stronger blue-gray activation, but still not correct color transfer

In the full-canvas variant, the used reference canvas is almost completely occupied by the Doraemon reference:

• near-black ratio: about 0.0040
• colorful ratio: about 0.2905

However, the generated result still does not recover the expected blue body / red nose / yellow bell colors.

What I expected

Since the target sketch and the reference both describe Doraemon, I expected the model to transfer at least the main reference colors:

• blue body
• red nose / collar
• yellow bell

But the output remains mostly gray or weakly blue-gray.

Questions

Is this expected behavior for MagicColor when the target sketch distribution differs from the official examples?

More specifically:

1. Does MagicColor rely strongly on the target sketch style or lineart distribution?
2. Is the reference mask used only to crop and place the reference instance on the reference canvas, or does it also affect semantic matching later?
3. Is there any recommended preprocessing for custom target sketches?
4. Should custom sketches be generated by a specific lineart extractor to better match the training distribution?
5. Are there any inference settings that are important for non-official custom sketches?
6. Is stretching the reference instance to fill the whole canvas a bad idea for this model, even if it removes the black background?

I can provide comparison images and input files if helpful.

Thanks!