I’m going to train LoRa for Z Image Turbo. But I don’t like the base output (background and colors). I want to do some color adjustment steps with NAG and Detail Daemon.
Is the base output recommended for LoRa training, or should I train with the most optimal images for my goals?
How will the two approaches affect my results?
When using it as a base model for LoRA training, Base might be fundamentally more suitable than Turbo? But Turbo came out first and is more widespread… making the choice difficult.
A LoRA does not “fix” a model in the abstract. It learns to reproduce the distribution of whatever you train it on (subject, lighting, palette, background types, contrast curve, etc.). This is why dataset choice matters more than most hyperparameters.
Z-Image Turbo adds a second complication: it is a few-step distilled model that does not rely on classifier-free guidance (CFG) at inference and, in the official prompting guidance, does not use negative prompts at all. (Hugging Face) Turbo is typically run at ~9 steps with guidance scale 0.0.
That pushes more of the “look” (including background/color tese model prior, and makes classic “negative prompt fixes” weaker unless you use something like NAG.
Using images that are clean, consistent, and not heavily “styled” is often the most robust path.
In this case, training on “base-ish” outputs can be acceptable, because you are not trying to reprogram the global aesthetic—just teach a concept.
Then training on base-like images is not recommended for your goal, because:
For a look-changing objective, your training images should already reflect your desired look.
If you want different backgrounds and color behavior, training on your “optimal” images is the direct route.
But “optimal” should mean:
Why the caution: Detail Daemon explicitly notes that pushing it too far produces an oversharpened and/or HDR effect. (GitHub) If you train on images that already have that signature and then also apply Detail Daemon at inference, you can get “double enhancement.”
What improves
What stays the same
Typical failure mode
What improves
What gets worse
Typical failure mode
Train on images that are 80–90% your target look (good grading, but not extreme), then do the last 10–20% with NAG/Detail Daemon.
NAG is explicitly motivated by the problem that CFG-based negative guidance collapses in few-step regimes, and it restores effective negative-style control by operating in attention space. (arXiv)
Implication for you
Detail Daemon adjusts sigma/sampling behavior to enhance detail and can reduce unwanted blur, but can produce an oversharpen/HDR look if pushed. (GitHub)
Implication for you
The official Z-Image repo recommends for the Base model:
That is exactly the environment where you can:
Turbo is distilled for fast steps and “no CFG/negative prompts.” (Hugging Face)
A commonly reported issue is that LoRAs trained directly on Turbo can lose the acceleration behavior—images become blurry at “fast” settings while looking normal at slower settings. (Hugging Face)
If you must train on Turbo anyway, there are two established mitigation paths:
Train on Z-Image (Base) with a dataset that reflects your desired palette/background, then:
This leverages Base’s controllability and avoids Turbo’s distillation fragility. (GitHub)
Train for Turbo, but plan the pipeline up front:
If you want the palette/background sometimes but not always:
my_grade_style) and keep other captions content-focused.Use Detail Daemon primarily at inference; if you must use it in dataset creation, keep it mild and consistent to avoid baking in HDR/sharpen artifacts. (GitHub)
Thank you very much. The base model is very inconsistent and produces poor quality images, or perhaps I’m not getting the desired results because I’m applying the same prompting technique to Turbo as I did to the original. In Turbo, the face changed significantly when the scene changed, so I created one portrait and then converted it into 15 different portraits from various angles using Nano Banana. I trained a Face LoRa with it, and now I can produce images with more consistent faces, almost exactly as I want in Turbo. My goal, as you suggested, is to produce 40 images with different locations, lighting, and elements. Then I will train a new LoRa with it. Currently, with a 1:4 LR, 16 rank, no quantization, and BF16 output, I get a sweet spot of 2400-3000 at 3k stepping. I previously tested it with 4k stepping, and the sweet spots were 2200 and 2600. I want to continue with this approach for now.
However, I still have some questions. For 40 images, is rank 16 ideal, or should I choose 32 (I’ll choose FP32 for quality anyway)? But since all parameters affect each other, finding the optimal solution is really difficult. Will 3-4k step be enough when I go up to 40 images? Should I use 1280 base resolution or 1536 when creating the images? Because Ostris has a 1536 resolution option. If I increase the rank, I think I’ll need to adjust the LR to 5e-5.
To be honest, it’s a very challenging process for someone who wants to achieve perfection.
There’s no absolute right answer for training parameters by nature…
It becomes a matter of choosing what kind of LoRA (model) behavior is likely to emerge. And often, you can’t say for sure without actually going through fine-tuning…
The inconsistency in prompts between Turbo and Base seems to be because their prompt quirks are completely different…
Z-Image Turbo and Z-Image Base want different prompting and sampling behaviors:
So if you applied “Turbo-style prompting” (short prompt, no negatives, guidance ~0) to Base, Base can look sloppy or inconsistent; and if you apply “Base-style prompting” (strong negatives + higher guidance) to Turbo, Turbo can drift (including face changes) because it’s not operating how it was distilled to operate. (Hugging Face)
Your “face drifts when the scene changes” observation is very consistent with Turbo’s few-step behavior, and using a face LoRA to stabilize identity is a reasonable workaround.
Turbo tends to degrade when stacking multiple LoRAs. This is widely reported in community testing and discussion. (Hugging Face)
That means your plan is actually strategically good because it can produce a single “combined” LoRA (identity + environment handling) so you don’t have to stack a face LoRA + a scene/style LoRA at inference.
The main risk is feedback-loop learning: if your 40 images are all synthetic outputs from the same pipeline, the second LoRA can learn the pipeline’s artifacts and biases very strongly. The fix is simple: curate hard, and force real diversity (lighting, lenses, backgrounds, compositions, distances, expressions).
Rule of thumb:
Community experience is mixed: some people find rank 16 “enough,” others report rank 32+ helps photorealistic micro-detail. Treat those reports as anecdotal, but directionally useful. (Reddit)
Recommendation for your 40-image “multi-scene” run
Steps depend on batch size/repeats, but you can reason with “steps per image”:
If (roughly) batch=1 and repeats=1:
So if your “sweet spot” previously appeared around 2,400–3,000 steps on a smaller set, the same “training intensity” on 40 images will usually occur later (often ~6k–8k steps).
This matches common Turbo LoRA baselines that treat ~2,500–3,000 as a starting point for small datasets, not a hard universal number. (Hugging Face)
Practical recommendation
There are two separate issues here:
(A) What Turbo was “in-domain” for
A Tongyi discussion on resolution/latent sizing strongly implies staying within a domain like 768–1280 around the “1024 grid,” and points to predefined resolution choices used in their app. (Hugging Face)
(B) What tooling exposes
Many workflows expose resolution presets including 1024 / 1280 / 1536. (GitHub)
What this means for you
Recommendation
A very common safe band for diffusion LoRA LR is 1e-4 to 5e-5. (Hugging Face)
If you go from rank 16 → 32, reducing LR (e.g., 1e-4 → 5e-5) is a reasonable safety move because you’re increasing capacity and the training can become more “aggressive.”
If your training is actually happening in bf16/fp16 (which is typical in these toolchains), saving the LoRA as fp32 does not recover lost precision; it mostly stores already-rounded values in a larger container. (Reddit)
BF16 is generally considered a strong default for stability/efficiency, and FP32 is rarely the deciding factor for visual quality compared to dataset quality and stopping at the right step. (RunComfy)
Recommendation
Training directly on Turbo can break the acceleration behavior; this is why “training adapter / de-distillation” and “DistillPatch” approaches exist.
If your LoRA starts looking fine at 20–30 steps but falls apart at 8–9 steps, that’s a classic sign you need one of these “preserve acceleration” approaches. (Hugging Face)
Hyperparameters interact, but you can make the search manageable by fixing everything except one axis at a time:
If you try to change rank/LR/steps/resolution together, it becomes almost impossible to tell what helped.
Because you’re using generated images as training targets, you should deliberately inject variation so the second LoRA learns identity rather than “that one portrait’s hidden artifacts”:
This reduces memorization and improves the “face stays the same when the scene changes” property you care about.
I can’t thank you enough for your detailed explanations. I really appreciate it. I’m going to follow this step exactly now. Face Lora is working well, only the ear isn’t closing 
because it was visible in all the images, so I added this and similar things to the caption and will train it again. This way I’ll get rid of this problem. Even at strength 1, it gives the perfect exposure for the composition (except for the ear, which requires a very specific prompt to close :D).
So, Face Lora seems to be working hypothetically. Now, the next step is to create 40 images with real variety. Also, before doing that, I want to train Look Lora, as you said. I’ve collected many images from Unsplash: travel, gym, cafe, streets, etc. They are all very high quality and high resolution, but I think I’ll need to resize them a bit first.
So, what are your suggestions for Look Lora? Since there will be a lot of detail here, and it won’t be learning just one subject, I think 32 or a higher rank would be logical. I currently have 60 images. Do you think this is enough to train Look Lora? I’m leaving a screenshot below. What are my chances of getting sharp images with the LoRa tool I’ll be creating from these images? I specifically tried to select images where a person might be present.
I also trained a LoRa with the base model, now I’m going to try it with the turbo.
Claude also suggested that applying an img2img with ZIT to the Nano Banana outputs might be a good idea. But I think the images are good, high-quality, realistic, and consistent (except for the watermark, which I remove with another tool). Do you think it’s necessary?
Meanwhile, I think training can be done on the base model and then the turbo can be used. But as you said, 3k steps were insufficient; more are needed.
Using the capabilities and knowledge of high-end commercial AI for training open-source models is generally a good idea, not just for Nano Banana.
A Look LoRA is best at learning global visual biases: color palette, contrast curve, lighting mood, lens/render “feel,” background complexity preference. It is not a reliable way to “upgrade sharpness” beyond what the base model + sampling can already produce—unless the base model is already capable and your LoRA is mainly nudging it toward consistently sharp photographic structure.
With Z-Image Turbo, keep in mind:
This affects your “chances of sharp images” more than rank alone.
60 high-quality photos is enough for one coherent look (e.g., “clean editorial travel photography, natural skin, balanced contrast, slightly warm highlights”). If your set spans many unrelated aesthetics (harsh HDR city night + soft pastel cafe + gritty gym flash + cinematic teal/orange), the LoRA tends to learn a blurry average of the styles and may feel like it “does nothing” or introduces instability.
You intentionally included scenes where people might be present. That is good if your Look LoRA must preserve:
Rank is capacity. Higher rank can learn more variation, but it also makes it easier to learn unwanted correlations and artifacts. Diffusers’ LoRA config explicitly treats r (rank) as a core capacity parameter. (Hugging Face)
Rank 16 is usually the best first run for a Look LoRA with ~60 images.
Move to Rank 32 only if you can clearly describe a “capacity failure,” such as:
Why not start at 32?
With only 60 images, Rank 32 can more easily overfit on specific scene content (“this kind of street = this color cast”) instead of learning a transferable grade.
When dataset size increases, a fixed step count means fewer updates per image. So if 3k steps was your “sweet spot” on a smaller set, that does not transfer directly.
A simple planning heuristic:
Recommendation (works with your “sweet spot hunting” approach)
If you increase rank to 32, reducing LR (e.g., toward 5e-5) is a common stability move. (This aligns with typical LoRA guidance that higher capacity usually wants gentler updates.) (Hugging Face)
Reasons:
Only if:
Given your stated goal (look + backgrounds + colors, not microtexture replication), 1280 is the safer choice.
High, if these conditions hold:
You preserve Turbo’s fast-step behavior
Your training set is consistently sharp
Your captions don’t teach the wrong thing
LoRA can reduce sharpness if it breaks Turbo distillation or if it learns “soft” images as part of the look.
When it helps
When it hurts
Practical compromise
Example:
If you can’t summarize the look, the dataset is probably too mixed.
Turbo doesn’t use negative prompts, so any “don’t do X” must be phrased as positive constraints in your prompt. (Hugging Face)
Detail Daemon can help detail, but don’t overuse it in training data or you risk teaching the HDR/oversharpen signature. (GitHub)
Use one base template and change only {location}, {lighting}, {angle}:
{trigger_face}, {trigger_look}, photorealistic editorial portrait photo,
identity consistent face, natural skin texture, sharp focus on eyes,
balanced exposure, realistic shadows, clean color science, neutral white balance,
{lighting}, {angle}, {location},
high detail, clean edges, no text, no watermark, no logo, no extra faces
Turbo expects you to put constraints in the positive prompt, because negatives aren’t used. (Hugging Face)
Hopefully this title will rank highly on Google because we’ve discussed everything necessary throughout this process and provided very detailed links.
Great question! For LoRA training on Z Image Turbo, I’d recommend using the base output as your training foundation.
Why base output works better:
Alternative approach:
If you specifically need the color-adjusted outputs in your LoRA, consider:
The key insight is that LoRA learns the “difference” from the base model - so if you train on adjusted images, your LoRA will essentially be learning to correct back to base. Hope this helps!
Thanks for the reply. However, there are no IP adapters or similar solutions on Zit yet. I’m currently researching trigger words. Also, when exactly is LoRa considered to have learned? When it responds to every prompt that includes a female description and a trigger word? Or when it responds to a neutral prompt that uses a trigger word without specifying gender?
@John6666 When do we assume LoRa has learned? When it responds to gender-specific prompts, or when it responds to neutral prompts as well?
The extent to which you should settle on a LoRA with strong influence as a result of fine-tuning will likely come down to personal preference and intended use. Taking checkpoints during training is probably a good idea for reliability. (Train them and choose later.)
A LoRA is best judged by validation prompts across checkpoints, not by one “lucky” prompt. The standard training workflow is to save intermediate checkpoints and compare them with a fixed validation prompt or prompt set. (Hugging Face)
I would treat it as “learned” when:
So, for your case:
trigger + portrait in a cafe still gives the same persontrigger + person in a cafe / gym / street / travel scene still gives the same personIf it only responds correctly to gender-specific prompts, that usually means the LoRA has learned the caption pattern too narrowly, not the identity robustly.
Because your Face LoRA represents a specific face, a neutral prompt should still produce that person’s default presentation.
It does not need to become gender-ambiguous.
That means:
In other words, if the subject is female-presenting, then:
trigger, portrait photo → should worktrigger, woman in streetwear → should worktrigger, man in streetwear → not an important test, and if it fully obeys that while losing identity, that can actually mean the LoRA is too weakIt works with:
If it passes those, then the LoRA has learned the face in a useful way.
A neutral content prompt starts to inherit the color / lighting / background mood of your target look, while still remaining editable.
Your LoRA is probably not learned cleanly yet if:
Your LoRA is probably overfit if:
Overfitting and “language drift” are well-known concerns in image finetuning, which is why prior-preservation style techniques exist in related workflows. (Hugging Face)
Because Turbo is a few-step distilled model and does not use negative prompts in the intended setup, your evaluation should be done with the same Turbo-style inference setup you actually plan to use. (Hugging Face)
So the best test is not:
The best test is:
For your case:
That is the checkpoint where I would say: the LoRA has actually learned what you wanted.
Thank you very much.
I concluded from my tests that the caption structure significantly impacts the learning process. If the dataset is very small and only has one face LORA, as you mentioned, if the caption is too detailed, the system becomes dependent on it. Therefore, in such cases, the CDO (Caption Drop Out) should be increased slightly. In a completely caption-less training, the LORA becomes completely locked onto the trigger word. So, a middle ground is needed. If you only increase the steps, it becomes an overfit. For example, to get the ideal result with a dataset of 15-20 images, you should use 4000-5000 steps. If you can’t, there’s a problem with your caption structure or the dataset. Using more steps increases the learning speed; I can say this clearly. This way, you can reach the sweet spot around 3k steps. If you start with low steps and then update them, I’ve found that it often results in regression. So, starting with a high step yields the ideal result (at least in my tests).
4k steps
For example, in one dataset, the first full-body samplers were male, but the first clear female figure appeared at 1200 steps, while it started to resemble my character at 2400-2600 steps. In another dataset, it still didn’t appear at 3800 steps. I think the reason for this is the visuals (the caption structure is exactly the same). Because the first dataset mainly shows only the face and a few upper body images. The second dataset, on the other hand, generally shows the below the shoulders. So, if you are going to train Face Lora using the method I used, your visuals should definitely be only the face and neck from various angles. They should be as close as possible. This increases the learning speed and prevents you from overfitting.
Also, there should definitely be 3-5 upper body images (I’m saying this for Face Lora; your character Lora should already have a lot of variety), otherwise you will never see your character when you want a full body image. Also, based on the results I obtained from my previous full character and current Face Lora tests, I can clearly say that the most consistent caption structure is exactly as follows.
[trigger_word], [framing], [pose/action], [facial expression], [clothing short], [accessories (if any)], [location], [lighting type], [background]
When you apply this caption structure, even if your faceLoRa dataset is full studio (which I think is more effective), you can get output in any environment you want. Because you give the caption inputs you don’t want to teach it, and LoRa only adapts to the identity.
I can’t say I’m very good at explaining and summarizing what I’ve learned, but these are my conclusions. If anyone needs help with this process in the future, I can give better answers and conclusions in a Q&A session.
I think this discussion will shed light on this for many people.
I also agree with John because when you describe a woman, LoRa already gives you what you want in 1k steps. But for example, when you want a scene where she’s angry, you see a man. Because you haven’t learned that your character can be angry, and I think in the base model, this association is predominantly associated with men. In short, if you want to ensure LoRa is fully trained, your sampling structure should definitely be as follows. Yes, taking too many samples slows down training. But what do you want? Quality? Or just getting it over with? The choice is yours.
- prompt: "[trigger] in a close-up portrait with a serious intense expression. Hard street lighting casts sharp shadows across the face from the left side. Standing on an urban sidewalk at night with the street visible behind"
- prompt: "[trigger] in a close-up portrait with a soft relaxed expression, her lips slightly parted. She has thick straight bangs cut just above the eyes with hair falling past the ears, framing the face. She is outdoors in a park with green trees and grass stretching behind her. Soft natural daylight wraps evenly around her face."
- prompt: "[trigger] laughing with eyes closed and mouth open in a genuine candid moment. Half body shot, standing in front of a suburban house with a white fence. Bright natural daylight fills the scene."
- prompt: "[trigger] crying with visible tears rolling down her cheeks, her brows furrowed and eyes glistening. Her hair is pulled up in a high messy bun with loose strands falling around the face. Half body shot, she is standing in front of a house with the front porch visible behind her. Soft overcast daylight with muted tones throughout."
- prompt: "[trigger] with yellow hair and round glasses, posing casually for the camera in a half body shot. Looking out over a volcanic landscape with the crater and steam visible in the distance. Wide-angle perspective."
- prompt: "A knee-up shot of [trigger], looking at the camera, wearing a plain white T-shirt and jeans, standing barefoot on a sandy beach. Bright midday sunlight, with the sharp blue ocean and horizon stretching behind."
height: 1248
width: 832
- prompt: "A knee-up shot of [trigger], looking at the camera, wearing a yellow floral sundress, standing outdoors in a green park. Her hair is tied back in a long ponytail swaying to one side. Bright natural daylight, with trees and a walking path visible in the background."
height: 1248
width: 832
- prompt: "A woman with vibrant dark brown hair playing chess at an outdoor park table. An explosion goes off in the far background with smoke and debris rising into the sky."
- prompt: "A Siberian man wearing traditional fur-lined clothing is crouching in front of an igloo, holding a steaming cup of hot tea with both hands. Snow covers the ground and the cold air is visible."
- prompt: "A native American woman wearing traditional clothing is standing in front of a teepee, holding a bow and arrow. The sun is setting behind her, casting a warm glow on the scene."
Also, the default settings are generally good. Of course, you may need to change them depending on your dataset and goals, but I think the biggest factor will always be your step, dataset, and caption structure. Because, as someone who has tested many different settings, no other setting changes the result as much as these three.
Yes, I just ran the test again. I updated the ratios of the photos showing below the shoulders using Birme and re-trained them. I’m currently getting the first female image at 2400.
I’ve currently trained with 45 visuals in 4500 steps. But I think it’s learning very erratically; even the Native American character is starting to resemble mine, but one with a neutral prompt still doesn’t respond after 3250 steps. I think I should try increasing the CDO. Or should I use DOP? Or is this many visuals too much for ZIT? Also, should I try training with the base model?
Ah, this process is really difficult; even my inferences remain unanswered, always giving very different results.
I disabled cache text embeddings and increased CDO, but nothing changed. Ugh, this is really driving me crazy.
I can prove it. For ZIT training, instead of using natural language flow, create tags that precisely follow the template I provided above. That’s the format that works best.