juhirats commited on
Commit
a0aa04c
Β·
verified Β·
1 Parent(s): 612a01f

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +317 -0
README.md ADDED
@@ -0,0 +1,317 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: openrail++
3
+ base_model: runwayml/stable-diffusion-v1-5
4
+ tags:
5
+ - stable-diffusion
6
+ - stable-diffusion-diffusers
7
+ - text-to-image
8
+ - diffusers
9
+ - lora
10
+ - lcm
11
+ - latent-consistency-model
12
+ datasets:
13
+ - Mercity/laion-subset
14
+ inference: true
15
+ widget:
16
+ - text: "a futuristic cyberpunk city at night with neon lights and rain reflections"
17
+ parameters:
18
+ num_inference_steps: 6
19
+ guidance_scale: 1.0
20
+ - text: "a portrait of a cat wearing a detective hat, film noir style"
21
+ parameters:
22
+ num_inference_steps: 6
23
+ guidance_scale: 1.0
24
+ - text: "a majestic lion standing on a rock, overlooking the african savannah at sunset"
25
+ parameters:
26
+ num_inference_steps: 6
27
+ guidance_scale: 1.0
28
+ ---
29
+
30
+ # LCM-LoRA SD1.5 - Checkpoint 800
31
+
32
+ ## Mid Training - Vibrant Style
33
+
34
+ <div align="center">
35
+ <img src="https://huggingface.co/Mercity/lcm-lora-sd1.5-800/resolve/main/comparison_grid.png" alt="Checkpoint 800 Comparison Grid">
36
+ </div>
37
+
38
+ ---
39
+
40
+ ## πŸ“ Part of Checkpoint Series
41
+
42
+ This is **Checkpoint 800** in our LCM-LoRA training series. Each checkpoint has different characteristics:
43
+
44
+ [Checkpoint 400](https://huggingface.co/Mercity/lcm-lora-sd1.5-400) β€’ **Checkpoint 800** (current) β€’ [Checkpoint 1200](https://huggingface.co/Mercity/lcm-lora-sd1.5-1200) β€’ [Checkpoint 1600](https://huggingface.co/Mercity/lcm-lora-sd1.5-1600)
45
+
46
+ ---
47
+
48
+ ## Model Description
49
+
50
+ This checkpoint represents training at **800 steps** in our LCM-LoRA progression for Stable Diffusion v1.5.
51
+
52
+ **Characteristics:**
53
+ - Mid-training checkpoint with vibrant, artistic outputs. Strong visual impact with saturated colors and expressive style.
54
+ - **Best for:** Artistic applications, vibrant aesthetic, expressive style
55
+ - **Quality:** High visual impact, strong artistic direction, vivid colors
56
+
57
+ **Key Features:**
58
+ - ⚑ **10x Faster**: Generate images in 4-6 steps vs 50 steps
59
+ - 🎯 **LoRA Adapter**: Only ~100MB, works with any SD1.5 model
60
+ - πŸ”§ **Easy Integration**: Drop-in replacement using diffusers
61
+ - πŸ“Š **Proven Quality**: See comparison grid above
62
+
63
+ ---
64
+
65
+ ## Checkpoint Comparison
66
+
67
+ This checkpoint is part of a training series. Compare with other checkpoints:
68
+
69
+ | Steps | Model | Characteristics |
70
+ |-------|-------|-----------------|
71
+ | 400 | [lcm-lora-sd1.5-400](Mercity/lcm-lora-sd1.5-400) | Early training checkpoint showing foundational LCM capabilities. Provides decent... |
72
+ | **800** | [lcm-lora-sd1.5-800](Mercity/lcm-lora-sd1.5-800) | Mid-training checkpoint with vibrant, artistic outputs. Strong visual impact wit... **β†’ This checkpoint** |
73
+ | 1200 | [lcm-lora-sd1.5-1200](Mercity/lcm-lora-sd1.5-1200) | Higher training with more refined outputs. Some prompts may show signs of overfi... |
74
+ | 1600 | [lcm-lora-sd1.5-1600](Mercity/lcm-lora-sd1.5-1600) | Final training checkpoint with mature, consistent outputs. Well-balanced and rel... |
75
+
76
+ ---
77
+
78
+ ## Sample Outputs
79
+
80
+ ### Installation
81
+
82
+ ```bash
83
+ pip install --upgrade diffusers transformers accelerate
84
+ ```
85
+
86
+ ### Basic Usage
87
+
88
+ ```python
89
+ import torch
90
+ from diffusers import StableDiffusionPipeline, LCMScheduler
91
+
92
+ # Load base SD1.5 model
93
+ pipe = StableDiffusionPipeline.from_pretrained(
94
+ "runwayml/stable-diffusion-v1-5",
95
+ torch_dtype=torch.float16
96
+ )
97
+ pipe.to("cuda")
98
+
99
+ # Load this LCM-LoRA checkpoint
100
+ pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-800")
101
+
102
+ # IMPORTANT: Use LCM scheduler
103
+ pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
104
+
105
+ # Generate with just 4-6 steps!
106
+ prompt = "a portrait of a cat wearing a detective hat, film noir style"
107
+ image = pipe(
108
+ prompt=prompt,
109
+ num_inference_steps=6,
110
+ guidance_scale=1.0
111
+ ).images[0]
112
+
113
+ image.save("output.png")
114
+ ```
115
+
116
+ ### Recommended Settings
117
+
118
+ ```python
119
+ num_inference_steps = 6 # Optimal for this checkpoint
120
+ guidance_scale = 1.0 # Required for LCM
121
+ ```
122
+
123
+ ---
124
+
125
+ ## Training Details
126
+
127
+ | Parameter | Value |
128
+ |-----------|-------|
129
+ | **Checkpoint** | 800 |
130
+ | **Base Model** | runwayml/stable-diffusion-v1-5 |
131
+ | **Training Steps** | 800 |
132
+ | **Dataset** | Mercity/laion-subset |
133
+ | **LoRA Rank** | 96 |
134
+ | **LoRA Alpha** | 96 |
135
+ | **Resolution** | 512Γ—512 |
136
+ | **Batch Size** | 64 |
137
+ | **Learning Rate** | 1e-4 |
138
+ | **Optimizer** | AdamW |
139
+
140
+ ---
141
+
142
+ ## Sample Outputs
143
+
144
+ The comparison grid above shows outputs from this checkpoint at 2, 4, and 6 inference steps, compared to standard SD1.5 at 50 steps.
145
+
146
+ **Prompts included:**
147
+ 1. Futuristic cyberpunk city with neon lights and rain reflections
148
+ 2. Portrait of a cat wearing a detective hat, film noir style
149
+ 3. Cozy coffee shop interior with warm lighting and plants
150
+ 4. Ancient Japanese temple in misty mountain landscape at sunrise
151
+ 5. Majestic lion on rock overlooking African savannah at sunset
152
+ 6. Magical forest with glowing blue mushrooms and fireflies
153
+ 7. Vintage red steam locomotive crossing stone viaduct over canyon
154
+
155
+ <details>
156
+ <summary>View individual samples</summary>
157
+
158
+ All sample images for this checkpoint are available in the `samples/` directory.
159
+
160
+ </details>
161
+
162
+ ---
163
+
164
+ ## Performance
165
+
166
+ ### Speed Comparison
167
+
168
+ | Method | Steps | Time (A100) | Time (RTX 3090) |
169
+ |--------|-------|-------------|-----------------|
170
+ | SD1.5 Default | 50 | ~15s | ~25s |
171
+ | SD1.5 Fast | 25 | ~8s | ~13s |
172
+ | **LCM-LoRA (this)** | **6** | **~2s** | **~3s** |
173
+ | **LCM-LoRA (this)** | **4** | **~1.5s** | **~2s** |
174
+
175
+ ### Quality Progression
176
+
177
+ - **2 steps**: Fast, captures main composition
178
+ - **4 steps**: Good balance, suitable for most cases
179
+ - **6 steps**: Best quality (recommended)
180
+ - **8 steps**: Slightly better, diminishing returns
181
+
182
+ ---
183
+
184
+ ## Advanced Usage
185
+
186
+ ### Speed Optimization
187
+
188
+ ```python
189
+ # Fuse LoRA for faster inference
190
+ pipe.fuse_lora(lora_scale=1.0)
191
+
192
+ # Use xformers for memory efficiency
193
+ pipe.enable_xformers_memory_efficient_attention()
194
+
195
+ # Compile model (PyTorch 2.0+)
196
+ pipe.unet = torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True)
197
+ ```
198
+
199
+ ### Multiple LoRAs
200
+
201
+ ```python
202
+ # Combine with other LoRAs
203
+ pipe.load_lora_weights("other_style.safetensors", adapter_name="style")
204
+ pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-800", adapter_name="lcm")
205
+
206
+ # Adjust weights
207
+ pipe.set_adapters(["style", "lcm"], adapter_weights=[0.8, 1.0])
208
+ ```
209
+
210
+ ### Switch Between Checkpoints
211
+
212
+ ```python
213
+ # Load different checkpoints from this series
214
+ pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-400")
215
+ pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-800")
216
+ pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-1200")
217
+ pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-1600")
218
+ ```
219
+
220
+ ---
221
+
222
+ ## Series Information
223
+
224
+ ### Training Progression
225
+
226
+ This checkpoint is part of a training series showing LCM-LoRA evolution:
227
+
228
+ ```
229
+ Training Steps: 400 ──→ 800 ──→ 1200 ──→ 1600
230
+ ↓ ↓ ↓ ↓
231
+ Quality: Baseline Peak Refined Mature
232
+ Style: Soft Vibrant Balanced Stable
233
+ ```
234
+
235
+ ### Download All Checkpoints
236
+
237
+ ```bash
238
+ # Download all checkpoints for comparison
239
+ huggingface-cli download Mercity/lcm-lora-sd1.5-400
240
+ huggingface-cli download Mercity/lcm-lora-sd1.5-800
241
+ huggingface-cli download Mercity/lcm-lora-sd1.5-1200
242
+ huggingface-cli download Mercity/lcm-lora-sd1.5-1600
243
+ ```
244
+
245
+ ---
246
+
247
+ ## Usage Tips
248
+
249
+ ### For Best Results
250
+
251
+ 1. **Always use `LCMScheduler`** - Required for LCM
252
+ 2. **Set `guidance_scale=1.0`** - CFG doesn't work with LCM
253
+ 3. **Use 4-8 steps** - Optimal range is 6 steps
254
+ 4. **Same prompts as SD1.5** - No special prompting needed
255
+
256
+ ### Checkpoint Selection
257
+
258
+ - **Testing/comparison?** Try different checkpoints to find your preference
259
+ - **Different characteristics:** Each checkpoint has unique qualities
260
+ - **Training progression:** See how the model evolves with more training
261
+
262
+ ---
263
+
264
+ ## Limitations
265
+
266
+ - Trained on 512Γ—512 resolution (best results at this size)
267
+ - Requires `LCMScheduler` - other schedulers won't work
268
+ - `guidance_scale` must be 1.0 (CFG incompatible with LCM)
269
+ - Each checkpoint has slightly different characteristics
270
+
271
+ ---
272
+
273
+ ## Citation
274
+
275
+ If you use this model in your research, please cite:
276
+
277
+ ```bibtex
278
+ @article{luo2023latent,
279
+ title={Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference},
280
+ author={Luo, Simian and Tan, Yiqin and Huang, Longbo and Li, Jian and Zhao, Hang},
281
+ journal={arXiv preprint arXiv:2310.04378},
282
+ year={2023}
283
+ }
284
+
285
+ @article{hu2021lora,
286
+ title={LoRA: Low-Rank Adaptation of Large Language Models},
287
+ author={Hu, Edward J and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},
288
+ journal={arXiv preprint arXiv:2106.09685},
289
+ year={2021}
290
+ }
291
+ ```
292
+
293
+ ---
294
+
295
+ ## License
296
+
297
+ This model is released under the same license as Stable Diffusion v1.5:
298
+ - **CreativeML Open RAIL-M License**
299
+ - Commercial use allowed with restrictions
300
+ - See: https://huggingface.co/spaces/CompVis/stable-diffusion-license
301
+
302
+ ---
303
+
304
+ ## Acknowledgments
305
+
306
+ - **Base Model**: [Stable Diffusion v1.5](https://huggingface.co/runwayml/stable-diffusion-v1-5)
307
+ - **LCM Method**: [Latent Consistency Models](https://arxiv.org/abs/2310.04378)
308
+ - **LoRA Method**: [Low-Rank Adaptation](https://arxiv.org/abs/2106.09685)
309
+ - **Training Framework**: [Diffusers](https://github.com/huggingface/diffusers)
310
+
311
+ ---
312
+
313
+ ## More Information
314
+
315
+ - **Other checkpoints in series**: [Checkpoint 400](https://huggingface.co/Mercity/lcm-lora-sd1.5-400) β€’ **Checkpoint 800** (current) β€’ [Checkpoint 1200](https://huggingface.co/Mercity/lcm-lora-sd1.5-1200) β€’ [Checkpoint 1600](https://huggingface.co/Mercity/lcm-lora-sd1.5-1600)
316
+ - **Discussions**: [Model discussions](https://huggingface.co/Mercity/lcm-lora-sd1.5-800/discussions)
317
+ - **Report issues**: [Community tab](https://huggingface.co/Mercity/lcm-lora-sd1.5-800/discussions)