lcm-lora-sd1.5-800 / README.md

juhirats

Upload README.md with huggingface_hub

a0aa04c verified about 2 months ago

preview code

raw

history blame

9.58 kB

metadata

license: openrail++
base_model: runwayml/stable-diffusion-v1-5
tags:
  - stable-diffusion
  - stable-diffusion-diffusers
  - text-to-image
  - diffusers
  - lora
  - lcm
  - latent-consistency-model
datasets:
  - Mercity/laion-subset
inference: true
widget:
  - text: a futuristic cyberpunk city at night with neon lights and rain reflections
    parameters:
      num_inference_steps: 6
      guidance_scale: 1
  - text: a portrait of a cat wearing a detective hat, film noir style
    parameters:
      num_inference_steps: 6
      guidance_scale: 1
  - text: >-
      a majestic lion standing on a rock, overlooking the african savannah at
      sunset
    parameters:
      num_inference_steps: 6
      guidance_scale: 1

LCM-LoRA SD1.5 - Checkpoint 800

Mid Training - Vibrant Style

📍 Part of Checkpoint Series

This is Checkpoint 800 in our LCM-LoRA training series. Each checkpoint has different characteristics:

Checkpoint 400 • Checkpoint 800 (current) • Checkpoint 1200 • Checkpoint 1600

Model Description

This checkpoint represents training at 800 steps in our LCM-LoRA progression for Stable Diffusion v1.5.

Characteristics:

Mid-training checkpoint with vibrant, artistic outputs. Strong visual impact with saturated colors and expressive style.
Best for: Artistic applications, vibrant aesthetic, expressive style
Quality: High visual impact, strong artistic direction, vivid colors

Key Features:

⚡ 10x Faster: Generate images in 4-6 steps vs 50 steps
🎯 LoRA Adapter: Only ~100MB, works with any SD1.5 model
🔧 Easy Integration: Drop-in replacement using diffusers
📊 Proven Quality: See comparison grid above

Checkpoint Comparison

This checkpoint is part of a training series. Compare with other checkpoints:

Steps	Model	Characteristics
400	lcm-lora-sd1.5-400	Early training checkpoint showing foundational LCM capabilities. Provides decent...
800	lcm-lora-sd1.5-800	Mid-training checkpoint with vibrant, artistic outputs. Strong visual impact wit... → This checkpoint
1200	lcm-lora-sd1.5-1200	Higher training with more refined outputs. Some prompts may show signs of overfi...
1600	lcm-lora-sd1.5-1600	Final training checkpoint with mature, consistent outputs. Well-balanced and rel...

Sample Outputs

Installation

pip install --upgrade diffusers transformers accelerate

Basic Usage

import torch
from diffusers import StableDiffusionPipeline, LCMScheduler

# Load base SD1.5 model
pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16
)
pipe.to("cuda")

# Load this LCM-LoRA checkpoint
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-800")

# IMPORTANT: Use LCM scheduler
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)

# Generate with just 4-6 steps!
prompt = "a portrait of a cat wearing a detective hat, film noir style"
image = pipe(
    prompt=prompt,
    num_inference_steps=6,
    guidance_scale=1.0
).images[0]

image.save("output.png")

Recommended Settings

num_inference_steps = 6        # Optimal for this checkpoint
guidance_scale = 1.0            # Required for LCM

Training Details

Parameter	Value
Checkpoint	800
Base Model	runwayml/stable-diffusion-v1-5
Training Steps	800
Dataset	Mercity/laion-subset
LoRA Rank	96
LoRA Alpha	96
Resolution	512×512
Batch Size	64
Learning Rate	1e-4
Optimizer	AdamW

Sample Outputs

The comparison grid above shows outputs from this checkpoint at 2, 4, and 6 inference steps, compared to standard SD1.5 at 50 steps.

Prompts included:

Futuristic cyberpunk city with neon lights and rain reflections
Portrait of a cat wearing a detective hat, film noir style
Cozy coffee shop interior with warm lighting and plants
Ancient Japanese temple in misty mountain landscape at sunrise
Majestic lion on rock overlooking African savannah at sunset
Magical forest with glowing blue mushrooms and fireflies
Vintage red steam locomotive crossing stone viaduct over canyon

View individual samples

All sample images for this checkpoint are available in the samples/ directory.

Performance

Speed Comparison

Method	Steps	Time (A100)	Time (RTX 3090)
SD1.5 Default	50	~15s	~25s
SD1.5 Fast	25	~8s	~13s
LCM-LoRA (this)	6	~2s	~3s
LCM-LoRA (this)	4	~1.5s	~2s

Quality Progression

2 steps: Fast, captures main composition
4 steps: Good balance, suitable for most cases
6 steps: Best quality (recommended)
8 steps: Slightly better, diminishing returns

Advanced Usage

Speed Optimization

# Fuse LoRA for faster inference
pipe.fuse_lora(lora_scale=1.0)

# Use xformers for memory efficiency
pipe.enable_xformers_memory_efficient_attention()

# Compile model (PyTorch 2.0+)
pipe.unet = torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True)

Multiple LoRAs

# Combine with other LoRAs
pipe.load_lora_weights("other_style.safetensors", adapter_name="style")
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-800", adapter_name="lcm")

# Adjust weights
pipe.set_adapters(["style", "lcm"], adapter_weights=[0.8, 1.0])

Switch Between Checkpoints

# Load different checkpoints from this series
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-400")  
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-800")  
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-1200") 
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-1600")

Series Information

Training Progression

This checkpoint is part of a training series showing LCM-LoRA evolution:

Training Steps:  400 ──→ 800 ──→ 1200 ──→ 1600
                  ↓       ↓        ↓        ↓
Quality:      Baseline  Peak   Refined   Mature
Style:        Soft    Vibrant  Balanced  Stable

Download All Checkpoints

# Download all checkpoints for comparison
huggingface-cli download Mercity/lcm-lora-sd1.5-400
huggingface-cli download Mercity/lcm-lora-sd1.5-800
huggingface-cli download Mercity/lcm-lora-sd1.5-1200
huggingface-cli download Mercity/lcm-lora-sd1.5-1600

Usage Tips

For Best Results

Always use LCMScheduler - Required for LCM
Set guidance_scale=1.0 - CFG doesn't work with LCM
Use 4-8 steps - Optimal range is 6 steps
Same prompts as SD1.5 - No special prompting needed

Checkpoint Selection

Testing/comparison? Try different checkpoints to find your preference
Different characteristics: Each checkpoint has unique qualities
Training progression: See how the model evolves with more training

Limitations

Trained on 512×512 resolution (best results at this size)
Requires LCMScheduler - other schedulers won't work
guidance_scale must be 1.0 (CFG incompatible with LCM)
Each checkpoint has slightly different characteristics

Citation

If you use this model in your research, please cite:

@article{luo2023latent,
  title={Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference},
  author={Luo, Simian and Tan, Yiqin and Huang, Longbo and Li, Jian and Zhao, Hang},
  journal={arXiv preprint arXiv:2310.04378},
  year={2023}
}

@article{hu2021lora,
  title={LoRA: Low-Rank Adaptation of Large Language Models},
  author={Hu, Edward J and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},
  journal={arXiv preprint arXiv:2106.09685},
  year={2021}
}

License

This model is released under the same license as Stable Diffusion v1.5:

CreativeML Open RAIL-M License
Commercial use allowed with restrictions
See: https://huggingface.co/spaces/CompVis/stable-diffusion-license

Acknowledgments

Base Model: Stable Diffusion v1.5
LCM Method: Latent Consistency Models
LoRA Method: Low-Rank Adaptation
Training Framework: Diffusers

More Information

Other checkpoints in series: Checkpoint 400 • Checkpoint 800 (current) • Checkpoint 1200 • Checkpoint 1600
Discussions: Model discussions
Report issues: Community tab