AutoLoraFactory

Automated Stable Diffusion LoRA pipeline with dataset extraction, training, generation, and evaluation.

AutoLoraFactory is a Python-based automation pipeline for creating high-quality LoRA models from videos.

It automatically:

• extracts frames from videos
• detects and crops faces
• removes duplicate images
• selects high-quality samples
• generates captions
• trains LoRA models
• generates images
• evaluates outputs with CLIP
• selects the best checkpoint automatically

---

Features

Dataset Automation

• Video frame extraction with FFmpeg
• Face detection using InsightFace
• Automatic face crop & resize
• Duplicate image removal
• Quality-based image filtering
• Automatic caption generation

LoRA Training

• Fully automated LoRA training
• Hyperparameter search
• Automatic checkpoint evaluation
• Best model selection
• Batch experiment support

Image Evaluation

• CLIP similarity scoring
• Simple aesthetic scoring
• Automatic ranking system

---

Pipeline Overview

Video (.mp4)
    ↓
Frame Extraction
    ↓
Face Detection & Crop
    ↓
Duplicate Removal
    ↓
Quality Filtering
    ↓
Caption Generation
    ↓
LoRA Training
    ↓
Image Generation
    ↓
CLIP Evaluation
    ↓
Best LoRA Selection

---

Project Structure

AutoLora/
├── input/                 # Input videos
├── dataset/
│   └── 10_person/         # Training dataset
├── lora/                  # Trained LoRA models
├── outputs/               # Generated images
├── best_lora/             # Best checkpoints
├── pipeline/
│   ├── extract.py
│   ├── train.py
│   ├── generate.py
│   ├── evaluate.py
│   └── utils.py
├── run.py
├── requirements.txt
├── LICENSE
└── README.md

---

Requirements

Environment

• Python 3.10+
• CUDA GPU
• FFmpeg
• Stable Diffusion WebUI
• kohya_ss / sd-scripts

Python Packages

pip install \
  torch \
  torchvision \
  diffusers \
  transformers \
  insightface \
  opencv-python \
  pillow \
  tqdm \
  imagehash \
  xformers

---

Setup

1. Install FFmpeg

Make sure FFmpeg is installed and added to PATH.

ffmpeg -version

---

2. Prepare Stable Diffusion Model

Place your base model here:

C:/stable-diffusion-webui/models/Stable-diffusion/

Example:

v1-5-pruned-emaonly.safetensors

---

3. Install sd-scripts

GitHub:

https://github.com/kohya-ss/sd-scripts

---

Usage

Step 1 — Add Videos

Place .mp4 files inside:

input/

---

Step 2 — Run Pipeline

python run.py

The pipeline automatically handles:

1. frame extraction
2. face detection & crop
3. duplicate removal
4. quality filtering
5. caption generation
6. LoRA training
7. image generation
8. CLIP evaluation
9. best checkpoint selection

---

Hyperparameter Search

Edit SEARCH in run.py.

SEARCH = {
    "dim": [8, 16, 32],
    "lr": [1e-4, 5e-5],
    "steps": [1000, 2000]
}

---

Evaluation Formula

score = CLIP * 0.7 + aesthetic * 0.3

Current metrics:

• CLIP similarity score
• simple brightness-based aesthetic score

---

Example Workflow

python run.py

That's it.

The pipeline handles the rest automatically.

---

Notes

• CUDA environment required
• Windows paths are currently hardcoded
• Designed for personal research / experimentation
• Path cleanup and config externalization recommended

---

Disclaimer

This project was built in 2 days.

I do not accept ugly code.
I will probably not maintain this repository.

Use it, modify it, break it — do whatever you want.

This project is intended for research and educational purposes only.
Users are responsible for complying with applicable laws and platform policies.

---

License

MIT License

---

Author

CyberSoulWing