FaceViT

Face attribute prediction (age, gender, race) using Vision Transformers with parameter-efficient fine-tuning (LoRA adapters).

For methodology, results, and analysis, see project_paper.pdf.

---

Project Layout

├── src/FaceViT/          # Installable Python package
│   ├── main.py           # Inference entry point
│   ├── models/           # Encoder implementations (CLIP, DINOv2, DINOv3) and factory
│   ├── data/             # Dataset loading and sampling utilities
│   └── utils/            # Shared helpers
├── models/               # Saved adapter weights (LoRA checkpoints)
│   ├── CLIP-80M/
│   ├── Dinov2-80M/
│   └── Dinov3-0.3B/
├── notebooks/            # Training & evaluation notebooks
│   ├── CLIP_Train.ipynb
│   ├── Dinov2_Train.ipynb
│   ├── Dinov3_Train.ipynb
├── logs/                 # TensorBoard logs from training runs
│   ├── CLIP-80m/
│   ├── Dinov2-80M/
│   └── Dinov3-0.3B/
├── documents/            # LaTeX source and figures for the paper
│   └── project_paper.tex
└── project_paper.pdf     # Final compiled paper

---

Setup

Install the package in editable mode so from FaceViT import ... works:

pip install -e .

This installs all dependencies listed in `pyproject.

Inference

Run main.py directly:

python src/FaceViT/main.py

Edit the top of main.py to set:

• image_paths – list of image files to classify.
• encoder_model – which backbone to use (ViTEncoders.DINOV3, DINOV2, or CLIP).
• device – "cuda" or "cpu".

The script outputs predicted age bucket, gender, and race for each image along with raw logit probabilities.

SAM3 Segmentation (Optional)

main.py includes an optional SAM3 segmentation step that masks the image before inference. Enable it by setting USE_SAM3 = True in the script.

Note: The models in this project were not trained or evaluated with segmentation. SAM3 integration is provided as a convenience for the user and may affect results.

---

Training

All training is done in the Jupyter notebooks under notebooks/:

Notebook	Backbone
CLIP_Train.ipynb	CLIP ViT-B/16
Dinov2_Train.ipynb	DINOv2
Dinov3_Train.ipynb	DINOv3

Trained adapter weights are saved to models/.

---

Logs

TensorBoard logs are written to logs/. To visualize training curves:

tensorboard --logdir logs/

---

Paper

The full project write-up is in project_paper.pdf at the repository root. LaTeX source and supporting figures are in documents/.