DDPM

PyTorch implementation of Denoising Diffusion Probabilistic Models (Ho et al., 2020).

Theory

Noise Schedule

A linear schedule is used, where $\beta_t$ increases linearly from $\beta_1 = 10^{-4}$ to $\beta_T = 0.02$ over $T = 1000$ timesteps. Precompute:

$$\alpha_t = 1 - \beta_t \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s$$

---

Forward Process — q_sample

Gradually adds noise to a clean image $\mathbf{x}_0$ in one shot using the closed-form:

$$q(\mathbf{x}_t \mid \mathbf{x}_0) = \mathcal{N}!\left(\sqrt{\bar{\alpha}_t},\mathbf{x}_0,\ (1 - \bar{\alpha}_t)\mathbf{I}\right)$$

$$\mathbf{x}_t = \sqrt{\bar{\alpha}_t},\mathbf{x}_0 + \sqrt{1 - \bar{\alpha}_t},\boldsymbol{\epsilon}, \qquad \boldsymbol{\epsilon} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$$

The U-Net is trained to predict $\boldsymbol{\epsilon}$ from $\mathbf{x}_t$ and timestep $t$, minimising:

$$\mathcal{L} = \mathbb{E}\left[\left\lVert \boldsymbol{\epsilon} - \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t) \right\rVert^2 \right]$$

---

Reverse Process — p_sample

Iteratively denoises from $\mathbf{x}_T \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$ back to $\mathbf{x}_0$:

$$p_\theta(\mathbf{x}_{t-1} \mid \mathbf{x}_t) = \mathcal{N}!\left(\boldsymbol{\mu}_\theta(\mathbf{x}_t, t),\ \tilde{\beta}_t \mathbf{I}\right)$$

$$\boldsymbol{\mu}_\theta = \frac{1}{\sqrt{\alpha_t}}\left(\mathbf{x}_t - \frac{\beta_t}{\sqrt{1 - \bar{\alpha}_t}},\boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t)\right) \qquad \tilde{\beta}_t = \frac{1 - \bar{\alpha}_{t-1}}{1 - \bar{\alpha}_t}\cdot\beta_t$$

---

Classes

Diffusion - main class with forward and backward sampling processes.

DDPM - main model class with diffusion class and unet model instantiated.

ImageSampler - callback to display denoising steps during training.