ResNet-50 & SE-ResNet-50, Built From Scratch

A from-scratch implementation of ResNet-50 and a Squeeze-and-Excitation variant, trained on ImageNet-100 without any pretrained weights. The goal wasn't just to hit a number — it was to build every piece by hand (bottleneck blocks, projection shortcuts, SE gating, the training loop, the interpretability tooling) and see how close a from-scratch model can get to fine-tuned, industry-standard baselines.

The SE-ResNet-50 variant adds channel-attention (squeeze-and-excitation) on top of the same backbone and outperforms the plain version on both Top-1 and Top-5 accuracy.

Table of Contents

• Architecture
• Results
• Visual Diagnostics
• Core Technical Features
• Installation & Usage

---

Architecture

ResNet-50 (baseline)

graph LR
    A["Input Image<br>3x224x224"] --> B("Stem<br>Conv 7x7, Stride 2")
    B --> C("BatchNorm + ReLU")
    C --> D("MaxPool 3x3")
    D --> E["Layer 1<br>3x Bottleneck Blocks<br>Output Depth: 256"]
    E --> F["Layer 2<br>4x Bottleneck Blocks<br>Output Depth: 512"]
    F --> G["Layer 3<br>6x Bottleneck Blocks<br>Output Depth: 1024"]
    G --> H["Layer 4<br>3x Bottleneck Blocks<br>Output Depth: 2048"]
    H --> I("AdaptiveAvgPool2d")
    I --> J("Flatten")
    J --> K["Linear Projection<br>100 Classes"]

Loading

SE-ResNet-50 — identical backbone, with a squeeze-and-excitation gate inserted into every bottleneck block:

graph LR
    A["Input Image<br>3x224x224"] --> B("Stem<br>Conv 7x7, Stride 2")
    B --> C("BatchNorm + ReLU")
    C --> D("MaxPool 3x3")
    D --> E["Layer 1<br>3x SE-Bottleneck Blocks<br>Output Depth: 256"]
    E --> F["Layer 2<br>4x SE-Bottleneck Blocks<br>Output Depth: 512"]
    F --> G["Layer 3<br>6x SE-Bottleneck Blocks<br>Output Depth: 1024"]
    G --> H["Layer 4<br>3x SE-Bottleneck Blocks<br>Output Depth: 2048"]

    subgraph SE ["Inside Each SE-Bottleneck Block"]
        direction TB
        X1["Conv Bottleneck<br>(1x1 → 3x3 → 1x1)"] --> X2["Global Average Pooling<br>(Squeeze)"]
        X2 --> X3["Fully Connected + ReLU<br>(Reduction Ratio r=16)"]
        X3 --> X4["Fully Connected + Sigmoid<br>(Excitation)"]
        X4 --> X5["Scale Feature Maps<br>(Element-wise Multiplication)"]
    end

    H --> I("AdaptiveAvgPool2d")
    I --> J("Flatten")
    J --> K["Linear Projection<br>100 Classes"]

Loading

---

Results

Metric	ResNet-50 (custom)	SE-ResNet-50 (custom)
Top-1 Accuracy	86.9%	87.09%
Top-5 Accuracy	96.2%	97.49%
Parameters	25,557,032	28,088,024
Inference Latency	5.94 ms/image	9.38 ms/image
Hardware	Dual NVIDIA T4, nn.DataParallel + AMP	Dual NVIDIA T4, nn.DataParallel + AMP

Note on latency: SE-ResNet-50 adds only ~10% more parameters but runs ~58% slower per image. This is expected — SE gates add several small, sequential, memory-bound ops (pool → FC → ReLU → FC → sigmoid → scale) per block, which carry fixed kernel-launch overhead that doesn't parallelize as well as convolutions. A fused CUDA implementation would likely close most of this gap.

---

Ablation study

A note on methodology: the two models were benchmarked against pretrained baselines (PT ResNet-50, VGG-16, MobileNetV2) under different protocols, so the two charts below aren't directly comparable to each other:

• ResNet-50 was compared against baselines using 5-epoch linear probing (backbone frozen, only the classification head trained) — a lighter-weight baseline.
• SE-ResNet-50 was compared against baselines using 20-epoch full end-to-end fine-tuning — a much stronger baseline, since the entire pretrained network is allowed to adapt.

Under that harder comparison, the custom SE-ResNet-50 (97.49% Top-5) essentially matched a fully fine-tuned pretrained ResNet-50 (97.45% Top-5) despite training from scratch with no ImageNet weights — which is the more meaningful result of the two studies.

ResNet-50 — 90-epoch convergence curve

SE-ResNet-50 — 90-epoch convergence curve

ResNet-50 vs. pretrained baselines — 5-epoch linear probing

SE-ResNet-50 vs. pretrained baselines — 20-epoch full fine-tuning

Both curves stay stable across 90 epochs despite the label/pixel noise introduced by CutMix and MixUp — a good sign the optimization schedule (warmup → cosine annealing) is doing its job.

---

Visual Diagnostics

Interpretability (Grad-CAM)

"Black box" predictions aren't acceptable in production. Gradient-weighted Class Activation Mapping (Grad-CAM) was implemented from scratch using forward/backward hooks attached to the final convolutional bottleneck (layer4[-1].conv3), to verify the network is actually keying on target morphology — not background texture or artifacts.

Best of Class: Water Ouzel (Dipper)	Best of Class: Rock Crab
Success: Tench (100% Confidence)	Success: Tench (Alternate View)
Diagnostic: Texture Bias (Predicted 84)	Best of Class: Wombat

---

Core Technical Features & Engineering

Architectural Design

• Native topology, hand-built — both the plain ResNet-50 and the SE-ResNet-50 were constructed from raw nn.Module primitives rather than torchvision.models, including bottleneck triplets (1×1 → 3×3 → 1×1), dynamically-sized projection shortcuts, and — for the SE variant — squeeze-and-excitation channel-attention gates inserted into every block.
• Kaiming (He) initialization — weight initialization tuned for deep ReLU networks, to keep gradients stable during the volatile early epochs of from-scratch training.

Regularization

• Probabilistic CutMix & MixUp — a custom data collator applies CutMix (spatial patch blending) and MixUp (feature/label interpolation) with randomized per-batch probability, pushing the network toward global structural cues instead of memorizing local pixel noise.
• LR schedule — linear warmup into cosine annealing (CosineAnnealingLR), to stabilize early high-variance gradients and then settle the optimizer into a narrow minimum.

MLOps, Profiling & Compute

• Inference profiling — custom timing decorators log throughput (images/sec) and per-image latency, to sanity-check real-time deployment viability.
• Multi-GPU + AMP — trained across dual NVIDIA T4 GPUs via nn.DataParallel, with torch.amp.autocast mixed precision cutting VRAM usage by roughly 40%.
• Fault-tolerant checkpointing — _latest.pth and _best.pth state dicts are written automatically during training, so a preemptible cloud instance dying mid-run doesn't cost you the session.

Interpretability

• Grad-CAM, built in-house — forward/backward hooks on layer4[-1].conv3 generate class activation heatmaps, used to confirm predictions are grounded in real object morphology rather than background shortcuts.

---

💻 Installation & Usage

1. Clone the repository

git clone https://github.com/Asmit159/ResNet-50.git
cd ResNet-50

2. Install dependencies

pip install -r requirements.txt

3. Run the training pipeline

python train.py --epochs 90 --batch_size 128 --mixed_precision True

4. Generate diagnostics (Grad-CAM & ablation)

python evaluate.py --checkpoint weights/resnet50_best.pth

---

Author

Asmit Mandal