The most structured deep learning curriculum on GitHub.
Every topic ships with rigorous mathematical theory, production-grade PyTorch code, and a line-by-line explanation — nothing is left unexplained.

**Made with ❤️ by Ayush Kumar Singh

---

📌 Table of Contents

• Why This Repository
• The 3-Component System
• Curriculum Map
• Phase Breakdown
• Live Results Snapshot
• Getting Started
• Repository Structure
• Prerequisites
• Progress Tracker
• Contributing
• Author

---

🎯 Why This Repository

Most deep learning resources give you one of three things: theory without code, code without intuition, or tutorials that don't scale. This repository gives you all three — for every single topic, without exception.

What you'll find here	What you won't find here
✅ Full mathematical derivations with LaTeX	❌ "Just run this Colab" tutorials
✅ Self-contained, runnable PyTorch code	❌ Code that requires hidden setup
✅ Line-by-line explanations of why	❌ Copy-paste snippets without context
✅ Inline tensor shape annotations	❌ Undocumented tensor ops
✅ Empirical proof of theoretical claims	❌ Theory disconnected from results
✅ CPU + GPU compatible, zero config	❌ Environment hell

---

🔩 The 3-Component System

Every topic in this repository follows an identical, non-negotiable 3-file structure:

topic-name/
├── theory.md           ← Mathematics, architecture, intuition
├── implementation.py   ← Production-ready PyTorch code
└── explanation.md      ← Line-by-line code breakdown

What each file contains

📐 theory.md Full mathematical derivations LaTeX-rendered equations ASCII/Mermaid architecture diagrams Geometric and visual intuition Historical context and paper references Common pitfalls and misconceptions

⚙️ implementation.py Fully executable PyTorch code Dataset loading + preprocessing Model definition + training loop Evaluation + metrics Matplotlib visualizations Best-weight saving & reproducibility

🔍 explanation.md Every non-obvious line explained Tensor shape traces [B, T, D] Design decisions justified Common bug table per topic no_grad() vs eval() distinctions Why each function call exists

---

🗺 Curriculum Map

deep-learning-mastery/
│
├── 📦 phase-1-foundations/              ← The bedrock of everything
│   ├── 01-perceptron-and-mlp/
│   ├── 02-activation-functions/
│   ├── 03-gradient-descent-and-backprop/
│   ├── 04-loss-functions-and-overfitting/
│   ├── 05-regularization-optimizers-batchnorm/
│   ├── 06-hyperparameter-tuning-augmentation/
│   └── 07-extra-linear-algebra-pytorch-tensors/
│
├── 🖼  phase-2-cnns/                    ← Vision intelligence
│   ├── 01-convolution-basics/
│   ├── 02-architectures-lenet-to-densenet/
│   ├── 03-object-detection-rcnn-yolo/
│   ├── 04-segmentation-unet-maskrcnn/
│   └── 05-transfer-learning-finetuning/
│
├── 🔄 phase-3-sequential/              ← Memory and sequences
│   ├── 01-rnns/
│   ├── 02-lstm-and-gru/
│   ├── 03-seq2seq-nlp/
│   └── 04-extra-state-space-models-mamba/
│
├── 🧠 phase-4-attention-transformers/  ← The modern backbone
│   ├── 01-attention-mechanisms/
│   ├── 02-transformer-architecture/
│   └── 03-vision-transformers-swin/
│
├── 🎨 phase-5-generative-ai/           ← Create, not just classify
│   ├── 01-autoencoders-and-vaes/
│   ├── 02-gans-dcgan-cyclegan/
│   ├── 03-llms-bert-gpt/
│   ├── 04-diffusion-models-ddpm/
│   └── 05-extra-lora-rag-rlhf/
│
└── 🚀 phase-6-advanced-deployment/     ← From model to production
    ├── 01-graph-neural-networks/
    ├── 02-deep-reinforcement-learning/
    └── 03-mlops-quantization-onnx-serving/

---

📚 Phase Breakdown

Phase 1 — Deep Learning Foundations (7 topics)

#	Topic	Key Concepts
01	Perceptron & MLP	Biological neuron, convergence theorem, Universal Approximation
02	Activation Functions	Sigmoid, ReLU, GELU, SELU, SiLU — vanishing gradient proof
03	Gradient Descent & Backprop	SGD, mini-batch, autodiff chain rule, computational graphs
04	Loss Functions & Overfitting	BCE, CE, MSE, bias-variance trade-off
05	Regularization & Optimizers	L1/L2, Dropout, BatchNorm, Adam, RMSprop, Early Stopping
06	Hyperparameter Tuning	LR schedules, batch size effects, data augmentation strategies
07 ★	Linear Algebra & PyTorch	Tensor ops, broadcasting, einsum, autograd mechanics

Phase 2 — Convolutional Neural Networks (5 topics)

#	Topic	Key Concepts
01	Convolution Basics	1D/2D/3D kernels, parameter sharing, receptive field, pooling
02	Architectures	LeNet → AlexNet → VGG → ResNet → DenseNet → GoogLeNet
03	Object Detection	Faster R-CNN, anchor boxes, RPN, YOLO v1–v8
04	Segmentation	U-Net skip connections, Mask R-CNN instance segmentation
05	Transfer Learning	Feature extraction vs fine-tuning, domain adaptation

Phase 3 — Sequential Modeling (4 topics)

#	Topic	Key Concepts
01	RNNs	Unrolled computation, BPTT, exploding/vanishing gradients
02	LSTM & GRU	Cell state, forget/input/output gates, GRU simplification
03	Seq2Seq	Encoder-decoder, teacher forcing, attention bridge
04 ★	State Space Models	Mamba, Jamba, selective scan, linear recurrence

Phase 4 — Attention & Transformers (3 topics)

#	Topic	Key Concepts
01	Attention Mechanisms	Scaled dot-product, multi-head, causal masking
02	Transformer Architecture	Encoder-decoder, positional encoding, LayerNorm placement
03	Vision Transformers	ViT patch embedding, Swin shifted windows, hierarchical features

Phase 5 — Generative AI & LLMs (5 topics)

#	Topic	Key Concepts
01	Autoencoders & VAEs	Bottleneck, ELBO, reparameterization trick
02	GANs	DCGAN, CycleGAN, mode collapse, Wasserstein loss
03	LLMs — BERT & GPT	Masked LM, causal LM, fine-tuning, summarization, translation
04	Diffusion Models	DDPM, forward/reverse diffusion, noise schedules, U-Net backbone
05 ★	LoRA / RAG / RLHF	Parameter-efficient fine-tuning, retrieval augmentation, DPO alignment

Phase 6 — Advanced Topics & Deployment (3 topics)

#	Topic	Key Concepts
01	Graph Neural Networks	Message passing, GCN, GAT, node/edge/graph classification
02	Deep Reinforcement Learning	DQN, PPO, policy gradients, replay buffer
03	MLOps	INT8 quantization, ONNX export, FastAPI serving, Dockerization

---

📊 Live Results Snapshot

These are actual empirical outputs from the implementations — not theoretical claims:

Phase 1 — Topic 1: Perceptron & MLP

Experiment	Result
Perceptron — AND gate	Converged in 4 epochs, 100% accuracy
Perceptron — XOR gate	Hit max 1000 epochs, stuck at 50% (proves single-layer limitation)
MLP Scratch — XOR	100% accuracy from epoch ~2000 onward
Production MLP — make_moons	99% val accuracy in 150 epochs, 6,465 parameters

Phase 1 — Topic 2: Activation Functions

Activation	Input-Layer Gradient Norm	Output-Layer Gradient Norm	Ratio
Sigmoid	2.82e-10	2.62e-01	1.1e-09 ← catastrophic vanishing
Tanh	1.78e-01	9.63e-02	1.8× ← manageable
ReLU	3.78e-03	3.16e-03	1.2× ← stable
SiLU/Swish	—	—	Lowest val loss (0.0155)

Every topic will include its own results table as it is completed.

---

⚡ Getting Started

1. Clone the repository

git clone https://github.com/Ayush-2703/deep-learning-mastery.git
cd deep-learning-mastery

2. Set up the environment

# Create and activate a virtual environment
python -m venv venv
source venv/bin/activate        # Linux/macOS
# venv\Scripts\activate         # Windows

# Install dependencies
pip install -r requirements.txt

3. Run any topic independently

Every implementation is fully self-contained — no shared state, no hidden dependencies:

# Example: Run the MLP implementation
python phase-1-foundations/01-perceptron-and-mlp/implementation.py

# Example: Run the Activation Functions comparison
python phase-1-foundations/02-activation-functions/implementation.py

4. Open in Google Colab (zero setup)

Click the Colab badge at the top of any implementation.py or browse directly:

https://colab.research.google.com/github/Ayush-2703/deep-learning-mastery

---

🛠 Prerequisites

Python knowledge assumed:

• Comfortable with Python classes, decorators, and list comprehensions
• Familiarity with NumPy arrays

Math assumed:

• High-school calculus (derivatives, chain rule)
• Basic linear algebra (matrices, dot products)

Everything beyond this is taught from scratch inside theory.md.

Software:

Python     ≥ 3.10
PyTorch    ≥ 2.0
NumPy      ≥ 1.24
Matplotlib ≥ 3.7
scikit-learn ≥ 1.3

---

✅ Progress Tracker

Phase	Topics	Status
Phase 1 — Foundations	7 topics	🟩 Completed
Phase 2 — CNNs	5 topics	🟩 Completed
Phase 3 — Sequential	4 topics	🟩 Completed
Phase 4 — Attention & Transformers	3 topics	🟩 Completed
Phase 5 — Generative AI & LLMs	5 topics	🟩 Completed
Phase 6 — Advanced & Deployment	3 topics	🟩 Completed

Overall: 27 / 27 topics completed

---

🤝 Contributing

Contributions are welcome, but this repository maintains a strict quality bar. Before submitting a PR, please read the guidelines:

1. Every PR must follow the 3-component structure (theory.md, implementation.py, explanation.md)
2. Code must run end-to-end without modification
3. Tensor shapes must be annotated inline: # [batch, seq_len, d_model]
4. No black-box functions — every non-obvious call gets a comment
5. Results must be empirically verified and included in the explanation

# Fork, clone, and create a branch
git checkout -b feature/phase-2-topic-1-convolutions

# After your changes
git commit -m "feat: add phase-2/01-convolution-basics (theory + impl + explanation)"
git push origin feature/phase-2-topic-1-convolutions

Then open a Pull Request with a brief description of what your implementation demonstrates empirically.

---

📜 License

Distributed under the MIT License. See LICENSE for details.
You're free to use, fork, and build on this for personal and commercial projects.

---

👤 Author

Ayush Kumar Singh

Researcher in Adversarial ML, Geospatial AI, and LLM/NLP Systems

---

If this repository helped you, please consider giving it a ⭐
It takes 2 seconds and helps others discover it.