The opinionated, interview-ready playbook for engineers crossing into AI — LLMs · RAG · Prompting · Agents · Evals · LLMOps · AI System Design.
9 chapters · 75+ lessons · 400+ checklist items · 100+ diagrams · 5 worked case studies · 30 practice problems
Most AI content is either academic papers or shallow "build a chatbot in 10 minutes" tutorials. Neither prepares you for the interview question "design an enterprise RAG system for 10M documents" — or for the on-call page when it breaks in production.
This guide is the missing middle: system architecture, deployment patterns, and operational rigor, written by an engineer for engineers. Every page is opinionated, includes a diagram, and ends with an interview cheat sheet.
It's for you if you're a software engineer (2+ yrs) moving into AI, prepping for AI/ML system-design interviews, or shipping LLM features and want to understand what breaks and why. It's not for you if you want ML theory and math (read Goodfellow instead) or academic paper summaries.
Every chapter opens with a ✅ Chapter Checklist: an interview syllabus of 400+ topics, each tracked at three honest levels of mastery —
☐ Learned → ☐ Can explain → ☐ Can design/debug
Don't mark a topic "done" because you read it. Push the ⭐ high-leverage topics to the third box before an interview. Pick your gap and go:
| 🐍 Python for AI | 🧬 LLM Internals | ✍️ Prompt & Context |
| 🔎 RAG & Retrieval | 🤖 Agents & MCP | 🛠️ Agent Harnesses |
| 📏 Evals | ⚙️ LLMOps & Ops | 🏗️ AI System Design |
- 📖 Read online — the live guide has full-text search, dark mode, and progress tracking.
- ✅ Study by checklist — treat the 9 checklists above as your syllabus; drill into the linked lesson for any topic you can't yet explain.
- 💻 Run it locally — the whole thing is Markdown + a tiny zero-dependency server:
git clone https://github.com/dipakkr/ai-engineering-guide
cd ai-engineering-guide/docs && python3 server.py
# open http://localhost:8080📋 Every chapter opens with a
✅ Chapter Checklist— an interview syllabus that tracks each topic at three levels: Learned → Can explain → Can design/debug. Use them to make sure nothing falls through the cracks. Start here: Python · LLM · Prompting · RAG · Agents · Evals · Ops · Harnesses · System Design.
The Python + CS foundation AI interviews assume: internals, async, backend, and the DSA screen.
- 00-checklist — ✅ Interview syllabus for this chapter
- what-is-python-for-ai — Why the Python screen still exists for AI roles
- 01-python-fundamentals — Mutability, generators, decorators, context managers
- 02-internals-and-performance — The GIL, memory, Big-O of collections
- 03-async-and-concurrency — Event loop, gather, semaphores, backpressure
- 04-ai-backend-python — Pydantic, FastAPI, streaming/SSE, retries
- 05-coding-interviews — The DSA screen: patterns, Big-O, classic problems
How transformers work, tokenization, context windows, when to fine-tune vs RAG.
- 00-checklist — ✅ Interview syllabus for this chapter
- 01-transformer-intuition — How transformers work, no math
- 02-tokenization — Tokens are money
- 03-attention-mechanisms — Self-attention, KV cache, Flash Attention
- 04-context-windows — Long context tradeoffs
- 05-training-pipeline — RAG vs fine-tune vs prompt (THE decision)
- 06-model-landscape — Model comparison table
- 07-small-language-models — When to use Phi/Gemma instead of GPT-4o
- 08-quantization — INT8/INT4, GGUF, running models on cheap hardware
- 09-fine-tuning — LoRA, QLoRA, when NOT to fine-tune
- 10-distillation-and-pruning — Making models cheaper
CoT, structured generation, prompt optimization, injection defense.
- 00-checklist — ✅ Interview syllabus for this chapter
- 01-prompting-patterns — Zero-shot to Tree of Thought
- 02-context-engineering — The underrated skill that separates good from great
- 03-structured-generation — Instructor, JSON mode, Outlines
- 04-prompt-optimization — DSPy, meta-prompting, eval-driven
- 05-prompt-security — Injection attacks and defenses
The complete RAG stack: chunking, embeddings, vector DBs, hybrid search, advanced patterns.
- 00-checklist — ✅ Interview syllabus for this chapter
- 01-rag-fundamentals — What/why/when, the naive pipeline
- 02-embedding-models — MTEB, dimensions, Matryoshka
- 03-vector-indexing — HNSW vs IVF, FAISS
- 04-vector-databases — Decision matrix, cost at scale
- 05-chunking-strategies — THE key lever most teams get wrong
- 06-hybrid-search — Dense + BM25 + RRF
- 07-reranking — Cross-encoders, Cohere, two-stage
- 08-query-transformation — HyDE, multi-query, decomposition
- 09-advanced-rag-patterns — GraphRAG, Agentic RAG, Self-RAG, CRAG
- 10-multimodal-rag — ColPali, PDFs with tables and images
- 11-rag-evaluation — RAGAS, debug flowchart
ReAct, tool use, MCP, LangGraph, multi-agent systems, memory.
- 00-checklist — ✅ Interview syllabus for this chapter
- 01-agent-fundamentals — ReAct, perception-action loop, failure modes
- 02-tool-use-and-function-calling — OpenAI vs Claude vs Gemini formats
- 03-mcp-protocol — Full MCP, server code, security
- 04-langchain-overview — What it does well and where it falls short
- 05-langgraph-deep-dive — Stateful graphs, persistence, human-in-loop
- 06-dspy-framework — Compile don't prompt
- 07-crewai-and-autogen — Honest assessment of multi-agent frameworks
- 08-llamaindex-haystack — Data frameworks vs orchestration frameworks
- 09-multi-agent-systems — When you actually need multiple agents
- 10-memory-and-state — Memory tiers, Mem0, Zep, checkpointing
- 11-agentic-patterns — Reflection, map-reduce, DAG patterns
- 12-browser-and-computer-use — Playwright, Claude Computer Use
How to actually measure if your system works: RAGAS, LLM-as-judge, production eval.
- 00-checklist — ✅ Interview syllabus for this chapter
- 01-eval-fundamentals — Why eval is hard, the eval pipeline
- 02-retrieval-and-rag-eval — Precision@K, MRR, NDCG, RAGAS
- 03-llm-as-judge — Pointwise vs pairwise, calibration
- 04-agent-and-e2e-eval — Task completion, A/B testing, continuous eval
Observability, guardrails, caching, inference infra, cost optimization.
- 00-checklist — ✅ Interview syllabus for this chapter
- 01-observability-and-tracing — LangSmith vs Langfuse, what to log
- 02-guardrails-and-safety — Defense-in-depth, NeMo, LlamaGuard, Presidio
- 03-caching-strategies — Multi-layer caching, semantic cache
- 04-inference-infrastructure — GPU table, vLLM vs TGI, auto-scaling
- 05-drift-and-monitoring — Drift types, detection, remediation
- 06-mlops-for-llms — CI/CD, prompt versioning, blue-green
- 07-cost-optimization — Token optimization, model routing, batch
The harness around the model: the loop, tool registry, context management, skills, and permissions that make coding/research agents work.
- 00-checklist — ✅ Interview syllabus for this chapter
- what-is-an-agent-harness — Model vs harness: why the same model feels different
- 01-harness-fundamentals — The model loop and the five pieces
- 02-context-management — Compaction, retrieval, budgeting across a run
- 03-skills — Instructions vs tools vs skills, progressive disclosure
- 04-coding-agent-harnesses — Filesystem, shell, patch/edit, git, sandboxing
- 05-execution — Planning, subagents, checkpoints, resumability
- 06-reliability — Verification loops, self-correction, loop detection
- 07-real-systems — Claude Code, Codex, LangGraph, CrewAI/AutoGen
Interview framework, 5 full case studies, 30 practice problems, 60+ conceptual questions.
- 00-checklist — ✅ Design practice log + coverage sequence
- 01-interview-framework — The 45-min structure. Worth the whole repo.
- 02-design-patterns-catalog — Full catalog with decision tree
- 03-architecture-templates — 6 reference architectures with cost models
- 04-case-enterprise-rag — Full worked design: enterprise knowledge base
- 05-case-code-assistant — Full worked design: GitHub Copilot-style
- 06-case-customer-support — Full worked design: support automation
- 07-case-doc-intelligence — Full worked design: document understanding
- 08-case-search-engine — Full worked design: AI-powered search
- 09-practice-problems — 30 problems with solution skeletons
- 10-conceptual-questions — 60+ questions with full conversational answers
Model pricing, glossary, cost formulas, essential papers.
- model-pricing-reference — Current pricing for all major models
- glossary — Terms defined in plain English
- cost-estimation-formulas — Spreadsheet-ready formulas
- essential-papers — The 20 papers worth reading
Working implementations: RAG pipeline, LangGraph agent, MCP server, eval pipeline.
- 01-basic-rag — Minimal RAG in 100 lines
- 02-advanced-rag — Hybrid search + reranking
- 03-langgraph-agent — Stateful agent with tools
- 04-mcp-server — Working MCP server
- 05-eval-pipeline — RAGAS + LLM-as-judge
- 06-semantic-cache — Semantic caching with Redis
- 07-structured-output — Instructor + Pydantic
The guide is intentionally opinionated. If you disagree with a recommendation, open an issue with your reasoning and production evidence. PRs welcome for:
- Factual errors or outdated information (especially model specs and pricing)
- Missing failure modes from your production experience
- New diagrams, checklist items, or worked examples
Adding a diagram? Prefer a
```mermaidblock. If you hand-author inline<svg>, keep it blank-line-free — the Markdown renderer injects<p>tags at blank lines, which breaks SVG rendering.