Agent-from-scratch

A hands-on learning repository documenting my journey of understanding AI agents and the technologies behind them.

1. AI Foundations

Learned the fundamentals of Artificial Intelligence and Generative AI, including:

• AI and Machine Learning fundamentals

• Generative AI concepts

• Large Language Models (LLMs)

• Prompt engineering

• AI agents and agentic workflows

• APIs and how AI applications interact with external systems

• Model inference and context

• Tool calling and function calling

• Structured outputs and JSON schemas

• AI model evaluation

• Hallucination and grounding

• AI application architecture

  2. RAG Foundations

Completed understanding of Retrieval-Augmented Generation:

• Data ingestion pipeline
• Document processing & chunking strategies
• Embeddings & how they represent meaning
• Vector databases (ChromaDB, Pinecone, FAISS)
• Semantic search vs keyword search
• Retrieval pipeline design
• How RAG reduces hallucination in production agents

3. Agent Evaluation Framework

Currently working on how to measure whether an agent is actually working — learned from building real agents for enterprise clients:

Output Quality

• Accuracy scoring against expected outputs
• Hallucination detection — spotting confident wrong answers
• Response consistency across repeated inputs

Edge Case Testing

• Empty or malformed inputs
• Ambiguous instructions
• Out-of-scope requests
• Integration failures mid-workflow

Performance

• Latency per agent run
• Token usage & cost per execution
• Failure rate tracking

Iteration Tracking

• Logging prompt versions and output changes
• Regression testing after every prompt update
• Client feedback loops as evaluation signal