🖼️ Image Caption Generator — Deep Learning Image Captioning System

An end-to-end deep learning image captioning and storytelling system featuring an interactive parallax web interface, Vision Transformer + GPT-2 encoder-decoder models, multi-style generation (Short, Balanced, Detailed Story), and custom PyTorch training pipelines for the Flickr30k dataset.

---

🌟 Key Capabilities

1. 🎭 Multi-Style Caption Generation

• Short: Crisp 8–10 word caption highlighting primary subjects, actions, and objects (ideal for Alt-Text and accessibility).
• Balanced: Structured multi-sentence paragraph analyzing the setting, atmosphere, color palette, and contextual cues.
• Detailed Narrative: Creative narrative story exploring emotional depth, environmental background, character motivation, and visual composition.

2. 🧠 Model Architectures & Pipelines

• Vision Transformer (ViT) & GPT-2: Pre-trained Vision-Encoder-Decoder model with beam search decoding.
• Dual Visual Feature Fusion (DEFF): Combines global scene context with localized visual embeddings.
• Story Generator: Fine-tuned Transformer decoder mapping visual cues into multi-paragraph stories.
• BLEU Evaluation: Automated BLEU-1 through BLEU-4 metrics for evaluation.

3. 🌐 Modern Parallax Web Dashboard

• Dynamic drag-and-drop image upload.
• Real-time generation with style toggle buttons.
• One-click copy, audio playback, and export.

---

🚀 Quick Start Guide

1. Clone the Repository

git clone https://github.com/harshrameshnerkar/Image-Caption-Generator.git
cd Image-Caption-Generator

2. Install Dependencies

pip install -r requirements.txt

3. Run the Web Application

python app.py

(Or double-click start_app.bat on Windows)

Open your browser at http://localhost:5000.

---

📂 Project Structure

Image-Caption-Generator/
├── app.py                      # Main Flask web application entry point
├── start_app.bat               # 1-click Windows launch script
├── requirements.txt            # Project dependencies
├── index.html                  # Showcase and portfolio page
├── README.md                   # Complete documentation
│
├── 🧠 Custom Model Architectures & Training
│   ├── model.py                # Visual feature fusion (DEFF) + Transformer Decoder
│   ├── story_model.py          # Multi-style story generation model
│   ├── dataset.py              # PyTorch Dataset loaders
│   ├── train_caption.py        # Caption model training script
│   ├── train_story.py          # Story model fine-tuning script
│   ├── generate_all_texts.py   # Multi-mode caption generation from checkpoints
│   ├── sample_captions.py      # Beam search sample generation
│   ├── extract_features.py     # Image feature extraction
│   └── eval_bleu.py            # BLEU evaluation
│
├── 📦 Model Weights & Tokenizers
│   ├── model_weights/          # Keras/PyTorch model weights
│   ├── tokenizer/              # Tokenizer vocabulary indices
│   └── storage/                # Feature mappings & vocabulary dictionary
│
├── 🌐 Web Interface Assets
│   ├── templates/              # Flask HTML templates
│   └── static/                 # CSS, JavaScript, and UI assets
│
└── 📓 Research Notebooks
    ├── image caption.ipynb     # Interactive exploration notebook
    └── working_code.ipynb      # Prototype implementation notebook

---

🛠️ Technical Requirements

• Python 3.10+
• PyTorch >= 2.0.0
• HuggingFace Transformers >= 4.30.0
• Flask >= 2.0.0
• 4 GB RAM minimum (GPU recommended for sub-second inference)

---

👤 Author

Harsh Nerkar

• GitHub: @harshrameshnerkar
• Email: harshrameshnerkar@gmail.com

---

📄 License

This project is licensed under the MIT License.