feat: Add multimodal document analysis with PDF and image support

[justinwilkin]

Summary

Adds a new analyze_document tool that enables Forge to analyze visual content including images (PNG/JPG) and PDF documents. This feature supports the excalidraw diagram generation workflow by allowing the agent to analyze mockups and validate generated diagrams, while also providing general document understanding capabilities for PDFs with visual elements.

The implementation uses the existing LLM provider abstraction with configurable vision models, includes smart pagination for large PDFs to optimize token usage, and outputs agent-optimized structured analysis rather than human-readable prose.

Changes

• New Tool: Added analyze_document tool in pkg/tools/coding/analyze_document.go with PDF and image analysis support
• Architecture Documentation: Added ADR 0053 documenting multimodal document analysis architecture decisions
• Product Documentation: Added feature documentation and scratch notes for multimodal analysis
• Configuration: New multimodal configuration options in pkg/config/multimodal.go for vision model selection
• LLM Provider: Extended OpenAI provider with vision capabilities and multimodal message support
• UI Settings: Added multimodal settings to TUI overlay for model configuration
• Integration: Registered analyze_document tool in both headless and TUI entry points
• Test Updates: Updated various tests to accommodate new tool registration

Testing

1. Test image analysis: analyze_document /path/to/diagram.png - verify agent can extract visual information
2. Test PDF analysis: analyze_document /path/to/document.pdf - verify text and visual content extraction
3. Test multi-page PDFs: verify pagination works and token usage is optimized
4. Test configuration: verify vision model selection works in settings UI
5. Verify security: ensure tool respects workspace isolation and file access controls
6. Test excalidraw workflow: analyze a hand-drawn mockup and validate a generated diagram PNG

[justinwilkin]

@anvxl docs

[anvxl]

🤖 Documentation Update Results

✅ Documentation updates completed successfully!

📝 A pull request with documentation updates has been created.

📊 Execution artifacts are available in the workflow run.

[Copilot]

Pull request overview

Adds first-class multimodal document understanding to Forge by introducing an analyze_document tool (images + PDFs) and wiring it through config, provider, and UI settings so agents can reason about visual artifacts (e.g., excalidraw mockups, diagram screenshots, diagram-heavy PDFs).

Changes:

• Introduces analyze_document coding tool with workspace-guarded file access and PDF page-range parameters.
• Extends llm.Provider with AnalyzeDocument and implements it for the OpenAI provider.
• Adds multimodal configuration + TUI settings support and accompanying docs/ADR updates.

Reviewed changes

Copilot reviewed 20 out of 21 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
pkg/tools/coding/analyze_document.go	New tool implementation for image/PDF analysis with page-range inputs and multimodal model override.
pkg/tools/browser/analyze_page_test.go	Updates mock provider to satisfy new Provider interface method.
pkg/llm/provider.go	Extends Provider interface with AnalyzeDocument.
pkg/llm/openai/openai.go	Implements AnalyzeDocument and optimizes Complete() accumulation.
pkg/executor/tui/overlay/settings_test.go	Updates mock provider to satisfy new Provider interface method.
pkg/executor/tui/overlay/settings.go	Adds “multimodal” section to settings UI load/save paths.
pkg/config/multimodal.go	New config section for multimodal model and PDF page limit.
pkg/config/config.go	Registers multimodal section and exposes GetMultimodal().
pkg/agent/longtermmemory/retrieval/helpers_test.go	Updates fake provider to satisfy new Provider interface method.
pkg/agent/longtermmemory/retrieval/engine_test.go	Updates counting provider to forward new Provider interface method.
pkg/agent/longtermmemory/capture/capture_test.go	Updates fake provider to satisfy new Provider interface method.
pkg/agent/default_test.go	Updates mock provider to satisfy new Provider interface method.
pkg/agent/context/tool_call_strategy_test.go	Updates mock provider to satisfy new Provider interface method.
go.sum	Adds pdfcpu + related indirect deps; bumps some indirect versions.
go.mod	Adds pdfcpu dependency and updates indirect dependencies.
docs/product/scratch/multimodal-analysis.md	Adds scratch notes describing intended multimodal workflow and pagination semantics.
docs/product/features/multimodal-analysis.md	Adds product/feature documentation for multimodal analysis capability.
docs/adr/README.md	Adds ADR index entries for 0052 and 0053.
docs/adr/0053-multimodal-document-analysis.md	Adds ADR describing architecture and design decisions for multimodal analysis.
cmd/forge/main.go	Registers analyze_document in TUI toolset.
cmd/forge-headless/main.go	Registers analyze_document in headless toolset.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.