Skip to content

Deterministic Grafana preflight tool (fallback for prompt-driven validation) #38

Description

@tallpsmith

The Why

Issue #10 implements Grafana datasource validation as a prompt-driven workflow — Claude calls mcp-grafana tools and compares URLs. This works but is non-deterministic: it relies on Claude correctly executing a multi-step comparison. If this proves unreliable in practice (wrong datasource selected, URL comparison flubbed, etc.), we need a deterministic fallback.

Depends on: #10 (prompt-driven preflight must be tried first)


Context from #10 Design

The #10 design chose "Option A: Prompt-driven preflight" over "Option B: Preflight tool with user-supplied datasource config" because it's simpler and requires no extra configuration. The design explicitly noted:

"If prompt-driven validation proves unreliable, fall back to a deterministic pcp_grafana_preflight tool with user-supplied config."

This issue is that fallback.


Proposed Feature

A new pmmcp tool: pcp_grafana_preflight

Configuration:

  • PMMCP_GRAFANA_DATASOURCE_UID — the expected Grafana datasource UID (user pre-configures this)

Behaviour:

  1. Claude calls mcp-grafana.get_datasource(uid) and passes the result to pcp_grafana_preflight
  2. The tool extracts the datasource's pmproxy URL from the response
  3. Compares it deterministically with pmmcp's configured PMPROXY_URL
  4. Returns a structured result: { "valid": true/false, "datasource_uid": "...", "pmproxy_url_match": true/false, "details": "..." }
  5. Caches the result for the session

Advantages over prompt-driven:

  • URL comparison is deterministic Python code, not LLM interpretation
  • Structured return value — prompts can branch on valid: true/false
  • Cacheable in server state (not just conversation context)

When to Build This

Don't build this proactively. Only implement if:

  • Users report that prompt-driven preflight is matching wrong datasources
  • URL comparison is failing (e.g. http://pmproxy:44322 vs http://localhost:44322 — same host, different names)
  • Session context loss causes repeated unnecessary preflight checks

The URL aliasing case (same pmproxy, different hostnames) is actually the most likely failure mode — worth watching for.


Open Questions

  • Should the tool accept the raw datasource JSON from mcp-grafana, or just the URL string?
  • How to handle URL aliasing (localhost vs hostname vs IP)? Normalize URLs before comparing?
  • Should this replace the prompt-driven flow entirely, or augment it (prompt discovers, tool validates)?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions