feat(metrics): measure protocol detection accuracy across all protocols

[mayankpande88]

Why

Detection accuracy is only observable for HTTP/2 today, via the invalid-frame ratio — and only because that ratio happened to be instrumented while chasing a specific bug.

That bug turned out to be a weak byte-pattern heuristic whose verdict is cached per connection, so one false positive poisoned an HTTPS/1.1 connection permanently. External HTTP/2 detection accuracy was roughly 16% before the fix and is 95% after.

Every other protocol is detected the same way. The ClickHouse check is three bytes; each protocol caches its verdict identically. The same failure mode is possible for Postgres, Redis, MySQL and Zookeeper and is currently unmeasured. There is no basis for assuming HTTP/2 was unique rather than simply the one we happened to investigate.

What this adds

node_agent_protocol_reclassified_total{protocol}
node_agent_connections_parsed_total{protocol}

A reclassification is a definitive misdetection: eBPF decided the protocol, and the parser then refused the payload parseFailThreshold times consecutively. trackParseFail already computes that verdict — it just was not counted.

Misdetection rate per protocol:

rate(node_agent_protocol_reclassified_total[1h])
/ (rate(node_agent_protocol_reclassified_total[1h]) + rate(node_agent_connections_parsed_total[1h]))

Per-connection, not per-event

Both counters increment once per connection — reclassified at the threshold, parsed on first success.

This is deliberate. The HTTP/2 invalid-frame ratio is per-event, so a single misdetected connection emitting thousands of invalid frames dominates it. That made the ratio a poor accuracy measure even though it was useful for spotting the bug. A per-connection rate answers the actual question: what fraction of connections did we tag wrongly.

trackParseOK replaces the bare parseFailCount = 0 resets at the five success sites, so the denominator is recorded everywhere the numerator can be.

Testing

gofmt, GOOS=linux gopls check clean; full CI suite passes. Measurement only — no behaviour change beyond recording the reset that already happened.

[gemini-code-assist]

Code Review

This pull request introduces tracking for protocol misdetection rates by adding two new Prometheus metrics, ProtocolReclassifiedTotal and ConnectionsParsedTotal, and updating connection parsing logic to record successful and failed parses per connection. The feedback suggests adding a nil check for the conn parameter in the new trackParseOK function to prevent potential runtime panics.

[gemini-code-assist]

To adhere to defensive programming best practices, we should ensure that 'conn' is not 'nil' before accessing its properties ('parseSucceeded' and 'parseFailCount'). Adding a nil check prevents potential runtime panics if this helper is ever called with a nil connection pointer.

func (c *Container) trackParseOK(conn *ActiveConnection, proto l7.Protocol) {
	if conn == nil {
		return
	}
	if !conn.parseSucceeded {

[mayankpande88]

Declining this one, with reasoning.

conn cannot be nil at any trackParseOK call site. All five sit inside the protocol switch in onL7RequestWithResult, which is reached only after conn has been dereferenced repeatedly — conn.DestinationKey, conn.srcWorkload, conn.Timestamp — so a nil conn would have panicked well before this point.

The adjacent trackParseFail, which this mirrors and which has existed for some time, has no nil check either. Adding one to only the new function would be inconsistent, and adding a guard that cannot fire suggests the invariant is uncertain when it is not — it makes the code marginally harder to reason about rather than easier.

If nil-safety here is wanted as a policy, it should go on both functions in a separate change, so the reasoning applies uniformly rather than to whichever function happened to be touched most recently.