fix(k8s/diagnose): keep the results payload out of the argument list - #232
Merged
Conversation
ignacioboud
force-pushed
the
fix/diagnose-argv-limit
branch
2 times, most recently
from
September 2, 2026 17:16
175a850 to
c65c801
Compare
notify_results passed the aggregated check results to jq as a single --argjson value and then to np as a single --body value. Linux caps one execve argument at 128 KiB, which ulimit does not lift, so once a check began publishing application log tails the step died with "jq: Argument list too long" and diagnose published nothing. Keep the payload on disk end to end: xargs for the file list (already the case), --slurpfile instead of --argjson, and a file path for np --body. The .json suffix on the temp files is required — np reads --body from disk only when the value ends in .json, and otherwise sends the string itself as the request body. Also cap log lines at 2000 characters in lines_to_json_array. kubectl's --tail bounds how many lines we collect, not how long each one is, so a single serialized stack trace could still dominate the payload. Tests use a 2 MB payload so they fail on macOS too, where the limit is a total argv size rather than a per-argument cap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ignacioboud
force-pushed
the
fix/diagnose-argv-limit
branch
from
September 2, 2026 17:34
c65c801 to
750eb12
Compare
fedemaleh
approved these changes
Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
diagnoseon k8s scopes dies right after the application-log check:notify_resultspassed the aggregated results tojqas one--argjsonvalue, then the body tonpas one--bodyvalue. Linux caps a single execve argument at 128 KiB, andulimitdoes not lift it — so there is no runtime escape.The
xargson the line above already guarded the file list. The payload four lines later was not guarded.It surfaced now because #181 added
logs/application_log_evidenceand raised that check'slogs[]cap from 20 to 200 lines.kubectl logs --tail=Nbounds line count, not line length, so two pods of structured logs clear 128 KiB easily.Net effect: diagnose collected everything and published nothing.
Fix
Keep the payload on disk end to end:
xargs(unchanged),--slurpfileinstead of--argjson, and a file path fornp --body.The
.jsonsuffix on the temp files is required —npreads--bodyfrom disk only when the value ends in.json, and otherwise sends the string itself. A baremktemppath would have shipped the path as the body: same outage, but silent.notify_resultsnow also propagates np's exit code and fails explicitly if grouping or body-building breaks, instead of continuing with an empty variable.Separately,
lines_to_json_arraytruncates lines pastEVIDENCE_LOG_LINE_MAX_CHARS(default 2000). Every log tail passes through it, so this coversread_log_tailandupdate_check_resultat once.Tests
3 new tests in
diagnose_utils.bats. The regression test uses a 2 MB payload so it fails on macOS too, where the limit is a total argv size rather than a per-argument cap. Itsnpmock mirrors the real.json-suffix rule — a looser[[ -f "$arg" ]]mock passes against a broken fix.146 tests, 0 failuresacrossdiagnose_utils,notify_diagnose_results,notify_check_running,evidence_schema,logs/application_log_evidence,build_context,kubectl_logs,kubectl_get.Not run locally: the
networking/,scope/andservice/suites shell out to realkubectland hang without a cluster — pre-existing, left to CI.Note
The
bash: line 1: 3: Bad file descriptorline in the original report is not from this repo (no fd 3 anywhere in the tree); it comes from the agent wrapper that runs the steps. Unrelated.🤖 Generated with Claude Code