feat: install a documented, rotating log for the configure server - #599
Merged
Conversation
Nothing in the package configured logging: every module took a logging.getLogger(__name__) and no handler was ever installed, so Python fell back to lastResort — WARNING+ went to stderr unformatted and every info/debug was discarded. The diagnostics the code deliberately preserves (the swallowed Meta token-refresh failure, the account-listing warning whose detail is stripped from the HTTP response on purpose) reached nobody, so "check the log" was advice no operator could act on. The one log file that existed belonged to the macOS LaunchAgent's stream redirect, on one platform, in one run mode, named in no documentation. mureo/logging_setup.py installs, from the configure entry point only: * a rotating file handler at <home>/.mureo/logs/configure.log (1 MiB x 3 backups, owner-only) on every platform; * a stderr handler at WARNING, preserving the console surface the lastResort fallback used to provide; * level INFO by default, raised with MUREO_LOG_LEVEL — an env var, not a flag, because the auto-start daemon is launched by a supervisor with a fixed argv. The handlers go on the `mureo` package logger and are installed by run_configure_wizard (the single funnel for interactive, --serve and all three service backends), never at import time and never on the root logger: importing mureo as a library still leaves logging to the host application, and no third-party logger is switched on — the Google Ads SDK logs whole request payloads at DEBUG and raising mureo's level cannot reach it. The path is deliberately not the LaunchAgent's stream capture file: launchd holds an open fd on it, and a rotation would leave the daemon appending to the rotated inode forever. Credential safety: turning a handler on makes previously-dead log lines reachable. Three BaseHTTPRequestHandler.log_message overrides logged the raw request line at DEBUG, and the OAuth callbacks land as GET /callback?code=<authorization code> — exchangeable for a token. They now route through safe_http_log_line, which drops the query string of every access line. No log line at any level carries a token, secret or credential value. Closes #581
Keeps both [Unreleased] entries in CHANGELOG.md (the only conflict). A conflicting PR gets no pull_request CI run, so merging main in is what lets the suite actually run against this branch.
Found by a security review of this PR. Installing a log handler is what makes these lines reach disk, so the fix belongs with it. GoogleAdsException does not curate its __str__: it never passes a message to super().__init__, so formatting it prints the underlying grpc.Call repr, which carries debug_error_string and with it the request metadata (developer token, authorization header). The three account-listing failure paths logged that with exc_info=True. Callers of the same function already logged the class only, and said why in a comment, so the defence existed but was applied one level too high to work. Also harden the mureo home directory itself: mkdir(parents=True, mode=...) applies the mode to the leaf only, so a fresh install created it at the umask default rather than the owner-only mode the docs describe.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Nothing in the package configured logging. There was no
basicConfig, noFileHandler, nodictConfiganywhere undermureo/: every module took alogging.getLogger(__name__)and no handler was ever installed. Under an interactivemureo configurerun Python therefore fell back tolastResort—WARNINGand above went to stderr unformatted, and everylogger.info/logger.debugwas discarded outright.So the diagnostics the code deliberately preserves reached nobody: the swallowed Meta token-refresh failure (
mureo/auth.py), the account-listing warning whose detail is stripped from the HTTP response on purpose (mureo/web/plugin_credentials.py), the record that a credential env var was written (mureo/web/env_var_writer.py). The UI's intentionally generic "Couldn't load accounts." was correct for the HTTP surface and left the cause recoverable nowhere.The only log file that existed was the macOS LaunchAgent's stdout/stderr redirect — one platform, one run mode, and only because launchd redirects the process's streams.
systemd.pyandwindows.pyset no redirection at all, and no doc named any of it.What this adds
mureo/logging_setup.py, installed from the configure entry point only:~/.mureo/logs/configure.logon every platform — 1 MiB × 3 backups (bounded at ~4 MiB however long the daemon runs), created owner-only (0600), re-applied after each rollover;WARNING, so the console surfacelastResortused to provide is preserved rather than silently removed (it only fires when a record finds no handler);INFOby default, raised or lowered withMUREO_LOG_LEVEL.mureo configureprints the path on startup, so "check the log" is now advice with an address.Decisions and why
Scope — the
mureopackage logger, fromrun_configure_wizard, never at import time. Installing on the root logger at import would hijack logging for anyone using mureo as a library, which a package must not do.run_configure_wizardis the single funnel every platform's configure server passes through — interactive,--serve, and all three auto-start backends exec the same CLI — and it already carries thehomeinjection seam, so a test never writes into the real home. Scoping tomureoalso excludes third-party loggers by construction: the Google Ads SDK logs whole request/response payloads (developer token included) at DEBUG, and raising mureo's level must never be able to turn that on. A test pins that.Path —
~/.mureo/logs/configure.log, deliberately not~/.mureo/configure.log. The latter is launchd'sStandardOutPath, and launchd holds an open fd on it. A rotation renames the file out from under launchd, which then appends to the rotated inode forever: two writers, one of them unbounded, and the rotation defeated. Keeping them apart gives the rotating file exactly one writer. On macOSconfigure.log/configure.errremain what they always were — raw stream capture for startup lines and escaped tracebacks — and the docs say which to read first. No change to any of the three service backends is needed, so an already-installed LaunchAgent/unit/task keeps working untouched.Level —
INFO, configurable viaMUREO_LOG_LEVEL. An env var rather than a CLI flag because the daemon is started by a supervisor with a fixed argv (plist / unit / Scheduled Task); a flag would only ever reach the interactive run, while the env var works in both and can be added to the unit. Unset, empty, unrecognised andNOTSETall resolve toINFO— a typo in the variable must not silence the log it was meant to open.Credential safety
Turning on a handler makes previously-dead log lines visible, so an existing line that leaks becomes a real leak on disk. I read every
logger.*call in the package. The deliberate properties hold:env_var_writer.pylogs the field name and target section only;plugin_credentials.pylogs the provider and the exception type; the Google/Meta account-listing failures log the exception class specifically because the SDK can embed the developer token or access token in the exception'sstr();handlers.pyscrubs and caps the Amazon exchange detail before it reaches either the response or the log.One real leak found and fixed. Three
BaseHTTPRequestHandler.log_messageoverrides —mureo/web/handlers.py(configure UI),mureo/cli/web_auth.py(the OAuth wizardmureo configurespawns) andmureo/auth_setup.py(the terminal wizard's callback server) — passed the raw access line tologger.debug. The OAuth callbacks land there asGET /callback?code=<authorization code>&state=…, and an authorization code is exchangeable for a token. That was inert while nothing had a handler; with one installed, and with the docs telling operators how to raise the level to DEBUG, it would have written authorization codes into a file. All three now route throughsafe_http_log_line, which drops the query string of every access line (path, status and size are what triage needs; the query never is — an allow/deny list of parameter names would silently outgrow the next flow).Docs
docs/cli.mdgains a Configure log section (path, rotation, what goes in it, how to raise the level including for the always-on service, the DEBUG caveat, and the macOS launchd distinction). Pointers added fromdocs/authentication.md(the generic-error-surface paragraph) and the troubleshooting sections of both getting-started guides.AGENTS.mdgains the "no log line may carry a credential value" rule and the new module in the tree;docs/architecture.mdlikewise.Tests
tests/test_configure_logging.py(33 tests, written first and failing first): path derivation andMUREO_HOME, no side effects from resolving the path, distinctness from launchd's files, level resolution incl. fallbacks, records reaching the file, root logger untouched, foreign loggers excluded, DEBUG withheld by default and enabled by the env var, stderr handler at WARNING, idempotency, rotation bounds and an actual rollover,0600on POSIX, an unwritable destination degrading to stderr instead of raising, no import-time handlers, the scrubber, all threelog_messageoverrides, and end-to-end wiring throughrun_configure_wizardand the CLI.tests/conftest.pygains an autouse fixture restoring themureologger's handlers and level after every test — without it, any test that reaches the configure entry point would leak a file handler pointing at a deletedtmp_pathand a level that silently filters records other tests assert on.Verification
python -m pytest(whole suite minus the four modules that fail on this machine from locally-installed plugins leaking into tests that assume none): 8821 passed, 8 skipped. Those four modules were then run separately: 12 failures, exactly the known pre-existing set intest_mcp_server_plugin_wiring.py,test_mcp_tool_provider.py,test_mcp_server.py,tests/analytics/builtin/test_live_clients.py.ruff check mureo/ tests/— clean.black --check— clean on every file touched.mypyon all six changed modules — no new errors (only the repo's pre-existing missing-stub errors for openpyxl / protobuf / jsonschema).Known limitation
Two configure servers running at once (single-instance reuse makes this rare, but
--portallows it) share oneRotatingFileHandlertarget, so a rollover happening in both processes can interleave or drop lines. It cannot crash the server, and it is the standard multi-process rotation caveat; noting it rather than adding a lock.Closes #581