Skip to content

feat: install a documented, rotating log for the configure server - #599

Merged
hyoshi merged 5 commits into
mainfrom
fix/configure-logging-handler
Aug 12, 2026
Merged

feat: install a documented, rotating log for the configure server#599
hyoshi merged 5 commits into
mainfrom
fix/configure-logging-handler

Conversation

@hyoshi

@hyoshi hyoshi commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

The problem

Nothing in the package configured logging. There was no basicConfig, no FileHandler, no dictConfig anywhere under mureo/: every module took a logging.getLogger(__name__) and no handler was ever installed. Under an interactive mureo configure run Python therefore fell back to lastResortWARNING and above went to stderr unformatted, and every logger.info / logger.debug was discarded outright.

So the diagnostics the code deliberately preserves reached nobody: the swallowed Meta token-refresh failure (mureo/auth.py), the account-listing warning whose detail is stripped from the HTTP response on purpose (mureo/web/plugin_credentials.py), the record that a credential env var was written (mureo/web/env_var_writer.py). The UI's intentionally generic "Couldn't load accounts." was correct for the HTTP surface and left the cause recoverable nowhere.

The only log file that existed was the macOS LaunchAgent's stdout/stderr redirect — one platform, one run mode, and only because launchd redirects the process's streams. systemd.py and windows.py set no redirection at all, and no doc named any of it.

What this adds

mureo/logging_setup.py, installed from the configure entry point only:

  • a rotating file handler at ~/.mureo/logs/configure.log on every platform — 1 MiB × 3 backups (bounded at ~4 MiB however long the daemon runs), created owner-only (0600), re-applied after each rollover;
  • a stderr handler at WARNING, so the console surface lastResort used to provide is preserved rather than silently removed (it only fires when a record finds no handler);
  • level INFO by default, raised or lowered with MUREO_LOG_LEVEL.

mureo configure prints the path on startup, so "check the log" is now advice with an address.

Decisions and why

Scope — the mureo package logger, from run_configure_wizard, never at import time. Installing on the root logger at import would hijack logging for anyone using mureo as a library, which a package must not do. run_configure_wizard is the single funnel every platform's configure server passes through — interactive, --serve, and all three auto-start backends exec the same CLI — and it already carries the home injection seam, so a test never writes into the real home. Scoping to mureo also excludes third-party loggers by construction: the Google Ads SDK logs whole request/response payloads (developer token included) at DEBUG, and raising mureo's level must never be able to turn that on. A test pins that.

Path — ~/.mureo/logs/configure.log, deliberately not ~/.mureo/configure.log. The latter is launchd's StandardOutPath, and launchd holds an open fd on it. A rotation renames the file out from under launchd, which then appends to the rotated inode forever: two writers, one of them unbounded, and the rotation defeated. Keeping them apart gives the rotating file exactly one writer. On macOS configure.log / configure.err remain what they always were — raw stream capture for startup lines and escaped tracebacks — and the docs say which to read first. No change to any of the three service backends is needed, so an already-installed LaunchAgent/unit/task keeps working untouched.

Level — INFO, configurable via MUREO_LOG_LEVEL. An env var rather than a CLI flag because the daemon is started by a supervisor with a fixed argv (plist / unit / Scheduled Task); a flag would only ever reach the interactive run, while the env var works in both and can be added to the unit. Unset, empty, unrecognised and NOTSET all resolve to INFO — a typo in the variable must not silence the log it was meant to open.

Credential safety

Turning on a handler makes previously-dead log lines visible, so an existing line that leaks becomes a real leak on disk. I read every logger.* call in the package. The deliberate properties hold: env_var_writer.py logs the field name and target section only; plugin_credentials.py logs the provider and the exception type; the Google/Meta account-listing failures log the exception class specifically because the SDK can embed the developer token or access token in the exception's str(); handlers.py scrubs and caps the Amazon exchange detail before it reaches either the response or the log.

One real leak found and fixed. Three BaseHTTPRequestHandler.log_message overrides — mureo/web/handlers.py (configure UI), mureo/cli/web_auth.py (the OAuth wizard mureo configure spawns) and mureo/auth_setup.py (the terminal wizard's callback server) — passed the raw access line to logger.debug. The OAuth callbacks land there as GET /callback?code=<authorization code>&state=…, and an authorization code is exchangeable for a token. That was inert while nothing had a handler; with one installed, and with the docs telling operators how to raise the level to DEBUG, it would have written authorization codes into a file. All three now route through safe_http_log_line, which drops the query string of every access line (path, status and size are what triage needs; the query never is — an allow/deny list of parameter names would silently outgrow the next flow).

Docs

docs/cli.md gains a Configure log section (path, rotation, what goes in it, how to raise the level including for the always-on service, the DEBUG caveat, and the macOS launchd distinction). Pointers added from docs/authentication.md (the generic-error-surface paragraph) and the troubleshooting sections of both getting-started guides. AGENTS.md gains the "no log line may carry a credential value" rule and the new module in the tree; docs/architecture.md likewise.

Tests

tests/test_configure_logging.py (33 tests, written first and failing first): path derivation and MUREO_HOME, no side effects from resolving the path, distinctness from launchd's files, level resolution incl. fallbacks, records reaching the file, root logger untouched, foreign loggers excluded, DEBUG withheld by default and enabled by the env var, stderr handler at WARNING, idempotency, rotation bounds and an actual rollover, 0600 on POSIX, an unwritable destination degrading to stderr instead of raising, no import-time handlers, the scrubber, all three log_message overrides, and end-to-end wiring through run_configure_wizard and the CLI.

tests/conftest.py gains an autouse fixture restoring the mureo logger's handlers and level after every test — without it, any test that reaches the configure entry point would leak a file handler pointing at a deleted tmp_path and a level that silently filters records other tests assert on.

Verification

  • python -m pytest (whole suite minus the four modules that fail on this machine from locally-installed plugins leaking into tests that assume none): 8821 passed, 8 skipped. Those four modules were then run separately: 12 failures, exactly the known pre-existing set in test_mcp_server_plugin_wiring.py, test_mcp_tool_provider.py, test_mcp_server.py, tests/analytics/builtin/test_live_clients.py.
  • ruff check mureo/ tests/ — clean. black --check — clean on every file touched.
  • mypy on all six changed modules — no new errors (only the repo's pre-existing missing-stub errors for openpyxl / protobuf / jsonschema).

Known limitation

Two configure servers running at once (single-instance reuse makes this rare, but --port allows it) share one RotatingFileHandler target, so a rollover happening in both processes can interleave or drop lines. It cannot crash the server, and it is the standard multi-process rotation caveat; noting it rather than adding a lock.

Closes #581

hyoshi added 3 commits August 13, 2026 06:35
Nothing in the package configured logging: every module took a
logging.getLogger(__name__) and no handler was ever installed, so Python
fell back to lastResort — WARNING+ went to stderr unformatted and every
info/debug was discarded. The diagnostics the code deliberately
preserves (the swallowed Meta token-refresh failure, the account-listing
warning whose detail is stripped from the HTTP response on purpose)
reached nobody, so "check the log" was advice no operator could act on.
The one log file that existed belonged to the macOS LaunchAgent's stream
redirect, on one platform, in one run mode, named in no documentation.

mureo/logging_setup.py installs, from the configure entry point only:

* a rotating file handler at <home>/.mureo/logs/configure.log (1 MiB x 3
  backups, owner-only) on every platform;
* a stderr handler at WARNING, preserving the console surface the
  lastResort fallback used to provide;
* level INFO by default, raised with MUREO_LOG_LEVEL — an env var, not a
  flag, because the auto-start daemon is launched by a supervisor with a
  fixed argv.

The handlers go on the `mureo` package logger and are installed by
run_configure_wizard (the single funnel for interactive, --serve and all
three service backends), never at import time and never on the root
logger: importing mureo as a library still leaves logging to the host
application, and no third-party logger is switched on — the Google Ads
SDK logs whole request payloads at DEBUG and raising mureo's level
cannot reach it. The path is deliberately not the LaunchAgent's stream
capture file: launchd holds an open fd on it, and a rotation would leave
the daemon appending to the rotated inode forever.

Credential safety: turning a handler on makes previously-dead log lines
reachable. Three BaseHTTPRequestHandler.log_message overrides logged the
raw request line at DEBUG, and the OAuth callbacks land as
GET /callback?code=<authorization code> — exchangeable for a token. They
now route through safe_http_log_line, which drops the query string of
every access line. No log line at any level carries a token, secret or
credential value.

Closes #581
Keeps both [Unreleased] entries in CHANGELOG.md (the only conflict).
A conflicting PR gets no pull_request CI run, so merging main in is
what lets the suite actually run against this branch.
Found by a security review of this PR. Installing a log handler is what
makes these lines reach disk, so the fix belongs with it.

GoogleAdsException does not curate its __str__: it never passes a message
to super().__init__, so formatting it prints the underlying grpc.Call
repr, which carries debug_error_string and with it the request metadata
(developer token, authorization header). The three account-listing
failure paths logged that with exc_info=True. Callers of the same
function already logged the class only, and said why in a comment, so the
defence existed but was applied one level too high to work.

Also harden the mureo home directory itself: mkdir(parents=True, mode=...)
applies the mode to the leaf only, so a fresh install created it at the
umask default rather than the owner-only mode the docs describe.
@hyoshi
hyoshi merged commit c15b387 into main Aug 12, 2026
13 checks passed
@hyoshi
hyoshi deleted the fix/configure-logging-handler branch August 12, 2026 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

configure server installs no logging handler, so emitted warnings are unreachable

1 participant