Conversation
jow-
marked this pull request as ready for review
June 20, 2024 22:07
jow-
force-pushed
the
debugger
branch
2 times, most recently
from
July 26, 2026 00:34
e0b5062 to
5454d3c
Compare
Add a new dict type that extends ucode objects by allowing arbitrary
value keys (not limited to strings). Key uniqueness follows uc_uniq()
semantics:
- Scalars (null, bool, int, double, string): compared by value
- Non-scalars (arrays, objects, etc.): compared by pointer equality
- NaN doubles are treated as equal
Dicts are distinguished from regular objects by their hash table
equal_fn function pointer, preserving ext_flag for is_constant semantics.
Provided functionality:
- dict() stdlib constructor accepting optional source object/dict/array
- keys() / values() returning actual value keys for dicts
- for...in iteration yielding value keys
- Spread operator support (dict->object converts keys to strings,
object->dict preserves strings as string values)
- Prototype chain lookup across dict/object boundaries
- GC marking for dict value keys
- JSON/stringification converting value keys to strings
Add stdlib test suite (tests/custom/03_stdlib/69_dict).
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
When an inner function invoked exit() in a ucode > native > ucode call stack situation, the VM did end up with an empty callframe stack after the native function returned, leading to subsequent invalid memory accesses. Properly deal with situation similar to how we also check for an empty call stack after processing I_RETURN and additionally translate the current exception type to a status return value. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Introduce low level facilities for registering breakpoints in the running VM context. The breakpoint primitives allow invoking provided callback functions when the VM reaches an associated instruction address. This functionality provides the foundation for building more thorough interactive debug functionality on top. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Publicly export the instruction format table in order to make it useable for libucode.so users, such as dynamically loaded debug libraries. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
When additional ucode scripts are loaded through require() or similar means, and the required code invokes exit(), the VM will clear the stack before returning, so we must ensure that the stack size matches our expectation before we're trying to pop values from it after returning from require. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Implement an interactive command line debugger within the debug module which can be started by invoking the `debugger()` function. The debugger offers common features such as the ability to set breakpoints, stepping through commands, examining call stacks and variables as well as byte code disassembly source code highlighting. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Introduce a new command line switch `-x` which loads the debug module and launches the given program or expression within the interactive debugger. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
- Add isatty() check to detect interactive vs non-interactive mode - Implement term_getline_fallback() for simple line reading when piped - Skip terminal raw mode and signal settings in non-interactive mode - Add EOF handling in term_getc_raw() to return -1 in non-interactive mode - Add quit -f flag for forced quit without confirmation in non-interactive mode - Fix filename_matches_pattern() to handle basenames without path separators - Fix BK_STEP breakpoint handling to properly free breakpoints after being hit - Fix cmd_continue() to return false after processing one command All 45 debugger tests pass. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
The remote debug protocol only exposed a hand-rolled subset of commands (continue/quit/help/print) against a small line-based dispatcher, while the local interactive debugger in lib/debug.c supports 16 commands (break, delete, list, next, step, continue, return, backtrace, variables, sources, print, lines, throw, disassemble, quit) with tab completion, history and ANSI-highlighted source/backtrace rendering. Rather than reimplementing all of that against the remote protocol, reuse it directly: term_getline()/term_printf() only ever do plain read()/write() on STDIN_FILENO/STDOUT_FILENO, with tty-specific tcgetattr/tcsetattr calls isolated in term_raw()/term_isig()/ term_reset(). So once a client connects, debug_cli_run_remote_session() dup2()s the socket onto stdin/stdout for the session and calls the same bk_enter_cli() dispatcher used locally, skipping just the tty ioctls via a new termstate.remote flag (a socket has no line discipline to configure; the remote peer manages its own local raw mode). No PTY is needed - raw single-key reads and ANSI rendering work identically over a plain socket once those ioctls are skipped. Breakpoints set during a session keep working across "continue" since they're dispatched directly from uc_vm_execute_chunk()'s per-instruction breakpoint check, nested inside the uc_vm_resume() call debug_cli_run_remote_session() makes after the initial CLI call returns - so they reenter bk_enter_cli() using the same fds. lib/debug_remote.c is now transport-only (socket create/accept/cleanup plus the EVENT push helpers); the old uloop-based line protocol (debug_handle_command, debug_remote_loop, the uloop fd callbacks) is gone. udbg.c changes from doing its own local line-editing to a transparent raw byte pump in both directions, since the server now drives all of the rendering - matching how a real terminal client should behave once the full CLI is exposed remotely. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Add a script-callable debug.listen() covering three cases: no argument arms SIGUSR1-triggered remote debugging on the PID-derived attach socket without blocking; a truish argument does the same but also blocks right here until a client connects (or a 30s timeout); a string argument binds an arbitrary caller-chosen socket path and blocks indefinitely. This is the counterpart to -X for host applications that embed the VM directly (uhttpd, uwsd, ...) and have no -X flag of their own. The SIGUSR1 case is dispatched through ucode's own signal() builtin rather than the break_requested/STATUS_BREAK mechanism -X uses, since the latter unwinds the entire C call stack back to whoever called uc_vm_execute()/ uc_vm_call(), which an embedding host has no way to handle safely. Along the way, consolidate the previously separate, largely duplicated debug.listen(path) (arbitrary path, blocking accept, inline bind/listen code) into this single function, sharing one bind/listen helper (debug_remote_bind_and_listen()) with the SIGUSR1 attach socket instead of two near-identical copies. debug_remote.c is now purely socket transport (bind/accept/cleanup, EVENT push helpers); the script-facing API and the interactive session driver live in debug.c. Also drops the now-redundant debug_remote_fns[]/uc_module_init_remote() registration path (debug.listen was already being registered a second time via debug_fns[] in debug.c). Testing this against a real embedding host (uwsd, which calls uc_vm_init(&ctx.vm, NULL)) surfaced a serious pre-existing bug: with no config, setup_signal_handlers defaults to false, so uc_vm_signal_handlers_setup() never wires up the signal self-pipe/handler array. Installing a handler through signal() in that state silently ends up with a NULL/SIG_DFL disposition, which terminates the process on the next occurrence of that signal instead of invoking the handler - confirmed by sending SIGUSR1 to a live uwsd worker and watching it die. This affected not just the new debug.listen() but also the pre-existing debug.attach() and the memory-dump signal handler (SIGUSR2 by default), for any host that doesn't opt into setup_signal_handlers. Fix: split the self-pipe/handler-array setup out of uc_vm_signal_handlers_setup() into a new uc_vm_signal_handlers_ensure(), exported so debug_setup() can call it unconditionally at debug module load time regardless of what the host configured. uc_vm_signal_dispatch() now checks whether the pipe was actually initialized rather than re-checking the original config flag, so signals raised this way are correctly dispatched too. Verified end-to-end against an isolated build of uwsd (linked against a temporary ucode install so as not to touch the system installation): a debug.listen()-armed handler script paused mid-request on SIGUSR1, udbg attached and ran backtrace/continue against it showing the real onBody(request=<uwsd.connection ...>) call stack, and the worker process remained healthy and resumed normally afterwards. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Fixes found while exercising the remote debugger end to end:
- bk_enter_cli() redid the whole listen/accept/splice dance on every
breakpoint hit, tearing down the live connection on each "next"/"step"
and requiring the client to reconnect within a 30s window or the script
would silently run to completion. It now reuses an already-connected
client across breakpoint hits, and on an unexpected disconnect (not an
explicit "quit") re-arms the listen socket instead of resuming the
paused script, so udbg can reconnect where it left off.
- insn_length()/cmd_disasm() mis-decoded the I_CALL operand's spread-count
bits, corrupting bytecode offset math for any call site (method calls
especially) and producing garbage disassembly.
- cmd_delete()'s "delete current breakpoint" path freed the very
debug_breakpoint_t bk_enter_cli() was still using for the rest of the
session (subsequent next/step reads ->depth; end-of-session cleanup
reads ->kind/->bk.ip) - a use-after-free and, once the session ended, a
double free. Deleting it now just unlinks it and defers the actual
free() until bk_enter_cli() is done with it.
- uc_debug_attach() unconditionally put the target's own controlling
terminal into raw mode at attach time, even though attach-mode sessions
only ever interact over the spliced remote socket once a client
connects - left the launching terminal echo-less with nothing to ever
restore it.
- format_context_header_backtrace()/format_context_header_callframe()
never terminated their breadcrumb-bar line, so the following source
snippet ran on directly after it instead of starting on its own line.
Also adds:
- debug.notifyExit(), pushing a JSON "EVENT exit" message to an attached
remote client right before the target exits (normal completion, exit(),
or an uncaught error, with the full exception object incl. stacktrace)
instead of leaving the client to infer it from the socket closing.
- debug_remote_notify_exception()/_notify_exit() now serialize the actual
{type, message, stacktrace} exception object (vm.c's
uc_vm_exception_object()) as JSON rather than a prose string.
- A dedicated BK_UNCAUGHT system breakpoint (vm.c) that fires right before
an exception nothing would catch starts unwinding the stack, with
callframes still fully intact - unlike hooking the existing exception
handler chain, which only runs after the real (destructive) unwind has
already popped the throwing frame. uc_vm_exception_would_be_caught()
non-destructively predicts whether anything would handle the exception
before deciding to break.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Replace the monolithic terminal debugger with a structured, line-based
protocol (VERB + optional JSON payload per line, see lib/debug_proto.h)
so the debug core never renders anything - no ANSI, no source text, no
formatted columns - and any client can drive it by speaking the wire
format alone.
- lib/debug_proto.c/h: protocol framing shared by the server.
- lib/debug.c: all CLI commands rewritten to emit structured responses;
bk_enter_session() replaces the old bk_enter_cli(), dispatching on VERB
over whatever fd it's handed (local socketpair or remote socket) rather
than raw terminal I/O over dup2'd stdio.
- udbg.c: rewritten as a real protocol client (typed commands, rendered
responses) instead of a dumb byte-forwarder; supports <pid>, a
socket path, or --fd N (used internally by local -x mode).
- debug_highlight.c/h: the original regex-based ucode/utpl syntax
highlighter and ANSI source renderer, ported out of lib/debug.c into a
standalone module with no ucode dependencies, adapted to the protocol's
per-line {file,line,col} coordinates instead of live source buffers.
- Local `-x` mode now forks and execs `udbg --fd 3` over a socketpair
instead of raw-tty'ing its own stdio, converging local and remote
sessions onto one code path.
- tests/custom/99_debugger: migrated to drive real subprocesses via
`-X<file>:1` and assert on parsed protocol messages instead of
rendered text.
- docs/debugger.md: rewritten for the protocol/client-server split.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Fix a real hang-looking bug: the "dbg > " prompt only reappeared on a fresh PAUSED message, never after other command responses (bt, lines, ...), and stdin could be read/sent before the connection's own initial PAUSED had even been drained - racing ahead of and garbling in-flight responses (most visibly, overlapping SOURCE fetches clobbering each other's pending state). Both are now gated on one "actually at a prompt" condition. Port the remaining original rendering pieces from lib/debug.c's git history into debug_highlight.c, adapted for the protocol: - format_context_header_backtrace()/format_context_header_callframe()'s full-width "[file] breadcrumb "/"[file] signature " status bar, shown above a paused location and above each backtrace frame. The server now sends the full call-chain breadcrumb in PAUSED's payload since the client no longer has the raw callframe stack to derive it from itself. - The single underlined "current instruction" character, in addition to the shaded statement span. Backtrace frames now render their own highlighted source snippet, header bar included - this needed a small async multi-file fetch queue in udbg.c since a backtrace can span several source files at once, unlike every other response which only ever needs one. Async EVENT messages (exception/exit/signal) are now rendered as human-readable, faint+italic lines instead of a raw JSON dump. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
udbg was unconditionally fetching source text from the server via SOURCE, even though the common cases are either fully local debugging (client and target share a filesystem) or a dev checkout driving a remote target where the *client* has the better source access, not the server. Try the exact reported path locally first; only fall back to asking the server if that fails. Add -s/--srcdir DIR for when the reported path doesn't exist as-is on this machine (different checkout/build root): DIR joined with just the reported path's basename is tried before the server fallback. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
…s gaps - The protocol's "col"/"from_col"/"to_col" fields are 1-based (human "line:col" display, see uc_source_get_line() in source.c), but debug_highlight's span/ip columns are 0-based byte indices - every consumer except render_paused() was feeding the raw 1-based value straight in, shifting the underline/highlight one character to the right. Added col0() and applied it everywhere a "col" field feeds a debug_highlight_span_t. - render_source_lines() passed the full terminal width to debug_highlight_print_source() without subtracting left_pad, so backtrace's indented (left_pad=2) snippets ran 2 columns past the terminal width instead of wrapping/padding to what's actually left. - Ported format_context_statement()'s multi-range/ellipsis-gap layout (a window at the start, a "…" gap, a window around the current instruction/end) for statements too long to show in full, via a new debug_highlight_print_source_ranges() and a client-side compute_context_ranges() that mirrors the original's split heuristic. Also: - Fix the prompt never reappearing after an unrecognized command (or "quit" declined at its confirmation prompt): send_command() now reports whether it actually dispatched anything to the server, and the main loop only waits for a response when it did. - Command matching is now shortest-unique-prefix against each command's full name (plus the existing short aliases that aren't literal prefixes, e.g. "bt", "ls"), matching the original interactive CLI's dispatch instead of requiring the full word or a fixed 1-letter alias. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
update_catchpoint() re-arms the exception catchpoint on every debugger pause, via update_breakpoint()'s "already armed at this ip -> invoke handler now" shortcut. That shortcut is fine for its BK_STEP use cases, but unsound for BK_CATCH: whenever two pauses land inside the same try/catch range (trivial to hit whenever a function's whole body is wrapped in one try, e.g. a test runner's loop), the second pause finds the target already armed and fires bk_handle_catch() -> bk_enter_session() synchronously and recursively, without the VM ever advancing - overflowing the stack. Arm the breakpoint directly instead of going through that fast path. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
…ace width Restores the pre-protocol interactive debugger's rendering richness on top of the new client/server split, in debug_highlight.c so any client can reuse it: - DISASSEMBLE: color-coded hex byte dump, semantic operand annotations (constants, local/upval/global names, closure/arrow index with capture lines, and now a decoded CALL mcall-flag/argcount) - the server now also ships raw instruction bytes, operand format and call/closure detail needed to render this client-side. - VARIABLES: kind shown via color only (bold cyan upvalue, faint this/ internal) matching the old CLI, with values rendered compact/single-line and truncated the old way (ellipsis placed before a synthetic closing bracket/quote) instead of always pretty-printed across multiple lines. - help: answered entirely client-side from a ported, word-wrapped usage table describing this client's own typed commands, instead of dumping the server's wire-protocol verb/payload reference. - backtrace: fixed each frame's header bar overflowing the terminal width by the "#N " prefix's length. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
- New debug_lineedit.c/.h: raw-mode, non-blocking terminal line editing for udbg's "dbg > " prompt - history (up/down), word-jump (Ctrl-Left/ Right), Ctrl-W, Tab command-name completion - ported from the pre-protocol interactive debugger's hand-rolled termline_t editor, with no external readline/editline dependency, matching that original choice. Falls back to plain fgets() when stdin isn't a tty. - New SET command: the idiomatic way to change a variable's value while paused (`set x expr`), instead of misusing PRINT with an assignment expression. Writes straight into the resolved local/upvalue slot or the same undeclared-global fallback plain assignment uses. - PRINT no longer rejects expressions that don't start with a variable/ this load (eval_expr's old chunk->entries[0] check) - it only ever blocked bare-literal-first expressions like "1+2" while already letting through identifier-first mutations, so it wasn't a meaningful restriction to begin with. print/set now evaluate like GDB's print: any expression, side effects included. - Fixed a real hang this unlocked: eval_expr() runs the compiled expression with a fresh, empty callframe/stack (so it can't see the paused program's real call stack), which makes an exception raised inside it look exactly like the whole program running out of callframes to unwind to - precisely what the debugger's "pause on uncaught exception" system breakpoint exists to catch. Left armed, a throwing print/set expression paused into a confusing nested session instead of just reporting the exception back as part of that command's own reply. eval_expr() now disarms that breakpoint (and BK_CATCH, defensively) for the duration of the call. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
- Replace the dedicated SET command with EVAL, mirroring the ucode CLI's -e/-p distinction: EVAL is PRINT's twin that discards the expression's result instead of reporting it. "set x.y 1" is just "eval x.y = 1" - ordinary assignment syntax already handles plain variables, property paths and array indices alike, so there is no separate name-resolution command needed for it. - Fixed eval_expr() to make assignment (and other side effects) actually reach the paused frame's real local/upvalue storage. It runs the expression against a temporary scope object pre-populated with *referenced* variables only, which missed a bare "x = 1": the compiler emits that as a plain SVAR with no preceding LVAR read (nothing to read first), so unlike "x + 1" it never looked up "x" against that scope at all - falling through to the real global scope chain and silently creating an unrelated global instead of touching the real local. The scope is now pre-populated with every declared local/upvalue in range, not just referenced ones, and eval_expr() writes any of them straight back into the real stack slot/upvalue after the call, the same way I_SLOC/I_SUPV do. - VARIABLES now marks a shadowed declaration (a same-named, less-nested local still holding a live value, just not what plain script code resolves the name to right now) instead of silently listing duplicates - rendered faint with a "(shadowed)" suffix, whose width is reserved from the value's truncation budget so it can't itself push the line past the terminal width. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Sitting at "dbg > " only ever happens while the debuggee is already paused - there was no way to get its attention while it was actually running (mid "continue"), short of waiting for the next breakpoint or killing it outright. - New BK_INTERRUPT system breakpoint: pre-created (inert) alongside BK_UNCAUGHT at session setup specifically so debug_break_signal_handler() never has to malloc from signal-handler context - arming it is just two field writes. Its "already attached" branch now arms it (fire on the very next instruction) instead of the old passive-only notification (debug_remote_notify_signal(), removed along with the now-dead "signal" client-side event case): any SIGUSR1 while attached is a deliberate act either way, and pausing for inspection is the reasonable universal response to it, from whoever sent it. bk_handle_interrupt() disarms itself before entering the session so it fires exactly once per request. eval_sandbox_enter()/_leave() (print/eval's exception sandboxing) also suspend and race-safely restore it, so a request arriving mid-eval isn't lost. - udbg resolves the debuggee's PID via SO_PEERCRED right after connecting - uniform across all three connection modes (<pid>, <socket-path>, inherited --fd) - and now watches stdin continuously (not just while at a prompt) whenever raw-mode editing is active. A lone Ctrl-C while the debuggee is running sends it SIGUSR1; anything else typed while running is discarded, same as before this feature. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a new interactive debug cli which can be started by invoking the ucode interpreter with the
-xflag.The main debugger functionality resides in the debug library while the cli frontend just gains some minimal amount of code to load and instantiate the debug library when the
-xoption is passed.As a precondition for the new debug functionality, the ucode VM has been extended with low level breakpoint support.