Add logfmt input format - #342
Merged
Merged
Conversation
Read space-separated key=value pairs (logfmt) as input, wired in the same way as the LTSV reader. Values can be bare tokens or double-quoted strings, and columns come from the keys in the preread rows. Enable it with -ilogfmt or a .logfmt file extension. Closes noborus#207
Owner
|
Thank you very much. |
There was a problem hiding this comment.
Pull request overview
Adds logfmt as a new input format to trdsql (requested in #207), enabling direct querying of space-separated key=value log lines commonly produced by Go services.
Changes:
- Introduces a new
LOGFMTinput format with CLI flag-ilogfmtand.logfmtextension detection. - Adds
LogfmtReaderimplementation (including quoted value parsing and bare-key handling) and accompanying tests + testdata. - Updates README documentation to describe logfmt usage and examples.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| trdsql.go | Adds LOGFMT to the Format enum and string conversion. |
| reader.go | Registers .logfmt extension mapping and NewLogfmtReader factory. |
| input_logfmt.go | Implements logfmt parsing/reading and preread column discovery. |
| input_logfmt_test.go | Adds unit tests mirroring LTSV coverage for logfmt parsing and preread/read-row behaviors. |
| cmd/cmd.go | Adds -ilogfmt flag wiring and format selection updates. |
| README.md | Documents logfmt input format and updates TOC/section numbering. |
| testdata/test.logfmt | Adds sample logfmt test data. |
| testdata/testnull.logfmt | Adds logfmt test data for NULL-replacement scenarios. |
| testdata/test_quote.logfmt | Adds quoted-value logfmt test data. |
| testdata/test_indefinite.logfmt | Adds logfmt test data with evolving keys across rows. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+167
to
+172
| key := line[start:i] | ||
| if key == "" { | ||
| // A stray '='; skip it so parsing makes progress. | ||
| i++ | ||
| continue | ||
| } |
| return r.types, nil | ||
| } | ||
|
|
||
| // PreReadRow is returns only columns that store preread rows. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds logfmt as an input format, which is what #207 asked for. logfmt is the space-separated
key=valueline format that a lot of Go services log in (logrus, the standard library's slog text handler, Heroku, etc.), so being able to point trdsql straight at those logs and query them is handy.It follows the same shape as the existing LTSV reader: a
LogfmtReaderthat reads the keys from the preread rows to build the columns, so-irworks the same way when later lines introduce new keys. Values can be bare tokens or double-quoted strings (with the usual escapes), and a bare key with no=is read as an empty value. You select it with-ilogfmtor by giving a file a.logfmtextension.Example:
I added tests mirroring the LTSV ones (quoted values, bare keys, an unterminated-quote error case, and the file-based preread/read-row cases) plus testdata files, and updated the README.
make testpasses andgo vet/gofmtare clean.