Skip to content

Add logfmt input format - #342

Merged
noborus merged 1 commit into
noborus:masterfrom
ChrisJr404:logfmt-input
Aug 22, 2026
Merged

Add logfmt input format#342
noborus merged 1 commit into
noborus:masterfrom
ChrisJr404:logfmt-input

Conversation

@ChrisJr404

@ChrisJr404 ChrisJr404 commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

This adds logfmt as an input format, which is what #207 asked for. logfmt is the space-separated key=value line format that a lot of Go services log in (logrus, the standard library's slog text handler, Heroku, etc.), so being able to point trdsql straight at those logs and query them is handy.

It follows the same shape as the existing LTSV reader: a LogfmtReader that reads the keys from the preread rows to build the columns, so -ir works the same way when later lines introduce new keys. Values can be bare tokens or double-quoted strings (with the usual escapes), and a bare key with no = is read as an empty value. You select it with -ilogfmt or by giving a file a .logfmt extension.

Example:

$ printf 'level=info msg="start request" status=200\n' | trdsql -ilogfmt -oat "SELECT level, status, msg FROM -"
+-------+--------+---------------+
| level | status |      msg      |
+-------+--------+---------------+
| info  |    200 | start request |
+-------+--------+---------------+

I added tests mirroring the LTSV ones (quoted values, bare keys, an unterminated-quote error case, and the file-based preread/read-row cases) plus testdata files, and updated the README. make test passes and go vet/gofmt are clean.

Read space-separated key=value pairs (logfmt) as input, wired in the
same way as the LTSV reader. Values can be bare tokens or double-quoted
strings, and columns come from the keys in the preread rows. Enable it
with -ilogfmt or a .logfmt file extension.

Closes noborus#207
@noborus

noborus commented Aug 20, 2026

Copy link
Copy Markdown
Owner

Thank you very much.
Adding logfmt was a TODO item I hadn't gotten around to yet, so I don't see any issue with merging it.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds logfmt as a new input format to trdsql (requested in #207), enabling direct querying of space-separated key=value log lines commonly produced by Go services.

Changes:

  • Introduces a new LOGFMT input format with CLI flag -ilogfmt and .logfmt extension detection.
  • Adds LogfmtReader implementation (including quoted value parsing and bare-key handling) and accompanying tests + testdata.
  • Updates README documentation to describe logfmt usage and examples.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
trdsql.go Adds LOGFMT to the Format enum and string conversion.
reader.go Registers .logfmt extension mapping and NewLogfmtReader factory.
input_logfmt.go Implements logfmt parsing/reading and preread column discovery.
input_logfmt_test.go Adds unit tests mirroring LTSV coverage for logfmt parsing and preread/read-row behaviors.
cmd/cmd.go Adds -ilogfmt flag wiring and format selection updates.
README.md Documents logfmt input format and updates TOC/section numbering.
testdata/test.logfmt Adds sample logfmt test data.
testdata/testnull.logfmt Adds logfmt test data for NULL-replacement scenarios.
testdata/test_quote.logfmt Adds quoted-value logfmt test data.
testdata/test_indefinite.logfmt Adds logfmt test data with evolving keys across rows.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread input_logfmt.go
Comment on lines +167 to +172
key := line[start:i]
if key == "" {
// A stray '='; skip it so parsing makes progress.
i++
continue
}
Comment thread input_logfmt.go
return r.types, nil
}

// PreReadRow is returns only columns that store preread rows.
@noborus
noborus merged commit 50b83d7 into noborus:master Aug 22, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants