control-plane-api: run publication tests on catalog-tests - #3418
Merged
Conversation
This was referenced Aug 27, 2026
jgraettinger
force-pushed
the
johnny/v2-tests-3-catalog-tests
branch
from
August 30, 2026 17:56
7c143a6 to
db1ce76
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
2 times, most recently
from
August 30, 2026 18:08
4d689b9 to
12b25e5
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-3-catalog-tests
branch
from
August 31, 2026 22:08
db1ce76 to
df38483
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
from
August 31, 2026 22:08
12b25e5 to
893de73
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-3-catalog-tests
branch
from
August 31, 2026 22:19
df38483 to
5dd646c
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
from
August 31, 2026 22:19
893de73 to
a7bf70c
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-3-catalog-tests
branch
from
August 31, 2026 22:46
5dd646c to
5106b77
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
2 times, most recently
from
September 1, 2026 00:51
a3a147b to
e99bf9f
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-3-catalog-tests
branch
from
September 1, 2026 00:51
5106b77 to
0b5782c
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
from
September 1, 2026 01:34
e99bf9f to
5727a84
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-3-catalog-tests
branch
from
September 1, 2026 04:46
0b5782c to
dae189d
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
from
September 1, 2026 04:46
5727a84 to
8c84db4
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-3-catalog-tests
branch
from
September 1, 2026 15:34
dae189d to
0094e7a
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
from
September 1, 2026 15:34
8c84db4 to
d7b8fb9
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-3-catalog-tests
branch
from
September 1, 2026 16:17
0094e7a to
a2e9974
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
from
September 1, 2026 16:17
d7b8fb9 to
f53a04b
Compare
`run_test_case` returned the scope of the last step reached, documented as being for error reporting, but no caller used it: the failure path takes its scope from `LiveDriver::last_scope` — which the driver tracks itself — and the success path uses the test's own first-step scope. The returned `String` was dead and its doc comment misleading. Return `()` instead, and drop the per-step tracking that fed it.
Publication tests now call `catalog_tests::run_tests` in-process, against the
`build::Output` the publication already produced.
That deletes most of the machinery: `data_plane()` and its `temp-data-plane`
shell-out, `wait_for_sockets`, the activate loop that stamped build labels onto
derivation specs and activated them at 3 splits, the `flowctl-go api test` job,
and the delete loop that tore it all down again. With no data plane there is
nothing to race, so the `tokio::select!` driving the two jobs goes too.
Failures now anchor precisely: each failing case becomes a `tables::Error` scoped
to the failing *step's* source URL and JSON pointer, where before a whole run
collapsed into one error at `flow://publication/test/api/test` with "View logs
for details". A case the run never reached is reported too, so a publication
never silently understates its test coverage. Logs still stream under the "test"
stream name existing consumers select on, now gated by each task's own
`shards: {logLevel}`.
`flowctl_go` is no longer threaded anywhere: dropped from `Publisher` and
`Publisher::new`, from the agent's `main`, from its integration-test harness, and
from `test_server`. The `locate-bin` crate stays — sops and flow-connector-init
still use it.
Tests: a passing and a failing derive-sqlite catalog, asserting the failing one
surfaces exactly one publication error naming the test and rendering the diff.
They need no database, since `test_catalog` never touches Postgres.
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
from
September 1, 2026 21:43
f53a04b to
875f483
Compare
bbartman
reviewed
Sep 2, 2026
bbartman
left a comment
bbartman
left a comment
Contributor
There was a problem hiding this comment.
I believe this looks OK. I don't see any problems with it.
bbartman
approved these changes
Sep 2, 2026
bbartman
left a comment
bbartman
left a comment
Contributor
There was a problem hiding this comment.
Sorry I missed a button.
github-actions
Bot
added
pending:flowctl
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
Stack 4 of 5, one commit. Publication tests now call
catalog_tests::run_testsin-process, against thebuild::Outputthe publication already produced.Workflow steps:
No change to how a user runs tests — they still publish a draft and see test results in publication logs. What changes is where those results come from and how precisely failures are reported.
Documentation links affected:
None.
Notes for reviewers:
Most of the surface is deletion. One behavior changes:
tables::Errorscoped to the failing step's source URL and JSON pointer, where before a whole run collapsed into one error atflow://publication/test/api/testwith "View logs for details". A case the run never reached is reported as an error too, so a publication never silently understates its test coverage.Stack — each PR is based on the one above it, so review only the top commits of each.
flowctl raw test#3417 — catalog-tests: the harness, andflowctl raw test