Fix #542: add a Citrus integration test for couchbase-source - #3008
Merged
Conversation
Follow-up to apache#3007, which shipped the Kamelet without a test because there was no Couchbase container in the toolchain. There is one -- it just needs setting up by hand. Citrus 5.0.0 has no built-in Couchbase container type, so this uses the generic container with couchbase/server:community-7.6.2. Three details were not optional: - Ports are bound one-to-one. The Couchbase SDK bootstraps from the management port and then reconnects using the addresses the cluster advertises, so a remapped host port leaves the client unable to reach the data service. - The wait strategy needs a full-line regex. Testcontainers matches log wait patterns against the whole line, so "Starting Couchbase Server" never matches and the container times out after five minutes. - initCluster.groovy avoids Groovy GStrings entirely. Citrus resolves dollar-brace expressions in these scripts as its own test variables, so a GString fails with "Unknown variable". The script initialises the cluster, creates a bucket, adds a primary index and seeds one document over plain HTTP, so the test module needs no Couchbase SDK on its classpath. It polls for the management port itself because the container logs its startup line well before that port serves requests. The statement in application.properties is "SELECT META().id AS __id, * FROM _default" rather than the more obvious "SELECT * FROM <bucket>". Both parts matter: CouchbaseConsumer skips any row without the __id alias, and it runs the query in the bucket scope, so the keyspace is the collection. Passes locally in about 35 seconds: Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 -- in CouchbaseIT SUCCESS (38088ms) couchbase-source-route-test Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tz352mt7yzGWLfvYTBo2Dk
oscerd
added a commit
that referenced
this pull request
Sep 7, 2026
The example shipped in #3007 -- SELECT * FROM `travel-sample` LIMIT 10 -- does not work, and fails in the worst way: the source polls happily and emits nothing. Two requirements, both found by running the Kamelet against a live cluster while writing the integration test in #3008: CouchbaseConsumer reads row.getString("__id") and skips any row without that field, logging a warning rather than failing. A query that does not alias the document id as __id therefore produces no exchanges at all. The consumer runs the query through scope.query(...), so the keyspace resolves as <bucket>._default.<what-you-wrote>. Naming the bucket gives <bucket>._default.<bucket> and fails with "Keyspace not found"; the collection, normally _default, is what belongs there. The example now matches what the integration test actually uses, and the description states both requirements so the next person does not have to rediscover them from a silent no-op. Claude-Session: https://claude.ai/code/session_01Tz352mt7yzGWLfvYTBo2Dk Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Author
|
CI is green, and the new test genuinely ran rather than being skipped: Adding the cost figure to go with the image-size caveat above, since it is the number that matters for the decision: 61 seconds on the CI runner, against 35 locally — the difference being the image pull on a cold cache. For context, that puts it mid-pack rather than at the top: So the 1.7 GB image costs about a minute of a suite that already runs for well over twenty. Still your call whether one Kamelet is worth that, but it is a smaller marginal cost than the image size alone suggests. Claude Code on behalf of Andrea Cosentino |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #3007, which shipped
couchbase-sourcewithout a test on the grounds that the project's Citrus toolchain had no Couchbase container. That was true only in the sense that there is no built-in one — the generic container works fine, it just needs setting up by hand.The test
Same shape as the existing
mongodb-source-routetest, so it should read familiarly.Three things that were not optional
Recording these because each cost a full run to find, and the next person adding a container-based test will hit at least one.
Ports are bound one-to-one. The Couchbase SDK bootstraps from the management port and then reconnects using the addresses the cluster advertises. With a remapped host port the client bootstraps and then cannot reach the data service. Hence explicit
8091:8091,8093:8093,11210:11210rather thanexposedPorts.The wait strategy needs a full-line regex. Testcontainers matches log wait patterns against the entire line, so
"Starting Couchbase Server"never matches the actual line and the container times out after five minutes. It has to be".*Starting Couchbase Server.*".No Groovy GStrings in the script. Citrus resolves dollar-brace expressions in these scripts as its own test variables, so a GString fails with
Unknown variable 'user'. The script uses string concatenation throughout — including, eventually, in the comment that explains why.Why the statement looks the way it does
couchbase.statement=SELECT META().id AS __id, * FROM _defaultNot the obvious
SELECT * FROM <bucket>, because both halves matter:CouchbaseConsumerreadsrow.getString("__id")and skips any row without it, logging a warning rather than failing. A query withoutMETA().id AS __idyields a source that polls happily and emits nothing.scope.query(...), so the keyspace is resolved as<bucket>._default.<what-you-wrote>.FROM <bucket>becomes<bucket>._default.<bucket>and fails withKeyspace not found.I found both by running the Kamelet against a live cluster, and they mean the example currently shipped in
couchbase-sourceis wrong —SELECT * FROM \travel-sample` LIMIT 10` would silently return nothing. That is my error from #3007. I am fixing it in a separate PR rather than folding a Kamelet change into a test PR.Verification
Passes locally, twice through (test and verify phases), about 35 seconds each:
mvn clean installpasses with tests from the repository root.For reviewers
The image is large —
couchbase/server:community-7.6.2is about 1.7 GB, materially bigger than redpanda or floci. On a cold CI cache that is a real cost for one test, and worth weighing against the value of covering this Kamelet.I did not add
camel.apache.org/kamelet.verified: "true". The contributor guide ties that label to having passing behaviour tests, but it is applied to only 21 Kamelets and notably not tomongodb-source, which does have a test — so it looks like a curation decision rather than an automatic consequence. Happy to add it if you want it applied here.Claude Code on behalf of Andrea Cosentino