Skip to content

Fix #542: add a Citrus integration test for couchbase-source - #3008

Merged
oscerd merged 1 commit into
apache:mainfrom
oscerd:ci-issue-542-test
Sep 7, 2026
Merged

Fix #542: add a Citrus integration test for couchbase-source#3008
oscerd merged 1 commit into
apache:mainfrom
oscerd:ci-issue-542-test

Conversation

@oscerd

@oscerd oscerd commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #3007, which shipped couchbase-source without a test on the grounds that the project's Citrus toolchain had no Couchbase container. That was true only in the sense that there is no built-in one — the generic container works fine, it just needs setting up by hand.

The test

testcontainers: start: container (couchbase/server:community-7.6.2)
  -> initCluster.groovy   (cluster init, bucket, primary index, seed one document)
  -> camel: cli: run      (couchbase-source -> log-sink)
  -> camel: cli: verify   (logMessage: "hello-from-couchbase")

Same shape as the existing mongodb-source-route test, so it should read familiarly.

Three things that were not optional

Recording these because each cost a full run to find, and the next person adding a container-based test will hit at least one.

Ports are bound one-to-one. The Couchbase SDK bootstraps from the management port and then reconnects using the addresses the cluster advertises. With a remapped host port the client bootstraps and then cannot reach the data service. Hence explicit 8091:8091, 8093:8093, 11210:11210 rather than exposedPorts.

The wait strategy needs a full-line regex. Testcontainers matches log wait patterns against the entire line, so "Starting Couchbase Server" never matches the actual line and the container times out after five minutes. It has to be ".*Starting Couchbase Server.*".

No Groovy GStrings in the script. Citrus resolves dollar-brace expressions in these scripts as its own test variables, so a GString fails with Unknown variable 'user'. The script uses string concatenation throughout — including, eventually, in the comment that explains why.

Why the statement looks the way it does

couchbase.statement=SELECT META().id AS __id, * FROM _default

Not the obvious SELECT * FROM <bucket>, because both halves matter:

  • CouchbaseConsumer reads row.getString("__id") and skips any row without it, logging a warning rather than failing. A query without META().id AS __id yields a source that polls happily and emits nothing.
  • The consumer runs the query through scope.query(...), so the keyspace is resolved as <bucket>._default.<what-you-wrote>. FROM <bucket> becomes <bucket>._default.<bucket> and fails with Keyspace not found.

I found both by running the Kamelet against a live cluster, and they mean the example currently shipped in couchbase-source is wrongSELECT * FROM \travel-sample` LIMIT 10` would silently return nothing. That is my error from #3007. I am fixing it in a separate PR rather than folding a Kamelet change into a test PR.

Verification

Passes locally, twice through (test and verify phases), about 35 seconds each:

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 -- in CouchbaseIT
  ✔ SUCCESS (38088ms) couchbase-source-route-test

mvn clean install passes with tests from the repository root.

For reviewers

The image is largecouchbase/server:community-7.6.2 is about 1.7 GB, materially bigger than redpanda or floci. On a cold CI cache that is a real cost for one test, and worth weighing against the value of covering this Kamelet.

I did not add camel.apache.org/kamelet.verified: "true". The contributor guide ties that label to having passing behaviour tests, but it is applied to only 21 Kamelets and notably not to mongodb-source, which does have a test — so it looks like a curation decision rather than an automatic consequence. Happy to add it if you want it applied here.


Claude Code on behalf of Andrea Cosentino

Follow-up to apache#3007, which shipped the Kamelet without a test because there
was no Couchbase container in the toolchain. There is one -- it just needs
setting up by hand.

Citrus 5.0.0 has no built-in Couchbase container type, so this uses the
generic container with couchbase/server:community-7.6.2. Three details were
not optional:

- Ports are bound one-to-one. The Couchbase SDK bootstraps from the
  management port and then reconnects using the addresses the cluster
  advertises, so a remapped host port leaves the client unable to reach the
  data service.
- The wait strategy needs a full-line regex. Testcontainers matches log
  wait patterns against the whole line, so "Starting Couchbase Server" never
  matches and the container times out after five minutes.
- initCluster.groovy avoids Groovy GStrings entirely. Citrus resolves
  dollar-brace expressions in these scripts as its own test variables, so a
  GString fails with "Unknown variable".

The script initialises the cluster, creates a bucket, adds a primary index
and seeds one document over plain HTTP, so the test module needs no
Couchbase SDK on its classpath. It polls for the management port itself
because the container logs its startup line well before that port serves
requests.

The statement in application.properties is
"SELECT META().id AS __id, * FROM _default" rather than the more obvious
"SELECT * FROM <bucket>". Both parts matter: CouchbaseConsumer skips any
row without the __id alias, and it runs the query in the bucket scope, so
the keyspace is the collection.

Passes locally in about 35 seconds:

  Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 -- in CouchbaseIT
  SUCCESS (38088ms) couchbase-source-route-test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tz352mt7yzGWLfvYTBo2Dk
@oscerd
oscerd merged commit 838ad91 into apache:main Sep 7, 2026
5 checks passed
oscerd added a commit that referenced this pull request Sep 7, 2026
The example shipped in #3007 -- SELECT * FROM `travel-sample` LIMIT 10 --
does not work, and fails in the worst way: the source polls happily and
emits nothing.

Two requirements, both found by running the Kamelet against a live cluster
while writing the integration test in #3008:

CouchbaseConsumer reads row.getString("__id") and skips any row without
that field, logging a warning rather than failing. A query that does not
alias the document id as __id therefore produces no exchanges at all.

The consumer runs the query through scope.query(...), so the keyspace
resolves as <bucket>._default.<what-you-wrote>. Naming the bucket gives
<bucket>._default.<bucket> and fails with "Keyspace not found"; the
collection, normally _default, is what belongs there.

The example now matches what the integration test actually uses, and the
description states both requirements so the next person does not have to
rediscover them from a silent no-op.


Claude-Session: https://claude.ai/code/session_01Tz352mt7yzGWLfvYTBo2Dk

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@oscerd

oscerd commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

CI is green, and the new test genuinely ran rather than being skipped:

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.15 s -- in CouchbaseIT

Adding the cost figure to go with the image-size caveat above, since it is the number that matters for the decision: 61 seconds on the CI runner, against 35 locally — the difference being the image pull on a cold cache.

For context, that puts it mid-pack rather than at the top:

CommonIT     398.5 s
AwsIT        408.3 s
AvroIT        86.99 s
KafkaIT       80.15 s
ProtobufIT    79.20 s
JiraIT        69.43 s
CouchbaseIT   61.15 s
MongoIT       50.78 s

So the 1.7 GB image costs about a minute of a suite that already runs for well over twenty. Still your call whether one Kamelet is worth that, but it is a smaller marginal cost than the image size alone suggests.


Claude Code on behalf of Andrea Cosentino

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant