Add authenticated database readiness check for Astra and Cassandra by erichare · Pull Request #2527 · stargate/data-api

erichare · 2026-07-27T15:30:01Z

What this PR does:

Adds an authenticated, database-backed readiness endpoint shared by Astra and Cassandra deployments.

Registers GET /v1/health/ready through Quarkus JAX-RS discovery; the existing /v1/* security policy requires authentication.
Uses the request’s tenant, token, and User-Agent to obtain a session through the normal CQLSessionCache. No separate readiness credentials are stored in application configuration.
Executes SELECT * FROM datastax_sla.check LIMIT 1 asynchronously at explicit LOCAL_QUORUM, using the table-read profile for the remaining read settings.
Uses Mutiny throughout session acquisition and query execution, with a five-second reactive timeout and no blocking await() or synchronous driver calls.
Returns HTTP 200 with {"status":"UP"} after a successful read, HTTP 401 for missing or invalid authentication, and HTTP 503 with {"status":"DOWN"} for database failures or timeouts.
Does not expose tokens, tenants, session details, or underlying exception messages in responses.
Implements the database check as a plain Java class rather than a CDI/MicroProfile health bean. CDI remains only at the existing JAX-RS request boundary.
Builds the immutable readiness statement once during check construction.
Removes explicit session-state inspection and manual eviction. The request User-Agent is passed through so a dedicated checker using the configured SLA User-Agent receives the shorter session-cache TTL.
Excludes readiness polling from normal tenant-request and collection HTTP metrics.
Updates the session outage integration test to provision datastax_sla.check and verify UP → DOWN → UP behavior using the authenticated endpoint.
Documents the deployment assumptions: a dedicated canary tenant and credentials, an appropriately replicated datastax_sla.check table, the Astra database hostname in Host, the exact configured SLA User-Agent, and a trusted per-pod caller.
Leaves the unauthenticated Quarkus health endpoints as process-only checks. Wiring the authenticated database check into Kubernetes or an SLA checker, including Secret delivery, remains deployment-specific.

Which issue(s) this PR fixes:

Fixes #2526

Validation:

./mvnw -Dtest=DatabaseReadinessCheckTest,TenantRequestMetricsFilterTest test — 6 tests passed.
./mvnw -q -Dtest=DatabaseReadinessResourceTest test — 5 tests passed.
Formatting, compilation, test compilation, and Quarkus augmentation succeeded during the focused runs.
SessionEvictionIntegrationTest was updated and compiled, but its Docker-backed run could not start locally because this host had no Docker socket. CI must validate the container outage/recovery path.

Checklist

Changes manually tested
Automated Tests added/updated
Documentation added/updated
CLA Signed: DataStax CLA

github-actions · 2026-07-27T15:33:20Z

Unit Test Coverage Report

Overall Project	53.34% `-0.08%`	🍏
Files changed	75.36%	🍏

File	Coverage
DatabaseReadinessResource.java	100%	🍏
DatabaseReadinessCheck.java	100%	🍏
CqlSessionCacheSupplier.java	96.12%	🍏
TenantRequestMetricsFilter.java	27.15% `-56.29%`	❌

github-actions · 2026-07-27T15:52:46Z

Integration Test Coverage Report (dse69-it)

Overall Project	71.47% `-0.14%`	🍏
Files changed	56.23%	❌

File	Coverage
CqlSessionCacheSupplier.java	99.06% `-0.94%`	🍏
DatabaseReadinessCheck.java	89.61% `-10.39%`	🍏
DatabaseReadinessResource.java	72.46% `-27.54%`	🍏
TenantRequestMetricsFilter.java	16.23% `-67.53%`	❌

github-actions · 2026-07-27T15:56:01Z

Integration Test Coverage Report (hcd-it)

Overall Project	72.8% `-0.14%`	🍏
Files changed	56.23%	❌

File	Coverage
CqlSessionCacheSupplier.java	99.06% `-0.94%`	🍏
DatabaseReadinessCheck.java	89.61% `-10.39%`	🍏
DatabaseReadinessResource.java	72.46% `-27.54%`	🍏
TenantRequestMetricsFilter.java	16.23% `-67.53%`	❌

amorton

-1 this only works in non-astra, uses blocking IO, and reads from a non replicated table so why the C* node will need to be marked as UP to respond it will not verify that its able to communicate with enough nodes to achieve quourm

need to rethink what we are trying to do here

amorton · 2026-08-04T01:07:09Z

+import org.slf4j.LoggerFactory;
+
+/**
+ * Health check that verifies Cassandra connectivity for the Data API readiness probe.


-1 comments should explain how this is registered, what url it is under etc.

the AI generated comments dont help

Cleaned it up and talked about how its registered

amorton · 2026-08-04T01:07:56Z

+ * types. In those deployments it reports UP without accessing the Cassandra session cache.
+ */
+@Readiness
+@ApplicationScoped


we are trying to remove CDI injection, is there a way to do this without it ?

I did some reading, correct me if i'm wrong about any of this... but Quarkus’s standard application health-check mechanism is CDI-based. There is an alternative tho, SmallRye’s programmatic HealthRegistry. https://quarkus.io/guides/smallrye-health https://smallrye.io/docs/smallrye-health/3.0.0/health-registry.html. giving it a try

amorton · 2026-08-04T01:09:07Z

+
+  @VisibleForTesting
+  CassandraConnectionHealthCheck(
+      CQLSessionCache sessionCache, OperationsConfig operationsConfig, Duration timeout) {


prefer to abstract out the OperationsConfig properties if we can, again trying to remove the CDI things

yep, good call. updated

amorton · 2026-08-04T01:10:52Z

+  public HealthCheckResponse call() {
+    var responseBuilder = HealthCheckResponse.named(HEALTH_CHECK_NAME);
+
+    if (operationsConfig.databaseConfig().type() != DatabaseType.CASSANDRA) {


this means it will not run for astra, is this correct ? Confusing that we are adding this but not using it

this was my mistake, i was misunderstanding the intent of the original issue. This now does get more complicated a bit though.... ahh, i see you already responded in the original GH issue as to why it gets more complicated.

I have a plan and ill update and explain it...

amorton · 2026-08-04T01:13:55Z

+          sessionCache
+              .getSession(healthCheckTenant, authToken, HEALTH_CHECK_USER_AGENT)
+              .await()
+              .atMost(timeout);


await() is blocking for the caller thread, we generally want to avoid making blocking calls. This should be using the UNI framework.

amorton · 2026-08-04T01:16:46Z

+              .setConsistencyLevel(operationsConfig.queriesConfig().consistency().reads())
+              .build();
+
+      var resultSet = session.execute(statement);


-1 for blocking IO call, we need to use async calls in the Uni framework

amorton · 2026-08-04T01:20:31Z

+
+  private static String createAuthToken(OperationsConfig.DatabaseConfig databaseConfig) {
+    return databaseConfig
+        .fixedToken()


-1 - the code comments for fixedToken explain what this is for it's not something we want to include in actual prod code its for testing

amorton · 2026-08-04T01:22:09Z

+    return false;
+  }
+
+  private void evictSession(Tenant tenant) {


-1 , the sessionCache evicts sessions when they TTL. It is also designed to evist sessions faster when they use the SLA checker user agent. Because this is not using the sla user agent they will be treated like regular user sessions and last for 10mins (by default I think).

Better to use the SLA checker user agent and not do this

Explained the new approach as a separate comment

amorton · 2026-08-04T01:22:57Z

    /**
-     * Username when connecting to cassandra database (when type is {@link DatabaseType#CASSANDRA})
-     * and fixedToken is used
+     * Username used for Cassandra connections when fixedToken is configured, and by the Cassandra


-1 - do not want to overload this

amorton · 2026-08-04T01:23:41Z


  private static final Logger LOGGER =
      LoggerFactory.getLogger(SessionEvictionIntegrationTest.class);
+  private static final String READINESS_PATH = "/stargate/health/ready";


-1 for anything using stargate in a path name

haha fair enough!

amorton · 2026-08-04T01:31:58Z

this needs a rethink about what we are trying to do

erichare · 2026-08-04T01:37:57Z

this needs a rethink about what we are trying to do

understood! i'll address the specific comments you made just for the practice, but yeah understood about rethinking the purpose

erichare · 2026-08-04T03:07:10Z

Okay, @amorton , so here's what i did...

First of all, renamed the endpoint to GET /v1/health/ready, no stargate in the path lol. Its an authenticated database readiness endpoint used for both Astra and Cassandra deployments. It gets a session through the normal session cache.

The endpoint executes SELECT * FROM datastax_sla.check LIMIT 1 with the table-read driver
profile. That profile uses LOCAL_QUORUM, so an UP response confirms that the coordinator can
complete a read from a replicated table at local quorum. It does not validate every tenant's
credentials, write availability, or cross-region availability.

Switched everything to async, response would look like {"status":"UP"}... and the readiness probe is pure java, no CDI.

I'll push shortly

Add Cassandra readiness health check

ad1a47b

erichare added 2 commits July 27, 2026 09:54

Address Cassandra readiness review feedback

143a006

Reuse shared Cassandra test fixtures

55a232a

erichare marked this pull request as ready for review July 27, 2026 17:19

erichare requested a review from a team as a code owner July 27, 2026 17:19

erichare requested review from amorton and clun July 27, 2026 17:19

amorton requested changes Aug 4, 2026

View reviewed changes

chore: refactor to support astra readiness

861d862

erichare requested a review from amorton August 4, 2026 03:23

erichare changed the title ~~Add Cassandra-based readiness probe~~ Add authenticated database readiness check for Astra and Cassandra Aug 4, 2026

fix: address review comments

3e7ff74

Uh oh!

Conversation

erichare commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Test Coverage Report

Uh oh!

github-actions Bot commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration Test Coverage Report (dse69-it)

Uh oh!

github-actions Bot commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration Test Coverage Report (hcd-it)

Uh oh!

amorton left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amorton commented Aug 4, 2026

Uh oh!

erichare commented Aug 4, 2026

Uh oh!

erichare commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

erichare commented Jul 27, 2026 •

edited

Loading

github-actions Bot commented Jul 27, 2026 •

edited

Loading

github-actions Bot commented Jul 27, 2026 •

edited

Loading

github-actions Bot commented Jul 27, 2026 •

edited

Loading