feat(playground): run ExtendDB on DocumentDB - #454
Draft
xgerman wants to merge 3 commits into
Draft
Conversation
Adds documentdb-playground/extenddb/, a playground demonstrating ExtendDB (open-source DynamoDB wire-protocol server) configured with its MongoDB storage backend pointed at a DocumentDB cluster, giving a DynamoDB-compatible API surface backed by DocumentDB. Includes: - Dockerfile building ExtendDB from source with the mongodb feature (no official Mongo-backend image is published upstream) - Kubernetes manifests: namespace/PVC, one-shot init Job, serve Deployment + Service - scripts/ for build, deploy, connectivity smoke test, and cleanup - demo/demo.py: small boto3 walkthrough of the classic DynamoDB Movies table - README covering architecture, quick start, config, the ddbsh interactive shell, and troubleshooting (notably the directConnection/replicaSet=rs0 interaction with DocumentDB's gateway) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 73ace353-38c7-44b2-8a9f-0ee833727c53 Signed-off-by: German Eichberger <geeichbe@microsoft.com>
…ve testing Fixes found by deploying and testing this playground end-to-end against a real DocumentDB instance (kind cluster) with the actual ExtendDB binary, following up on a rubber-duck review of the original implementation: - deploy.sh: strip replicaSet=rs0 from the connection string (matching lightrag/keda-autoscaling). A prior mongosh-based test suggested keeping it was safe with directConnection=true, but the real ExtendDB Rust mongodb v3 driver strictly validates the requested replica set name and rejects the gateway's response even in direct-connection mode. - deploy.sh: append tlsAllowInvalidCertificates=true -- the Rust driver's default certificate verifier rejects DocumentDB's self-signed gateway cert with UnknownIssuer. - deploy.sh: restrict envsubst substitution to explicit variable lists so it stops blanking out unrelated variable references in the manifests. - deploy.sh: capture full init Job logs with a fallback so admin credentials (printed once) are never silently lost by a failed keyword grep. - manifests/init-job.yaml: extenddb init --backend mongodb reads the Mongo connection string from the --config file itself, not from the connection-string env var override (that override only applies at serve time) -- write a seed TOML first, then run init --overwrite. Use bash instead of sh (dash doesn't support set -o pipefail). - manifests/init-job.yaml, manifests/serve.yaml: set HOME to the PVC state dir so the self-signed TLS cert/key extenddb init generates under the home-relative .extenddb/tls/ path land on the PVC-backed state dir instead of the init Job's ephemeral container home dir, so the separate serve Deployment pod can find them. - manifests/init-job.yaml, manifests/serve.yaml: add securityContext (runAsUser/runAsGroup/fsGroup 10001) so the non-root container can reliably write to the PVC regardless of the CSI driver's default mount ownership. - manifests/init-job.yaml: add ttlSecondsAfterFinished to release the shared ReadWriteOnce PVC promptly. - manifests/serve.yaml: document the readiness/liveness probe caveat -- extenddb healthcheck doesn't verify backend connectivity. - Dockerfile: install libssl3 in the runtime stage defensively. - demo.py: use Decimal instead of float literals -- boto3's DynamoDB resource rejects native Python floats. - README: document a confirmed, currently-blocking upstream compatibility gap -- ExtendDB's MongoDB backend unconditionally uses MongoDB's snapshot read concern for CreateTable/PutItem/etc., which this operator's DocumentDB gateway does not support (Error code 115: CommandNotSupported). CreateTable succeeds; PutItem and most other data-plane operations currently fail with InternalServerError. Also documents the extenddb manage create-account/create-user/ put-user-policy/create-access-key workflow needed to get a SigV4 access key (the init admin credentials are management-API only, not a SigV4 key), and updates the replicaSet/directConnection troubleshooting writeup with the corrected, doubly-verified explanation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 73ace353-38c7-44b2-8a9f-0ee833727c53 Signed-off-by: German Eichberger <geeichbe@microsoft.com>
Configure majority transaction reads, add local-source image builds, and expand the playground smoke test to cover CRUD and transactional operations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c9b74922-c628-425e-bf1b-484b223c52b5 Signed-off-by: German Eichberger <geeichbe@microsoft.com>
documentdb-triage-tool
Bot
added
documentation
|
🤖 Auto-triaged by documentdb-triage-tool. Applied: Reasoningcomponent from path globs (playground, docs); effort from diff stats (1235+0 LOC, 12 files); LLM: Adds a new ExtendDB-backed playground with multi-file changes spanning build scripts, Kubernetes manifests, IAM setup, smoke tests, and read-concern configuration. If a label is wrong, remove it manually and ping |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
majority, since DocumentDB does not supportsnapshotread concernTransactWriteItems, andTransactGetItemsin the smoke testValidation
CommandNotSupported/ read-concern error 115 failuresbash -npassed for all playground scriptsRelated