MySQL and SQL Server adapters, outbound delivery to external brokers, and a per-engine guide - #300
Conversation
…really checked Each is what the plan said it would be — a driver, a connection string, a catalog query and a capability list over the existing core — and the core grew two hooks to fit them. MySQL, which serves MariaDB. A database IS the schema, so discovery with no schema means the one this connection opened rather than every database on a shared server. No sequences, no MERGE, no RETURNING, no array type; a procedure returns rows by SELECTing, which is the capability PostgreSQL declares false. SQL Server, which serves Azure SQL. The default schema belongs to the LOGIN rather than the connection, so a data source that names one applies it per connection — through the new OnConnectionOpenedAsync hook, and not fatally, because impersonating a schema's owner is a privilege many service accounts do not have. MERGE, OUTPUT and snapshot isolation are all real here. The second hook is the one that mattered. The shared statement check prepares the SQL, and two engines cannot be asked that way: - SqlCommand.Prepare refuses unless every parameter has an explicit type, which a caller checking somebody else's SQL does not know. It asks sp_describe_undeclared_parameters instead. - ODP.NET's Prepare is a client-side NO-OP — Oracle compiles a statement when it is executed, not when it is prepared. So Oracle has been reporting every statement as valid without looking at one, including a select from a table that does not exist, and its connection test's "prepares every statement" was vacuous. It uses DBMS_SQL.PARSE now, which compiles and resolves names while running nothing, and the PL/SQL frames that raises are stripped so the answer is the one ORA- line about the operator's SQL. Two more things the engines made visible: - The schema browser wrote @name and `limit 100` whatever the connection was. That is right for two engines and wrong for the other two — Oracle binds :name and takes `fetch first`, SQL Server puts `top` before the column list. The adapter reports its own prefix and limit style now, so a draft cannot use a syntax the connection will refuse. It also drafted "call proc(@A)" for a procedure, which made a statement whose procedure NAME was that whole string. - A statement saved while its adapter was not running was never checked, and the save said nothing — indistinguishable from one that had been verified. Create and Update now answer with whether the database actually looked, and both forms say "Saved, but not checked" when it did not. 472 unit, 431 integration (35 new, across both engines) and 313 client tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…base The settings tables said what each field is. They did not say which four you have to fill in, which defaults to leave alone, how the engine writes a placeholder or a row limit, what its types arrive as, whether a procedure can return rows, or what its error messages mean. That is the whole of what someone needs on the day they point Bitween at a database they have not met. One section per engine, each answering the same five questions, plus the parts that are the same everywhere, how to choose a receive mode, and a minimal grant per engine. The reference links to it and it links back. Writing it turned up two things that were not true: - PostgreSQL claimed MERGE whatever the server was. It arrived in 15, and 13 and 14 are both still widely run — so a capability list that said yes was a statement somebody writes and cannot parse. Capabilities can now be narrowed once the version is known, which is the only point at which the difference can be told, and the rule is unit-tested rather than needing a container per release. - A PostgreSQL table that has never been ANALYZEd reports reltuples as -1, and that was clamped to 0 — so the browser said "no rows" about a table that might hold millions. Null means unknown, and null is what it says now. And one thing the doc got wrong before the code was checked: Oracle reports every NUMBER as decimal, including NUMBER(10,0). The hint is coarse on purpose. 475 unit, 431 integration, 313 client tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Publish had existed in both bus adapters since they did, and nothing could reach
it: Bitween's pipeline calls Handle on a handler, and neither class had one. So
an integration could drain a customer's queue and had no way to answer on it.
Both adapters now implement IInfolinkHandler and declare [AdapterKind("handler")],
so a subscription's delivery stage can target a queue.
Adding Handle was the smaller half. A broker data source is EXCLUSIVE — one node
holds the connection so the queue is drained once — while a delivery runs on
whichever node picked the message up. Publishing would therefore have failed on
every node but the owner, which is to say almost always.
Exclusivity is about consuming, not about connecting. When the owned instance is
not on this node, the runtime opens a send-only connection instead: built from
the data source's own settings, Consume=false, no Endpoints, its own pool key so
it can never be confused with the consuming one. It sends and never subscribes,
so nothing is processed twice. That is the same Consume=false the connection test
already uses, for the same reason.
The rented path also had to carry the call's properties. Startup values are the
connection's; where to publish is the subscription's, and passing only the spec
sent the message to an endpoint the connection had never heard of.
Two bugs this turned up:
- WriteBackHealthAsync keyed a dictionary by InstanceKey, which is unique for an
exclusive instance and not for a POOLED one. Publish-only instances made that
routine, and a duplicate key took the whole reconcile pass down. It now looks
only at instances that ARE a data source.
- Subscriptions/Get stopped projecting DataSourceId — dropped when the r10 merge
took their version of the file, which predates the field. Nothing failed: the
API answered null while the row held an id, so the UI read every bound
subscription as unbound and saving one from that screen wrote the null back.
A test now asserts the read, because a projection is only as good as the thing
that notices a field missing from it.
UI: a broker is offered in a delivery (never in a receiver — ingress comes
through a bus gateway), with Publish to / Exchange / Routing key, and a warning
when the endpoint is empty. The connection is subscription-wide, so choosing a
broker where another stage needs a database now says so rather than saving
something that fails on its first message. The SQL hint no longer names two
engines out of four — it reports the prefix the adapter itself declares.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
r10 moved by one commit: the merge of PR #299, which is this branch's own earlier work. So the merge changes no file — `git diff --cached` is empty and the tree is identical to the branch head. What it records is ancestry, so the next pull request carries only the three commits since: the MySQL and SQL Server adapters, the per-engine guide, and outbound delivery to a customer's broker. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 37147666 | Triggered | Generic Password | 2e3d40b | SW.Bitween.IntegrationTests/Fixtures/MySqlDbFixture.cs | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Repository: simplify9/coderabbit/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Follows #299. Three things: the two remaining database engines, outbound delivery to a customer's broker, and a guide for a developer meeting one of these databases for the first time.
Along the way it fixes four bugs that were already in
releases/r10.0, two of them silent.MySQL and SQL Server adapters
Each is what the plan predicted — a driver, a connection string, a catalog query and a capability list over the existing core — and the core grew two hooks to fit them.
MySQL (serves MariaDB). A database is the schema, so discovery with no schema means the one this connection opened rather than every database on a shared server. No sequences, no
MERGE, noRETURNING, no array type; a procedure returns rows bySELECTing, which is the capability PostgreSQL declares false.SQL Server (serves Azure SQL). The default schema belongs to the login, not the connection, so a named schema is applied per connection through a new
OnConnectionOpenedAsynchook — and not fatally, because impersonating a schema's owner is a privilege many service accounts lack.MERGE,OUTPUTand snapshot isolation are all real here.Outbound: a delivery publishes to the customer's broker
Publishhad existed in both bus adapters since they did, and nothing could reach it — the pipeline callsHandleon a handler and neither class had one. An integration could drain a customer's queue and had no way to answer on it. Both adapters now implementIInfolinkHandlerand declare[AdapterKind("handler")].Adding
Handlewas the smaller half. A broker data source is exclusive — one node holds the connection so the queue is drained once — while a delivery runs on whichever node picked the message up. Publishing would have failed on every node but the owner.Exclusivity is about consuming, not connecting. When the owned instance is not on this node, the runtime opens a send-only connection instead: the data source's own settings,
Consume=false, noEndpoints, its own pool key. It sends and never subscribes, so nothing is processed twice. SameConsume=falsethe connection test already uses, for the same reason.Integration-tested on both brokers, including a test that publishes from a host holding no instance for the data source and asserts ownership was not taken on the way past.
Four bugs this found in r10
Prepare()is a client-side no-op — Oracle compiles at execute time — so the shared statement check passed everything, includingselect 1 from no_such_table. Its connection test's "prepares every statement" was vacuous. Now usesDBMS_SQL.PARSE, with the PL/SQL frames stripped so the answer is the oneORA-line about the operator's SQL.Subscriptions/Getstopped projectingDataSourceId— dropped when an earlier r10 merge took their version of the file, which predates the field. Nothing failed: the API answerednullwhile the row held an id, so the UI read every bound subscription as unbound and saving one from that screen wrote the null back. A test now asserts the read.WriteBackHealthAsynckeyed a dictionary byInstanceKey, unique for an exclusive instance and not for a pooled one. Publish-only instances made that routine, and a duplicate key took the whole reconcile pass down.MERGEon every version. It arrived in 15; 13 and 14 are still widely run. Capabilities can now be narrowed once the server version is known — unit-tested rather than needing a container per release.Plus: a never-
ANALYZEd PostgreSQL table reported 0 rows (reltuplesis-1there, clamped to zero), so the browser said "no rows" about a table that might hold millions. It says unknown now.UI
@nameandlimit 100. Oracle binds:nameand takesfetch first; SQL Server putstopbefore the columns. Each adapter reports its own dialect now. It also draftedcall proc(@a)for a procedure, which made a statement whose procedure name was that entire string.Docs
docs/database-adapters-per-engine.md— for a developer who knows SQL but not this database. One section per engine answering the same five questions: what you must set, how to write a statement, what the types arrive as, how to call a procedure, and what the errors mean. Plus receive-mode selection and a minimal grant per engine.external-bus-providers.mdgains an Egress section: the properties, why it needed more than a method, and the one thing it cannot do.database-adapters-api.mddocuments both new engines and both bespoke statement checks.dev-warehouse-{mysql,sqlserver,oracle}.sql), same schema and rows as the PostgreSQL one so the differences you meet are the real ones.Verified
475 unit · 431 integration (53 new) · 313 client.
End to end on a live two-node stack: ownership split correct (brokers on one node, relational pools on every node), no duplication with both live, failover with the fencing term incrementing (RabbitMQ 29→30, SQS 1→2), and messages flowing after takeover. All four databases polled through the real pipeline with cursors scoped per subscription.
Notes for review
dataSourceId, applied to every slot — so one subscription cannot read from a database and publish to a broker. Chain two withResponseSubscriptionId. The UI now warns rather than letting it save.--log-bin-trust-function-creators=1: creating a function requires SUPER while binary logging is on.🤖 Generated with Claude Code