Skip to content

TCP N10: lift composite children's element types on read and write - #564

Draft
alex-clickhouse wants to merge 9 commits into
tcp/epic-b9-tlsfrom
tcp/epic-n10-composite-lift
Draft

TCP N10: lift composite children's element types on read and write#564
alex-clickhouse wants to merge 9 commits into
tcp/epic-b9-tlsfrom
tcp/epic-n10-composite-lift

Conversation

@alex-clickhouse

@alex-clickhouse alex-clickhouse commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #563 (tcp/epic-b9-tls), the current tip of the TCP stack.

Summary

This PR lets TCP composite codecs expose and accept the CLR types supported by their children. The composition works recursively through Array, Map, Tuple, Nullable, and LowCardinality without enumerating every possible combination.

Examples:

  • Array(DateTime('UTC')) can read and write DateTime[] instead of requiring raw uint[] epoch seconds.
  • Map(String, Array(DateTime('UTC'))) can map to KeyValuePair<string, DateTime[]>[].
  • Array(Array(Tuple(DateTime('UTC'), String))) can map to (DateTime, string)[][].
  • Nullable(Tuple(DateTime('UTC'), String)) can write (DateTime, string)? when the server feature is enabled.
  • LowCardinality(DateTime('America/New_York')) can be written from an ergonomic ArrayColumn<DateTime>.

The review also found and fixed two write-path correctness issues:

  • Nullable asks its inner codec whether a CLR type is writable instead of searching a finite diagnostic list.
  • LowCardinality builds its dictionary on an equality relation that agrees with the encoding, supplied by the inner codec.

Nested, Variant, and Dynamic keep their existing fixed object-based surfaces. Their child CLR types do not survive at the outer surface, so there is no type shape to lift.

How the contracts fit together

Contract Purpose
ElementType Canonical CLR type produced by the binary reader.
ReadableElementTypes Common readable types used for planning and diagnostics. It is not exhaustive for composites.
TryProjectRead Authoritative read question: can a canonical value be projected to this requested CLR type? Composite codecs inspect the requested shape and ask their children.
WritableElementTypes Preferred writable types used for planning and diagnostics. It remains finite and may omit composite combinations.
CanWriteElementType Authoritative type-level write question. Composite and wrapper codecs decompose the candidate type and ask their children recursively.
CanWrite Final check against a concrete column, including specialized dense layouts.
NullPlaceholderAs Returns a serializable hidden value in the requested CLR write shape for a null row.
BeginWrite Prepares columns and child state once, then shares that state between serialization-prefix and body writes.
WireEqualityComparer Which values a LowCardinality dictionary may merge: a comparer over a write surface that agrees with this codec's encoding, or null for a codec that offers none.

The enumerated type lists are intentionally not authoritative. Enumerating a seven-field tuple whose children each accept three CLR types would require materializing 3^7 constructed tuple types. The interrogative contracts only inspect the exact shape a caller requests.

Read path

  1. The codec reads its canonical ElementType from the Native stream.
  2. POCO planning asks TryProjectRead for the target property type.
  3. A composite codec validates the outer shape, then asks each child for its field or element projection.
  4. The resulting expression is compiled once and applied while rows are materialized.

For Array(DateTime('UTC')), the canonical value is uint[]. When the target is DateTime[], Array recognizes the array shape and asks the DateTime codec to project each uint to DateTime.

Write path

  1. POCO planning asks CanWriteElementType whether the property type is accepted.
  2. The gathered column retains the caller-facing CLR type; no eager converted array is required.
  3. The outer codec resolves a write shape from the column's actual element type and delegates each child column to its codec.
  4. BeginWrite prepares flattened or projected child columns and child state once.
  5. The block writer emits child serialization prefixes, then child bodies, using the same prepared state.

Nullable composites

The Native encoding of Nullable(T) contains a null map followed by an encoded T value for every row, including null rows. The hidden value has no semantic meaning, but the inner codec must still be able to serialize it.

For a null (DateTime, string)? row:

  1. Nullable writes 1 to the null map.
  2. It asks Tuple for a writable (DateTime, string) placeholder.
  3. Tuple asks its DateTime and String children for valid placeholders and constructs one tuple.
  4. The tuple codec writes that placeholder through its normal child writers.

Nullable(Tuple(...)) is gated to ClickHouse 26.6 or newer. Its real-server test enables enable_nullable_tuple_type=1 only for that query and is skipped on older servers.

LowCardinality dictionaries

A LowCardinality column writes a block-local dictionary of the distinct values plus one key per row. Deciding which rows share an entry means choosing an equality relation, and the only correct one is "encodes to the same bytes". CLR equality is a different relation for several inner types:

  • DateTime.Equals compares ticks and ignores Kind. An Unspecified value is read in the column's timezone while a Utc value already names an instant, so two ticks-equal values can encode to different seconds. Merging them changes what the server stores.
  • +0f equals -0f, and every NaN equals every other. Neither holds of their bit patterns.
  • byte[] compares by reference, so a FixedString dictionary keyed on values holds close to one entry per row.

IColumnCodec.WireEqualityComparer(Type writeType) answers this. It returns a comparer over that write surface which agrees with the codec's own encoding, or null. Each comparer is derived from the conversion the write path already uses, through WireEquality.Projected, so the comparison and the bytes cannot drift apart:

Inner codec Relation
String, UUID, Date, Date32, Enum8/16, integer and Bool fixed-width CLR equality
Float32, Float64 bit pattern
BFloat16 the top 16 bits that are written
DateTime, DateTime64 the instant at the column's scale
Time, Time64 the count at the column's scale
FixedString byte content
IPv4, IPv6 the encoded address, which drops ScopeId and folds an IPv4-mapped address onto its IPv4 form
Decimal the mantissa at the column's scale

Returning null keeps a type out of a LowCardinality dictionary and is the safe default: a codec that stays silent is rejected when the write is planned rather than deduplicated on the wrong relation. It also matches the server, which without allow_suspicious_low_cardinality_types accepts only String and FixedString as an inner type. The composite codecs return null.

The dense path is unchanged. A LowCardinality column read back off the wire already carries its dictionary and keys, so it is re-emitted with no rebuild and no comparer involved.

Performance

Measured against the previous revision of this PR (28c2223), 10,000 rows with 100 distinct values, .NET 9, minimum of four runs.

Case 28c2223 This revision
LowCardinality(String) 409 us 408 us
LowCardinality(UInt32) 460 us 434 us
LowCardinality(Nullable(Int32)) 425 us 442 us
LowCardinality(FixedString(8)) 815 us 554 us
LowCardinality(DateTime) 581 us 1014 us
LowCardinality(Float32) 294 us 736 us

Treat these as directional only. They come from a stopwatch harness, not BenchmarkDotNet, and run-to-run spread was roughly +-40%. The harness is not included in the PR.

FixedString improves because a byte-content comparer is cheaper than the wrapper type it replaces. DateTime and Float32 regress because a Dictionary with a custom comparer invokes the projection about three times per duplicate row, once to hash and twice to compare, where the previous revision converted once per row into a projected column. Both types need allow_suspicious_low_cardinality_types to exist in a table at all; the two inner types reachable at default server settings, String and FixedString, are at parity and better respectively.

Outside LowCardinality, removing the canonical branch from FixedWidthColumnCodec.WriteColumn drops a ProjectedColumn allocation and a per-element delegate call from every scattered Float32/Float64 write, such as a Nullable(Float32) column or a float tuple field, for byte-identical output.

Known ceiling

The remaining overhead is the repeated projection described above. Closing it means keeping the converted key per row instead of recomputing it inside the comparer: the codec would return a small keyed writer generic in both the surface type and the converted type, so the dictionary is keyed on the converted value and a parallel array holds the values to encode. The codec knows both type parameters statically, so this needs no reflection. Not done here; it is a contained follow-up.

Review

An independent review of this revision checked each codec's comparer against what that codec's WriteColumn emits and found no relation coarser than its encoding, which is the case that would corrupt data. It raised the repeated-projection cost above as its only substantive finding.

Validation

  • Full TCP test suite on net9.0: 3,126 passed, 0 failed, including 1,380 integration tests against a real server.
  • Wire output is byte-identical to 28c2223 across 16 LowCardinality cases, each through both the ergonomic write and the dense re-emit path. Covers lifted DateTime/DateTime64/Time/BFloat16, +-0 floats, FixedString, IPv4/IPv6, nullable variants, and non-zero slices.
  • CHANGELOG.md and RELEASENOTES.md are unchanged; this is TCP-client work.

ClickHouse.Driver.Tcp.Tests/Types/CanonicalWriteProjectionTests.cs is removed: it pinned the equivalence of two write paths, and there is now one.

Checklist

  • Compose read projections through Array, Map, Tuple, Nullable, and LowCardinality.
  • Compose write acceptance through the same wrappers and containers.
  • Keep readable and writable type lists finite and diagnostic-only.
  • Prepare child write state once for prefix and body serialization.
  • Build writable null placeholders for composite inner types.
  • Give LowCardinality an equality relation that agrees with the encoding, from the inner codec.
  • Derive each comparer from the conversion the write path already uses.
  • Preserve dense LowCardinality re-emission and non-zero slice indexes.
  • Gate Nullable Tuple server coverage to ClickHouse 26.6+ with its Beta setting.
  • Run full tests and an independent correctness review.
  • Keep TCP-only work out of the main client changelog and release notes.

@alex-clickhouse
alex-clickhouse marked this pull request as draft August 17, 2026 17:28
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from d463021 to cd10534 Compare August 17, 2026 17:40
@alex-clickhouse
alex-clickhouse requested a balanced review from Copilot August 17, 2026 17:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends TCP codecs so composite child types lift recursively across read and write paths.

Changes:

  • Adds interrogative write-type acceptance and column element-type discovery.
  • Implements recursive lifting for arrays, maps, tuples, and low-cardinality columns.
  • Adds broad codec, POCO, equivalence, and integration coverage.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
ClickHouse.Driver.Tcp/Types/IColumnCodec.cs Adds write-type interrogation.
ClickHouse.Driver.Tcp/Types/IColumn.cs Exposes column element types.
ClickHouse.Driver.Tcp/Types/CompositeElementProjections.cs Builds composite read projections.
ClickHouse.Driver.Tcp/Types/Codecs/VariantColumnCodec.cs Preserves variant write gating.
ClickHouse.Driver.Tcp/Types/Codecs/ValueNullableShape.cs Replaces value-type probes.
ClickHouse.Driver.Tcp/Types/Codecs/TupleColumnCodec.cs Lifts tuple fields recursively.
ClickHouse.Driver.Tcp/Types/Codecs/ReferenceNullableShape.cs Replaces reference-type probes.
ClickHouse.Driver.Tcp/Types/Codecs/NullableLowCardinalityShape.cs Uses interrogative inner acceptance.
ClickHouse.Driver.Tcp/Types/Codecs/NullableColumnCodec.cs Gates nullable write types.
ClickHouse.Driver.Tcp/Types/Codecs/NothingColumnCodec.cs Rejects all write types.
ClickHouse.Driver.Tcp/Types/Codecs/NestedColumnCodec.cs Rejects row-shaped writes.
ClickHouse.Driver.Tcp/Types/Codecs/MapColumnCodec.cs Lifts map keys and values.
ClickHouse.Driver.Tcp/Types/Codecs/LowCardinalityShape.cs Replaces inner write probes.
ClickHouse.Driver.Tcp/Types/Codecs/LowCardinalityColumnCodec.cs Adds lifted write shapes.
ClickHouse.Driver.Tcp/Types/Codecs/IArrayWriteShape.cs Adds lazy array write shapes.
ClickHouse.Driver.Tcp/Types/Codecs/ArrayColumnCodec.cs Lifts array elements.
ClickHouse.Driver.Tcp/Poco/PocoWriteConversion.cs Uses codec write interrogation.
ClickHouse.Driver.Tcp/Poco/PocoColumnBuilder.cs Updates composite diagnostics.
ClickHouse.Driver.Tcp.Tests/Utilities/InsertRoundTripCase.cs Documents canonical array cases.
ClickHouse.Driver.Tcp.Tests/Types/WritePathEquivalenceTests.cs Verifies lifted wire equivalence.
ClickHouse.Driver.Tcp.Tests/Types/TupleColumnCodecTests.cs Covers tuple rejection paths.
ClickHouse.Driver.Tcp.Tests/Types/CompositeLiftMatrixTests.cs Sweeps nested lift combinations.
ClickHouse.Driver.Tcp.Tests/Types/CompositeElementProjectionTests.cs Tests projection behavior.
ClickHouse.Driver.Tcp.Tests/Types/ColumnWriteAcceptanceTests.cs Tests write-type acceptance.
ClickHouse.Driver.Tcp.Tests/Types/ColumnElementTypeTests.cs Tests element-type resolution.
ClickHouse.Driver.Tcp.Tests/Types/ArrayColumnCodecTests.cs Covers lifted null-row errors.
ClickHouse.Driver.Tcp.Tests/Poco/PocoReadPlanTests.cs Tests lifted POCO reads.
ClickHouse.Driver.Tcp.Tests/Integration/PocoWriteIntegrationTests.cs Adds server round trips.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread ClickHouse.Driver.Tcp/Types/CompositeElementProjections.cs Outdated
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch 2 times, most recently from eef7be6 to c8b2167 Compare August 18, 2026 07:03
@alex-clickhouse
alex-clickhouse changed the base branch from tcp/epic-n9-poco-write to tcp/epic-b9-tls August 18, 2026 07:03
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from c8b2167 to 7ce2650 Compare August 18, 2026 07:29
@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from 7ce2650 to eb7ddf6 Compare August 18, 2026 08:11
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from eb7ddf6 to 7aead06 Compare August 21, 2026 13:07
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from 7aead06 to 216408d Compare August 22, 2026 16:47
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from 216408d to 007d917 Compare August 22, 2026 17:06
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from 007d917 to 4e8c058 Compare August 22, 2026 17:25
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch 3 times, most recently from 995f3f4 to 86aeff2 Compare August 26, 2026 15:40
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from 86aeff2 to 8d2e631 Compare August 26, 2026 16:38
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from 39409bd to 67f8c87 Compare August 28, 2026 11:03
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from 67f8c87 to 9c377d1 Compare August 28, 2026 12:18
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from b43646b to 1de32e9 Compare August 30, 2026 09:07
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from 1de32e9 to 8d2a9ac Compare August 31, 2026 08:50
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from 8d2a9ac to 6dcfa6b Compare September 2, 2026 09:00
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch 2 times, most recently from 29944e2 to d895727 Compare September 3, 2026 09:25
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch 2 times, most recently from 215bca9 to d1ebeac Compare September 3, 2026 14:11
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from d1ebeac to f38528d Compare September 4, 2026 09:00
alex-clickhouse and others added 8 commits September 4, 2026 11:17
PR 548 made the read contract interrogative so a container could recurse
into its children without enumerating their cartesian product. Only the two
wrappers used it. Array, Map and Tuple kept the identity-only default, so
Array(DateTime) read as uint[] and nothing else, and Tuple(DateTime, String)
only as ValueTuple<uint, string>.

Each container now overrides TryProjectRead and asks its children. The
structural half is shared: a map row is an array of pairs, so both reduce to
one element-wise loop in CompositeElementProjections, with the caller's
element projection building the new pair. A tuple rebuilds field-wise and
needs no loop.

The loop binds the row expression to a local, so a projection evaluates the
expression it was handed once however many elements the row has.

Containers recurse through containers, so Array(Array(DateTime)) and a tuple
with an array field both lift. Nested, Variant and Dynamic cannot and never
will: their element type is a fixed object[][] or object, so no per-child CLR
type survives to the surface to lift into.

ReadableElementTypes stays canonical-only. The honest list for a container is
its children's cartesian product, which costs a materialized Type per
combination on a failure path; TryProjectRead is the authority and answers
targets the list omits.

One plan test pinned the removed limitation. It becomes two: the lifted
property now fills, and a reading no child offers still reports what the
column does read as.

Co-Authored-By: Claude <noreply@anthropic.com>
Seven places wanted to know whether a codec accepts a given CLR element
type, and all seven asked by building a throwaway one-element column and
calling CanWrite on it. PocoWriteConversion went further and built its probe
reflectively, through Activator.CreateInstance over a MakeGenericType.

CanWriteElementType(Type) is that question asked directly. It defaults to
membership of WritableElementTypes, so every leaf keeps its answer, and the
four nullable/low-cardinality shapes, Map's shape, Array, Tuple and Variant
all drop their probe columns.

Two codecs have to override it: Nothing has no values to encode, and Nested
is written only from its own wire-shaped NestedColumn. Neither is describable
as an element type, which is what keeps a row-oriented insert reporting that
no property type can fill such a column.

The member is interrogative rather than a longer list for the same reason
TryProjectRead is: a composite will answer by asking its children about the
matching part of the type, so it never enumerates the cartesian product of
what they each accept.

No behavior change; this is the contract the composite write lifting needs.

Co-Authored-By: Claude <noreply@anthropic.com>
Reads lifted, writes did not: a DateTime[] property could be read from an
Array(DateTime) column but not inserted into one, and PocoWriteConversion's
doc recorded a LowCardinality(DateTime) column reading into a DateTime
property while accepting only raw epoch seconds.

Each container now answers CanWriteElementType by asking its children about
the matching part of the type, and resolves its write shape from the column's
own element type rather than its canonical one. So an Array(DateTime) column
takes an IColumn<DateTime[]>, flattens it into a ConcatColumn<DateTime>, and
the inner codec converts as it writes -- through the case IColumn<DateTime>
that already existed. Nothing is copied and nothing is allocated per row: the
flattening view is lazy, as it already was for the canonical type.

Most of the generic machinery was already in the right shape. MapShape's write
members were parameterized on the key and value types and took the codecs as
arguments, so lifting Map was resolving a different shape. TupleFieldColumn<T>
is generic over the field type and reads through ITuple, so lifting Tuple was
closing its per-field projection builders over the source tuple's field types.
Only Array needed the ergonomic branch extracted, into ArrayWriteShape<TWrite>;
its dense branch stays on the codec's own type argument, a dense column being
canonical by definition.

Shapes resolve lazily and cache per element type, following MapShapes.For and
LowCardinalityShapes.For. That is what keeps the accepted set unmaterialized: a
seven-field tuple of DateTime64 accepts 3^7 element types and builds a shape
only for those actually written.

CanWrite has to interrogate a column whose element type it does not know
statically, so IColumn gains ElementType: a default interface member resolving
the T of the implemented IColumn<T>, cached per column type. Enumerating shapes
instead would not compose, because a container child reports only its canonical
type.

The consumer keeps its existing walk first -- those types a caller already holds
in the shape the writer wants -- and falls back to asking the codec, in which
case the write type is the property's own type and no conversion is emitted.

Write stays inside read, which is what keeps a POCO round-tripping. A column
with null rows refuses a bare value type, so Array(Nullable(Int32)) is not
writable from int[]: it would insert and then fail to select back into the same
property. A sweep over a candidate pool asserts that invariant for every shape
in the matrix.

Co-Authored-By: Claude <noreply@anthropic.com>
Three reachable failures had no test. The Array null-row message moved into
ArrayWriteShape when the ergonomic branch was extracted, and nothing asserted
it -- it is the message that names the column, the row and the two ways out, so
it is worth pinning, and pinning for a lifted element type too, since the
offsets are now computed by a shape resolved from the row's own type.

The other two are contract guards with no other way to observe them: the
element variable ProjectArray is handed must match the row's elements, and a
column class surfacing two element types has no single one and is refused
rather than resolved to whichever interface reflection listed first.

IColumn.ElementType is read through the interface in these tests, which is how
every codec reaches it: a default interface member is not visible on the
implementing class without a cast.

Co-Authored-By: Claude <noreply@anthropic.com>
Coverage found the lifted LowCardinality write path untested: the acceptance
tests only ask CanWriteElementType, which never reaches the shape resolution, so
the headline case -- a LowCardinality(DateTime) column written from a DateTime
property -- had no test at all. It now round-trips through a real server, with
uniqExact asserting the dictionary really deduplicated and the raw epoch seconds
asserted on the wire, so a wrong dictionary shows up as a wrong value rather
than merely a different encoding.

WritePathEquivalenceTests gains seven lifted cases. Its property is the
strongest available here: the dense read-back of a column is always in the
canonical CLR type, so byte equality between it and a lifted ergonomic write is
what proves lifting changes the CLR surface and nothing else. One case lifts
through two levels over an inner that has a state prefix of its own --
Array(LowCardinality(DateTime)) -- where the Array resolves a shape for DateTime
and the LowCardinality it hands the flattened view resolves one too.

These also cover the state-free WriteStatePrefix and WriteColumn overloads with
a lifted shape, which nothing reached before.

Two Tuple guards behind CanWrite: a dense column of a different arity, refused
on its own CLR tuple type before any child is consulted, and a flat tuple column
of field types no child accepts.

The remaining uncovered lines in the changed files are an Array.MaxLength
overflow guard, which needs more elements than one array can hold, and
pre-existing read paths.

Co-Authored-By: Claude <noreply@anthropic.com>
Nullable and Variant refuse a column when their child cannot be written at
all -- innerCanWrite, allChildrenWritable -- but neither overrode
CanWriteElementType, so the interface default answered from
WritableElementTypes and skipped that gate. Since every container now asks the
interrogative question, the gate was bypassed transitively, breaking the
contract stated one member above: the two must agree wherever both can answer.

This was a regression, not a missing feature. The server accepts
Map(String, Nullable(Nothing)) and Array(Variant(String, Nested(...))) as real
table columns, and the insert gate used to refuse them before any byte went
out. With the gate bypassed the write faults part-way through a block, leaving
a half-written INSERT on the wire, and the POCO path lost its accurate
plan-build message for a generic one.

Both overrides now apply the same condition their CanWrite applies. Eleven
cases cover the wrappings the previous test missed -- a bare Nothing and a
bare Nested were covered, Nullable(Nothing) and Variant(..., Nested(...)) were
not -- and a new test asserts the contract itself rather than case by case, so
the next codec to gate on extra state is caught.

The default also no longer builds a Type[] to answer the canonical type, which
is the common answer and is asked once per column per slice.

Four stale doc comments: a cref to a member that never shipped, the
"LowCardinality is asymmetric today" paragraph -- which the revert restored,
and which prescribed a remedy this change did not need -- and two references to
probe columns that no longer exist.

Co-Authored-By: Claude <noreply@anthropic.com>
TryGetArrayElement tested IsArray plus rank one, which also admits the
non-zero-based T[*] that Type.MakeArrayType(1) builds: rank one, right element
type, distinct type. Both directions were wrong for it, and Map inherited the
fault through the same helper.

On a read the projection builds its result with MakeArrayType(), which is
always zero-based, so the codec returned true with an expression whose type was
not the one asked for -- the one thing the contract promises it will not do. On
a write, acceptance returned true and the failure moved to ArrayWriteShape<T>
casting the column to IColumn<T[]>, turning a plan-build refusal into a cast
failure with the insert already open.

IsSZArray is exactly the predicate: single dimension and zero-based. One helper,
so the fix covers Array's read projection, Array's write acceptance and Map's
pair test at once.

Four tests, one per direction per container; all four fail without the fix.

Co-Authored-By: Claude <noreply@anthropic.com>
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-n10-composite-lift branch from f38528d to 28c2223 Compare September 4, 2026 09:35
A LowCardinality dictionary must hold two rows equal exactly when they
encode to the same bytes. CLR equality is a different relation for
several types: DateTime.Equals ignores Kind although Kind changes the
instant encoded, +0f equals -0f with different bit patterns, and byte[]
compares by reference.

IColumnCodec gains one member, WireEqualityComparer(Type writeType). It
returns a comparer over that write surface which agrees with the codec's
own encoding, or null when the codec offers none. Null keeps the type out
of a LowCardinality dictionary, which matches the server: without
allow_suspicious_low_cardinality_types it accepts only String and
FixedString as an inner type.

Each comparer is derived from the conversion the write path already uses,
so the two cannot drift. Decimal compares on the mantissa at the column
scale, IPv6 on the 16 encoded bytes, FixedString on byte content, and the
floats on their bit patterns.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants