fix: require explicit pure-Python source builds

[vsolano9]

Summary

Fixes #994.

Source builds now stop when Cython preparation or extension compilation fails instead of silently retrying setup() without extensions. CLICKHOUSE_CONNECT_SKIP_CYTHON=1 remains the explicit opt-out for a pure-Python source build, and the installation documentation and changelog describe that behavior.

Regression tests cover missing Cython, Cython preparation failure, propagation of setup/compiler failures without a retry, and the explicit pure-Python opt-out.

Validation:

• Red/green regression: 3 failures and 1 pass before the fix; all 4 pass after it.
• Full suite against ClickHouse 25.8: 2,827 passed, 62 skipped, 16 subtests passed.
• Ruff formatting/linting and mypy pass.
• Setup regressions pass on Python 3.10, 3.11, 3.12, 3.13, and 3.14.
• A deliberately failing compiler exits the build immediately without a pure-Python retry.
• Compiled and explicit pure-Python wheels both build, install in clean environments, and pass a live SELECT 13 smoke test; the compiled wheel contains the expected extensions and the pure wheel does not.

Checklist

• Unit and integration tests covering the common scenarios were added
• A human-readable description of the changes was provided to include in CHANGELOG
• Relevant installation documentation was updated with the explicit pure-Python build procedure

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[socketpair]

I did not check, but tests should check all the functionality in both cases. I guess, there should be an option to choose implementation at runtime (at least for tests).

[socketpair]

I think, this file is extra. doesn't need to be tested.

[vsolano9]

This file is the regression boundary for #994 rather than a duplicate runtime-implementation test. It proves the removed broad fallback cannot return, that unrelated packaging errors propagate, and that the explicit source-build opt-out still produces no extensions. Runtime C/Python parity is already covered separately in tests/unit_tests/test_driver/test_c_parity.py, with CLICKHOUSE_CONNECT_USE_C=0 providing runtime selection.

[vsolano9]

I checked this against the current package paths.

• Runtime selection already exists in clickhouse_connect/driver/ctypes.py: CLICKHOUSE_CONNECT_USE_C=0 forces the pure-Python modules even in an install that contains the extensions.
• tests/unit_tests/test_driver/test_c_parity.py imports both implementations directly and compares the C and Python buffer/conversion paths.
• tests/test_setup.py covers a different boundary introduced by this PR: failed Cython preparation must abort, unrelated setup() failures must not be swallowed, and only CLICKHOUSE_CONNECT_SKIP_CYTHON=1 may produce an extension-free source build. Removing it would leave Don't silently fall back to a pure-Python build when the C extensions fail to compile #994 without regression coverage.

I also rebuilt the PR source archive in clean Python 3.12 environments. A normal install loaded driverc, the same compiled install with CLICKHOUSE_CONNECT_USE_C=0 loaded driver, and a cache-isolated CLICKHOUSE_CONNECT_SKIP_CYTHON=1 install contained no .so files and loaded driver. All three returned [1, None, 3] for the same nullable-column conversion. No additional runtime switch is needed in this PR.

[joe-clickhouse]

Hi @vsolano9 & @socketpair thanks for putting this together the discussion. I agree that the blanket except around setup() is too broad and can swallow unrelated packaging errors and retry them as a pure Python build, which we should fix.

However, I'm not comfortable with successful C extension compilation becoming a default requirement. The pure Python fallback is existing documented behavior and changing a source install from success to failure would be a breaking change. Normal wheel installs already contain the compiled extensions and are unaffected.

My preference would be to:

• Keep the current default. Have source builds try the extensions, warn on a compiler or linker failure, and fall back to pure Python
• Keep CLICKHOUSE_CONNECT_SKIP_CYTHON=1 for an explicitly pure Python build
• Add something like CLICKHOUSE_CONNECT_REQUIRE_C=1 for users and CI that need compilation to succeed
• Let unrelated setup and packaging errors propagate normally instead of treating them as extension failures
• Enable the required mode in our release-wheel jobs so we cannot accidentally publish a wheel without the extensions

This will allows us to preserve compatibility while still giving deployments that require the optimized path a reliable way to ensure it does work.

[socketpair]

@joe-clickhouse thanks — agreed on the broad except. But I'd push back on keeping the fallback as the default.

Nothing that works today breaks. Wheels are unaffected; source builds with a working toolchain are unaffected. The only builds that flip to failure are the ones already producing a silently degraded artifact the user never asked for. That's not breaking a working setup — it's surfacing a broken one, with a one-line documented fix printed in the error message.

"Warn and fall back" == "silently fall back". pip suppresses setup.py output on successful builds. The warning lands in CI/Docker logs nobody reads, so the people who'd act on it are exactly the ones who never see it.

Opt-in safety protects the wrong people. CLICKHOUSE_CONNECT_REQUIRE_C=1 only helps those who already know a silent pure-Python build is possible — but the harm falls on those who don't. Defaults should be correct-by-default with an escape hatch for the informed minority, which CLICKHOUSE_CONNECT_SKIP_CYTHON=1 already is. Two flags also create an ambiguous three-state matrix for no gain. And enabling the required mode in your own release jobs concedes the point: a silent fallback isn't acceptable when the result matters — it matters for users too.

Downstream. I package clickhouse-connect as an RPM for Fedora and currently carry a local patch that makes the build fail hard on Cython/extension failure, because a distro package silently missing its C extensions is a defect shipped to every user. I'd rather not carry that patch — and neither would Debian, Arch, conda-forge, or anyone building internal wheels.

So: fail by default (this PR), SKIP_CYTHON=1 as the sole opt-out named in the error message, unrelated setup() errors propagating untouched (already the case here). If you want a migration path, a changelog note in the next minor release is enough — no third flag. Happy to help with docs wording.