Skip to content

Phase 0 follow-up: integration + Alembic migration tests for the CI baseline #8

Description

@kosminus

Context

The Phase 0 plan calls for a CI baseline of integration tests covering the pipeline, connectors, semantic context builder, SQL sanitizer, and Alembic up/down — before the refactors land.

PR #7 delivered DB/LLM-free unit tests (secrets, jobs/registry/arq, rate limiter, telemetry, tracing, SQL sanitizer, health-probe shape) and a CI workflow that gates on pytest. Of the integration targets, only the SQL sanitizer is covered. This issue tracks closing the rest so the stated baseline is fully met.

Scope

  • Alembic round-tripupgrade head then downgrade base (and back) against a disposable Postgres+pgvector, asserting no errors and a clean schema diff.
  • Query pipelineexecute_nl_query end-to-end with a fake/stub LLM provider and a real (or sqlite/pg) target DB: context build → compose → validate → execute → interpret, plus the retry path.
  • ConnectorsPostgreSQLConnector against a real Postgres (read-only enforcement, timeout, max-rows truncation); BigQuery/Databricks behind recorded/mocked clients.
  • Semantic context builderbuild_context with seeded glossary/metrics/dictionary/knowledge, asserting prompt assembly and relevance selection.

Infra needed

These require services CI doesn't yet provide:

  • A postgres:16 + pgvector service container in the workflow.
  • A deterministic fake LLM provider (registered via provider_registry) to avoid network/key dependencies and keep tests hermetic.
  • Test fixtures/factories for connections + metadata (the existing tests/fixtures/sample_seed.sql is a starting point).

Notes

  • Keep the existing unit tests as the fast, no-dependency gate; add integration tests as a separate (possibly slower) CI job.
  • Once green, consider flipping the advisory ruff/mypy CI steps to gating after the pre-existing lint debt is addressed.

Surfaced during review of #7.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions