Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions AGENTS.MD
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,15 @@
This repo pushes millions of legacy rows through SQLAlchemy. When Codex or any other agent has to work on
these transfers, keep the following rules in mind to avoid hour-long runs:

## Branching
- **Always branch off `staging`**, never off `main` or `production`. This applies to every `chore/`, `fix/`,
and `feat/` (feature) branch, including ticket branches named after a Jira key.
- Fetch first so the new branch starts from the current remote tip:
`git fetch origin staging && git switch -c <branch> origin/staging`.
- Open the PR against `staging`. Promotion to `production` happens later via a separate promotion PR — see
`docs/release-flow.md`.
- The only exception is `hotfix/vX.Y.Z`, which branches off a release tag (`hotfix-start.yml`), not `staging`.

## 1. Skip ORM object construction once volume climbs
- **Do not call `session.bulk_save_objects`** for high frequency tables (e.g., transducer observations,
water-levels, chemistry results). It still instantiates every mapped class and kills throughput.
Expand Down
13 changes: 12 additions & 1 deletion cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -1209,12 +1209,23 @@ def data_migrations_run(
force: bool = typer.Option(
False, "--force", help="Re-run even if already applied."
),
dry_run: bool = typer.Option(
False,
"--dry-run",
help="Report the planned changes without writing anything.",
),
theme: ThemeMode = typer.Option(
ThemeMode.auto, "--theme", help="Color theme: auto, light, dark."
),
):
from db.engine import session_ctx
from data_migrations.runner import run_migration_by_id
from data_migrations.runner import dry_run_migration_by_id, run_migration_by_id

if dry_run:
with session_ctx() as session:
dry_run_migration_by_id(session, migration_id)
typer.echo("dry run complete; nothing written")
return

with session_ctx() as session:
ran = run_migration_by_id(session, migration_id, force=force)
Expand Down
7 changes: 6 additions & 1 deletion data_migrations/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# limitations under the License.
# ===============================================================================
from dataclasses import dataclass
from typing import Callable
from typing import Callable, Optional

from sqlalchemy.orm import Session

Expand All @@ -27,3 +27,8 @@ class DataMigration:
description: str
run: Callable[[Session], None]
is_repeatable: bool = False
# Optional read-only preview. Migrations that delete rows or re-point
# foreign keys should provide one so the planned changes can be reviewed
# before anything is written. It must not commit. Any return value is for
# the caller's own use -- the runner ignores it.
dry_run: Optional[Callable[[Session], object]] = None
Loading
Loading