Branching CI/CD — validate schema changes on a Lakebase branch

A reference GitHub CI/CD workflow that uses Lakebase branching to ship database schema changes with confidence. Every pull request that touches migrations/ forks a short-lived Lakebase branch off production — a copy-on-write copy of the production schema and data, ready in seconds — runs the changed SQL there, has an AI Gateway model analyze the impact, and posts the report on the PR. Merging applies the migration to production exactly once; closing the PR deletes the branch.

Schema changes never touch production directly, and reviewers see a real execution result plus an AI risk assessment before they approve.

Architecture

            ┌───────────────── your repo ─────────────────┐
            │ migrations/*.sql   scripts/   .github/        │
            └──────┬───────────────────────────┬───────────┘
   PR opened       │                merge to main
        │          │                           │
        ▼          │                           ▼
  ┌──────────────┐ │                  apply pending migrations
  │ Lakebase     │◀┘ forked from      to the production branch
  │ branch:      │   production       (exactly-once via
  │ pr-<n>-<slug>│                     schema_migrations)
  └──────┬───────┘
         │
         ▼
  run changed SQL  →  AI Gateway impact analysis  →  PR comment
         │
         ▼
  PR closed  →  delete branch

How the workflow runs

  1. PR validate (lakebase-pr-validate.yml) — on a PR touching migrations/, fork a Lakebase branch pr-<n>-<slug> off production (24h TTL), run the changed SQL via psql, collect row-count stats, and send the PR context plus execution results to an AI Gateway model. It returns an intent summary, risk analysis (data loss, locking, constraints), and an APPROVE / REQUEST_CHANGES / BLOCK recommendation — posted as a PR comment. The check fails if any statement failed.
  2. Deploy on merge (lakebase-deploy.yml) — a push to main touching migrations/ applies pending migrations to the production branch. A schema_migrations table (checksum-tracked) makes this exactly-once and flags drift if an already-applied file changed.
  3. Cleanup — closing the PR deletes its Lakebase branch.

You can run the identical validation locally, without GitHub:

databricks auth login
./.github/scripts/run_local.sh migrations/003_your_change.sql

Inside the PR validation

lakebase_pr_validate.py drives the whole check against the Lakebase Postgres API:

  • Fork the branch — POST /api/2.0/postgres/projects/{project}/branches with {"spec": {"source_branch": "…/production", "ttl": "86400s"}}. The branch is named pr-<number>-<slug> (the head ref, slugified, capped at 48 chars). If it already exists (a PR update), it’s reused rather than recreated — the run is idempotent.
  • Mint a short-lived credential — POST /api/2.0/postgres/credentials for the branch’s READ_WRITE endpoint (it falls back to any endpoint if none is marked read-write).
  • Run only the changed files — detected with git diff --diff-filter=AM $PR_BASE_SHA $PR_HEAD_SHA -- 'migrations/**/*.sql', executed with psql -v ON_ERROR_STOP=1 -X -a over sslmode=require. stdout and stderr are captured (tail 8 000 chars) and each file’s duration recorded.
  • Collect table stats — table names are parsed out of the SQL and a SELECT count(*) runs on each (up to 10), giving the reviewer before/after row counts.
  • AI review — PR title/body + SQL + execution output + stats go to the model (AI_GATEWAY_MODEL, default databricks-claude-opus-4-7) under the system prompt “You are a senior database reviewer for production Postgres changes.” It returns an intent summary, a per-statement success check, an assessment of any test cases stated in the PR body, risk flags (data loss, locking, constraints, RLS, indexes), and an APPROVE / REQUEST_CHANGES / BLOCK call. The call tries /serving-endpoints/{model}/invocations and falls back to the AI Gateway chat-completions route.
  • Comment + exit code — the report is posted as a PR comment with a PASS/FAIL badge, a results table, and collapsible error output. The job’s exit code mirrors SQL success, so a failed statement fails the check.

Exactly-once, and drift detection

Both the merge-to-main deploy and any local run share scripts/migrations_runner.py, so dev and CI behave identically. It tracks applied files in a table:

CREATE TABLE IF NOT EXISTS schema_migrations (
    filename    TEXT PRIMARY KEY,
    checksum    TEXT NOT NULL,           -- SHA-256 of the file content
    applied_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
    applied_by  TEXT NOT NULL DEFAULT current_user
);

Files run in filename order (hence the zero-padded 001_, 002_ convention), each inside its own transaction — a failure rolls that file back entirely and stops the run. Already-applied files are skipped. If a file’s content changed after it was applied, its checksum no longer matches and the runner reports drift and skips it — it will not silently re-apply edited history, so the fix is a new migration, never an edit to an old one. The deploy workflow also runs under a cancel-in-progress: false concurrency group, so production migrations are serialized and never race.

Prerequisites and secrets

You provision the Lakebase project the workflow acts on, then add the GitHub Actions secrets the workflows read:

Secret Example
DATABRICKS_HOST https://<workspace>.cloud.databricks.com
DATABRICKS_TOKEN PAT or SP OAuth token with Lakebase permissions
LAKEBASE_PROJECT your Lakebase project id
LAKEBASE_SOURCE_BRANCH production
LAKEBASE_DATABASE databricks_postgres
LAKEBASE_PG_USER the Postgres role / user email
AI_GATEWAY_MODEL databricks-claude-opus-4-7

You also need a long-lived production Lakebase branch with a READ_WRITE endpoint, and a model-serving / AI Gateway chat endpoint for the review.

Edit this page →