Skip to content

feat: Option B1 staging→production Neon promotion pipeline#1405

Merged
jaypatrick merged 10 commits intomainfrom
copilot/fix-neon-branch-parent-to-staging
Mar 25, 2026
Merged

feat: Option B1 staging→production Neon promotion pipeline#1405
jaypatrick merged 10 commits intomainfrom
copilot/fix-neon-branch-parent-to-staging

Conversation

Copy link
Contributor

Copilot AI commented Mar 25, 2026

Implements the agreed Neon branch hierarchy (production → staging → pr-NNNN) and a GitHub Releases–gated promotion pipeline so migrations reach production only on an explicit release, not on every merge to main.

Description

Implements the agreed Neon branch hierarchy (production → staging → pr-NNNN) and a GitHub Releases–gated promotion pipeline so migrations reach production only on an explicit release, not on every merge to main.

Changes

neon-branch-create.yml

  • parent: productionparent: staging — PR branches now fork from staging, not production
  • Updated header comment to clarify NEON_DATABASE_URL is used to derive DB name/role only; the parent branch is set explicitly as staging (not derived from the URL)
  • Updated inline step comments to reflect staging-as-parent

db-migrate.yml

  • New trigger: release: types: [published] — the Option B1 promotion gate; applies Prisma migrations to production Neon only when a GitHub Release is published
  • New workflow_dispatch input: target: choice (staging | production | both) for manual promotion control
  • Split PostgreSQL apply into two jobs:
    • pg_migrate_staging — runs on push to main; uses DIRECT_DATABASE_URL_STAGING
    • pg_migrate_production — runs on GitHub Release; uses DIRECT_DATABASE_URL; fails fast with a clear error if DIRECT_DATABASE_URL is not configured
  • migrate job narrowed to D1-only, push-to-main only (D1 has no release-gated promotion)
  • dry-run job: HAS_DIRECT_DB_URL now checks both staging and production secrets; PostgreSQL dry-run step selects DIRECT_DATABASE_URL (production) when workflow_dispatch target is production, and DIRECT_DATABASE_URL_STAGING for all other cases (pull requests, target=staging|both)

docs/database-setup/branching-strategy.md (new)

  • Documents the full branch hierarchy, per-branch migration triggers, required secrets, and the Option B1 promotion sequence diagram
  • Clarified NEON_DATABASE_URL secret description to distinguish it from the migration secrets
  • Fixed sequence diagram to use a dedicated Neon pr-NNNN branch participant to avoid ambiguity
  • Corrected local dev setup note to reference .env.local for DIRECT_DATABASE_URL

Testing

  • Unit tests added/updated
  • Manual testing performed
  • CI passes

Zero Trust Architecture Checklist

This PR touches only CI workflow YAML files and documentation — no worker/ or frontend/ code was modified.

Worker / Backend

  • Every handler verifies auth before executing business logic — N/A
  • CORS origin allowlist enforced (not *) on write/authenticated endpoints — N/A
  • All secrets accessed via Worker Secret bindings (not [vars]) — N/A
  • All external inputs Zod-validated before use — N/A
  • All D1 queries use parameterized .prepare().bind() (no string interpolation) — N/A
  • Security events emitted to Analytics Engine on auth failures — N/A

Frontend / Angular

  • Protected routes have functional CanActivateFn auth guards — N/A
  • Auth tokens managed via Clerk SDK (not localStorage) — N/A
  • HTTP interceptor attaches ****** (no manual token passing) — N/A
  • API responses validated with Zod schemas before consumption — N/A
Original prompt

Context

This PR implements Option B1: a GitHub Releases–triggered staging → production promotion pipeline, along with a fix to the Neon branch parent hierarchy (staging instead of production) and all supporting workflow updates.

The agreed architecture is:

Neon:   production  (deployed Worker uses env.HYPERDRIVE → Neon pooler → production branch)
            └── staging     (GitHub main branch points here; DIRECT_DATABASE_URL_STAGING secret)
                    └── pr-NNNN  (each PR forks from staging, not from production)
                    └── dev/jayson

GitHub:

  • Every push to main applies Prisma migrations to staging Neon branch
  • A GitHub Release (v* tag) triggers a promotion job that applies migrations to production Neon branch
  • PRs fork their Neon branch from staging

Changes Required

1. Fix neon-branch-create.yml — change PR parent from production to staging

In the create-neon-branch job, wherever the Neon branch is created (the step that calls the Neon API or uses neondatabase/create-branch-action), change:

parent: production

to:

parent: staging

Also update the header comment (lines 14–22 area) that says NEON_DATABASE_URL – Connection string for the *production* branch to clarify that the parent of PR branches is now staging. Update the NEON_DATABASE_URL secret description to explain it should point to the staging branch (used as the PR parent).


2. Update db-migrate.yml — split apply into staging vs. production

Currently db-migrate.yml applies migrations on push to main. Under Option B1 the workflow needs to:

  1. On push to main → apply migrations to staging Neon (using DIRECT_DATABASE_URL_STAGING secret)
  2. On GitHub Release published (on: release: types: [published]) → promote migrations to production Neon (using DIRECT_DATABASE_URL secret, which points to Neon production)

Required trigger block changes

Add a release trigger alongside the existing push and pull_request triggers:

on:
    push:
        branches: [main]
        paths:
            - 'migrations/**'
            - 'admin-migrations/**'
            - 'prisma/migrations/**'
            - 'prisma/schema.prisma'
    pull_request:
        branches: [main]
        paths:
            - 'migrations/**'
            - 'admin-migrations/**'
            - 'prisma/migrations/**'
            - 'prisma/schema.prisma'
    release:
        types: [published]
    workflow_dispatch:
        inputs:
            dry_run:
                description: 'Perform a dry-run only (validate, no writes)'
                type: boolean
                default: false
            target:
                description: 'Migration target (staging | production | both)'
                type: choice
                options: [staging, production, both]
                default: staging

Required job changes

The existing migrate job (PostgreSQL apply step) should be split into two jobs:

pg_migrate_staging — runs on push to main or workflow_dispatch with target == staging || target == both:

pg_migrate_staging:
    name: Apply Prisma Migrations → Staging
    runs-on: ubuntu-latest
    timeout-minutes: 15
    needs: [validate]
    if: >-
        (github.event_name == 'push' && github.ref == 'refs/heads/main') ||
        (github.event_name == 'workflow_dispatch' && (inputs.target == 'staging' || inputs.target == 'both'))
    env:
        DIRECT_DATABASE_URL: ${{ secrets.DIRECT_DATABASE_URL_STAGING }}
    steps:
        - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
        - name: Setup Deno environment
          uses: ./.github/actions/setup-deno-env
          with:
              deno-version: ${{ env.DENO_VERSION }}
        - name: Apply Prisma migrations to staging
          if: env.DIRECT_DATABASE_URL != ''
          run: deno run -A npm:prisma migrate deploy

pg_migrate_production — runs on GitHub Release published or workflow_dispatch with target == production || target == both:

pg_migrate_production:
    name: Apply Prisma Migrations → Production
    runs-on: ubuntu-latest
    timeout-minutes: 15
    needs: [validate]
    if: >-
        github.event_name == 'release' ||
        (github.event_name == 'workflow_dispatch' && (inputs.target == 'production' || inputs.target == 'both'))
    env:
        DIRECT_DATABASE_URL: ${{ secrets.DIRECT_DATABASE_URL }}
    steps:
        - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
        - name: Setup Deno environment
          uses: ./.github/actions/setup-deno-env
          with:
              deno-version: ${{ env.DENO_VERSION }}
        - name: Apply Prisma migrations to production
          if: env.DIRECT_DATABASE_URL != ''
          run: deno run -A npm:prisma migrate deploy

The existing D1 migration apply steps (d1_migrate_main and d1_migrate_admin) ...

This pull request was created from Copilot chat.


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

- neon-branch-create.yml: change PR parent from production to staging;
  update header comment and inline comments accordingly
- db-migrate.yml: add release trigger (Option B1 promotion gate);
  add target choice input to workflow_dispatch; split PostgreSQL
  migrations into pg_migrate_staging (push to main) and
  pg_migrate_production (GitHub Release); keep D1 in migrate job
  (push to main only); update dry-run to use DIRECT_DATABASE_URL_STAGING;
  update header comment
- docs/database-setup/branching-strategy.md: new doc explaining the
  full Neon branching hierarchy, promotion flow, secrets, and triggers

Co-authored-by: jaypatrick <1800595+jaypatrick@users.noreply.github.com>
Agent-Logs-Url: https://github.com/jaypatrick/adblock-compiler/sessions/715714c1-534f-4323-aaa1-dfaa1b668d69
Copilot AI changed the title [WIP] Fix Neon branch parent hierarchy to staging feat: Option B1 staging→production Neon promotion pipeline Mar 25, 2026
Copilot AI requested a review from jaypatrick March 25, 2026 20:02
@jaypatrick jaypatrick added the database Neon Postgre label Mar 25, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements Option B1 for Neon environment promotion by introducing a GitHub Releases–gated path for applying Prisma migrations to production, while keeping staging updated on merges to main, and updating PR branch creation to fork from staging.

Changes:

  • Update Neon PR branch creation to fork from staging (not production) and refresh related workflow comments.
  • Split DB migration workflow so Prisma migrations apply to staging on push to main and to production on release.published, with workflow_dispatch target control.
  • Add documentation describing the Neon branching hierarchy, triggers, secrets, and the promotion flow diagram.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File Description
docs/database-setup/branching-strategy.md New documentation for the production→staging→PR branching model and Option B1 promotion flow.
.github/workflows/neon-branch-create.yml Changes PR branch parent from production to staging and updates workflow commentary accordingly.
.github/workflows/db-migrate.yml Adds release trigger and splits staging vs production Prisma migration jobs; narrows D1 apply to push-to-main.

jaypatrick and others added 6 commits March 25, 2026 16:56
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@jaypatrick jaypatrick marked this pull request as ready for review March 25, 2026 21:06
Copilot AI review requested due to automatic review settings March 25, 2026 21:06
@jaypatrick jaypatrick merged commit 29e54f6 into main Mar 25, 2026
28 checks passed
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

Comment on lines +431 to +434
if: >-
github.event_name == 'release' ||
(github.event_name == 'workflow_dispatch' && (inputs.target == 'production' || inputs.target == 'both'))
env:
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as staging: workflow_dispatch runs with inputs.dry_run: true can still reach this job and apply production migrations (if target=production|both). Gate the workflow_dispatch branch of the condition on inputs.dry_run == false to ensure dry-run dispatches cannot mutate production.

Copilot uses AI. Check for mistakes.
## Local Development

For local development, use a personal `dev/<name>` Neon branch forked from `staging`.
Set `DIRECT_DATABASE_URL` in your `.dev.vars` to point at your dev branch.
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section says to set DIRECT_DATABASE_URL in .dev.vars, but local dev setup uses CLOUDFLARE_HYPERDRIVE_LOCAL_CONNECTION_STRING_HYPERDRIVE in .dev.vars and DIRECT_DATABASE_URL in .env.local for Prisma CLI (see docs/database-setup/local-dev.md). Update these instructions to match the existing local-dev doc to avoid misconfiguration.

Suggested change
Set `DIRECT_DATABASE_URL` in your `.dev.vars` to point at your dev branch.
In your `.dev.vars`, set `CLOUDFLARE_HYPERDRIVE_LOCAL_CONNECTION_STRING_HYPERDRIVE` to the connection string for your dev branch (used by the local Worker/Hyperdrive).
In your `.env.local`, set `DIRECT_DATABASE_URL` to the same dev-branch connection string for use by the Prisma CLI.

Copilot uses AI. Check for mistakes.
Comment on lines 226 to 230
if echo "$STATUS" | grep -q "P3009"; then
echo "⚠️ P3009 detected — restoring branch to production tip for a clean slate."
echo "⚠️ P3009 detected — restoring branch to staging tip for a clean slate."

if [ -z "$BRANCH_ID" ]; then
echo "::error::BRANCH_ID is empty. Cannot restore branch. Check create-branch step outputs."
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log message was updated to say the branch is restored to the staging tip, but the nearby comment still says the parent is production. Update the comment to match the new parent: staging behavior so future readers don’t restore/diagnose against the wrong branch.

Copilot uses AI. Check for mistakes.
Comment on lines +410 to +413
if: >-
(github.event_name == 'push' && github.ref == 'refs/heads/main') ||
(github.event_name == 'workflow_dispatch' && (inputs.target == 'staging' || inputs.target == 'both'))
env:
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

workflow_dispatch with inputs.dry_run: true will still satisfy this job condition and can apply migrations to staging. Add an inputs.dry_run == false guard (only for workflow_dispatch) so dry-run dispatches never run migrate deploy side effects.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

database Neon Postgre

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants