Skip to content

ENG-3299: Add CorrespondenceMetadata model and table#7867

Open
JadeCara wants to merge 19 commits intoENG-3299/enums-scopes-encryptionfrom
ENG-3299/correspondence-metadata
Open

ENG-3299: Add CorrespondenceMetadata model and table#7867
JadeCara wants to merge 19 commits intoENG-3299/enums-scopes-encryptionfrom
ENG-3299/correspondence-metadata

Conversation

@JadeCara
Copy link
Copy Markdown
Contributor

@JadeCara JadeCara commented Apr 8, 2026

Ticket ENG-3299

Dependency: Requires #7866 to be merged first (stacked PR)

Description Of Changes

PR 2 of 2 for ENG-3299. Creates the CorrespondenceMetadata table with a 1:1 relationship to Comment for delivery tracking and email threading.

Table fields:

  • comment_id — FK to comment.id, UNIQUE (1:1), CASCADE delete
  • message_id — UNIQUE, indexed (idempotency key for deduplication)
  • in_reply_to — threading reference to parent message
  • reply_to_address — indexed for inbound routing
  • delivery_status — varchar with default pending (values: pending/sent/delivered/bounced/failed)
  • delivered_at, bounce_reason, sender_email, recipient_email

Design decision: delivery_status uses varchar (not a PG enum) to avoid cross-migration enum type dependencies. Validation happens at the Python model layer via CorrespondenceDeliveryStatus enum.

Code Changes

  • src/fides/api/models/comment.py — Added CorrespondenceMetadata model with 1:1 relationship to Comment
  • src/fides/api/db/base.py — Registered CorrespondenceMetadata for Alembic discovery
  • New migration: creates correspondence_metadata table with indexes and constraints

Steps to Confirm

  1. Run alembic upgrade head — verify table created
  2. Run alembic downgrade -1 then alembic upgrade head — verify full cycle
  3. Verify table schema: \d correspondence_metadata shows all columns, UNIQUE on comment_id and message_id, FK to comment.id with CASCADE
  4. Run pytest tests/ctl/models/test_comment.py — all pass

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
    • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
    • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change
    • Updates unreleased work already in Changelog, no new entry necessary
  • UX feedback:
    • All UX related changes have been reviewed by a designer
    • No UX review needed
  • Followup issues:
    • Followup issues created
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
    • No migrations
  • Documentation:
    • Documentation complete, PR opened in fidesdocs
    • Documentation issue created in fidesdocs
    • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • No documentation updates required

JadeCara and others added 2 commits April 8, 2026 17:16
Create correspondence_metadata table (1:1 with comment) for delivery
tracking and threading. Fields: message_id (unique, idempotency key),
in_reply_to, reply_to_address, delivery_status, delivered_at,
bounce_reason, sender_email, recipient_email.

CASCADE delete from comment ensures cleanup. delivery_status uses
varchar (not PG enum) to avoid cross-migration enum dependencies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown
Contributor

vercel bot commented Apr 8, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
fides-plus-nightly Ready Ready Preview, Comment Apr 15, 2026 10:35pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
fides-privacy-center Ignored Ignored Apr 15, 2026 10:35pm

Request Review

JadeCara and others added 3 commits April 8, 2026 17:22
Tests the constraints the TLA+ spec relies on:
- message_id UNIQUE (idempotency guard)
- comment_id UNIQUE (1:1 relationship)
- CASCADE delete (PR deletion cleanup)
- Default delivery_status is "pending" (FSM initial state)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 8, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.07%. Comparing base (249d24c) to head (7267554).

Additional details and impacted files
@@                        Coverage Diff                        @@
##           ENG-3299/enums-scopes-encryption    #7867   +/-   ##
=================================================================
  Coverage                             85.07%   85.07%           
=================================================================
  Files                                   629      630    +1     
  Lines                                 40879    40904   +25     
  Branches                               4748     4749    +1     
=================================================================
+ Hits                                  34776    34800   +24     
  Misses                                 5030     5030           
- Partials                               1073     1074    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JadeCara JadeCara marked this pull request as ready for review April 10, 2026 18:18
@JadeCara JadeCara requested a review from a team as a code owner April 10, 2026 18:18
@JadeCara JadeCara requested review from adamsachs and removed request for a team April 10, 2026 18:18
Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — PR #7867: Add CorrespondenceMetadata table

The overall design is solid: the 1:1 relationship between Comment and CorrespondenceMetadata, the cascade behavior, and the deletion guard for correspondence comments are all well-structured and well-tested. There are a couple of bugs that would cause runtime failures and one structural issue to address before merge.

Critical (must fix)

1. server_default generates invalid DDL in both the model and the migration.

server_default in SQLAlchemy treats its argument as a raw SQL expression, not a Python string. Passing "pending" (or CorrespondenceDeliveryStatus.pending, which evaluates to the string "pending") generates DEFAULT pending in the DDL — an unquoted identifier. PostgreSQL will reject this because pending is not a known column or function. The migration will fail to apply.

  • In the migration: change server_default="pending"server_default=sa.text("'pending'")
  • In the model: change server_default=CorrespondenceDeliveryStatus.pendingserver_default="'pending'"

The pattern is already used correctly above in the same migration for created_at/updated_at (sa.text("now()")).

2. CorrespondenceDeliveryStatus is defined twice.

The enum is defined in comment.py (pre-existing) and again in correspondence_metadata.py (new). These are separate Python classes — they happen to have the same values but are not equal. correspondence_metadata.py should import the enum from comment.py rather than redefining it.

Minor

3. Deprecated import + unconventional declared_attr usage.

from sqlalchemy.ext.declarative import declared_attr is deprecated (moved to sqlalchemy.orm). Additionally, @declared_attr for __tablename__ is a mixin pattern; a concrete model should use a plain class attribute (__tablename__ = "correspondence_metadata").


What looks good

  • The UniqueConstraint("reference_id") + UniqueConstraint("message_id") correctly enforce the 1:1 relationship and idempotency invariants.
  • cascade="all, delete-orphan" on Comment.correspondence_metadata is correctly set and tested.
  • The deletion guard in Comment.delete() and the filter in delete_comments_for_reference_and_type() are both correct and have good test coverage.
  • The CORRESPONDENCE_COMMENT_TYPES frozenset is a clean way to centralise the type guard.
  • Dataset YAML and changelog are both present.

🔬 Codegraph: connected (46297 nodes)


💡 Write /code-review in a comment to re-run this review.

Comment thread src/fides/api/models/correspondence_metadata.py Outdated
Comment thread src/fides/api/models/correspondence_metadata.py
Comment thread src/fides/api/models/correspondence_metadata.py
JadeCara and others added 3 commits April 13, 2026 15:17
…atus enum

Import CorrespondenceDeliveryStatus from comment.py instead of redefining it
in correspondence_metadata.py, avoiding two separate incompatible enum classes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@JadeCara JadeCara requested review from erosselli and removed request for adamsachs April 13, 2026 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant