Skip to content

Add event receiver framework for webhook-driven graph updates #508

@ravisuhag

Description

@ravisuhag

Context

Not every metadata change needs a full extraction cycle. Many sources can push notifications — a GitHub webhook on a schema migration merge, a PagerDuty alert when an incident opens, a cloud event when a resource is created. Meteor should be able to receive these events and convert them into graph updates.

Scope

  • Lightweight HTTP server for receiving webhooks (meteor receive or built into meteor run)
  • Pluggable event handler interface — map incoming events to asset/relationship mutations
  • Built-in handlers for common sources:
    • GitHub webhooks (push, PR merge, release)
    • Generic cloud events (GCP Pub/Sub, AWS EventBridge)
  • Feed received events into the same processing pipeline as extracted records
  • Authentication and validation for incoming webhooks

Design Considerations

  • Event receivers complement extractors, they don't replace them. Full extraction provides the baseline; events provide real-time updates.
  • Should support running alongside batch extraction in the same Meteor process
  • Events should carry enough metadata to route to the right graph update

Why

Batch extraction on a 1-hour schedule means the graph can be up to 1 hour stale. Event receivers close this gap for sources that support push notifications, making the graph near-real-time for critical changes.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions