-
Notifications
You must be signed in to change notification settings - Fork 44
Open
Description
Context
Not every metadata change needs a full extraction cycle. Many sources can push notifications — a GitHub webhook on a schema migration merge, a PagerDuty alert when an incident opens, a cloud event when a resource is created. Meteor should be able to receive these events and convert them into graph updates.
Scope
- Lightweight HTTP server for receiving webhooks (
meteor receiveor built intometeor run) - Pluggable event handler interface — map incoming events to asset/relationship mutations
- Built-in handlers for common sources:
- GitHub webhooks (push, PR merge, release)
- Generic cloud events (GCP Pub/Sub, AWS EventBridge)
- Feed received events into the same processing pipeline as extracted records
- Authentication and validation for incoming webhooks
Design Considerations
- Event receivers complement extractors, they don't replace them. Full extraction provides the baseline; events provide real-time updates.
- Should support running alongside batch extraction in the same Meteor process
- Events should carry enough metadata to route to the right graph update
Why
Batch extraction on a 1-hour schedule means the graph can be up to 1 hour stale. Event receivers close this gap for sources that support push notifications, making the graph near-real-time for critical changes.
References
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels