Generalize object storage sink to support S3 and Azure Blob #511
Description
Context
The current GCS sink only supports Google Cloud Storage. For archival and compliance use cases, the object storage sink should support all major cloud providers.
Scope
- Refactor the GCS sink into a general object storage sink
- Add support for:
- Amazon S3 (and S3-compatible stores like MinIO)
- Azure Blob Storage
- Maintain GCS support and backward-compatible configuration
- Unified configuration with provider selection
Why
Object storage is one of the four core sinks going forward (Compass, Kafka, object storage, HTTP). It should work with whatever cloud provider the organization uses.