Document required Kafka ACLs

[hpedrorodrigues]

First of all, thanks for this project! 🚀

Summary

I just started working with this project and had to configure ACLs for the components that connect to Kafka. I could not find this information anywhere. So, this may be a good starting point:

It assumes:

• Bulker is configured with BULKER_KAFKA_TOPIC_PREFIX=<t-prefix>
• Rotor is configured with ROTOR_KAFKA_CONSUMER_GROUP_ID=Rotor (using this name for simplicity here)
• And all components are configured with:

  KAFKA_DESTINATIONS_TOPIC_NAME: '<t-prefix>-destination-messages'
  KAFKA_DESTINATIONS_RETRY_TOPIC_NAME: '<t-prefix>-destination-messages-retry'
  KAFKA_DESTINATIONS_DEAD_LETTER_TOPIC_NAME: '<t-prefix>-destination-messages-dead-letter'

Overview:

Topic prefix: <t-prefix>
  Producers: <t-prefix>, Ingest
  Consumers: <t-prefix>, Rotor

<t-prefix> here is Bulker. This is just to make the configuration a bit easier since Bulker declares consumer groups based on topic IDs.

ACLs:

• Producers (<t-prefix>, Ingest) need WRITE and CREATE in topics prefixed by <t-prefix>, and DESCRIBE and WRITE in transactional IDs prefixed by <t-prefix>.
• Consumers (<t-prefix>, Rotor) need READ, DESCRIBE, and DESCRIBE_CONFIGS in topics prefixed by <t-prefix>.
• Consumers (<t-prefix>, Rotor) need READ and DESCRIBE in their respective groups (it's easier to declare this in groups prefixed by <t-prefix> and Rotor respectively)

Example (resource pattern is PREFIXED in all rows):

<t-prefix> = data-platform

• User:data-platform on TOPIC data-platform*: WRITE, CREATE, READ, DESCRIBE, DESCRIBE_CONFIGS
• User:data-platform on TRANSACTION_ID data-platform*: DESCRIBE, WRITE
• User:ingest on TOPIC data-platform*: WRITE, CREATE
• User:rotor on TOPIC data-platform*: READ, DESCRIBE, DESCRIBE_CONFIGS
• User:data-platform on GROUP data-platform*: READ, DESCRIBE
• User:rotor on GROUP rotor*: READ, DESCRIBE

Note: this configuration could be a bit simpler if we could use static consumer groups (instead of using topic IDs).

System configuration and versions

Docker images: 2.11.0 (Jitsu Next)
Deployed using the Helm chart: stafftastic/jitsu-chart

Artifacts (logs, etc)

N/A

Notes

Not sure if I'm missing something here. If so, please let me know. Thanks!