Add SQL Engine (Oxla) workshop documentation#1566
Conversation
doc review feedback Co-authored-by: Michele Cyran <michele@redpanda.com>
Co-authored-by: Michele Cyran <michele@redpanda.com>
Co-authored-by: Michele Cyran <michele@redpanda.com>
doc review feedback Co-authored-by: Michele Cyran <michele@redpanda.com>
# Conflicts: # modules/reference/partials/properties/cluster-properties.adoc
Add comprehensive workshop guide for provisioning and using Redpanda SQL Engine (Oxla) with BYOC clusters. The workshop covers: - Overview of SQL Engine architecture and capabilities - Limitations and prerequisites - BYOC cluster provisioning with SQL Engine - User management and authentication - Creating tables from Redpanda topics with Schema Registry integration - Installing and using psql client - Scaling operations via Fleet Management API - Cluster deletion procedures Organized for a 2-3 hour hands-on workshop with practical examples. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
❌ Deploy Preview for redpanda-docs-preview failed. Why did it fail? →
|
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the
📝 WalkthroughWalkthroughThis PR restructures topic management documentation by reorganizing the documentation hierarchy under a new "manage-topics" folder and creating new documentation pages for Cloud Topics. It updates numerous cross-references throughout the codebase to point to the new documentation locations, removes "exclude_from_docs" flags from several cloud-related properties to make them publicly visible, and enriches property metadata with expanded descriptions and related topic references. Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (6)
modules/develop/pages/manage-topics/config-topics.adoc (1)
110-115:⚠️ Potential issue | 🟡 MinorFix CLI syntax: em dash and ellipsis break copy/paste.
Line 114 uses a Unicode em dash and ellipsis; this will fail in shells. Use ASCII
--setand....✅ Proposed fix
-rpk topic alter-config [TOPICS…] —-set cleanup.policy=compact +rpk topic alter-config [TOPICS...] --set cleanup.policy=compactmodules/reference/partials/properties/topic-properties.adoc (1)
1-1:⚠️ Potential issue | 🟠 MajorDo not edit autogenerated properties partials directly.
This file is autogenerated; updates should be done via the doc-tools overrides so they aren’t overwritten. Based on learnings, never directly edit files in/modules/reference/partials/properties/because they are auto-generated and will be overwritten.Also applies to: 950-985
modules/reference/partials/properties/cluster-properties.adoc (4)
904-957:⚠️ Potential issue | 🟠 MajorAvoid editing autogenerated property partials; move these updates to overrides.
Edits in this file will be overwritten. Please apply the Cloud Topics text/related topics changes indocs-data/property-overrides.jsoninstead, and let the generator update this file. Based on learnings: "Applies to /modules/reference/partials/properties/*.adoc : Never directly edit files in/modules/reference/partials/properties/- they are auto-generated and will be overwritten".Use empty-bracket xrefs in the overrides.
In the overrides, preferxref:...[]so link text is pulled from the target title. Based on learnings: "AsciiDoc linking: prefer using xref links with empty brackets (e.g., xref:section/target.adoc[]) because the title is pulled from the referenced document automatically."
1192-1242:⚠️ Potential issue | 🟠 MajorAutogenerated file edit — move to overrides.
The Cloud Topics upload interval description/related topics should be set indocs-data/property-overrides.jsonrather than this generated partial. Based on learnings: "Applies to /modules/reference/partials/properties/*.adoc : Never directly edit files in/modules/reference/partials/properties/- they are auto-generated and will be overwritten".Use empty-bracket xrefs in the overrides.
The related topics should bexref:...[](no hard‑coded link text) when you move them into overrides. Based on learnings: "AsciiDoc linking: prefer using xref links with empty brackets (e.g., xref:section/target.adoc[]) because the title is pulled from the referenced document automatically."
1246-1295:⚠️ Potential issue | 🟠 MajorAutogenerated file edit — move to overrides.
The reconciliation interval copy/related topics should be placed indocs-data/property-overrides.json. Based on learnings: "Applies to /modules/reference/partials/properties/*.adoc : Never directly edit files in/modules/reference/partials/properties/- they are auto-generated and will be overwritten".Use empty-bracket xrefs in the overrides.
Please switch toxref:...[]when defining the related topics in overrides. Based on learnings: "AsciiDoc linking: prefer using xref links with empty brackets (e.g., xref:section/target.adoc[]) because the title is pulled from the referenced document automatically."
19837-19842:⚠️ Potential issue | 🟠 MajorAutogenerated file edit — move to overrides.
The updated write caching related topic link should be applied viadocs-data/property-overrides.json, not directly here. Based on learnings: "Applies to /modules/reference/partials/properties/*.adoc : Never directly edit files in/modules/reference/partials/properties/- they are auto-generated and will be overwritten".Use empty-bracket xrefs in the overrides.
Usexref:develop:manage-topics/config-topics.adoc#configure-write-caching[]in overrides to avoid hard‑coded text. Based on learnings: "AsciiDoc linking: prefer using xref links with empty brackets (e.g., xref:section/target.adoc[]) because the title is pulled from the referenced document automatically."
🤖 Fix all issues with AI agents
In `@modules/get-started/pages/sql-engine-workshop.adoc`:
- Line 82: The load balancer type values used in the redpanda-proto JSON (the
"load_balancer": {"type": 1} / {"type": 2} entries) are undocumented; update the
content around the JSON examples to explicitly state what each numeric value
means (e.g., type: 1 = internal cluster-only load balancer, type: 2 =
external/public load balancer) and either add a short explanatory sentence near
the snippet or convert the example to include inline comments showing the
mapping so readers know which value to use; ensure you update both occurrences
that use "type": 1 / "type": 2.
- Line 96: Replace the vague step with a specific command and location: on the
agent VM change into the infrastructure module directory (e.g., cd
./terraform/infrastructure or the module path matching "infrastructure" in the
repo), then run the Terraform workflow: terraform init && terraform apply
-var-file=secrets.tfvars -auto-approve (or omit -auto-approve if you want
interactive confirmation); if your setup uses a specific workspace run terraform
workspace select <name> first. Mention that module path may vary and give the
example pattern "./terraform/<module-name>" and note that any required variable
file (e.g., secrets.tfvars) or environment variables must be present before
running the command.
In `@modules/reference/attachments/redpanda-properties-v25.3.3.json`:
- Around line 10967-10976: The cross-reference uses a mismatched anchor
<<cloudtopicsenabled,`cloud_topics_enabled`>> causing a broken link; update the
inline xref to use the actual property name anchor (e.g.
<<cloud_topics_enabled,`cloud_topics_enabled`>>) and ensure the target property
definition has a matching anchor like [[cloud_topics_enabled]]; modify the JSON
entry for "name": "redpanda.cloud_topic.enabled" to replace the related_topics
xref string accordingly and add or rename the anchor for the cluster property
`cloud_topics_enabled` so the link resolves.
🧹 Nitpick comments (14)
modules/manage/pages/cluster-maintenance/disk-utilization.adoc (1)
88-88: Prefer xref with empty brackets to auto-title.This link hard-codes the label; use empty brackets so the published title is pulled automatically.
♻️ Proposed change
-* xref:develop:manage-topics/config-topics.adoc#delete-records-from-a-topic[Delete records from a topic] +* xref:develop:manage-topics/config-topics.adoc#delete-records-from-a-topic[]Based on learnings: “AsciiDoc linking: prefer using xref links with empty brackets (e.g., xref:section/target.adoc[]) because the title is pulled from the referenced document automatically. Avoid hard-coding link text; use xref:...[] to let the target document's title render as the link text when publishing.”
modules/reference/pages/rpk/rpk-redpanda/rpk-redpanda-mode.adoc (1)
13-13: Consider using empty brackets for automatic title rendering.Per AsciiDoc best practices, prefer using empty brackets in xref links to automatically pull the section title from the target document. Based on learnings, this helps maintain consistency when target titles change.
📝 Suggested improvement
-** Enables write caching, which is a relaxed mode of `acks=all` that acknowledges a message as soon as it is received and acknowledged on a majority of brokers, without waiting for it to fsync to disk. This provides lower latency while still ensuring that a majority of brokers acknowledge the write. For more information, or to disable this, see xref:develop:manage-topics/config-topics.adoc#configure-write-caching[write caching]. +** Enables write caching, which is a relaxed mode of `acks=all` that acknowledges a message as soon as it is received and acknowledged on a majority of brokers, without waiting for it to fsync to disk. This provides lower latency while still ensuring that a majority of brokers acknowledge the write. For more information, or to disable this, see xref:develop:manage-topics/config-topics.adoc#configure-write-caching[].Based on learnings: "AsciiDoc linking: prefer using xref links with empty brackets (e.g., xref:section/target.adoc[]) because the title is pulled from the referenced document automatically."
modules/manage/pages/cluster-maintenance/topic-property-configuration.adoc (1)
221-224: Prefer empty-bracket xref here.Use
xref:develop:manage-topics/config-topics.adoc[]so the title is sourced from the target document.♻️ Proposed fix
-* xref:develop:manage-topics/config-topics.adoc[Manage Topics] - Create and manage topics +* xref:develop:manage-topics/config-topics.adoc[] - Create and manage topicsBased on learnings: AsciiDoc linking: prefer using xref links with empty brackets (e.g., xref:section/target.adoc[]) because the title is pulled from the referenced document automatically. Avoid hard-coding link text; use xref:...[] to let the target document's title render as the link text when publishing.
modules/develop/pages/produce-data/configure-producers.adoc (1)
72-78: Use empty-bracket xref.Prefer
xref:develop:manage-topics/config-topics.adoc#configure-write-caching[]and keep the label outside the link text if needed.♻️ Proposed fix
-using xref:develop:manage-topics/config-topics.adoc#configure-write-caching[write caching], which is a relaxed mode of `acks=all` ... +using xref:develop:manage-topics/config-topics.adoc#configure-write-caching[] (write caching), which is a relaxed mode of `acks=all` ...Based on learnings: AsciiDoc linking: prefer using xref links with empty brackets (e.g., xref:section/target.adoc[]) because the title is pulled from the referenced document automatically. Avoid hard-coding link text; use xref:...[] to let the target document's title render as the link text when publishing.
modules/manage/pages/cluster-maintenance/compaction-settings.adoc (1)
13-16: Prefer empty-bracket xref.Use
xref:develop:manage-topics/config-topics.adoc#change-the-cleanup-policy[]and keep the label outside the link.♻️ Proposed fix
-NOTE: If your application requires consuming every record for a given key, consider using the `delete` xref:develop:manage-topics/config-topics.adoc#change-the-cleanup-policy[cleanup policy] instead. +NOTE: If your application requires consuming every record for a given key, consider using the `delete` xref:develop:manage-topics/config-topics.adoc#change-the-cleanup-policy[] instead.Based on learnings: AsciiDoc linking: prefer using xref links with empty brackets (e.g., xref:section/target.adoc[]) because the title is pulled from the referenced document automatically. Avoid hard-coding link text; use xref:...[] to let the target document's title render as the link text when publishing.
modules/manage/pages/disaster-recovery/shadowing/overview.adoc (1)
171-174: Prefer empty-bracket xref.Use
xref:develop:manage-topics/config-topics.adoc#configure-write-caching[]and keep the label outside the link.♻️ Proposed fix
-* **Avoid write caching on source topics**: Do not shadow source topics that have xref:develop:manage-topics/config-topics.adoc#configure-write-caching[write caching] enabled. +* **Avoid write caching on source topics**: Do not shadow source topics that have xref:develop:manage-topics/config-topics.adoc#configure-write-caching[] (write caching) enabled.Based on learnings: AsciiDoc linking: prefer using xref links with empty brackets (e.g., xref:section/target.adoc[]) because the title is pulled from the referenced document automatically. Avoid hard-coding link text; use xref:...[] to let the target document's title render as the link text when publishing.
modules/develop/pages/kafka-clients.adoc (1)
67-67: Use empty-bracket xref for auto-titled links.
Replace the explicit link text withxref:...[]to keep titles sourced from the target page. As per coding guidelines, prefer xref links with empty brackets so the title is pulled from the referenced document automatically.Suggested fix
-* Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group] using AlterClientQuotas and DescribeClientQuotas APIs are supported. +* Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[] using AlterClientQuotas and DescribeClientQuotas APIs are supported.modules/develop/pages/manage-topics/cloud-topics.adoc (1)
13-13: Switch xrefs to empty brackets for auto-titled links.
Several new xrefs use explicit link text. Please usexref:...[]to keep titles consistent and localized automatically. As per coding guidelines, prefer xref links with empty brackets so the title is pulled from the referenced document automatically.Suggested fix (examples)
-... Redpanda xref:config-topics.adoc[topics] ... +... Redpanda xref:config-topics.adoc[] ...-For details, see xref:manage:tiered-storage.adoc#enable-tiered-storage-for-a-cluster[Enable Tiered Storage for a cluster]. +For details, see xref:manage:tiered-storage.adoc#enable-tiered-storage-for-a-cluster[].-For more details about using Docker Compose with Redpanda, see the xref:get-started:quick-start.adoc[Redpanda Self-Managed Quickstart]. +For more details about using Docker Compose with Redpanda, see the xref:get-started:quick-start.adoc[].-... recommends using xref:../consume-data/follower-fetching.adoc[Follower Fetching] ... +... recommends using xref:../consume-data/follower-fetching.adoc[] ...Also applies to: 34-34, 56-56, 92-92
modules/deploy/pages/redpanda/manual/production/dev-deployment.adoc (1)
11-11: Use empty-bracket xref for auto-titled links.
Update the link toxref:...[]to follow the doc-linking convention. As per coding guidelines, prefer xref links with empty brackets so the title is pulled from the referenced document automatically.Suggested fix
-... see xref:develop:manage-topics/config-topics.adoc#configure-write-caching[write caching]. +... see xref:develop:manage-topics/config-topics.adoc#configure-write-caching[].modules/get-started/pages/quick-start.adoc (1)
446-446: Use empty-bracket xref for auto-titled links.
Switch toxref:...[]to keep titles consistent and localized. As per coding guidelines, prefer xref links with empty brackets so the title is pulled from the referenced document automatically.Suggested fix
-... see xref:develop:manage-topics/config-topics.adoc#choose-the-replication-factor[Choose the replication factor]. +... see xref:develop:manage-topics/config-topics.adoc#choose-the-replication-factor[].modules/reference/partials/properties/topic-properties.adoc (1)
980-985: Use empty-bracket xrefs in the generated content source/overrides.
These xrefs should usexref:...[]to follow the docs style. Apply this in the overrides/source that generates this file. As per coding guidelines, prefer xref links with empty brackets so the title is pulled from the referenced document automatically.Suggested fix (in overrides/source)
-* xref:develop:manage-topics/cloud-topics.adoc[Cloud Topics] +* xref:develop:manage-topics/cloud-topics.adoc[]-* xref:develop:manage-topics/config-topics.adoc#choose-the-replication-factor[Choose the replication factor] +* xref:develop:manage-topics/config-topics.adoc#choose-the-replication-factor[]-* xref:develop:manage-topics/config-topics.adoc#configure-write-caching[Write caching] +* xref:develop:manage-topics/config-topics.adoc#configure-write-caching[]Also applies to: 1718-1721, 2114-2114
docs-data/property-overrides.json (2)
1693-1702: New xrefs look good; consider removing duplicate link.The new cross-references to
manage-topics.adocuse proper full Antora resource IDs. However, there's a pre-existing duplicate in thisrelated_topicsarray:
- Line 1697:
xref:reference:properties/cluster-properties.adoc#default_topic_replication[...]- Line 1700:
xref:reference:properties/cluster-properties.adoc#default_topic_replication[...]Both point to the same target. Since you're already modifying this array, consider removing the duplicate.
Proposed fix
"related_topics": [ "xref:reference:rpk/rpk-topic/rpk-topic-describe.adoc[`rpk topic describe`]", "xref:reference:rpk/rpk-topic/rpk-topic-alter-config.adoc[`rpk topic alter-config`]", "xref:reference:properties/cluster-properties.adoc#default_topic_replication[`default_topic_replication`]", "xref:develop:manage-topics.adoc#choose-the-replication-factor[Choose the replication factor]", - "xref:develop:manage-topics.adoc#change-the-replication-factor[Change the replication factor]", - "xref:reference:properties/cluster-properties.adoc#default_topic_replication[default_topic_replication]" + "xref:develop:manage-topics.adoc#change-the-replication-factor[Change the replication factor]" ],As per coding guidelines: "Remove duplicate links from related_topics lists in property overrides to keep them clean."
2078-2086: New xref looks good; consider normalizing and deduplicating existing links.Line 2080 correctly uses a full Antora resource ID. However, there are pre-existing issues in this
related_topicsarray:
- Non-normalized xref: Line 2083 uses
xref:cluster-properties.adoc#...which is missing thereference:properties/module prefix.- Duplicate link: Lines 2082 and 2083 both point to
write_caching_default.Since you're modifying this array, consider cleaning these up.
Proposed fix
"related_topics": [ "xref:develop:manage-topics.adoc#configure-write-caching[Write caching]", "xref:manage:tiered-storage.adoc[Tiered Storage]", - "xref:reference:properties/cluster-properties.adoc#write_caching_default[`write_caching_default`]", - "xref:cluster-properties.adoc#write_caching_default[`write_caching_default`]" + "xref:reference:properties/cluster-properties.adoc#write_caching_default[`write_caching_default`]" ],As per coding guidelines: "Always use full Antora resource IDs with module prefixes in xref links" and "Remove duplicate links from related_topics lists."
modules/get-started/pages/sql-engine-workshop.adoc (1)
115-118: Use placeholder text for password examples.The example uses
'secure_password'which may be copied literally by workshop attendees. Consider using a more obvious placeholder that signals it should be replaced.🔐 Proposed fix to clarify password placeholder
-CREATE ROLE data_analyst WITH LOGIN PASSWORD 'secure_password'; +CREATE ROLE data_analyst WITH LOGIN PASSWORD '<your-secure-password>';Optionally, add a note reminding users to use strong passwords:
Connect to SQL Engine and create users with the `CREATE ROLE` statement: +NOTE: Replace `<your-secure-password>` with a strong password. Use a password manager or generate a secure random password. + [,sql]
| ---- | ||
| fleetmgmt operation create --type update \ | ||
| --update-paths 'spec.redpanda_oxla' \ | ||
| --redpanda-proto '{"spec": {"redpanda_oxla": {"enabled": true, "replicas": 1, "load_balancer": {"enabled": true, "type": 1}}}}' \ |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Document the load balancer type values.
The "type": 1 and "type": 2 values for load balancer configuration are not explained in the documentation. Workshop attendees may not understand which value corresponds to internal vs. external load balancers without this context.
📝 Proposed addition to clarify load balancer types
Add an explanation before or after the code examples:
CONNECTION redpanda_prod;
----
+NOTE: Load balancer types: `1` = internal (VPC-only access), `2` = external (public access).
+
**Enable with internal load balancer (VPC access):**Alternatively, add inline comments in the JSON:
- --redpanda-proto '{"spec": {"redpanda_oxla": {"enabled": true, "replicas": 1, "load_balancer": {"enabled": true, "type": 1}}}}' \
+ --redpanda-proto '{"spec": {"redpanda_oxla": {"enabled": true, "replicas": 1, "load_balancer": {"enabled": true, "type": 1}}}}' \ # type 1 = internalAlso applies to: 92-92
🤖 Prompt for AI Agents
In `@modules/get-started/pages/sql-engine-workshop.adoc` at line 82, The load
balancer type values used in the redpanda-proto JSON (the "load_balancer":
{"type": 1} / {"type": 2} entries) are undocumented; update the content around
the JSON examples to explicitly state what each numeric value means (e.g., type:
1 = internal cluster-only load balancer, type: 2 = external/public load
balancer) and either add a short explanatory sentence near the snippet or
convert the example to include inline comments showing the mapping so readers
know which value to use; ensure you update both occurrences that use "type": 1 /
"type": 2.
| --cluster-ids <cluster-id> | ||
| ---- | ||
|
|
||
| After enabling an external load balancer, manually run the appropriate Terraform apply for the infrastructure module on the agent VM. |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Clarify the Terraform apply instruction.
The instruction "manually run the appropriate Terraform apply for the infrastructure module on the agent VM" is too vague for a workshop setting. Attendees need specific guidance on:
- Which Terraform module to apply
- The exact command to run
- Where to run it (which directory)
- Whether additional variables or flags are needed
📝 Proposed fix to add specific instructions
-After enabling an external load balancer, manually run the appropriate Terraform apply for the infrastructure module on the agent VM.
+After enabling an external load balancer, SSH to the agent VM and apply the infrastructure module:
+
+[,bash]
+----
+ssh <agent-vm>
+cd /path/to/terraform/infrastructure-module
+terraform apply
+----
+
+Wait for the operation to complete before connecting to the external endpoint.If the exact path or module name is variable, at least provide a pattern or example.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| After enabling an external load balancer, manually run the appropriate Terraform apply for the infrastructure module on the agent VM. | |
| After enabling an external load balancer, SSH to the agent VM and apply the infrastructure module: | |
| [,bash] | |
| ---- | |
| ssh <agent-vm> | |
| cd /path/to/terraform/infrastructure-module | |
| terraform apply | |
| ---- | |
| Wait for the operation to complete before connecting to the external endpoint. |
🤖 Prompt for AI Agents
In `@modules/get-started/pages/sql-engine-workshop.adoc` at line 96, Replace the
vague step with a specific command and location: on the agent VM change into the
infrastructure module directory (e.g., cd ./terraform/infrastructure or the
module path matching "infrastructure" in the repo), then run the Terraform
workflow: terraform init && terraform apply -var-file=secrets.tfvars
-auto-approve (or omit -auto-approve if you want interactive confirmation); if
your setup uses a specific workspace run terraform workspace select <name>
first. Mention that module path may vary and give the example pattern
"./terraform/<module-name>" and note that any required variable file (e.g.,
secrets.tfvars) or environment variables must be present before running the
command.
| "defined_in": "src/v/config/configuration.cc", | ||
| "description": null, | ||
| "is_deprecated": true, | ||
| "is_enterprise": false, | ||
| "name": "enable_coproc", | ||
| "needs_restart": true, | ||
| "nullable": false, | ||
| "type": "deprecated_property" | ||
| }, | ||
| "enable_developmental_unrecoverable_data_corrupting_fe |
There was a problem hiding this comment.
Fix the Cloud Topics enablement anchor to avoid a broken link.
<<cloudtopicsenabled>> doesn’t match the property name and is inconsistent with the rest of the file; it likely won’t resolve. Use the property name anchor instead.
💡 Suggested fix
- "description": "Enable this topic as a Cloud Topic. Cloud Topics are optimized for high-throughput, cost-sensitive workloads that can tolerate higher latencies compared to standard Kafka topics.\n\nIMPORTANT: You can only set this property when creating a topic. After a topic is created as a Cloud Topic, it cannot be converted back to a standard Kafka topic. Before creating Cloud Topics, you must enable the cluster-wide <<cloudtopicsenabled,`cloud_topics_enabled`>> property.",
+ "description": "Enable this topic as a Cloud Topic. Cloud Topics are optimized for high-throughput, cost-sensitive workloads that can tolerate higher latencies compared to standard Kafka topics.\n\nIMPORTANT: You can only set this property when creating a topic. After a topic is created as a Cloud Topic, it cannot be converted back to a standard Kafka topic. Before creating Cloud Topics, you must enable the cluster-wide <<cloud_topics_enabled,`cloud_topics_enabled`>> property.",Based on learnings: In the Redpanda documentation, topic property cross-references like <<max.compaction.lag.ms>> and <<min.compaction.lag.ms>> require corresponding property definition sections with anchors like [[maxcompactionlagms]] and [[mincompactionlagms]] to prevent broken links.
🤖 Prompt for AI Agents
In `@modules/reference/attachments/redpanda-properties-v25.3.3.json` around lines
10967 - 10976, The cross-reference uses a mismatched anchor
<<cloudtopicsenabled,`cloud_topics_enabled`>> causing a broken link; update the
inline xref to use the actual property name anchor (e.g.
<<cloud_topics_enabled,`cloud_topics_enabled`>>) and ensure the target property
definition has a matching anchor like [[cloud_topics_enabled]]; modify the JSON
entry for "name": "redpanda.cloud_topic.enabled" to replace the related_topics
xref string accordingly and add or rename the anchor for the cluster property
`cloud_topics_enabled` so the link resolves.
| @@ -0,0 +1,444 @@ | |||
| = Redpanda SQL Engine Workshop | |||
There was a problem hiding this comment.
| = Redpanda SQL Engine Workshop | |
| = Workshop: Getting Started with Redpanda SQL Engine in BYOC |
There was a problem hiding this comment.
'Oxla' vs 'SQL Engine' - I think Oxla has been approved as Redpand SQL Engine name due to various reasons, one of them is possibility to uniquely identify the product in search engines or using AI-assisted search. Hence I'd allow for using 'Oxla' along with 'SQL Engine' or interchangeably, across the doc
| ** Map types | ||
| ** Variant types (coming in future release) | ||
| ** Geometry and geography types (limited support) | ||
| * **No consumer groups:** SQL Engine does not use Kafka consumer groups; partition assignment is handled by the query planner |
There was a problem hiding this comment.
CREATE TABLE as well as any ops on tables, like populating them with COPY, altering, dropping etc is disabled in Beta / Preview release. This is an explicit Product requirement to ingest data solely from Redpanda topics. The users can create / alter / drop KAFKA CONNECTIONs and KAFKA SOURCEs, instead. When such a source is created it can be queried in the manner of querying tables
| . Choose your cloud provider (AWS, GCP, or Azure) and region | ||
| . Configure networking details (VPC/VNet, CIDR ranges) | ||
| . Download the Terraform bundle generated by the wizard | ||
| . Extract the bundle locally and run the bootstrap script: |
There was a problem hiding this comment.
When creating BYOC cluster the users have an option to enable Oxla with 'Oxla' Enable/Disable switch
| * `UPDATE` - Modify existing data (not supported for Redpanda-backed tables) | ||
| * `DELETE` - Remove data (not supported for Redpanda-backed tables) | ||
| * `USAGE` - Access schemas and other objects | ||
| * `CREATE` - Create new objects (tables, connections, sources) |
There was a problem hiding this comment.
Regular tables are disabled for Beta/Preview release. Only Redpanda-backed tables are available
| \q | ||
| ---- | ||
|
|
||
| == Scale up or down using Data Plane APIs |
There was a problem hiding this comment.
Dynamic scaling up/down Oxla clusters is not available, yet
|
Closing this PR. SQL Engine is Cloud BYOC-only, so this documentation has been moved to cloud-docs repository. See new PR: redpanda-data/cloud-docs#500 |
|
|
||
| == What is Redpanda SQL Engine? | ||
|
|
||
| Redpanda SQL Engine is a SQL query engine that runs alongside your Redpanda cluster, allowing you to query streaming data using familiar SQL syntax instead of building custom consumers or streaming applications. It transforms your Redpanda cluster into a SQL-addressable analytics surface, making it easier to explore, analyze, and join streaming data with minimal integration overhead. |
There was a problem hiding this comment.
Repetition: 'Redpanda SQL Engine is a SQL query engine '
|
|
||
| SQL Engine has the following limitations in its current release: | ||
|
|
||
| * **Read-only access:** You cannot write to Redpanda-backed tables using `INSERT`, `UPDATE`, or `DELETE` statements |
There was a problem hiding this comment.
It should be mentioned in the limitations section that the user cannot create regular tables with CREATE TABLE, DROP / UPDATE INSERT or COPY on the regular tables os not working, too
| * **Flat structures only:** The first release supports only the `FLATTEN` struct mapping policy; compound types, variants, and JSON mapping are not yet supported | ||
| * **Single-dimensional arrays:** Arrays of arrays and nested complex types are not supported | ||
| * **No streaming queries:** Queries execute in on-demand mode and return results when complete; continuous streaming queries are not supported | ||
| * **Limited type support:** Some Redpanda data types are not supported: |
There was a problem hiding this comment.
There is more types that are are not supported at the moment. Please refer to the proto file I've sent separately. In general any repeated, packed/unpacked, nested and map types are not supported, as well as structs or unions. Singular types are, Timestamp and Date are supported, groups are supported
4 participants
Description
Adds comprehensive workshop documentation for Redpanda SQL Engine (formerly Oxla) in BYOC environments.
This workshop guide is designed for a 2-3 hour hands-on session and covers:
Structure Changes
Reorganized sections based on workshop flow:
Page Preview
Once deployed, the page will be available at:
get-started/sql-engine-workshop.htmlChecks
🤖 Generated with Claude Code