diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 422c6f540..0de25b417 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -24,6 +24,7 @@ ** xref:get-started:cluster-types/create-dedicated-cloud-cluster.adoc[] * xref:ai-agents:index.adoc[Agentic AI] +** xref:ai-agents:adp-overview.adoc[ADP Overview] ** xref:ai-agents:mcp/index.adoc[MCP] *** xref:ai-agents:mcp/overview.adoc[MCP Overview] *** xref:ai-agents:mcp/remote/index.adoc[Remote MCP] diff --git a/modules/ai-agents/pages/adp-overview.adoc b/modules/ai-agents/pages/adp-overview.adoc new file mode 100644 index 000000000..7427b9817 --- /dev/null +++ b/modules/ai-agents/pages/adp-overview.adoc @@ -0,0 +1,74 @@ += Redpanda Agentic Data Plane Overview +:description: Enterprise-grade infrastructure for building, deploying, and governing AI agents at scale with compliance-grade audit trails. +:page-topic-type: overview +:page-categories: AI, Agentic AI +:personas: evaluator, ai_agent_developer, platform_admin +:learning-objective-1: Identify the key components of Redpanda ADP and their purposes +:learning-objective-2: Describe how each component addresses enterprise governance and reliability requirements +:learning-objective-3: Determine whether Redpanda ADP fits your organization's requirements for AI agent deployment + +glossterm:AI agent[,AI agents] are moving from demos to production. Enterprises need governance, reliability, and cost control to deploy them safely. Redpanda Agentic Data Plane (ADP) combine a streaming-native immutable log, 300+ proven data connectors, and declarative glossterm:AI agent[,AI agents] into a unified platform with built-in compliance-grade audit trails. + +After reading this page, you will be able to: + +* [ ] {learning-objective-1} +* [ ] {learning-objective-2} +* [ ] {learning-objective-3} + +NOTE: The Agentic Data Plane is supported on BYOC clusters running with AWS and Redpanda version 25.3 and later. + +== AI Agents + +Instead of writing Python or JavaScript, you declare the agent behavior you want and Redpanda handles execution and orchestration. You define behaviors in YAML, orchestrate multiple specialized glossterm:subagent[,sub-agents], or bring your own frameworks like LangChain or LlamaIndex. + +What makes this practical at scale is xref:develop:connect/about.adoc[Redpanda Connect]. More than 300 connectors with built-in filtering, enrichment, and routing give declarative definitions real power. Upcoming templates will provide default behaviors for common domains such as customer success, legal, and finance. + +The result is faster time-to-production, lower maintenance (declarative definitions instead of imperative code), and organizational consistency across teams. + +== MCP Servers + +glossterm:MCP server[,MCP servers] translate agent intent into connections to databases, queues, HRIS, CRMs, and other business systems. They are the simplest way to give agents context and capabilities without writing glue code. + +Under the hood, MCP servers wrap the same proven connectors that power some of the world's largest e-commerce, EV, electricity, and AI companies. Built on xref:develop:connect/about.adoc[Redpanda Connect], they are lightweight, support OIDC-based authentication, and enforce deterministic policies at the tool level. You define tools in YAML, and policy enforcement programmatically prevents prompt injection, SQL injection, and other agent-based attacks. + +With over 300 connectors out-of-the-box and real-time debugging capabilities, you reduce integration time while getting enterprise-grade security. You can reuse your existing infrastructure and data sources rather than building new integrations from scratch. + +== Transcripts + +Every agent action is recorded in an end-to-end execution log. A single glossterm:transcript[] can span multiple agents, tools, and models, covering interactions that last minutes to days. + +Transcripts are the keystone of agent governance. They are built on Redpanda's immutable log with Raft consensus and TLA+ correctness proofs. No gaps, no tampering. For regulated industries that require multi-year audit trails, this provides a compliance-grade record of every decision an agent makes and every data source it uses. + +Redpanda captures 100% of agent actions through OpenTelemetry standards, with end-to-end lineage across the entire execution chain. You can materialize execution logs to Iceberg tables for long-term retention and analysis, or replay them to evaluate and improve agent performance over time. + +== AI Gateway + +The AI Gateway manages LLM provider access with two priorities: keeping your application up and keeping costs under control. + +For high availability, the gateway provides provider-agnostic routing with intelligent failover. Your users don't care which provider serves a request. They care that the application stays up. For fiscal control, you get per-tenant budgets and rate limiting, so there are no runaway costs and no surprise bills. + +The gateway also supports tenancy modeling for teams, individuals, applications, and service accounts, giving you chargeback transparency for internal cost allocation. You can proxy both models and MCP gateways, centralizing compliance for all LLM interactions without locking into any single provider. + +== Enterprise governance + +Redpanda ADP addresses critical enterprise requirements across all components. + +*Security by design*:: MCP servers enforce policies at the tool level, programmatically preventing prompt injection, SQL injection, and other agent-based attacks. Policy enforcement is deterministic and controlled. Agents cannot bypass security constraints even through creative prompting. + +*Unified authorization*:: All components use OIDC-based authentication with an on-behalf-of authorization model. When a user invokes an agent, the agent inherits the intersection of its own permissions and the user's permissions. This ensures proper data access scoping. + +*Complete observability*:: Redpanda ADP provides two levels of inspection. Execution logs (transcripts) capture every agent action with 100% sampling using OpenTelemetry standards. Real-time debugging tools allow you to inspect individual MCP server calls down to individual tool invocations with full timing data. You can view detailed agent actions in glossterm:Redpanda Console[] and replay data for agent evaluations. + +*Compliance and audit*:: For industries requiring multi-year audit trails, Redpanda ADP records every agent action and data source used in decision-making. Execution logs are stored in Redpanda topics and can be materialized to Iceberg tables for long-term retention and analysis. + +== Use cases + +Organizations use Redpanda ADP to: + +* *Automate operational workflows*: Create specialized agents for building management, infrastructure monitoring, compliance reporting, and other domain-specific tasks. +* *Monitor manufacturing and operations*: Deploy multi-agent systems that analyze factory machine telemetry in real-time, detect anomalies, search equipment manuals, and create maintenance tickets automatically. +* *Extend enterprise productivity tools*: Integrate Microsoft Copilot or other workplace agents with internal data sources and systems that are otherwise inaccessible. + +== Next steps + +* xref:ai-agents:mcp/overview.adoc[] diff --git a/modules/ai-agents/pages/index.adoc b/modules/ai-agents/pages/index.adoc index 9ac867a96..73ea4dc12 100644 --- a/modules/ai-agents/pages/index.adoc +++ b/modules/ai-agents/pages/index.adoc @@ -1,8 +1,4 @@ -= AI Agents in Redpanda Cloud -:description: Learn about AI agents and the tools Redpanda Cloud provides for building them. += Redpanda Agentic Data Plane +:description: Redpanda Agentic Data Plane (ADP) provides enterprise-grade infrastructure for building, deploying, and governing AI agents at scale with enterprise governance, cost controls, and compliance-grade audit trails. :page-layout: index :page-aliases: develop:agents/about.adoc, develop:ai-agents/about.adoc - -AI agents are configurable assistants that autonomously perform specialist tasks by leveraging large language models (LLMs) and connecting to external data sources and tools. - -Redpanda Cloud provides two complementary Model Context Protocol (MCP) options to help you build AI agents. diff --git a/modules/get-started/pages/cloud-overview.adoc b/modules/get-started/pages/cloud-overview.adoc index 57b85e371..6602ce9f7 100644 --- a/modules/get-started/pages/cloud-overview.adoc +++ b/modules/get-started/pages/cloud-overview.adoc @@ -1,14 +1,27 @@ = Redpanda Cloud Overview -:description: Learn about Redpanda Serverless, Bring Your Own Cloud (BYOC), and Dedicated clusters. +:description: Learn about the Redpanda Agentic Data Plane (ADP) and deployment options including BYOC, Dedicated, and Serverless clusters. :page-aliases: cloud:dedicated-byoc.adoc, deploy:deployment-option/cloud/dedicated-byoc.adoc, deploy:deployment-option/cloud/cloud-overview.adoc -Redpanda Cloud is a complete data streaming platform delivered as a fully-managed service. It provides automated upgrades and patching, data balancing, and support while continuously monitoring your clusters and underlying infrastructure to meet strict performance, availability, reliability, and security requirements. All Redpanda Cloud clusters are deployed with an integrated glossterm:Redpanda Console[], and all clusters have access to unlimited retention and 300+ data connectors with xref:develop:connect/about.adoc[Redpanda Connect]. +Redpanda Cloud is a complete data streaming and agentic data plane platform delivered as a fully-managed service. It provides automated upgrades and patching, data balancing, and support while continuously monitoring your clusters and underlying infrastructure to meet strict performance, availability, reliability, and security requirements. All Redpanda Cloud clusters are deployed with an integrated glossterm:Redpanda Console[], and all clusters have access to unlimited retention and 300+ data connectors with xref:develop:connect/about.adoc[Redpanda Connect]. -TIP: For more detailed information about the Redpanda platform, see xref:get-started:intro-to-events.adoc[] and xref:get-started:architecture.adoc[]. +== Redpanda Agentic Data Plane (ADP) -== Redpanda Cloud cluster types +Redpanda Agentic Data Plane (ADP) is enterprise-grade infrastructure for building, deploying, and governing glossterm:AI agent[,AI agents] at scale. It combines Redpanda's streaming-native immutable log, 300+ proven data connectors, and declarative agent definitions into a unified platform with built-in governance, cost controls, and compliance-grade audit trails. -Redpanda Cloud offers three fully-managed cloud deployment options, each designed for different use cases: +Redpanda ADP includes the following key components: + +* *AI Agents*: Declare the behavior you want instead of writing code. Redpanda powers declarative definitions with 300+ connectors. +* *MCP Servers*: Translate agent intent into connections to your business systems using proven connectors, no glue code required. +* *Transcripts*: End-to-end execution records built on an immutable log with formal correctness guarantees. Transcripts are the keystone of agent governance. +* *AI Gateway*: High-availability model routing with fiscal controls and per-tenant cost attribution across LLM providers. + +NOTE: The Agentic Data Plane is supported on BYOC clusters running with AWS and Redpanda version 25.3 and later. + +To learn what makes Redpanda's approach unique, see xref:ai-agents:adp-overview.adoc[]. + +== Redpanda Cloud deployment options + +Redpanda Cloud applications are supported by three fully-managed deployment options: * **<>**: Fastest way to get started with automatic scaling * **<>**: Production clusters in Redpanda's cloud with enhanced isolation @@ -33,6 +46,11 @@ Redpanda Cloud offers three fully-managed cloud deployment options, each designe | Redpanda's cloud (AWS/Azure/GCP) | Your cloud account (AWS/Azure/GCP) +| *Redpanda ADP* +| ✗ +| ✗ +| ✓ + | *Tenancy* | Multi-tenant | Single-tenant @@ -127,8 +145,7 @@ include::get-started:partial$get-started-serverless.adoc[] === Dedicated -With Dedicated clusters, you host your data on Redpanda Cloud resources (AWS, GCP, or Azure), and Redpanda handles provisioning, operations, and maintenance. Dedicated clusters are single-tenant deployments that support private networking (for example, VPC peering to talk over private IPs) for better data isolation. -When you create a Dedicated cluster, you select the supported xref:reference:tiers/dedicated-tiers.adoc[tier] that meets your compute and storage needs. +With Dedicated clusters, you host your data on Redpanda Cloud resources (AWS, GCP, or Azure), and Redpanda handles provisioning, operations, and maintenance. When you create a Dedicated cluster, you select the supported xref:reference:tiers/dedicated-tiers.adoc[tier] that meets your compute and storage needs. ==== Sign up for Dedicated @@ -152,6 +169,8 @@ Redpanda creates a cloud organization for you and sends you a welcome email. === Bring Your Own Cloud (BYOC) +With BYOC clusters, the Redpanda data plane (including Redpanda ADP components and Redpanda brokers) deploys into your existing VPC or VNet, ensuring all data remains in your environment. + With BYOC clusters, you deploy the Redpanda glossterm:data plane[] into your existing VPC (for AWS and GCP) or VNet (for Azure), and all data is contained in your own environment. This provides an additional layer of security and isolation. (See xref:get-started:byoc-arch.adoc[].) Redpanda manages provisioning, monitoring, upgrades, and security policies, and it manages required resources in your VPC or VNet, including subnets (subnetworks in GCP), IAM roles, and object storage resources (for example, S3 buckets or Azure Storage accounts). @@ -161,6 +180,8 @@ With BYOVPC/BYOVNet clusters, you take full control of the networking lifecycle. The BYOC infrastructure that Redpanda manages should not be used to deploy any other workloads. +For details about the control plane - data plane framework in BYOC, see xref:get-started:byoc-arch.adoc[BYOC architecture]. + ==== Sign up for BYOC To start using BYOC, contact https://redpanda.com/try-redpanda?section=enterprise-trial[Redpanda sales^] to request a private offer with possible discounts. You are billed directly or through Google Cloud Marketplace or AWS Marketplace. @@ -176,6 +197,7 @@ Serverless clusters are a good fit for the following use cases: Consider BYOC or Dedicated if you need more control over the deployment or if you have workloads with consistently-high throughput. BYOC and Dedicated clusters offer the following features: +* Redpanda Agentic Data Plane (ADP): BYOC only * Multiple availability zones (AZs). A multi-AZ cluster provides higher resiliency in the event of a failure in one of the zones. * Role-based access control (RBAC) in the data plane * Kafka Connect @@ -185,7 +207,11 @@ Consider BYOC or Dedicated if you need more control over the deployment or if yo When you sign up for a Redpanda account, Redpanda creates an organization for you. Your organization contains all your Redpanda resources, including your clusters and networks. Within your organization, Redpanda creates a default resource group to contain your resources. You can rename this resource group, and you can create more resource groups. For example, you may want different resource groups for production and testing. -For details about the control plane - data plane framework in BYOC, see xref:get-started:byoc-arch.adoc[BYOC architecture]. +[TIP] +==== + +For more detailed information about the Redpanda platform, see xref:get-started:intro-to-events.adoc[] and xref:get-started:architecture.adoc[]. +==== == Shared responsibility model @@ -388,14 +414,14 @@ Features in beta are available for testing and feedback. They are not covered by The following features are currently in beta in Redpanda Cloud: -* Serverless on GCP * BYOVPC for AWS * BYOVNet for Azure * Secrets Management for BYOVPC on GCP and AWS -* xref:ai-agents:index.adoc[AI agents with MCP servers] +* xref:ai-agents:index.adoc[Redpanda ADP] including AI agents, MCP servers, AI Gateway, and Transcripts * Several Redpanda Connect components == Next steps +* xref:ai-agents:index.adoc[Build AI agents with Redpanda ADP] * xref:manage:maintenance.adoc[Learn about upgrades and maintenance] * xref:get-started:cluster-types/serverless.adoc[Create a Serverless cluster] * xref:get-started:cluster-types/byoc/index.adoc[Create a BYOC cluster] diff --git a/modules/get-started/pages/whats-new-cloud.adoc b/modules/get-started/pages/whats-new-cloud.adoc index 1231b4d32..c1174c9ef 100644 --- a/modules/get-started/pages/whats-new-cloud.adoc +++ b/modules/get-started/pages/whats-new-cloud.adoc @@ -8,6 +8,12 @@ This page lists new features added to Redpanda Cloud. == February 2026 +=== Agentic Data Plane (ADP) + +Redpanda Agentic Data Plane (ADP) provides enterprise-grade infrastructure for building, deploying, and governing AI agents at scale. Key capabilities include declarative agents, MCP servers backed by 300+ connectors, an AI Gateway with model failover and fiscal controls, and compliance-grade transcripts built on Redpanda's immutable log. Redpanda ADP is now available on BYOC clusters (AWS, Redpanda 25.3+). + +See xref:ai-agents:adp-overview.adoc[Agentic Data Plane Overview]. + === Serverless on AWS: GA xref:get-started:cluster-types/serverless.adoc[Serverless] on AWS is now generally available (GA). This release includes private networking with AWS PrivateLink. You can use the Cloud Console, the Cloud API, or the Redpanda Terraform provider to create and manage Serverless private links. Serverless is the easiest and fastest way to begin streaming data with Redpanda.