Topics — Confluent Self-Service Docs

Overview

Topics are the fundamental unit of organization in Apache Kafka. They represent a category or feed name to which records are published. Topics in Kafka are always multi-subscriber; that is, a topic can have zero, one, or many consumers that subscribe to the data written to it.

The Self-Service API allows you to create, configure, and manage topics within your business unit's Confluent Cloud environment.

Confluent Documentation

Topic Naming Convention

Topic names are automatically generated based on your cluster type:

Dedicated Clusters


        {business_unit}-{stage}-{id}

Example: sales-dev-crm-orders

Shared Clusters


        {landing_zone}-{business_unit}-{stage}-{id}

Example: edh-shared-scada-dev-crm-orders

landing_zone: Shared cluster landing zone identifier (e.g., "edh-shared")

business_unit: Your organization's identifier (e.g., "scada", "sales")

stage: Environment stage (dev, qas, or run)

id: Descriptive name for your topic (e.g., "crm-orders", "user-events")

Note

When creating a topic, you only need to provide the identifier. The system automatically normalizes it (lowercase, hyphens instead of underscores/dots/spaces) and adds the appropriate prefix based on your cluster configuration.

Creating Topics

When creating a topic, you can configure the following settings:

Partitions

The number of partitions determines parallelism. More partitions allow more consumers to read in parallel. Default: 1. Valid range: 1-100. Consider your throughput requirements when setting this value.

Retention Period

How long messages are retained before being deleted. Set via retention.ms in milliseconds. Default: 604800000 (7 days).

Cleanup Policy

delete: Old segments are deleted when retention time/size is reached.
compact: Kafka retains only the latest value for each key.
compact,delete: Combination of both policies.

Topic Configuration

Common configuration options for topics:

Configuration	Description	Default
`retention.ms`	Message retention time in milliseconds	604800000 (7 days)
`retention.bytes`	Maximum size of the log before deletion	-1 (unlimited)
`cleanup.policy`	Log cleanup policy (delete/compact)	delete
`max.message.bytes`	Maximum size of a message	1048588

Tableflow

Topics with a registered value schema (Data Contract) can be continuously materialized into an open table format (Apache Iceberg or Delta Lake) using Tableflow. Once enabled, analytics engines can query the topic data directly — without a Kafka consumer.

The Enable Tableflow button is available on each topic's detail page. See the Tableflow documentation for full setup instructions, prerequisites, and storage options.

Requires platform activation

Tableflow must be enabled at the landing zone level before it can be used on any topic. This is a one-time setup that the Platform Team must perform for each environment. If the Enable Tableflow button is disabled on your topic's detail page, reach out to the Platform Team at streaming-platform@eon.com and specify the target environment (dev / qas / run). You can check whether Tableflow has been activated for your current environment on the Info page.

Best Practices

Use meaningful, descriptive identifiers (e.g., "crm-orders", "user-events")
Start with fewer partitions and scale up as needed (partitions cannot be reduced)
Set appropriate retention based on your data requirements and compliance needs
Use compacted topics for stateful data where only the latest value matters
Consider message size limits when designing your event schemas
Register a value schema for topics that will use Tableflow
Document your topics and their intended use cases