How-to
Apply a topic action
Stream processing without SQL — worked example: remove duplicates
What you'll achieve
A new topic derived from an existing one, produced by a topic action — a guided wizard, no SQL. This walkthrough uses Deduplicate, which creates an exact copy of a topic with duplicate records removed; Transform and Mask Fields work the same way.
Before you start
- Your role badge says Developer or higher and the environment allows changes — otherwise the Actions button doesn't appear at all.
- The topic has a value schema (data contract) — actions read the field list from it. If the menu item is greyed out saying "Needs a value schema", do Attach a schema to a topic first.
- Your environment has a Flink compute pool and a Flink Developer service account — the wizard tells you plainly if either is missing (both are admin territory; see Create a service account).
Steps
- Open your topic and click Actions (top right), then Deduplicate.
- In Deduplicate on, pick the field that identifies duplicates — records sharing this field's value count as the same record, and the earliest one is kept. Click Continue.
- On the Output step: type an Output topic name (your team's prefix is added automatically — the New topic name preview shows the result), pick the Compute pool, and under Run as pick the Flink service account. Click Continue.
- The Review & deploy step describes the outcome in plain language — read it; the curious can peek at the generated SQL under View SQL. Click Deploy action.
What success looks like
A success screen — "Running" (or "Starting up") — tells you new records are appearing in your output topic. View statement opens the running job's page; it also shows up under Flink → Statements with a "Topic action" badge, and the new topic appears in your Topics list. The job keeps running — and costing — until you stop it, so treat it as something you own.
Something went wrong?
- A red note on the Output step: "A compute pool is required to run this action." or "No Flink service account exists for this business unit yet." — both are one-time setup for your team's admin.
- "Couldn't start" after deploying — read the message shown; Edit in SQL Workspace is the escape hatch, or contact the platform team.
- Done with the job? Open it via Flink → Statements and use Stop (pausable) or Delete. Note: deleting the statement does not delete the output topic — remove that separately with Delete a topic if you no longer need it.
Background — all three actions, what they generate, and costs? Read the Flink guide.