Manage pipelines

Learn how to:

Create a pipeline

Pipelines execute SQL statements that define how data flows from streams to sinks.

To create a pipeline, run the pipelines create command:

npx wrangler pipelines create my-pipeline \
  --sql "INSERT INTO my_sink SELECT * FROM my_stream"

You can also provide SQL from a file:

npx wrangler pipelines create my-pipeline \
  --sql-file pipeline.sql

Alternatively, to use the interactive setup wizard that helps you configure a stream, sink, and pipeline, run the pipelines setup command:

npx wrangler pipelines setup

Pipelines support SQL statements for data transformation. For complete syntax, supported functions, and data types, see the SQL reference.

Common patterns include:

Transfer all data from stream to sink:

INSERT INTO my_sink SELECT * FROM my_stream

Filter events based on conditions:

INSERT INTO my_sink
SELECT * FROM my_stream
WHERE event_type = 'purchase' AND amount > 100

Choose only the fields you need:

INSERT INTO my_sink
SELECT user_id, event_type, timestamp, amount
FROM my_stream

Apply transformations to fields:

INSERT INTO my_sink
SELECT
  user_id,
  UPPER(event_type) as event_type,
  timestamp,
  amount * 1.1 as amount_with_tax
FROM my_stream

In the Cloudflare dashboard, go to the Pipelines page.
Select a pipeline to view its SQL transformation, connected streams/sinks, and associated metrics.

To view a specific pipeline, run the pipelines get command:

npx wrangler pipelines get <PIPELINE_ID>

To list all pipelines in your account, run the pipelines list command:

npx wrangler pipelines list

Deleting a pipeline stops data flow from the connected stream to sink.

In the Cloudflare dashboard, go to the Pipelines page.
Select the pipeline you want to delete. 3. In the Settings tab, and select Delete.

To delete a pipeline, run the pipelines delete command:

npx wrangler pipelines delete <PIPELINE_ID>

Pipeline SQL cannot be modified after creation. To change the SQL transformation, you must delete and recreate the pipeline.