Pipeline Settings
This guide explains how to manage pipeline schedules and execution options. Settings are accessible through the Settings (gear icon) on the workflow editor's top toolbar.
Settings Dialog
The Settings dialog consists of two tabs.
Schedule Tab
Configure the periodic execution schedule for the pipeline. This is exclusive to Batch pipelines; this tab is hidden for Event pipelines.
- Preset Selection: Choose from Every Minute, Hourly, Daily, Weekly, Monthly, or Custom
- Cron Expression: Enter a Cron expression directly when Custom is selected
- Delete Schedule: Remove a registered schedule with the Remove button
For detailed schedule configuration, refer to the Scheduling documentation.
Options Tab
Configure the pipeline execution environment.
| Item | Description |
|---|---|
| Docker Image | Container image to use for pipeline execution |
If custom packages are needed, you can build and specify a separate Docker image. The default image includes major data processing libraries such as pandas, polars, and pyarrow.
Pipeline Metadata
Basic information such as the pipeline name and description can be edited directly from the editor's top toolbar.
Renaming
Click the pipeline title directly for inline editing.
AI Auto-generation
AI-powered automatic description and tag generation is also available for pipelines.
Individual Node Settings
The write mode can be configured individually for each Dataset output node. Check the following options in the Options tab of the Inspector panel:
| Mode | Description |
|---|---|
| Append | Add to existing data |
| Overwrite | Overwrite existing data |
| Merge | Merge by key (Upsert) |
Write mode is configured in the Options tab of each individual Dataset output node.