PipelineUpdate

Payload used to update Pipeline. Optional fields include name, alias, type, collection_id, folder_id, and 5 more.

name object

anyOf

• string
• null

string

alias object

anyOf

• string
• null

string

type object

anyOf

• string
• null

string

Possible values: [batch, event]

collection_id object

anyOf

• string
• null

string

Possible values: non-empty

folder_id object

anyOf

• string
• null

string

tags object

anyOf

• string[]
• null

Array [

string

]

description object

anyOf

• string
• null

string

steps object

anyOf

• object[]
• null

Array [

nameName (string)required

Step name

alias object

anyOf

• string
• null

string

description object

Step-owned description, independent of the referenced Code's description so steps sharing one Code can be labeled distinctly.

anyOf

• string
• null

string

script object

Code resource ID to execute (code step)

anyOf

• string
• null

string

transform object

Native transform spec (transform step)

anyOf

• object
• null

oprequired

Possible values: [aggregate, join, rename_cast, select_filter, union]

aggregate

versionVersion (integer)

Constant value: 1

Default value: 1

config objectrequired

Config for the aggregate op: group-by + allowlisted aggregate functions. An empty group_by aggregates the whole input to a single row. input is a port key.

inputInput (string)required

Input port key

group_bystring[]

Grouping columns (empty = whole-input aggregate)

aggregations object[]required

Aggregations (non-empty)

Array [

functionFunction (string)required

Allowlisted aggregate function

Possible values: [sum, count, avg, min, max]

column object

Aggregated column (omit only for count)

anyOf

• string
• null

string

aliasAlias (string)required

Output column name

]

join

versionVersion (integer)

Constant value: 1

Default value: 1

config objectrequired

Two-input equi-join configuration. left_input and right_input are pipeline port keys, keys contains at least one equality pair, and columns explicitly selects the output. Other join predicates are not accepted.

left_inputLeft Input (string)required

Left input port key

right_inputRight Input (string)required

Right input port key

join_typeJoin Type (string)required

inner | left | right | full

Possible values: [inner, left, right, full]

keys object[]required

Equi-join column pairs (non-empty)

Array [

leftLeft (string)required

Join column on the left input

rightRight (string)required

Join column on the right input

]

columns object[]required

Output projection (non-empty)

Array [

inputInput (string)required

Source input port key (left_input or right_input)

nameName (string)required

Source column name on that input

alias object

Output column name (defaults to name)

anyOf

• string
• null

string

]

rename_cast

versionVersion (integer)

Constant value: 1

Default value: 1

config objectrequired

Config for the rename_cast op. Listed columns are renamed / cast; columns not listed pass through unchanged. input is a port key.

inputInput (string)required

Input port key

columns object[]required

Column rename / cast rules (non-empty)

Array [

nameName (string)required

Source column name

alias object

New column name (defaults to name)

anyOf

• string
• null

string

cast object

Allowlisted target type

anyOf

• string
• null

string

Possible values: [utf8, int64, float64, bool]

]

select_filter

versionVersion (integer)

Constant value: 1

Default value: 1

config objectrequired

Select-and-filter transform configuration accepted in a request. input is a pipeline port key, not a resource identifier; columns=null keeps every column, and where=null applies no predicate.

inputInput (string)required

Input port key (a PipelineStep.inputs key)

columns object

Column names returned for this resource.

anyOf

• object[]
• null

Array [

nameName (string)required

Source column name on the input

alias object

Output column name (defaults to name)

anyOf

• string
• null

string

]

where object

Structured row predicate

anyOf

• RowFilter
• null

kindKind (string)

Possible values: [and, or]

Default value: and

conditions object[]

Nested conditions (leaf RowFilterCondition or group RowFilter)

Array [

anyOf

• RowFilterCondition
• Option 2

kindKind (string)

Constant value: condition

Default value: condition

columnColumn (string)required

Column name to filter on

operatorRowFilterOperator (string)required

Allowed Row Filter Operator values. Values: eq, ne, gt, gte, lt, lte, in, not_in, and 4 more.

Possible values: [eq, ne, gt, gte, lt, lte, in, not_in, between, like, is_null, is_not_null]

value object

Filter value. Single value for scalar operators, list for 'in'/'not_in', 2-element list for 'between'. Omit for 'is_null'/'is_not_null'.

anyOf

• Option 1
• null

]

union

versionVersion (integer)

Constant value: 1

Default value: 1

config objectrequired

Union configuration for two or more pipeline input ports. Inputs are aligned by column name, missing values become null, and duplicate rows are retained.

inputsstring[]required

Input port keys to union (>= 2)

inputs object

Input dataset mapping

anyOf

• object
• null

object

outputs object

Output dataset mapping

anyOf

• object
• null

object

dependencies object

List of dependent step names

anyOf

• string[]
• null

Array [

string

]

options object

anyOf

• object
• null

object

secrets object

Secret values to inject into contexts['secrets']

anyOf

• object[]
• null

Array [

nameName (string)required

contexts['secrets'] key the step reads

refRef (string)required

Secret reference, e.g. secret://secret./token

]

]

options object

anyOf

• object
• null

object

metadata object

anyOf

• object
• null

object

PipelineUpdate

{
  "name": "string",
  "alias": "string",
  "type": "batch",
  "collection_id": "string",
  "folder_id": "string",
  "tags": [
    "string"
  ],
  "description": "string",
  "steps": [
    {
      "name": "string",
      "alias": "string",
      "description": "string",
      "script": "string",
      "transform": {
        "version": 1,
        "op": "select_filter",
        "config": {
          "input": "string",
          "columns": [
            {
              "name": "string",
              "alias": "string"
            }
          ],
          "where": {
            "kind": "and",
            "conditions": [
              {
                "kind": "condition",
                "column": "string",
                "operator": "eq"
              },
              null
            ]
          }
        }
      },
      "inputs": {},
      "outputs": {},
      "dependencies": [
        "string"
      ],
      "options": {},
      "secrets": [
        {
          "name": "string",
          "ref": "string"
        }
      ]
    }
  ],
  "options": {},
  "metadata": {}
}