PipelineCreate

Payload used to create Pipeline. Required fields are name, type, collection_id, steps.

nameName (string)필수

alias object

하나 이상(anyOf)

• string
• null

string

typeType (string)필수

Pipeline execution mode: 'batch' or 'event' (required)

Possible values: [batch, event]

collection_idCollection Id (string)필수

Parent collection ID (required)

Possible values: non-empty

folder_id object

하나 이상(anyOf)

• string
• null

string

tags object

하나 이상(anyOf)

• string[]
• null

Array [

string

]

description object

하나 이상(anyOf)

• string
• null

string

steps object[]필수

Execution steps

Array [

nameName (string)필수

Step name

alias object

하나 이상(anyOf)

• string
• null

string

description object

Step-owned description, independent of the referenced Code's description so steps sharing one Code can be labeled distinctly.

하나 이상(anyOf)

• string
• null

string

script object

Code resource ID to execute (code step)

하나 이상(anyOf)

• string
• null

string

transform object

Native transform spec (transform step)

하나 이상(anyOf)

• object
• null

op필수

Possible values: [aggregate, join, rename_cast, select_filter, union]

aggregate

versionVersion (integer)

상수 값: 1

기본값: 1

config object필수

Config for the aggregate op: group-by + allowlisted aggregate functions. An empty group_by aggregates the whole input to a single row. input is a port key.

inputInput (string)필수

Input port key

group_bystring[]

Grouping columns (empty = whole-input aggregate)

aggregations object[]필수

Aggregations (non-empty)

Array [

functionFunction (string)필수

Allowlisted aggregate function

Possible values: [sum, count, avg, min, max]

column object

Aggregated column (omit only for count)

하나 이상(anyOf)

• string
• null

string

aliasAlias (string)필수

Output column name

]

join

versionVersion (integer)

상수 값: 1

기본값: 1

config object필수

Two-input equi-join configuration. left_input and right_input are pipeline port keys, keys contains at least one equality pair, and columns explicitly selects the output. Other join predicates are not accepted.

left_inputLeft Input (string)필수

Left input port key

right_inputRight Input (string)필수

Right input port key

join_typeJoin Type (string)필수

inner | left | right | full

Possible values: [inner, left, right, full]

keys object[]필수

Equi-join column pairs (non-empty)

Array [

leftLeft (string)필수

Join column on the left input

rightRight (string)필수

Join column on the right input

]

columns object[]필수

Output projection (non-empty)

Array [

inputInput (string)필수

Source input port key (left_input or right_input)

nameName (string)필수

Source column name on that input

alias object

Output column name (defaults to name)

하나 이상(anyOf)

• string
• null

string

]

rename_cast

versionVersion (integer)

상수 값: 1

기본값: 1

config object필수

Config for the rename_cast op. Listed columns are renamed / cast; columns not listed pass through unchanged. input is a port key.

inputInput (string)필수

Input port key

columns object[]필수

Column rename / cast rules (non-empty)

Array [

nameName (string)필수

Source column name

alias object

New column name (defaults to name)

하나 이상(anyOf)

• string
• null

string

cast object

Allowlisted target type

하나 이상(anyOf)

• string
• null

string

Possible values: [utf8, int64, float64, bool]

]

select_filter

versionVersion (integer)

상수 값: 1

기본값: 1

config object필수

Select-and-filter transform configuration accepted in a request. input is a pipeline port key, not a resource identifier; columns=null keeps every column, and where=null applies no predicate.

inputInput (string)필수

Input port key (a PipelineStep.inputs key)

columns object

Column names returned for this resource.

하나 이상(anyOf)

• object[]
• null

Array [

nameName (string)필수

Source column name on the input

alias object

Output column name (defaults to name)

하나 이상(anyOf)

• string
• null

string

]

where object

Structured row predicate

하나 이상(anyOf)

• RowFilter
• null

kindKind (string)

Possible values: [and, or]

기본값: and

conditions object[]

Nested conditions (leaf RowFilterCondition or group RowFilter)

Array [

하나 이상(anyOf)

• RowFilterCondition
• Option 2

kindKind (string)

상수 값: condition

기본값: condition

columnColumn (string)필수

Column name to filter on

operatorRowFilterOperator (string)필수

Allowed Row Filter Operator values. Values: eq, ne, gt, gte, lt, lte, in, not_in, and 4 more.

Possible values: [eq, ne, gt, gte, lt, lte, in, not_in, between, like, is_null, is_not_null]

value object

Filter value. Single value for scalar operators, list for 'in'/'not_in', 2-element list for 'between'. Omit for 'is_null'/'is_not_null'.

하나 이상(anyOf)

• Option 1
• null

]

union

versionVersion (integer)

상수 값: 1

기본값: 1

config object필수

Union configuration for two or more pipeline input ports. Inputs are aligned by column name, missing values become null, and duplicate rows are retained.

inputsstring[]필수

Input port keys to union (>= 2)

inputs object

Input dataset mapping

하나 이상(anyOf)

• object
• null

object

outputs object

Output dataset mapping

하나 이상(anyOf)

• object
• null

object

dependencies object

List of dependent step names

하나 이상(anyOf)

• string[]
• null

Array [

string

]

options object

하나 이상(anyOf)

• object
• null

object

secrets object

Secret values to inject into contexts['secrets']

하나 이상(anyOf)

• object[]
• null

Array [

nameName (string)필수

contexts['secrets'] key the step reads

refRef (string)필수

Secret reference, e.g. secret://secret./token

]

]

options object

하나 이상(anyOf)

• object
• null

object

metadata object

하나 이상(anyOf)

• object
• null

object

PipelineCreate

{
  "name": "string",
  "alias": "string",
  "type": "batch",
  "collection_id": "string",
  "folder_id": "string",
  "tags": [
    "string"
  ],
  "description": "string",
  "steps": [
    {
      "name": "string",
      "alias": "string",
      "description": "string",
      "script": "string",
      "transform": {
        "version": 1,
        "op": "select_filter",
        "config": {
          "input": "string",
          "columns": [
            {
              "name": "string",
              "alias": "string"
            }
          ],
          "where": {
            "kind": "and",
            "conditions": [
              {
                "kind": "condition",
                "column": "string",
                "operator": "eq"
              },
              null
            ]
          }
        }
      },
      "inputs": {},
      "outputs": {},
      "dependencies": [
        "string"
      ],
      "options": {},
      "secrets": [
        {
          "name": "string",
          "ref": "string"
        }
      ]
    }
  ],
  "options": {},
  "metadata": {}
}