Python Extension Development (Labs)

Nodes

class knime.extension.PythonNode

Extend this class to provide a pure Python based node extension to KNIME Analytics Platform.

Users can either use the decorators @knext.input_table, @knext.input_binary, @knext.output_table, @knext.output_binary, and @knext.output_view, or populate the input_ports, output_ports, and output_view attributes.

Use the Python logging facilities and its .warning and .error methods to write warnings and errors to the KNIME console. .info and .debug will only show up in the KNIME console if the log level in KNIME is configured to show these.

Examples

>>> import logging
... import knime.extension as knext
...
... LOGGER = logging.getLogger(__name__)
...
... category = knext.category("/community", "mycategory", "My Category", "My category described", icon="icons/category.png")
...
... @knext.node(name="Pure Python Node", node_type=knext.NodeType.LEARNER, icon_path="icons/icon.png", category=category)
... @knext.input_table(name="Input Data", description="We read data from here")
... @knext.output_table(name="Output Data", description="Whatever the node has produced")
... class TemplateNode(knext.PythonNode):
...     # A Python node has a description.
...
...     def configure(self, configure_context, table_schema):
...         LOGGER.info(f"Configuring node")
...         return table_schema
...
...     def execute(self, exec_context, table):
...         return table

abstract configure(config_context: ConfigurationContext, *inputs)

Configure this Python node.

Parameters:

• config_context (ConfigurationContext) – The ConfigurationContext providing KNIME utilities during execution

• *inputs – Each input table spec or binary port spec will be added as parameter, in the same order that the ports were defined.

Returns:

Either a single spec, or a tuple or list of specs. The number of specs must match the number of defined output ports, and they must be returned in this order. Alternatively, instead of a spec, a knext.Column can be returned (if the spec shall only consist of one column).

Return type:

Union[Spec, List[Spec], Tuple[Spec, …], Column]

Raises:

InvalidParametersError – If the current input parameters do not satisfy this node’s requirements.

abstract execute(exec_context: ExecutionContext, *inputs)

Execute this Python node.

Parameters:

• exec_context (ExecutionContext) – The ExecutionContext providing KNIME utilities during execution.

• *inputs (tuple) – Each input table or binary port object will be added as a parameter, in the same order that the ports were defined. Tables will be provided as a kn.Table, while binary data will be a plain Python bytes object.

Returns:

Either a single output object (table or binary), or a tuple or list of objects. The number of output objects must match the number of defined output ports, and they must be returned in this order. Tables must be provided as a kn.Table or kn.BatchOutputTable, while binary data should be returned as plain Python bytes object.

Return type:

object or tuple/list of objects

A node is part of a category:

knime.extension.category(path: str, level_id: str, name: str, description: str, icon: str, after: str = '', locked: bool = True)

Register a new node category.

A node category must be created only once. Use the string that represents the absolute category path to add nodes to an existing category.

Examples

>>> node_category = knext.category(
...     "/testing", "python-nodes", "Python Nodes", "Python testing nodes", "icon.png"
... )
...
... @knext.node(
...     name="Simple Python Node",
...     node_type=knext.NodeType.MANIPULATOR,
...     icon_path="icon.png",
...     category=node_category
... )
... class SimpleNode(knext.PythonNode):
...     pass

Parameters:

• path (str) – The absolute path that leads to this category, e.g., “/io/read”. The segments are the category level-IDs, separated by a slash (“/”). Categories that contain community nodes should be placed in the “/community” category.

• level_id (str) – The identifier of the level which is used as a path-segment and must be unique at the level specified by “path”.

• name (str) – The name of this category, e.g., “File readers”.

• description (str) – A short description of the category.

• icon (str) – File path to a 16x16 pixel PNG icon for this category. The path must be relative to the root of the extension.

• after (str, optional) – Specifies the level-id of the category after which this category should be sorted. Defaults to “”.

• locked (bool, optional) – Set this to False to allow extensions from other vendors to add sub-categories or nodes to this category. Defaults to True.

Returns:

The full path of the category, which can be used to create nodes inside this category.

Return type:

str

A node has a type:

class knime.extension.NodeType(value)

Defines the different node types that are available for Python based nodes.

LEARNER = 'Learner'

A node learning a model that is typically consumed by a PREDICTOR.

MANIPULATOR = 'Manipulator'

A node that manipulates data.

OTHER = 'Other'

A node that doesn’t fit one of the other node types.

PREDICTOR = 'Predictor'

A node that predicts something typically using a model provided by a LEARNER.

SINK = 'Sink'

A node consuming data.

SOURCE = 'Source'

A node producing data.

VISUALIZER = 'Visualizer'

A node that visualizes data.

A node’s configure method receives a configuration context that lets you interact with KNIME

class knime.extension.ConfigurationContext(java_ctx, flow_variables, input_ports=None, output_ports=None)

The ConfigurationContext provides utilities to communicate with KNIME during a node’s configure() method.

property flow_variables: Dict[str, Any]

The flow variables coming in from KNIME as a dictionary with string keys.

Notes

The dictionary can be edited and supports flow variables of the following types:

• bool

• list[bool]

• float

• list[float]

• int

• list[int]

• str

• list[str]

get_connected_input_port_numbers() → List[int]

Gets the number of connected input ports for each port.

This method can be used to know how many input ports are connected to the node for each port.

Returns:

A list of the number of connected input ports for each port type.

Return type:

list of int

get_connected_output_port_numbers() → List[int]

Gets the number of connected output ports for each port.

This method can be used to know how many output ports are connected to the node for each port. This is relevant when using PortGroups to determine which ports have to be populated with data.

Examples

>>> @node(name="Example Node with Group Ports")
... @input_table(name="Input Data", description="The data to process in my node")
... @output_table_group(name="Output Data", description="Multiple outputs from my node")
... class ExampleNode(PythonNode):
...     # ...
...     def execute(self, exec_context, table)->List[knext.Schema]:
...         output_table = table
...         connected_output_ports  = exec_context.get_connected_output_port_numbers()
...         # When 2 output ports are connected, this will return [2].
...         output_tables = [output_table] * connected_output_ports[0]
...         return output_tables # Thus, one list with two tables has to be returned.

Returns:

A list of the number of connected output ports for each port type.

Return type:

list of int

get_credential_names()

Returns the identifier (flow variable name) for each credential.

Returns:

A list of credential names.

Return type:

list

get_credentials(identifier: str) → Credential

Returns the credentials dataclass for the given identifier.

Parameters:

identifier (str) – The identifier of the credentials to retrieve.

Returns:

A dataclass containing the credentials.

Return type:

Credential

set_warning(message: str) → None

Sets a warning on the node.

Parameters:

message (str) – The warning message to display on the node.

A node’s execute method receives an execution context that lets you interact with KNIME and e.g. check whether the user has cancelled the execution of your Python node.

class knime.extension.ExecutionContext(java_ctx, flow_variables, input_ports=None, output_ports=None)

The ExecutionContext provides utilities to communicate with KNIME during a node’s execute() method.

property flow_variables: Dict[str, Any]

The flow variables coming in from KNIME as a dictionary with string keys.

Notes

The dictionary can be edited and supports flow variables of the following types:

• bool

• list[bool]

• float

• list[float]

• int

• list[int]

• str

• list[str]

get_connected_input_port_numbers() → List[int]

Gets the number of connected input ports for each port.

This method can be used to know how many input ports are connected to the node for each port.

Returns:

A list of the number of connected input ports for each port type.

Return type:

list of int

get_connected_output_port_numbers() → List[int]

Gets the number of connected output ports for each port.

This method can be used to know how many output ports are connected to the node for each port. This is relevant when using PortGroups to determine which ports have to be populated with data.

Examples

>>> @node(name="Example Node with Group Ports")
... @input_table(name="Input Data", description="The data to process in my node")
... @output_table_group(name="Output Data", description="Multiple outputs from my node")
... class ExampleNode(PythonNode):
...     # ...
...     def execute(self, exec_context, table)->List[knext.Schema]:
...         output_table = table
...         connected_output_ports  = exec_context.get_connected_output_port_numbers()
...         # When 2 output ports are connected, this will return [2].
...         output_tables = [output_table] * connected_output_ports[0]
...         return output_tables # Thus, one list with two tables has to be returned.

Returns:

A list of the number of connected output ports for each port type.

Return type:

list of int

get_credential_names()

Returns the identifier (flow variable name) for each credential.

Returns:

A list of credential names.

Return type:

list

get_credentials(identifier: str) → Credential

Returns the credentials dataclass for the given identifier.

Parameters:

identifier (str) – The identifier of the credentials to retrieve.

Returns:

A dataclass containing the credentials.

Return type:

Credential

get_knime_home_dir() → str

Returns the local absolute path to the directory in which KNIME stores its configuration as well as log files.

Returns:

The local absolute path to the KNIME directory.

Return type:

str

get_workflow_data_area_dir() → str

Returns the local absolute path to the current workflow’s data area folder. This folder is meant to be part of the workflow, so its contents are included whenever the workflow is shared.

get_workflow_temp_dir() → str

Returns the local absolute path where temporary files for this workflow should be stored. Files created in this folder are not automatically deleted by KNIME.

By default, this folder is located in the operating system’s temporary folder. In that case, the contents will be cleaned by the OS.

is_canceled() → bool

Returns true if this node’s execution has been canceled from KNIME. Nodes can check for this property and return early if the execution does not need to finish. Raising a RuntimeError in that case is encouraged.

set_progress(progress: float, message: str | None = None)

Set the progress of the execution.

Note that the progress that can be set here is 80% of the total progress of a node execution. The first and last 10% are reserved for data transfer and will be set by the framework.

Parameters:

• progress (float) – A floating point number between 0.0 and 1.0.

• message (str, optional) – An optional message to display in KNIME with the progress.

set_warning(message: str) → None

Sets a warning on the node.

Parameters:

message (str) – The warning message to display on the node.

The dialog creation context is used to create dialogs for the configuration of the node. It can be accessed indirectly, by passing its method’s as arguments to specific parameters (see the example below).

class knime.extension.DialogCreationContext(java_ctx, flow_variables, specs_to_python_converter)

The DialogCreationContext provides utilities to communicate with KNIME during the dialog creation phase. It enables access to the flow variables, the specs of the input tables and the credentials. These can be used to create the dialog elements, by passing the respective method as lambda function to the constructor of the string parameter class. The lambdas will receive the dialog creation context as parameter which should be passed as first parameter to the fully qualified method calls of DialogCreationContext as below:

Examples

>>> class ExampleNode:
...     # This dialog element displays a dropdown with all available credentials
...     string_param = knext.StringParameter(label="Credential parameter", description="Choices is a callable",
...                                 choices=lambda a: knext.DialogCreationContext.get_credential_names(a))

property flow_variables: Dict[str, Any]

The flow variables coming in from KNIME as a dictionary with string keys.

Notes

The dictionary can be edited and supports flow variables of the following types:

• bool

• list[bool]

• float

• list[float]

• int

• list[int]

• str

• list[str]

get_credential_names()

Returns the identifier (flow variable name) for each credential.

Returns:

A list of credential names.

Return type:

list

get_credentials(identifier: str) → Credential

Returns the credentials dataclass for the given identifier.

Parameters:

identifier (str) – The identifier of the credentials to retrieve.

Returns:

A dataclass containing the credentials.

Return type:

Credential

get_flow_variables()

Returns the flow variables coming in from KNIME as a dictionary with string keys. The dictionary cannot be edited.

Returns:

The flow variables dictionary with string keys.

Return type:

dict

Notes

The supported flow variable types are: - bool - list(bool) - float - list(float) - int - list(int) - str - list(str)

get_input_specs() → List[PortObjectSpec]

Returns the specs for all input ports of the node.

Returns:

A list of specs for all input ports.

Return type:

List

Tables

Table and Schema are the two classes that are used to communicate tabular data (Table) during execute, or the table structure (Schema) in configure between Python and KNIME.

class knime.extension.Table

This class serves as public API to create KNIME tables either from pandas or pyarrow. These tables can than be sent back to KNIME. This class has to be instantiated by calling either from_pyarrow() or from_pandas()

__getitem__(slicing: slice | List[int] | List[str] | Tuple[slice | List[int] | List[str], slice]) → _TabularView

Creates a view of this Table by slicing rows and columns. The slicing syntax is similar to that of numpy arrays, but columns can also be addressed as index lists or via a list of column names.

Notes

The syntax is [column_slice, row_slice]. Note that this is the exact opposite order than in the deprecated scripting API’s ReadTable.

Parameters:

• column_slice (int, str, slice, list) – A column index, a column name, a slice object, a list of column indices, or a list of column names.

• row_slice (slice, optional) – A slice object describing which rows to use.

Returns:

A _TabularView representing a slice of the original Table.

Return type:

TabularView

Examples

>>> row_sliced_table = table[:, :100] # Get the first 100 rows
... column_sliced_table = table[["name", "age"]] # Get all rows of the columns "name" and "age"
... row_and_column_sliced_table = table[1:5, :100] # Get the first 100 rows of columns 1,2,3,4

append(other: _Columnar | Sequence[_Columnar]) → _ColumnarView

Append another _Columnar object (e.g. Table, Schema) or a sequence of _Columnar objects to the current _Columnar object.

Parameters:

other (Union["_Columnar", Sequence["_Columnar"]]) – The _Columnar object or a sequence of _Columnar objects to be appended.

Returns:

A _ColumnarView object representing the current _Columnar object after the append operation.

Return type:

_ColumnarView

batches() → Iterator[Table]

Returns a generator over the batches in this table. A batch is part of the table with all columns, but only a subset of the rows. A batch should always fit into memory (max size currently 64mb). The table being passed to execute() is already present in batches, so accessing the data this way is very efficient.

Returns:

A generator object that yields batches of the table.

Return type:

generator

Examples

>>> output_table = BatchOutputTable.create()
... for batch in my_table.batches():
...     input_batch = batch.to_pandas()
...     # process the batch
...     output_table.append(Table.from_pandas(input_batch))

abstract property column_names: list

Get the names of the columns in a dataset.

static from_pandas(data: pandas.DataFrame, sentinel: str | int | None = None, row_ids: str = 'auto')

Factory method to create a Table given a pandas.DataFrame. The index of the data frame will be used as RowKey by KNIME.

Examples

>>> Table.from_pandas(my_pandas_df, sentinel="min")

Parameters:

• data (pandas.DataFrame) – A pandas DataFrame.

• sentinel (str, optional) –

  Interpret the following values in integral columns as missing value:

  • "min": min int32 or min int64 depending on the type of the column

  • "max": max int32 or max int64 depending on the type of the column

  • a special integer value that should be interpreted as missing value

• row_ids ({'keep', 'generate', 'auto'}, optional) –

  Defines what RowID should be used. Must be one of the following values:

  • "keep": Keep the DataFrame.index as the RowID. Convert the index to strings if necessary.

  • "generate": Generate new RowIDs of the format f"Row{i}" where i is the position of the row (from 0 to length-1).

  • "auto": If the DataFrame.index is of type int or unsigned int, use f"Row{n}" where n is the index of the row. Else, use “keep”.

Returns:

The created Table object.

Return type:

Table

static from_pyarrow(data: pyarrow.Table, sentinel: str | int | None = None, row_ids: str = 'auto')

Factory method to create a Table given a pyarrow.Table.

All batches of the table must have the same number of rows. Only the last batch can have less rows than the other batches.

Examples

>>> Table.from_pyarrow(my_pyarrow_table, sentinel="min")

Parameters:

• data (pyarrow.Table) – A pyarrow.Table

• sentinel (str) –

  Interpret the following values in integral columns as missing value:

  • "min" min int32 or min int64 depending on the type of the column

  • "max" max int32 or max int64 depending on the type of the column

  • a special integer value that should be interpreted as missing value

• row_ids (str) –

  Defines what RowID should be used. Must be one of the following values:

  • "keep": Use the first column of the table as RowID. The first column must be of type string.

  • "generate": Generate new RowIDs of the format f"Row{i}" where i is the position of the row (from 0 to length-1).

  • "auto": Use the first column of the table if it has the name “<RowID>” and is of type string or integer.

    • If the “<RowID>” column is of type string, use it directly

    • If the “<RowID>” column is of an integer type use f"Row{n} where n is the value of the integer column.

    • Generate new RowIDs ("generate") if the first column has another type or name.

Returns:

The created Table instance.

Return type:

pyarrow.Table

insert(other: _Columnar, at: int) → _Columnar

Insert a column or another _Columnar object (e.g. Table, Schema) into the current _Columnar object at a specific position.

Parameters:

• other (_Columnar or Column) – The column or _Columnar object to be inserted.

• at (int) – The index at which the insertion should occur.

Returns:

The _Columnar object after the insertion.

Return type:

_Columnar

Raises:

TypeError – If other is not of type _Columnar or Column.

Notes

The insertion is done in-place, meaning the current _Columnar object is modified.

abstract property num_columns: int

Get the number of columns in the dataset.

remove(slicing: str | int | List[str])

Implements remove method for Columnar data structures. The input can be a column index, a column name or a list of column names.

If the input is a column index, the column with that index will be removed. If it is a column name, then the first column with matching name is removed. Passing a list of column names will filter out all (including duplicate) columns with matching names.

Parameters:

slicing (int | list | str) – Can be of type integer representing the index in column_names to remove. Or a list of strings removing every column matching from that list. Or a string of which first occurrence is removed from the column_names.

Returns:

_ColumnarView

Return type:

A View missing the columns to be removed.

Raises:

• ValueError – If no matching column is found given a list or str.:

• IndexError – If column is accessed by integer and is out of bounds.:

• TypeError – If the key is neither an integer nor a string or list of strings.:

abstract property schema: Schema

The schema of this table, containing column names, types, and potentially metadata

to_batches() → Iterator[Table]

Alias for Table.batches()

to_pandas(sentinel: str | int | None = None) → pandas.DataFrame

Access this table as a pandas.DataFrame.

Parameters:

sentinel (str or int) –

Replace missing values in integral columns by the given value. It can be one of the following:

• "min" min int32 or min int64 depending on the type of the column

• "max" max int32 or max int64 depending on the type of the column

• An integer value that should be inserted for each missing value

to_pyarrow(sentinel: str | int | None = None) → pyarrow.Table

Access this table as a pyarrow.Table.

Parameters:

sentinel (str or int) –

Replace missing values in integral columns by the given value, which can be one of the following:

• ”min”: minimum value of int32 or int64 depending on the type of the column

• ”max”: maximum value of int32 or int64 depending on the type of the column

• An integer value that should be inserted for each missing value