Skip to content

Feature/implement polars code to flowfile function#370

Merged
Edwardvaneechoud merged 8 commits intomainfrom
feature/implement-polars-code-to-flowfile-function
Mar 29, 2026
Merged

Feature/implement polars code to flowfile function#370
Edwardvaneechoud merged 8 commits intomainfrom
feature/implement-polars-code-to-flowfile-function

Conversation

@Edwardvaneechoud
Copy link
Copy Markdown
Owner

This pull request significantly enhances the expression system in flowfile_frame/expr.py by introducing a new internal representation (ff_repr) for expressions, enabling conversion to a "flowfile function" format. It adds mapping logic and propagation for this representation across binary operators, string and datetime methods, casting, and more. Additionally, new string trimming methods are introduced, and support for abs and round operations is added.

Flowfile function representation improvements:

  • Added ff_repr attribute to Expr and Column classes, which tracks a flowfile function representation of expressions, and ensures it is propagated through all relevant methods and operations. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19]

  • Introduced mapping dictionaries (FF_OPERATOR_MAP, CAST_FF_MAP, STRING_METHOD_FF_MAP, DT_METHOD_FF_MAP) and helper _get_ff_repr to convert Python/Polars expressions and values into flowfile function syntax.

Operator and method support:

  • Implemented flowfile function representation for all binary operators, including both left- and right-side (reverse) operations, as well as for casting, string, and datetime methods. [1] [2] [3] [4] [5]

  • Added new methods to Expr for string trimming (strip_chars_start, strip_chars_end), and numeric operations (abs, round), each with their corresponding flowfile function representation. [1] [2]

Utility and consistency improvements:

  • Ensured that methods like alias, fill_null, is_null, and is_not_null correctly propagate or compute the ff_repr attribute. [1] [2] [3] [4]

  • Updated literal creation (lit) to include flowfile function representation for literal values.

@netlify
Copy link
Copy Markdown

netlify bot commented Mar 26, 2026

Deploy Preview for flowfile-wasm canceled.

Name Link
🔨 Latest commit 54bb87e
🔍 Latest deploy log https://app.netlify.com/projects/flowfile-wasm/deploys/69c8e22199eff20008b199b2

@Edwardvaneechoud Edwardvaneechoud merged commit ae4e024 into main Mar 29, 2026
23 checks passed
@Edwardvaneechoud Edwardvaneechoud deleted the feature/implement-polars-code-to-flowfile-function branch March 29, 2026 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant