Skip to content

[fluss-client] Support Complex Data Types on the Java Typed API (Map and Array)#2814

Merged
polyzos merged 14 commits into
apache:mainfrom
nhuantho:client-2805
Mar 11, 2026
Merged

[fluss-client] Support Complex Data Types on the Java Typed API (Map and Array)#2814
polyzos merged 14 commits into
apache:mainfrom
nhuantho:client-2805

Conversation

@nhuantho

@nhuantho nhuantho commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #2805

  • This PR adds support for ARRAY and MAP types when converting between Fluss internal types and POJO types.

Brief change log

  • New Utilities

    • Created FlussTypeToPojoTypeConverter: shared utilities for converting Fluss types to POJO types.
    • Moved charLengthExceptionMessage to ConverterCommons, since it is used by both:
      • toBinaryStringForText in ConverterCommons.
      • convertTextValue in FlussTypeToPojoTypeConverter.
      • validateCompatibility logic supported for MAP and ARRAY types
  • New Adapter Classes:

    • Created FlussArrayToPojoArray: converts Fluss InternalArray → POJO array.
    • Created FlussMapToPojoMap: converts Fluss InternalMap → POJO Map.
    • Created PojoArrayToFlussArray: converts POJO array → Fluss InternalArray.
    • Created PojoMapToFlussMap: converts POJO Map → Fluss InternalMap.
  • Converter Updates

    • Added support for ARRAY and MAP in:
      • PojoToRowConverter.
      • RowToPojoConverter.

Tests

  • Updated ConvertersTestFixtures to include logic for MAP and ARRAY types.

    • Added FlussArrayToPojoArrayTest to verify all array conversion cases.
    • Added PojoArrayToFlussArrayTest to verify all array conversion cases.
  • Added testMapType in:

    • PojoToRowConverterTest.
    • RowToPojoConverterTest.
  • Removed testUnsupportedSchemaFieldTypeThrows from PojoToRowConverterTest.

API and Format

Documentation

Comment thread website/docs/_configs/_partial_config.mdx Outdated
Comment thread website/docs/_configs/_partial_config.mdx Outdated
@polyzos

polyzos commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

@nhuantho thank you for your contribution. It's almost there, and I left a few comments.
One design issue worth resolving alongside the above: the PojoMaptoFlussMap lazy InternalMap approach should be replaced with an eager conversion consistent with how PojoArrayToFlussArray works.

@nhuantho

nhuantho commented Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

Hi @polyzos, could you review it again for me?

  • Changed the PojoMapToFlussMap lazy InternalMap approach to an eager conversion, consistent with how PojoArrayToFlussArray works.
  • Updated the validateCompatibility logic for MAP and ARRAY types.
  • Handled the primitive array types.

@nhuantho nhuantho requested a review from polyzos March 11, 2026 04:12
@polyzos

polyzos commented Mar 11, 2026

Copy link
Copy Markdown
Contributor

@nhuantho Thank you for this.. i pushed some small improvements and added test to the existing FlussTypedClientITCase for end2end scenarios.
I will merge once CI passes

@polyzos polyzos merged commit 2f8cada into apache:main Mar 11, 2026
9 of 11 checks passed
hemanthsavasere pushed a commit to hemanthsavasere/fluss that referenced this pull request Mar 14, 2026
…and Array) (apache#2814)

* [fluss-client] Add pojo array to fluss array

* [fluss-client] Add pojo map to fluss map

* [fluss-client] Create shared utilities for POJO type and Fluss type

* [fluss-client] Create shared utilities for Fluss type to Pojo type

* [fluss-client] Add Fluss array to Pojo array

* [fluss-client] Add Fluss map to Pojo map

* [ci] Fix Compile Java 8

* Revert _partial_config.mdx

* [fluss-client] Hanle primitive arrays

* [fluss-client] Refactor PojoMapToFlussMap to follow the same conversion logic as PojoArrayToFlussArray

* [fluss-client] Change validateCompatibility logic for MAP and ARRAY types

* small fixes and add tests to FlussTypedClient

* fix checkstyle

* fix checkstyle

---------

Co-authored-by: ipolyzos <ipolyzos.se@gmail.com>
wxplovecc pushed a commit to tongcheng-elong/fluss that referenced this pull request Apr 17, 2026
…and Array) (apache#2814)

* [fluss-client] Add pojo array to fluss array

* [fluss-client] Add pojo map to fluss map

* [fluss-client] Create shared utilities for POJO type and Fluss type

* [fluss-client] Create shared utilities for Fluss type to Pojo type

* [fluss-client] Add Fluss array to Pojo array

* [fluss-client] Add Fluss map to Pojo map

* [ci] Fix Compile Java 8

* Revert _partial_config.mdx

* [fluss-client] Hanle primitive arrays

* [fluss-client] Refactor PojoMapToFlussMap to follow the same conversion logic as PojoArrayToFlussArray

* [fluss-client] Change validateCompatibility logic for MAP and ARRAY types

* small fixes and add tests to FlussTypedClient

* fix checkstyle

* fix checkstyle

---------

Co-authored-by: ipolyzos <ipolyzos.se@gmail.com>
wxplovecc pushed a commit to tongcheng-elong/fluss that referenced this pull request Apr 20, 2026
…and Array) (apache#2814)

* [fluss-client] Add pojo array to fluss array

* [fluss-client] Add pojo map to fluss map

* [fluss-client] Create shared utilities for POJO type and Fluss type

* [fluss-client] Create shared utilities for Fluss type to Pojo type

* [fluss-client] Add Fluss array to Pojo array

* [fluss-client] Add Fluss map to Pojo map

* [ci] Fix Compile Java 8

* Revert _partial_config.mdx

* [fluss-client] Hanle primitive arrays

* [fluss-client] Refactor PojoMapToFlussMap to follow the same conversion logic as PojoArrayToFlussArray

* [fluss-client] Change validateCompatibility logic for MAP and ARRAY types

* small fixes and add tests to FlussTypedClient

* fix checkstyle

* fix checkstyle

---------

Co-authored-by: ipolyzos <ipolyzos.se@gmail.com>
Ugbot pushed a commit to Ugbot/fluss that referenced this pull request Apr 26, 2026
…and Array) (apache#2814)

* [fluss-client] Add pojo array to fluss array

* [fluss-client] Add pojo map to fluss map

* [fluss-client] Create shared utilities for POJO type and Fluss type

* [fluss-client] Create shared utilities for Fluss type to Pojo type

* [fluss-client] Add Fluss array to Pojo array

* [fluss-client] Add Fluss map to Pojo map

* [ci] Fix Compile Java 8

* Revert _partial_config.mdx

* [fluss-client] Hanle primitive arrays

* [fluss-client] Refactor PojoMapToFlussMap to follow the same conversion logic as PojoArrayToFlussArray

* [fluss-client] Change validateCompatibility logic for MAP and ARRAY types

* small fixes and add tests to FlussTypedClient

* fix checkstyle

* fix checkstyle

---------

Co-authored-by: ipolyzos <ipolyzos.se@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Complex Data Types on the Java Typed API

2 participants