Skip to content

[Java] Added creating MapWriter in ComplexWriter. #36375

@xxlaykxx

Description

@xxlaykxx

Describe the enhancement requested

Add new method rootAsMap() to ComplexWriter and implement it in ComplexWriterImpl for supporting Map type. Because currently it's possible to create MapWriter but like a part of list or struct.
Previously in dremio side:

When i trying to return map like @output ComplexWrite
with this code:

org.apache.arrow.vector.complex.writer.BaseWriter.MapWriter mapWriter = out.rootAsList().map(false);

mapWriter.startMap();
for (java.util.Map.Entry<java.lang.Integer, java.lang.Integer> element : map.entrySet()) {
mapWriter.startEntry();
mapWriter.key().integer().writeInt((Integer) element.getKey());
mapWriter.value().integer().writeInt((Integer) element.getValue());
mapWriter.endEntry();
}
mapWriter.endMap();

It use UnionMapWriter and generate schema like:
EXPR$0: Map(false)<$data$: Union(Sparse, [1, 39])<struct: Struct<key: Int(32, true) not null, value: Int(32, true) not null> not null, map: Map(false)<entries: Struct<key: Int(32, true) not null, value: Int(32, true)> not null>>>
But in OutputDerivation impl class where i should create output Complete type

List children = Arrays.asList( CompleteType.INT.toField("key", false), CompleteType.INT.toField("value", false));
return new CompleteType(CompleteType.MAP.getType(), CompleteType.struct(children).toField(MapVector.DATA_VECTOR_NAME, false));

(This is only one valid case, because MapVector.initializeChildrenFromFields())
return
EXPR$0::map<key::int32, value::int32> I found a place where it start using union - PromotableWriter.promoteToUnion.
And in the end i have

SCHEMA_CHANGE ERROR: Schema changed during projection. Schema was
schema(EXPR$0::map<key::int32, value::int32>)
but then changed to
schema(EXPR$0::map<struct::struct<key::int32, value::int32>, map::map<key::int32, value::int32>>)

Component(s)

Java

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions