Skip to content

[flink] Union Read fails when meet timestamp type #1126

Description

@leonardBang

When I verify RC2 of 0.7 version following quick start, I try to test timestamp type but meet following issue:

4e916ccdabf955ce_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from RUNNING to FAILED with failure cause:
java.lang.RuntimeException: Failed to deserialize record: +Icom.alibaba.fluss.flink.lakehouse.paimon.reader.PaimonRowWrapper@3b9544ff@-1. Cause: class com.alibaba.fluss.row.TimestampNtz cannot be cast to class com.alibaba.fluss.row.TimestampLtz (com.alibaba.fluss.row.TimestampNtz and com.alibaba.fluss.row.TimestampLtz are in unnamed module of loader 'app')
	at com.alibaba.fluss.flink.source.emitter.FlinkRecordEmitter.processAndEmitRecord(FlinkRecordEmitter.java:93) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.flink.lakehouse.LakeRecordRecordEmitter.emitRecord(LakeRecordRecordEmitter.java:49) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.flink.source.emitter.FlinkRecordEmitter.emitRecord(FlinkRecordEmitter.java:81) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.flink.source.emitter.FlinkRecordEmitter.emitRecord(FlinkRecordEmitter.java:42) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:203) ~[flink-connector-files-1.20.0.jar:1.20.0]
	at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:422) ~[flink-dist-1.20.0.jar:1.20.0]
	at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) ~[flink-dist-1.20.0.jar:1.20.0]
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-dist-1.20.0.jar:1.20.0]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:638) ~[flink-dist-1.20.0.jar:1.20.0]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.20.0.jar:1.20.0]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:973) ~[flink-dist-1.20.0.jar:1.20.0]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:917) ~[flink-dist-1.20.0.jar:1.20.0]
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:970) ~[flink-dist-1.20.0.jar:1.20.0]
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:949) [flink-dist-1.20.0.jar:1.20.0]
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:763) [flink-dist-1.20.0.jar:1.20.0]
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) [flink-dist-1.20.0.jar:1.20.0]
	at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.lang.ClassCastException: class com.alibaba.fluss.row.TimestampNtz cannot be cast to class com.alibaba.fluss.row.TimestampLtz (com.alibaba.fluss.row.TimestampNtz and com.alibaba.fluss.row.TimestampLtz are in unnamed module of loader 'app')
	at com.alibaba.fluss.row.GenericRow.getTimestampLtz(GenericRow.java:152) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.flink.lakehouse.paimon.reader.ScanRecordWrapper.getTimestamp(ScanRecordWrapper.java:120) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.flink.lakehouse.paimon.reader.PaimonRowWrapper.getTimestampNtz(PaimonRowWrapper.java:99) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.row.InternalRow.lambda$createFieldGetter$9c1fe9f3$1(InternalRow.java:258) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.row.InternalRow.lambda$createFieldGetter$fbda14b8$1(InternalRow.java:274) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.flink.utils.FlussRowToFlinkRowConverter.toFlinkRowData(FlussRowToFlinkRowConverter.java:70) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.flink.utils.FlussRowToFlinkRowConverter.toFlinkRowData(FlussRowToFlinkRowConverter.java:59) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.flink.source.deserializer.RowDataDeserializationSchema.deserialize(RowDataDeserializationSchema.java:85) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.flink.source.deserializer.RowDataDeserializationSchema.deserialize(RowDataDeserializationSchema.java:47) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]
	at com.alibaba.fluss.flink.source.emitter.FlinkRecordEmitter.processAndEmitRecord(FlinkRecordEmitter.java:90) ~[fluss-flink-1.20-0.7.0.jar:0.7.0]

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions