Skip to content

Add Reproduction Test Case for Scan Panic on Schema with map<string, list<string>>#736

Closed
alexandre-normand wants to merge 2 commits intoapache:mainfrom
alexandre-normand:alex.normand/scanner-problem
Closed

Add Reproduction Test Case for Scan Panic on Schema with map<string, list<string>>#736
alexandre-normand wants to merge 2 commits intoapache:mainfrom
alexandre-normand:alex.normand/scanner-problem

Conversation

@alexandre-normand
Copy link
Copy Markdown
Contributor

@alexandre-normand alexandre-normand commented Feb 18, 2026

This test (not actually meant for merging) exercises a path that we found triggered scanning/reading to panic. The test case is meant to make for the simplest reproduction: it includes a schema with a single column called tags which is a nullable map of string to a list of strings (i.e. map<string, list<string>>).

The issue is that doing a scan on a table with such a schema (where it's possible for some rows to have an empty tags field value) panics.

The runtime panic looked like this:

panic: arrow/array: map array child array should have two fields

goroutine 4357 [running]:
github.com/apache/arrow-go/v18/arrow/array.(*Map).validateData(0xfd25dd002f01?, 0x40035c8d20)
        /Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/arrow/array/map.go:68 +0x110
github.com/apache/arrow-go/v18/arrow/array.(*Map).setData(0x400089a5d0, 0x40035c8d20)
        /Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/arrow/array/map.go:77 +0x24
github.com/apache/arrow-go/v18/arrow/array.NewMapData({0x2811350, 0x40035c8d20})
        /Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/arrow/array/map.go:41 +0xb8
github.com/apache/arrow-go/v18/arrow/array.init.0.func33({0x2811350?, 0x40035c8d20?})
        /Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/arrow/array/array.go:170 +0x24
github.com/apache/arrow-go/v18/arrow/array.MakeFromData({0x2811350, 0x40035c8d20})
        /Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/arrow/array/array.go:120 +0x5c
github.com/apache/arrow-go/v18/parquet/pqarrow.(*listReader).BuildArray(0x40065fb860, 0x20000)
        /Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/parquet/pqarrow/column_readers.go:469 +0x658
github.com/apache/arrow-go/v18/parquet/pqarrow.(*ColumnReader).NextBatch(0x40007c3c30, 0x20000)
        /Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/parquet/pqarrow/file_reader.go:169 +0x5c
github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next.func1(0x44, 0x4004f1ff38?)
        /Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/parquet/pqarrow/file_reader.go:786 +0x54
github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next.func2()
        /Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/parquet/pqarrow/file_reader.go:839 +0xe0
created by github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next in goroutine 333
        /Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/parquet/pqarrow/file_reader.go:830 +0x2ac

Which is the same as this test:

=== RUN   TestTableWriting/TestScanPanicOnMapStringKeyStringListValue
panic: arrow/array: map array child array should have two fields

goroutine 54 [running]:
github.com/apache/arrow-go/v18/arrow/array.(*Map).validateData(0x1083d0101?, 0x140009875e0)
	/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/arrow/array/map.go:68 +0x110
github.com/apache/arrow-go/v18/arrow/array.(*Map).setData(0x1400085a3f0, 0x140009875e0)
	/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/arrow/array/map.go:77 +0x24
github.com/apache/arrow-go/v18/arrow/array.NewMapData({0x1054c1e30, 0x140009875e0})
	/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/arrow/array/map.go:41 +0xb8
github.com/apache/arrow-go/v18/arrow/array.init.0.func33({0x1054c1e30?, 0x140009875e0?})
	/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/arrow/array/array.go:170 +0x24
github.com/apache/arrow-go/v18/arrow/array.MakeFromData({0x1054c1e30, 0x140009875e0})
	/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/arrow/array/array.go:120 +0x5c
github.com/apache/arrow-go/v18/parquet/pqarrow.(*listReader).BuildArray(0x14000e46b40, 0x20000)
	/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/parquet/pqarrow/column_readers.go:469 +0x658
github.com/apache/arrow-go/v18/parquet/pqarrow.(*ColumnReader).NextBatch(0x14000e49290, 0x20000)
	/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/parquet/pqarrow/file_reader.go:169 +0x5c
github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next.func1(0x0, 0x14000113738?)
	/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/parquet/pqarrow/file_reader.go:786 +0x54
github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next.func2()
	/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/parquet/pqarrow/file_reader.go:839 +0xe0
created by github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next in goroutine 16
	/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/v18@v18.5.1/parquet/pqarrow/file_reader.go:830 +0x2ac

@alexandre-normand alexandre-normand mentioned this pull request Feb 18, 2026
Closed
@alexandre-normand alexandre-normand force-pushed the alex.normand/scanner-problem branch from 449eae0 to 2606c10 Compare February 18, 2026 02:38
@zeroshade
Copy link
Copy Markdown
Member

zeroshade commented Feb 20, 2026

Superceded and fixed by #748

@zeroshade zeroshade closed this Feb 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zeroshade zeroshade Awaiting requested review from zeroshade zeroshade will be requested when the pull request is marked ready for review zeroshade is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@alexandre-normand @zeroshade