Describe the bug, including details regarding any error messages, version, and platform.
I'm testing using the variant encoding for JSON fields, and can't seem to read back the values which has been created from JSON using the parquet/variant package.
I've managed to narrow it down to a simple reproducer, the issue occurs when I have an array of objects over a certain size. See below:
package variant
import (
"testing"
"github.com/apache/arrow-go/v18/parquet/variant"
)
func TestVariantJSONRoundTrip(t *testing.T) {
jsonBytes := []byte(`{
"object_array": [
{
"a_field_name_1": "some value AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
"a_field_name_2": "some value AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
"a_field_name_3": "some value",
"a_field_name_4": "some value",
"a_field_name_5": "some value",
"a_field_name_6": "some value",
"a_field_name_7": "some value",
"a_field_name_8": "some value",
"a_field_name_9": "some value",
"a_field_name_10": "some value"
}
]
}`)
variantFromJSON, err := variant.ParseJSONBytes(jsonBytes, false)
if err != nil {
panic(err)
}
t.Log(variantFromJSON.String()) // Calls json.Marshal, which iterates all fields.
// panic: runtime error: slice bounds out of range [234881546:269] [recovered, repanicked]
// github.com/apache/arrow-go/v18/parquet/variant/variant.go:313
}github.com/apache/arrow-go/v18 v18.5.0
go version go1.25.0 darwin/arm64
Component(s)
Parquet
Describe the bug, including details regarding any error messages, version, and platform.
I'm testing using the variant encoding for JSON fields, and can't seem to read back the values which has been created from JSON using the
parquet/variantpackage.I've managed to narrow it down to a simple reproducer, the issue occurs when I have an array of objects over a certain size. See below:
Component(s)
Parquet