I don't have a minimal example yet, but it looks like if you never append to an array, it won't have a values buffer (poking at it in the debugger). I think this is invalid, and the Arrow Go library crashes when it tries to import this.
https://arrow.apache.org/docs/format/Columnar.html explicitly allows omitting the validity buffer, but this language is not carried into other sections, hence I think it's required.