[C#][Integration] C# C Data interface does not release ArrowArray imports where a non-validity buffer is NULL #40898
Description
Describe the bug, including details regarding any error messages, version, and platform.
In #39302, the integration tests where nanoarrow is producing an C# is consuming are reporting memory leaks. These leaks are based on allocations/frees reported by nanoarrow's special buffer allocator, which I instrumented to do some printing of the results. Those adventures are in a few gists ( https://gist.github.com/paleolimbot/257be29c926c3467b174fd83c3baf2b1 , https://gist.github.com/paleolimbot/8183f50a1c1425db4a744903ba3f905b ) as well as later in this thread, and the output that led me to this conclusion is below: basically, I instrumented the special allocator to print to stdout and did some scraping of the output. I noticed that no buffers are freed when csharp is responsible for releasing, but all the buffers are freed when nanoarrow is (and presumably other implementations too, since no memory leaks are reported for those).
This only happens for custom_metadata, extension, primitive_zerolength, recursive_nested, and union cases, and the patterns seems to be that something happens and then the release callback is just never called. This only happens when nanoarrow is exporting.
#> exporting importing file batch within_batch_id op
#> 1 nanoarrow C# primitive_zerolength 0 1 reallocate
#> 2 nanoarrow C# primitive_zerolength 0 2 reallocate
#> 3 nanoarrow C# primitive_zerolength 0 3 reallocate
#> 4 nanoarrow C# primitive_zerolength 0 4 reallocate
#> 5 nanoarrow nanoarrow primitive_zerolength 0 1 reallocate
#> 6 nanoarrow nanoarrow primitive_zerolength 0 2 reallocate
#> 7 nanoarrow nanoarrow primitive_zerolength 0 3 reallocate
#> 8 nanoarrow nanoarrow primitive_zerolength 0 4 reallocate
#> 9 nanoarrow nanoarrow primitive_zerolength 0 5 reallocate
#> 10 nanoarrow nanoarrow primitive_zerolength 0 6 reallocate
#> 11 nanoarrow nanoarrow primitive_zerolength 0 7 reallocate
#> 12 nanoarrow nanoarrow primitive_zerolength 0 8 reallocate
#> 13 nanoarrow nanoarrow primitive_zerolength 0 53 free
#> 14 nanoarrow nanoarrow primitive_zerolength 0 56 free
#> 15 nanoarrow nanoarrow primitive_zerolength 0 59 free
#> 16 nanoarrow nanoarrow primitive_zerolength 0 62 free
#> 17 nanoarrow nanoarrow primitive_zerolength 0 116 free
#> 18 nanoarrow nanoarrow primitive_zerolength 0 119 free
#> 19 nanoarrow nanoarrow primitive_zerolength 0 122 free
#> 20 nanoarrow nanoarrow primitive_zerolength 0 125 free
The commit that eliminated the leaks from nanoarrow's side was a bit of code that ensured that there were no NULL buffers that were exported (except validity buffers, which can be null).
Component(s)
C#, Integration