Skip to content

Zstd decompression fails due to unknown frame content size  #625

@mkitti

Description

@mkitti

Neuroglancer appears to be having difficulty decompressing a Zstd compressed dataset:
https://neuroglancer-demo.appspot.com/#!https://s3proxy.janelia.org/shroff-public/shroff_20220523_Janelia_embryo2/20220523i_Janelia_embryo2_OTO_6x6x6nm_2xbin.zstd.zarr/s6/ng.json

The Google Chrome developer tools console reports the following error:

Error retrieving chunk [object Object]:0,0,0: Error: Invalid typed array length: 4294967226

Raw or Gzip compressed Zarr v2 datasets seem to load just fine:

Raw compressed Zarr v2 dataset:
https://neuroglancer-demo.appspot.com/#!https://s3proxy.janelia.org/shroff-public/shroff_20220523_Janelia_embryo2/20220523i_Janelia_embryo2_OTO_6x6x6nm_2xbin.raw.zarr/s6/ng.json

Gzip compressed Zarr v2 dataset:
https://neuroglancer-demo.appspot.com/#!https://s3proxy.janelia.org/shroff-public/shroff_20220523_Janelia_embryo2/20220523i_Janelia_embryo2_OTO_6x6x6nm_2xbin.gzip.zarr/s6/ng.json

The value 4294967226 corresponds to the 32-bit value 0xFFFFFFBA. The 64-bit value 0xFFFFFFFFFFFFFFBA corresponds to the Zstd error code for "Destination buffer is too small".

The dataset was encoded by Tensorstore 0.1.62 via conda-forge package with build number py312h7e2185d_0 and zstd version 1.5.6 ha6fb4c9_0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions