Skip to content

Intermittent "snappy: corrupt input" with plain CSVs #2157

@tmtmtmtm

Description

@tmtmtmtm

From time to time (somewhere on the order of 1% of runs), with no pattern I can discern, I get a io error: snappy: corrupt input (expected stream header but got unexpected chunk type byte 112) error when reading plain CSV files. I can never replicate this: on a subsequent run of the same command everything works fine. It usually happens somewhere in the middle of a chain of piped commands, so I also can't tell which of the commands is blowing up, or if there's any pattern to it.

I have been holding off on reporting in the hope that I could pin it down at least a little more, but I've been unable to do so. I have no .sz files anywhere, so I'm assuming that either qsv is running a check for snappy-ness somewhere (in which case perhaps there's a way I could explicitly turn that off?), or one of the sub-commands is producing a temporary snappy file. But I don't really understand what's going on well enough so hopefully there's enough info here for someone else to pick up a useful clue.

(I'm currently on qsv 0.134.0-mimalloc-apply;fetch;foreach;geocode;Luau 0.640;to;polars-0.42.0-fe04390;self_update-8-8;12.80 GiB-677.75 MiB-0 B-16.00 GiB (aarch64-apple-darwin compiled with Rust 1.81) prebuilt but it's been happening with other recent versions too)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions