Skip to content

Auto-detect failure on ZNG input: buffer exceeded max size #4586

@philrz

Description

@philrz

Repro is with GA Zed tagged v1.7.0.

A community zync user was using the Zed lake migration tools and reported the following failure:

image

This failure was at the step where it attempted to zed load the ZNG dump of the old-format pool to make the new-format pool. Since we knew the expected input was ZNG, they were able to easily work around the problem by adding an explicit -i zng to the zed load. However, since it's Zed's own format, we'd expect we should be able to auto-detect ZNG pretty reliably, so we'd like to better understand what broke here.

Unfortunately, due to the sensitive nature of the data, it could not be shared for us to reproduce the problem ourselves. Thankfully the zed dev dig frames command can be used to show the structure of the ZNG file without revealing its contents. They shared the following output:

image

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions