Skip to content

Parquet arrow-writer tests take very long to run (over 60 seconds) #3318

@alamb

Description

@alamb

Describe the bug
Many arrow writer tests take more than 60 seconds to run "has been running for over 60 seconds"

test arrow::arrow_writer::tests::all_null_primitive_single_column has been running for over 60 seconds
test arrow::arrow_writer::tests::arrow_writer_primitive_dictionary has been running for over 60 seconds
test arrow::arrow_writer::tests::arrow_writer_string_dictionary has been running for over 60 seconds
test arrow::arrow_writer::tests::arrow_writer_string_dictionary_unsigned_index has been running for over 60 seconds
test arrow::arrow_writer::tests::binary_column_bloom_filter has been running for over 60 seconds
test arrow::arrow_writer::tests::binary_single_column has been running for over 60 seconds
test arrow::arrow_writer::tests::bool_single_column has been running for over 60 seconds
test arrow::arrow_writer::tests::date32_single_column has been running for over 60 seconds
test arrow::arrow_writer::tests::date64_single_column has been running for over 60 seconds
test arrow::arrow_writer::tests::empty_string_null_column_bloom_filter has been running for over 60 seconds
test arrow::arrow_writer::tests::f32_single_column has been running for over 60 seconds
test arrow::arrow_writer::tests::f64_single_column has been running for over 60 seconds
test arrow::arrow_writer::tests::fixed_size_binary_single_column has been running for over 60 seconds
test arrow::arrow_writer::tests::i16_single_column has been running for over 60 seconds
test arrow::arrow_writer::tests::i32_column_bloom_filter has been running for over 60 seconds
test arrow::arrow_reader::tests::test_utf8_single_column_reader_test ... ok
test arrow::arrow_writer::tests::i32_single_column has been running for over 60 seconds

To Reproduce

cargo test -p parquet --features=arrow

Expected behavior
The tests should pass quickly

Additional context
I believe it also affected the codecov jobs that now start timing out - #3302
https://github.com/apache/arrow-rs/actions/runs/3650327783/jobs/6166169908

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugparquetChanges to the parquet crate

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions