Skip to content

Add SZ3 compression codec#67161

Merged
rschu1ze merged 55 commits intoClickHouse:masterfrom
scanhex12:sz_attempt_2
Jul 2, 2025
Merged

Add SZ3 compression codec#67161
rschu1ze merged 55 commits intoClickHouse:masterfrom
scanhex12:sz_attempt_2

Conversation

@scanhex12
Copy link
Copy Markdown
Member

@scanhex12 scanhex12 commented Jul 25, 2024 •
edited by rschu1ze
Loading

Fixes #49888

This PR adds a new codec with syntax

  • CODEC(SZ3) (automatic mode) or
  • CODEC(SZ3(algorithm, error_bound_mode, error_bound)) (expert mode).

SZ3 is a lossy yet error-bound codec for columns of type Float32 and Float64. See its homepage for details. Supported values for 'algorithm' are ALGO_LORENZO_REG, ALGO_INTERP_LORENZO, and ALGO_INTERP. Supported values for errror_bound_mode are ABS, REL, PSNR and ABS_AND_REL. Argument error_bound is the maximum error and of type Float64.

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add SZ3 as a lossy yet error-bounded compression codec for columns of type Float32 and Float64.

@scanhex12 scanhex12 changed the title first version SZ3 compression codec Jul 25, 2024
@alexey-milovidov alexey-milovidov added the can be tested Allows running workflows for external contributors label Jul 25, 2024
@robot-clickhouse-ci-2 robot-clickhouse-ci-2 added pr-feature Pull request with new product feature submodule changed At least one submodule changed in this PR. labels Jul 26, 2024
@robot-clickhouse
Copy link
Copy Markdown
Member

robot-clickhouse commented Jul 26, 2024 •
edited by robot-ch-test-poll3
Loading

This is an automated comment for commit 05a9690 with description of existing statuses. It's updated for the latest CI running

❌ Click here to open a full report in a separate page

Check nameDescriptionStatus
BuildsThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS❌ failure
Successful checks
Check nameDescriptionStatus
Fast testNormally this is the first check that is ran for a PR. It builds ClickHouse and runs most of stateless functional tests, omitting some. If it fails, further checks are not started until it is fixed. Look at the report to see which tests fail, then reproduce the failure locally as described here✅ success
Style checkRuns a set of checks to keep the code style clean. If some of tests failed, see the related log from the report✅ success

@rschu1ze rschu1ze self-assigned this Jul 30, 2024
@rschu1ze

This comment was marked as resolved.

Sign in to view
@rschu1ze rschu1ze changed the title SZ3 compression codec Add SZ3 compression codec Jul 30, 2024
@rschu1ze rschu1ze force-pushed the sz_attempt_2 branch 3 times, most recently from 8276f61 to 0633807 Compare August 8, 2024 22:10
@rschu1ze

This comment was marked as resolved.

Sign in to view
@scanhex12

This comment was marked as resolved.

Sign in to view
@rschu1ze

This comment was marked as resolved.

Sign in to view
@rschu1ze

This comment was marked as resolved.

Sign in to view
@scanhex12

This comment was marked as resolved.

Sign in to view
@rschu1ze

This comment was marked as resolved.

Sign in to view
@rschu1ze rschu1ze force-pushed the sz_attempt_2 branch 2 times, most recently from 363db19 to a945528 Compare September 4, 2024 19:03
@Algunenano

This comment was marked as resolved.

Sign in to view
@rschu1ze rschu1ze mentioned this pull request Sep 6, 2024
Closed
@alexey-milovidov

This comment was marked as resolved.

Sign in to view
@clickhouse-gh

This comment was marked as resolved.

Sign in to view
@alexey-milovidov

This comment was marked as resolved.

Sign in to view
@scanhex12

This comment was marked as resolved.

Sign in to view
@clickhouse-gh clickhouse-gh bot added the manual approve Manual approve required to run CI label May 28, 2025
rschu1ze

This comment was marked as resolved.

Sign in to view

@scanhex12 scanhex12 requested a review from rschu1ze July 1, 2025 10:39
rschu1ze
rschu1ze reviewed Jul 1, 2025
View reviewed changes
rschu1ze
rschu1ze reviewed Jul 1, 2025
View reviewed changes
@scanhex12 scanhex12 requested a review from rschu1ze July 1, 2025 12:45
rschu1ze
rschu1ze reviewed Jul 1, 2025
View reviewed changes
@rschu1ze
Copy link
Copy Markdown
Member

rschu1ze commented Jul 2, 2025

Stateless tests (amd_binary, old analyzer, s3 storage, DatabaseReplicated, 2/2): #74734

@rschu1ze rschu1ze added this pull request to the merge queue Jul 2, 2025
Merged via the queue into ClickHouse:master with commit 17cfcf6 Jul 2, 2025
118 of 123 checks passed
@tavplubix tavplubix mentioned this pull request Jul 2, 2025
Merged
@robot-clickhouse-ci-2 robot-clickhouse-ci-2 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jul 2, 2025
@rschu1ze rschu1ze mentioned this pull request Jul 10, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rschu1ze rschu1ze rschu1ze approved these changes

Assignees

@rschu1ze rschu1ze

Labels

can be tested Allows running workflows for external contributors manual approve Manual approve required to run CI pr-feature Pull request with new product feature pr-synced-to-cloud The PR is synced to the cloud repo submodule changed At least one submodule changed in this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add support for SZ Compression

6 participants

@scanhex12 @robot-clickhouse @rschu1ze @Algunenano @alexey-milovidov @robot-clickhouse-ci-2