Intern task: Quantization by nikita4109 · Pull Request #77018 · ClickHouse/ClickHouse

nikita4109 · 2025-03-02T16:27:50Z

Changelog category (leave one):

New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Functions for quantizations

Added new functions for packing and unpacking floating-point vector arrays into/from FixedStrings with various quantization levels (16-bit, 8-bit, 4-bit, and 1-bit)
Implemented optimized distance calculation functions (L2 distance, cosine similarity) that operate directly on quantized vectors
These functions significantly improve performance for vector embedding search operations while reducing memory usage

These additions enable more efficient storage and searching of vector embeddings, which are essential for semantic search, recommendation systems, and other machine learning applications.

Documentation entry for user-facing changes

Documentation is written (mandatory for new features)

Performance Benchmark Results

Random vectors

Quantization Level	L2 Distance (seconds)	Quantized L2 Distance (seconds)
16-bit	0.5948	0.5588
8-bit	0.5823	0.3221
4-bit	0.5866	0.1694
1-bit	0.5836	0.0353

Quantization Level	Cosine Distance (seconds)	Quantized Cosine Distance (seconds)
16-bit	0.9454	0.6906
8-bit	0.9288	0.5229
4-bit	0.9437	0.2794
1-bit	0.9239	0.0576

Hacker News comments

Quantization Level	L2 Distance (seconds)	Quantized L2 Distance (seconds)
16-bit	26.6642	20.4774
8-bit	26.8680	6.8990
4-bit	20.1942	4.1797
1-bit	29.3374	1.0389

Quantization Level	Cosine Distance (seconds)	Quantized Cosine Distance (seconds)
16-bit	32.4704	24.6458
8-bit	38.5771	10.2922
4-bit	33.7280	6.0128
1-bit	32.7070	1.4331

Quality Comparison

Experimental Setup

Input Data: 500 random vectors with 2048 dimensions
Query Set: First 10 vectors used as query vectors
Evaluation Metric: Precision and recall based on top-10 nearest neighbors

Methods Tested

Quantization Methods:
- 16-bit quantization (half-precision float)
- 8-bit quantization (two formats: SFP and Minifloat)
- 4-bit quantization
- 1-bit quantization (binary)
Distance Metrics:
- L2 (Euclidean) distance
- Cosine distance

Quantization Method	Distance Metric	Precision	Recall	Avg. Matches (of 10)	Storage Reduction
16-bit	L2	99.11%	99.11%	9.82	50%
SFP 8-bit	L2	7.64%	7.34%	0.56	75%
Minifloat 8-bit	L2	4.07%	4.06%	0.38	75%
4-bit	L2	1.98%	1.98%	0.20	87.5%
1-bit	L2	1.98%	1.98%	0.20	97.5%
16-bit	Cosine	99.11%	99.11%	9.82	50%
SFP 8-bit	Cosine	92.33%	92.33%	9.14	75%
Minifloat 8-bit	Cosine	52.95%	52.99%	5.23	75%
4-bit	Cosine	8.89%	8.99%	0.88	87.5%
1-bit	Cosine	1.98%	1.98%	0.20	97.5%

clickhouse-gh · 2025-03-02T17:42:11Z

Workflow [PR], commit [126c869]

alexey-milovidov · 2025-03-03T23:01:38Z

Thanks! Please take a look at the bugs found by fuzzers.
Can we also add a comparison on the quality?

clickhouse-gh · 2025-09-11T15:06:55Z

Workflow [PR], commit [174dcc9]

Summary: ❌
15 failures out of 106 shown:

job_name	test_name	status
Fast test		failure
	02415_all_new_functions_must_be_documented	FAIL
Build (amd_debug)		dropped
Build (amd_release)		dropped
Build (amd_asan)		dropped
Build (amd_tsan)		dropped
Build (amd_msan)		dropped
Build (amd_ubsan)		dropped
Build (amd_binary)		dropped
Build (arm_release)		dropped
Build (arm_asan)		dropped
Build (arm_coverage)		dropped
Build (arm_binary)		dropped
Build (amd_darwin)		dropped
Build (arm_darwin)		dropped
Build (arm_v80compat)		dropped

rienath · 2025-09-11T16:29:03Z

src/Functions/FunctionQuantizedDistance.h

+#include <Columns/ColumnConst.h>
+#include <Columns/ColumnFixedString.h>
+#include <DataTypes/DataTypeFixedString.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <Functions/IFunction.h>
+#include <base/types.h>


Not all includes are actually used. Removing unnecessary ones (in all committed .cpp/h files) would improve readability, build times and prevent transitive includes

rienath · 2025-09-11T16:30:46Z

src/Functions/FunctionQuantize.cpp

+REGISTER_FUNCTION(Quantize16Bit)
+{
+    FunctionDocumentation::Description description = " ";
+    FunctionDocumentation::Syntax syntax = " ";
+    FunctionDocumentation::Arguments argument = {{" ", " "}};
+    FunctionDocumentation::ReturnedValue returned_value = {" "};
+    FunctionDocumentation::Examples examples = {{" ", " ", " "}};
+    FunctionDocumentation::IntroducedIn introduced_in = {25, 10};
+    FunctionDocumentation::Category categories = FunctionDocumentation::Category::Unknown;
+    FunctionDocumentation documentation = {description, syntax, argument, returned_value, examples, introduced_in, categories};
+    factory.registerFunction<FunctionQuantize16Bit>(documentation);
+}


I added documentation templates in all files where needed. Please fill them up to help the users to use this feature

rienath · 2025-09-11T16:59:54Z

src/Functions/FunctionDequantize.h

+#include <cstdint>
+
+
+namespace DB


It is difficult to understand what is happening without any context. To make it easier, add a comment to explain what this file is for at the top of the.h files. Refer to any of these for examples:

https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/FunctionTokens.h

https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/extractAllGroups.h

https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/array/arrayAll.h

rienath · 2025-09-11T17:06:04Z

src/Functions/FunctionDequantize.h

+
+    String getName() const override { return name; }
+    size_t getNumberOfArguments() const override { return 1; }
+    bool isInjective(const ColumnsWithTypeAndName &) const override { return false; }


It's false by default, we don't need to override

rienath · 2025-09-12T14:50:49Z

tests/performance/qnt_8bit_cosine_distance_function.xml

+    <fill_query>      
+
+    ALTER TABLE test.vectors
+    UPDATE vector_quantized = quantize8Bit(vector, 2048)


This function does not exist

rienath · 2025-09-12T14:51:12Z

tests/performance/qnt_8bit_l2_distance_function.xml

+    <fill_query>      
+
+    ALTER TABLE test.vectors
+    UPDATE vector_quantized = quantize8Bit(vector, 2048)


See https://github.com/ClickHouse/ClickHouse/pull/77018/files#r2344492075

rienath · 2025-09-12T15:04:31Z

tests/performance/qnt_16bit_cosine_distance_function.xml

+    ADD COLUMN vector_quantized FixedString(4096);
+
+    </fill_query>
+
+    <fill_query>      
+
+    ALTER TABLE test.vectors
+    UPDATE vector_quantized = quantize16Bit(vector, 2048)


Why do we need FixedString(4096) if we then use only 2048 bytes?

rienath · 2025-09-12T15:26:19Z

src/Functions/FunctionQuantizedDistance.h

+        return std::make_shared<DataTypeFloat32>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const override


:) CREATE TABLE sfp8 (`id` String, `quantized` FixedString(384)) ENGINE = MergeTree ORDER BY id; :) INSERT INTO sfp8 SELECT id, quantizeSFP8Bit(vector, 384) FROM hackernews; Code: 49.DB::Exception: Block structure mismatch in function connect between ApplySquashingTransform and ConvertingTransform stream: different columns: quantized FixedString(384) FixedString(size = 0) quantized FixedString(384) FixedString(size = 0). (LOGICAL_ERROR)

But with an extra byte it works

:) CREATE TABLE sfp8 (`id` String, `quantized` FixedString(385)) ENGINE = MergeTree ORDER BY id;

Same story with quantizeMini8Bit

You can use the small version of hackernews so that you don't have to download the huge one

rienath · 2025-11-24T11:02:03Z

Closing for now due to missing documentation and a bug, but hope someone can build on this work later. @nikita4109 maybe you will finish it one day :)

nikita4109 added 15 commits January 18, 2025 15:51

16 bit

8e7eb7a

default

0e8f1d9

fixed

1161975

fixed

c6b0da4

fixed

590528d

fixed

e51b6ca

1-bit and 4-bit quantization

228c62f

refactored

bc84341

fixed

d1c0c90

fixed

9ce9717

1-bit & 4-bit tests

3988255

cosine distance

e4d4745

fixed

9022056

tests

674cd06

fixed

a66b05c

thevar1able added the can be tested Allows running workflows for external contributors label Mar 2, 2025

clickhouse-gh bot added the pr-feature Pull request with new product feature label Mar 2, 2025

nikita4109 added 4 commits March 2, 2025 20:30

fixed

43005dc

fixed

c955a75

fixed

ab8286d

fixed

dbf674a

nikita4109 added 2 commits March 4, 2025 21:47

fixed

5d460b2

fixed

d449c91

rschu1ze mentioned this pull request Mar 5, 2025

Intern Tasks 2024/2025 #71175

Closed

nikita4109 added 4 commits March 6, 2025 17:39

fixed

bd328cd

fixed

af72fb0

fixed

bdd1bee

fixed

f2cd12f

nikita4109 added 6 commits March 7, 2025 20:01

fixed

7935dbe

fixed

e54af85

minifloat added

7f22923

fixed

1ec0870

fixed

5d7048d

fixed

e21dc92

rschu1ze changed the title ~~Quantization~~ Intern task: Quantization Mar 22, 2025

nikita4109 and others added 3 commits March 24, 2025 18:36

fixed

22e2b96

fixed

126c869

Merge branch 'master' into quantization

6e11c93

rienath added 2 commits September 11, 2025 15:54

Fix includes style

f090d14

Add templates for documentation

174dcc9

rienath reviewed Sep 11, 2025

View reviewed changes

rienath reviewed Sep 12, 2025

View reviewed changes

rienath self-assigned this Nov 24, 2025

rienath closed this Nov 24, 2025

rienath added the unfinished code label Dec 19, 2025

		#include <cstdint>


		namespace DB

Conversation

nikita4109 commented Mar 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Functions for quantizations

Documentation entry for user-facing changes

Performance Benchmark Results

Random vectors

Hacker News comments

Quality Comparison

Experimental Setup

Methods Tested

Uh oh!

clickhouse-gh bot commented Mar 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexey-milovidov commented Mar 3, 2025

Uh oh!

clickhouse-gh bot commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rienath Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rienath commented Nov 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

nikita4109 commented Mar 2, 2025 •

edited

Loading

clickhouse-gh bot commented Mar 2, 2025 •

edited

Loading

clickhouse-gh bot commented Sep 11, 2025 •

edited

Loading

rienath Sep 12, 2025 •

edited

Loading