Skip to content

Replace curl http client with poco http client for azure#83294

Merged
alesapin merged 15 commits intomasterfrom
new_azure_client
Jul 18, 2025
Merged

Replace curl http client with poco http client for azure#83294
alesapin merged 15 commits intomasterfrom
new_azure_client

Conversation

@alesapin
Copy link
Copy Markdown
Member

@alesapin alesapin commented Jul 4, 2025

Changelog category (leave one):

  • Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Replace curl http client with poco http client for azure blob storage. Introduced multiple settings for this clients which mirror settings from S3. Introduced aggressive connect timeouts for both Azure and S3. Improved introspection into Azure profile events and metrics. New client is enabled by default, provide much better latencies for cold queries on top of Azure Blob Storage. Old Curl client can be returned back by setting azure_sdk_use_native_client=false.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Jul 4, 2025

Workflow [PR], commit [00c0d7b]

Summary:

@clickhouse-gh clickhouse-gh bot added the pr-improvement Pull request with some product improvements label Jul 4, 2025
EXTERN_TYPES[ErrorCodes]=int
EXTERN_TYPES[ProfileEvents]=Event
EXTERN_TYPES[CurrentMetrics]=Metric
EXTERN_TYPES[LatencyBuckets]=LatencyEvent
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea how style check worked before for IO/S3/PocoHTTPClient.h

M(AzurePutRequestThrottlerSleepMicroseconds, "Total time a query was sleeping to conform Azure PUT request throttling.", ValueType::Microseconds) \
M(DiskAzurePutRequestThrottlerCount, "Number of Azure disk PUT requests passed through throttler.", ValueType::Number) \
M(DiskAzurePutRequestThrottlerSleepMicroseconds, "Total time a query was sleeping to conform Azure disk PUT request throttling.", ValueType::Microseconds) \
M(RemoteReadThrottlerBytes, "Bytes passed through 'max_remote_read_network_bandwidth_for_server'/'max_remote_read_network_bandwidth' throttler.", ValueType::Bytes) \
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird diff

extern const Event S3FirstByteReadAttempt1Microseconds;
extern const Event S3FirstByteReadAttempt2Microseconds;
extern const Event S3FirstByteReadAttemptNMicroseconds;
extern const LatencyEvent S3FirstByteReadAttempt1Microseconds;
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to do this renaming, otherwise style check is confused.

Copy link
Copy Markdown
Member

@CheSema CheSema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general it is good.

@alesapin
Copy link
Copy Markdown
Member Author

Tests are green, I just resolved conflicts.

@alesapin
Copy link
Copy Markdown
Member Author

All failed checks are actually green.

@alesapin alesapin enabled auto-merge July 18, 2025 16:35
@alesapin alesapin added this pull request to the merge queue Jul 18, 2025
Merged via the queue into master with commit 2dcadc9 Jul 18, 2025
224 of 255 checks passed
@alesapin alesapin deleted the new_azure_client branch July 18, 2025 16:52
@robot-ch-test-poll4 robot-ch-test-poll4 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jul 18, 2025
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Jul 18, 2025
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Jul 20, 2025
baibaichen pushed a commit to apache/gluten that referenced this pull request Jul 20, 2025
* [GLUTEN-1632][CH]Daily Update Clickhouse Version (20250720)

* Fix gtest due to ClickHouse/ClickHouse#83599

* fxi build due to ClickHouse/ClickHouse#83631

* Fix Build due to ClickHouse/ClickHouse#83294

* Fix Build due to ClickHouse/ClickHouse#82881

---------

Co-authored-by: kyligence-git <gluten@kyligence.io>
Co-authored-by: Chang chen <chenchang@apache.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants