fix(clp-rust-utils): Configure AWS region and credentials in base config to avoid default provider resolution (fixes #1915). by LinZhihao-723 · Pull Request #1931 · y-scope/clp

LinZhihao-723 · 2026-01-28T20:54:38Z

Description

As the title suggests, this PR implements the suggested fix in #1915 to avoid unnecessary IMDS lookups.

Checklist

The PR satisfies the contribution guidelines.
This is a breaking change and that has been indicated in the PR title, OR this isn't a
breaking change.
Necessary docs have been updated, OR no docs need to be updated.

Validation performed

Ensure all workflows pass.

Created both S3 scanning and SQS listening jobs. Ensure that with the fix applied, there is no IMDS lookup overhead. log-ingestor log:

{"timestamp":"2026-01-28T20:34:02.705660Z","level":"INFO","fields":{"message":"Server started at 0.0.0.0:3002"},"filename":"components/log-ingestor/src/bin/log_ingestor.rs","line_number":103}
{"timestamp":"2026-01-28T20:34:16.492677Z","level":"INFO","fields":{"message":"Create SQS listener ingestion job.","config":"SqsListenerConfig { base: BaseConfig { bucket_name: NonEmptyString(\"lzh-test\"), key_prefix: NonEmptyString(\"test-1769632456\"), region: Some(NonEmptyString(\"us-east-2\")), endpoint_url: None, dataset: None, timestamp_key: None, unstructured: false }, queue_url: NonEmptyString(\"https://sqs.us-east-2.amazonaws.com/568954113123/log-ingestor-sqs-test\") }"},"filename":"components/log-ingestor/src/routes.rs","line_number":218}
{"timestamp":"2026-01-28T20:34:16.520597Z","level":"INFO","fields":{"message":"Created SQS listener ingestion job.","job_id":"5dd9d9a2-5f4d-471b-9b9f-d0a40051d08d"},"filename":"components/log-ingestor/src/routes.rs","line_number":226}
{"timestamp":"2026-01-28T20:34:23.139606Z","level":"INFO","fields":{"message":"Received new object metadata from SQS.","object":"ObjectMetadata { bucket: NonEmptyString(\"lzh-test\"), key: NonEmptyString(\"test-1769632456/log_1769632457.clp.zstd\"), size: 6748 }"},"filename":"components/log-ingestor/src/ingestion_job/sqs_listener.rs","line_number":103}
{"timestamp":"2026-01-28T20:34:27.732227Z","level":"INFO","fields":{"message":"Received new object metadata from SQS.","object":"ObjectMetadata { bucket: NonEmptyString(\"lzh-test\"), key: NonEmptyString(\"test-1769632456/log_1769632462.clp.zstd\"), size: 6731 }"},"filename":"components/log-ingestor/src/ingestion_job/sqs_listener.rs","line_number":103}
{"timestamp":"2026-01-28T20:34:36.143491Z","level":"INFO","fields":{"message":"Received new object metadata from SQS.","object":"ObjectMetadata { bucket: NonEmptyString(\"lzh-test\"), key: NonEmptyString(\"test-1769632456/log_1769632467.clp.zstd\"), size: 6776 }"},"filename":"components/log-ingestor/src/ingestion_job/sqs_listener.rs","line_number":103}
{"timestamp":"2026-01-28T20:34:41.682829Z","level":"INFO","fields":{"message":"Received new object metadata from SQS.","object":"ObjectMetadata { bucket: NonEmptyString(\"lzh-test\"), key: NonEmptyString(\"test-1769632456/log_1769632475.clp.zstd\"), size: 6754 }"},"filename":"components/log-ingestor/src/ingestion_job/sqs_listener.rs","line_number":103}
{"timestamp":"2026-01-28T20:34:46.256316Z","level":"INFO","fields":{"message":"Received new object metadata from SQS.","object":"ObjectMetadata { bucket: NonEmptyString(\"lzh-test\"), key: NonEmptyString(\"test-1769632456/log_1769632481.clp.zstd\"), size: 6721 }"},"filename":"components/log-ingestor/src/ingestion_job/sqs_listener.rs","line_number":103}
{"timestamp":"2026-01-28T20:35:10.734166Z","level":"INFO","fields":{"message":"Create S3 scanner ingestion job.","config":"S3ScannerConfig { base: BaseConfig { bucket_name: NonEmptyString(\"lzh-test\"), key_prefix: NonEmptyString(\"test-1769632511\"), region: Some(NonEmptyString(\"us-east-2\")), endpoint_url: None, dataset: None, timestamp_key: None, unstructured: false }, scanning_interval_sec: 30, start_after: None }"},"filename":"components/log-ingestor/src/routes.rs","line_number":167}
{"timestamp":"2026-01-28T20:35:10.734596Z","level":"INFO","fields":{"message":"Created S3 scanner ingestion job.","job_id":"5b7dac86-74ec-4a9b-88b8-dc111be87654"},"filename":"components/log-ingestor/src/routes.rs","line_number":175}
{"timestamp":"2026-01-28T20:40:16.514338Z","level":"INFO","fields":{"message":"Submitting CLP compression job..."},"filename":"components/log-ingestor/src/compression/compression_job_submitter.rs","line_number":132}
{"timestamp":"2026-01-28T20:40:16.516975Z","level":"INFO","fields":{"message":"Compression job submitted with ID: 1"},"filename":"components/log-ingestor/src/compression/compression_job_submitter.rs","line_number":144}
{"timestamp":"2026-01-28T20:40:19.520901Z","level":"INFO","fields":{"message":"Compression job 1 completed successfully."},"filename":"components/log-ingestor/src/compression/compression_job_submitter.rs","line_number":187}

Summary by CodeRabbit

Chores
- Streamlined internal AWS client configuration for S3 and SQS to simplify initialization and ownership.
- Adjusted internal handling of region and endpoint setup and corrected an internal credential provider identifier.
- No changes to functionality, public interfaces, or exported declarations.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

coderabbitai · 2026-01-28T20:54:53Z

Walkthrough

Applied minor refactor to AWS client initialization in S3 and SQS: credential provider id updated, credentials and region are applied before calling .load().await, and builders are created from the loaded config; endpoint handling unchanged. (49 words)

Changes

Cohort / File(s)	Summary
AWS client config (S3 & SQS) `components/clp-rust-utils/src/s3/client.rs`, `components/clp-rust-utils/src/sqs/client.rs`	Renamed local variable `credential` → `credentials`; updated provider id `"clp-credential-provider"` → `"clp-credentials-provider"`; apply `.credentials_provider(...)` and `.region(Region::new(...))` via `aws_config::defaults(...)` before `.load().await`; construct `Builder::from(base_config)` from the loaded config; move `force_path_style(true)` to builder stage; endpoint URL handling unchanged.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title check	✅ Passed	The title clearly summarizes the main change: configuring AWS region and credentials in base config to avoid default provider resolution, which aligns with the actual modifications across both S3 and SQS client files.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

junhaoliao

validated the fix worked correctly:

No IMDS delays: Job creation + S3 client initialization completed in ~230-307ms (vs. 3+ seconds before the fix)
No spurious WARN logs: Zero "failed to load region from IMDS" messages

task docker-images:package

then modify tools/deployment/package-helm/set-up-test.sh

# Load the locally-built clp-package image into kind
repo_root="${script_dir}/../../../"
iid_file="${repo_root}/build/clp-package-image.id"
if [[ -f "$iid_file" ]]; then
    image_id=$(<"$iid_file")
    user="${USER:-$(id -un 2>/dev/null || whoami 2>/dev/null || echo unknown)}"
    short_id="${image_id#sha256:}"
    short_id="${short_id:0:4}"
    image_tag="dev-${user}-${short_id}"
    echo "Loading clp-package:${image_tag} into kind cluster..."
    kind load docker-image "clp-package:${image_tag}" --name "${CLUSTER_NAME}"
else
    echo "ERROR: No locally-built image found at ${iid_file}. Run 'task docker-images:package' first."
    exit 1
fi

Then Helm install with overrides to use the local image:

helm install test "${script_dir}" \
    --set "image.clpPackage.repository=clp-package" \
    --set "image.clpPackage.tag=${image_tag}" \
    --set "image.clpPackage.pullPolicy=Never"

then tested with the instructions at #1913

junhaoliao

we can rename let credential -> let credentials

the rest lgtm

junhaoliao

for the title, how about:

fix(clp-rust-utils): Configure AWS region and credentials in base config to avoid default provider resolution (fixes #1915).

…fig to avoid default provider resolution (fixes y-scope#1915). (y-scope#1931)

fix

d5392a1

LinZhihao-723 requested a review from a team as a code owner January 28, 2026 20:54

Merge branch 'main' into fix-1915

8d3c820

LinZhihao-723 requested a review from junhaoliao January 28, 2026 20:55

LinZhihao-723 added this to the January 2026 milestone Jan 28, 2026

Merge branch 'main' into fix-1915

88a6f01

junhaoliao requested changes Jan 29, 2026

View reviewed changes

Comment thread components/clp-rust-utils/src/s3/client.rs

Comment thread components/clp-rust-utils/src/s3/client.rs Outdated

Comment thread components/clp-rust-utils/src/sqs/client.rs Outdated

Merge branch 'main' into fix-1915

d7520cf

junhaoliao assigned junhaoliao and LinZhihao-723 Jan 30, 2026

LinZhihao-723 added 2 commits January 29, 2026 20:43

Apply code review comments.

2cd9457

Fix.

7a395ef

LinZhihao-723 requested a review from junhaoliao January 30, 2026 01:44

junhaoliao previously approved these changes Jan 30, 2026

View reviewed changes

Fix...

8446526

LinZhihao-723 dismissed junhaoliao’s stale review via 8446526 January 30, 2026 03:02

LinZhihao-723 requested a review from junhaoliao January 30, 2026 03:02

junhaoliao approved these changes Jan 30, 2026

View reviewed changes

LinZhihao-723 changed the title ~~fix(clp-rust-utils): Set AWS region in base config to avoid unnecessary IMDS lookups (fixes #1915).~~ fix(clp-rust-utils): Configure AWS region and credentials in base config to avoid default provider resolution (fixes #1915). Jan 30, 2026

LinZhihao-723 merged commit 8ae9873 into y-scope:main Jan 30, 2026
24 checks passed

junhaoliao pushed a commit to junhaoliao/clp that referenced this pull request May 17, 2026

fix(clp-rust-utils): Configure AWS region and credentials in base con…

a01cc23

…fig to avoid default provider resolution (fixes y-scope#1915). (y-scope#1931)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(clp-rust-utils): Configure AWS region and credentials in base config to avoid default provider resolution (fixes #1915).#1931

fix(clp-rust-utils): Configure AWS region and credentials in base config to avoid default provider resolution (fixes #1915).#1931
LinZhihao-723 merged 7 commits into
y-scope:mainfrom
LinZhihao-723:fix-1915

LinZhihao-723 commented Jan 28, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jan 28, 2026 •

edited

Loading

Uh oh!

junhaoliao left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

junhaoliao left a comment

Uh oh!

junhaoliao left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

LinZhihao-723 commented Jan 28, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Validation performed

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Uh oh!

junhaoliao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

junhaoliao left a comment

Choose a reason for hiding this comment

Uh oh!

junhaoliao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

LinZhihao-723 commented Jan 28, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jan 28, 2026 •

edited

Loading