Enhance system stability for registering massive resources#2327
Merged
cb-github-robot merged 1 commit intocloud-barista:mainfrom Feb 24, 2026
Merged
Enhance system stability for registering massive resources#2327cb-github-robot merged 1 commit intocloud-barista:mainfrom
cb-github-robot merged 1 commit intocloud-barista:mainfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request enhances system stability for registering massive resources by implementing comprehensive rate limiting, memory leak prevention, and improved concurrency control. The changes address scalability issues when dealing with large numbers of cloud resources across multiple Cloud Service Providers (CSPs).
Changes:
- Centralized CSP-specific rate limiting configuration in a single location with per-CSP settings for registration, VM creation, and status fetching operations
- Implemented RequestMap memory management with TTL-based cleanup, max-entries eviction, and periodic snapshots to prevent unbounded memory growth
- Refactored resource registration and inspection to use hierarchical rate limiting (global cap → per-CSP cap → per-connection processing) with channel-based result collection to avoid data races
- Added request skip patterns for high-frequency polling endpoints to prevent memory leaks from UI polling operations
- Improved DataDisk status handling to avoid infinite API call loops when status is "Unknown"
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/core/model/csp/csp.go | Adds centralized RateLimitConfig structure with per-CSP rate limits for all concurrent operations |
| src/core/infra/utility.go | Refactors RegisterCspNativeResourcesAll and InspectResourcesOverview to use hierarchical rate limiting and channel-based result collection |
| src/core/infra/provisioning.go | Updates VM creation rate limiting to use centralized CSP configuration |
| src/core/infra/manageInfo.go | Updates VM status fetching rate limiting to use centralized CSP configuration |
| src/core/resource/common.go | Adds "Unknown" status check for DataDisk to prevent infinite API retry loops; only updates status when changed |
| src/core/common/client/client.go | Implements RequestMap memory management with TTL-based cleanup (1 hour), max-entries eviction (1000 limit), and auto-save functionality |
| src/core/common/logfilter/patterns.go | Adds skip patterns for high-frequency GET endpoints (MCI status, k8sCluster, resource lists) to prevent RequestMap memory leaks |
| src/interface/rest/server/middlewares/requestIdAndDetailsIssuer.go | Adds IncrementRequestMapCount call after storing to RequestMap for accurate entry tracking |
| docker-compose.yaml | Updates cb-spider image version from 0.12.5 to 0.12.6 |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Seokho Son <shsongist@gmail.com>
Member
Author
|
/approve |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
To support registering massive resources..