Problem

The DevRank.services.Spider incorrectly triggers a "Graceful Exit" (stopping all operations) when the GitHub API Search Rate Limit (30 req/min) is reached, even if the Core Rate Limit (5000 req/hr) is still healthy.

This happens because GitHub.mjs currently tracks a single rateLimit object, which gets overwritten by the headers of the last request.

1. Spider runs a search query -> rateLimit updated to ~29/30.
2. Spider tries to process repositories (fetching contributors via Core API).
3. Spider sees rateLimit.remaining is 29 (from Search) and incorrectly thinks it's below the safety threshold (50), triggering a shutdown.

Goal

Decouple rate limit tracking to respect GitHub's x-ratelimit-resource header.

Proposed Changes

1. Refactor GitHub.mjs:

   • Change rateLimit config to store multiple buckets:

     rateLimit: {
         core: { remaining: 5000, reset: null, limit: 5000 },
         search: { remaining: 30, reset: null, limit: 30 },
         graphql: { remaining: 5000, reset: null, limit: 5000 },
         integration_manifest: { remaining: 5000, reset: null, limit: 5000 }
     }

   • Update #updateRateLimit to read x-ratelimit-resource and update the corresponding bucket.

2. Update Spider.mjs:

   • Use GitHub.rateLimit.search for runSearch.
   • Use GitHub.rateLimit.core for fetchContributors / processRepositories.

3. Update Updater.mjs:

   • Ensure it uses the correct bucket (likely graphql or core).

Acceptance Criteria

• Spider continues processing repositories (Core API) even if Search API limit is exhausted.
• Spider pauses only searching when Search limit is low.
• Logs clearly distinguish between Core and Search limits.