Skip to content

Tune Spider Strategy Probabilities and Enhance Documentation #9133

@tobiu

Description

@tobiu

We are tuning the Spider's discovery probability distribution to prioritize the new "Network Walker" strategy. Since the repository search space is becoming saturated (15k users found), traversing the social graph of these qualified users is now the highest-yield strategy.

Changes:

  1. Rebalance Probabilities:
    • Network Walker: Increase from ~15% to ~30%.
    • High Stars: Decrease from ~30% to ~25%.
    • Bio Signals / Community: Decrease to ~5%.
    • Stargazer: Decrease to ~5%.
  2. Knowledge Base Enhancement: Add rich, intent-driven JSDoc to Spider.mjs explaining the "Filter Bubble" problem and why the Network Walker is the solution for the current stage of the index.

This shift marks a transition from "Breadth-First Repository Scanning" to "Depth-First Network Traversal".

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions