Skip to content

Refactor: DevRank Spider Checkpointing #9033

@tobiu

Description

@tobiu

Implement incremental saving in the Spider service to prevent data loss during long discovery runs.

Changes:

  • Spider.mjs: Save tracker.json (candidates) and visited.json (graph) after processing every page of search results (or every N repositories).

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestperformancePerformance improvements and optimizations

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions