Skip to content

[Core] [Feature] Better observability for lineage reconstruction #21427

@stephanie-wang

Description

@stephanie-wang

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

Right now, you can only see how many objects have been recovered in the past. This is not very useful when trying to debug correctness or performance issues, such as if object reconstruction is slow/hanging.

Some useful things to collect are:

  • ObjectRefs that are currently being reconstructed (not including any intermediate ObjectRefs that had to be recursively reconstructed)
  • the current frontier of ObjectRefs that are being reconstructed
  • how long each object reconstruction is taking, maybe compared to the initial execution

It would also be nice to integrate an ObjectRef's current status (e.g., re-executing, pending dependency reconstruction, etc) with ray memory/the dashboard.

Use case

Debugging correctness or performance issues in object reconstruction.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1Issue that should be fixed within a few weeksenhancementRequest for new feature and/or capability

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions