Skip to content

[Core] Ray doesn't respect object_store_memory when spilling is disabled #53086

@AdrienVannson

Description

@AdrienVannson

What happened + What you expected to happen

I am starting Ray with a small object store memory size, while preventing object spilling. Then, I do ray.put on a large numpy array.

I expect this to fail, since the array is much larger than the object store. However, it works fine.

If I look at the Ray processes, I can see that the data has been written to the disk at some point.

Image

However, I don't see anything in /tmp/ray/session... like when spilling is enabled, so I'm not really sure what is happening.

Versions / Dependencies

Ray 2.46.0
Python 3.12.3
Ubuntu

Reproduction script

import ray
import numpy as np
import time

ray.init(object_store_memory=100 * 1024 * 1024, _system_config={'automatic_object_spilling_enabled': False})

ref = ray.put(np.random.random((200, 1024, 1024)))

time.sleep(1000)

This is the method suggested in #21998

Issue Severity

Medium: It is a significant difficulty but I can work around it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1Issue that should be fixed within a few weeksbugSomething that is supposed to be working; but isn'tcommunity-backlogcoreIssues that should be addressed in Ray Coregood-first-issueGreat starter issue for someone just starting to contribute to Rayusability

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions