Skip to content

Cleanup worker.py #2016

@suquark

Description

@suquark

Describe the problem

Currently, ray/worker.py seems too heavy. It contains ~3000 lines of code (maybe 1/3 of core python code of ray).

And its a bit messy (contains nearly all kinds of structure-breaking reference🙃) :

global_worker = Worker()

class Worker:
    def func_a(self):
        foo(worker=self)
        
    def func_b(self):
        foo()
        
    def func_c(self):
        put()

    def func_d(self):
        bar(worker=self)

    def func_d(self):
        bar(worker=global_worker)

def foo(worker=global_worker):
    pass

def bar(worker):
    pass

def put():
    do_something(global_worker)

What's more, worker.py contains all these things:

  • logging utils
  • serialization utils
  • core worker logics
  • ray APIs
  • ray exceptions
  • constants

Since a possible correct dependency may look like:

image

, it could be better to divide worker.py into several independent modules. We could keep ray's APIs in worker.py so it may not affect other parts of ray.

I want to know if a PR for refactoring worker.py is welcomed? (I understand that such a PR is heavy and may be unnecessary now.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Issue moderate in impact or severity

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions