Use source code in hash where possible (fix #2089) by AdamGleave · Pull Request #2090 · ray-project/ray

AdamGleave · 2018-05-18T02:31:33Z

What do these changes do?

Rather than attempting to infer when source code for a function is available, we always try and omit it only if we find an inspection.

Related issue number

Fixes #2089 and maybe #1446 (do not have the setup to confirm)

AmplabJenkins · 2018-05-18T03:01:51Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5466/
Test FAILed.

AdamGleave · 2018-05-18T04:43:55Z

@AmplabJenkins build seems to have failed due to lack of resources on the cluster, is there a way to poke it to retry?

AdamGleave · 2018-05-18T04:44:23Z

retest this please

robertnishihara · 2018-05-18T05:07:09Z

python/ray/remote_function.py

+        # in the hash.
+        source = inspect.getsource(function).encode("ascii")
+        function_id_hash.update(source)
+    except OSError:


This doesn't seem to work for me in a Python 2 interpreter. It raises an IOError, which is related, but slightly different.

22:05 $ python2 Python 2.7.13 |Anaconda 4.3.1 (x86_64)| (default, Dec 20 2016, 23:05:08) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://anaconda.org >>> import inspect >>> def f(): ... pass ... >>> try: ... inspect.getsource(f) ... except OSError as e: ... pass ... Traceback (most recent call last): File "<stdin>", line 2, in <module> File "/Users/rkn/anaconda2/lib/python2.7/inspect.py", line 700, in getsource lines, lnum = getsourcelines(object) File "/Users/rkn/anaconda2/lib/python2.7/inspect.py", line 689, in getsourcelines lines, lnum = findsource(object) File "/Users/rkn/anaconda2/lib/python2.7/inspect.py", line 537, in findsource raise IOError('could not get source code') IOError: could not get source code

Ah yes, this changed in Python 3.3 it seems: https://docs.python.org/3/library/inspect.html

I'll change it to catch both for compatibility.

AmplabJenkins · 2018-05-18T06:03:36Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5468/
Test PASSed.

AmplabJenkins · 2018-05-18T06:32:04Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5471/
Test PASSed.

robertnishihara · 2018-05-18T06:40:17Z

Looks like inspect.getsource actually raises a TypeError when run on a Cython function (see error below). I pushed a fix I think.

======================================================================
ERROR: test_numpy (__main__.CythonTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/cython_test.py", line 51, in test_numpy
    result = get_ray_result(cyth.masked_log, array)
  File "test/cython_test.py", line 13, in get_ray_result
    func = ray.remote(cython_func)
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/worker.py", line 2647, in remote
    return make_decorator(worker=worker)(args[0])
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/worker.py", line 2608, in decorator
    num_return_vals, max_calls)
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/remote_function.py", line 74, in __init__
    self._function_id = compute_function_id(self._function).id()
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/remote_function.py", line 33, in compute_function_id
    source = inspect.getsource(function).encode("ascii")
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 968, in getsource
    lines, lnum = getsourcelines(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 955, in getsourcelines
    lines, lnum = findsource(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 768, in findsource
    file = getsourcefile(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 684, in getsourcefile
    filename = getfile(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 666, in getfile
    'function, traceback, frame, or code object'.format(object))
TypeError: <cyfunction masked_log at 0x7fb078fb5778> is not a module, class, method, function, traceback, frame, or code object

======================================================================
ERROR: test_simple_func (__main__.CythonTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/cython_test.py", line 28, in test_simple_func
    self.assertEqualHelper(cyth.simple_func, 6, 1, 2, 3)
  File "test/cython_test.py", line 25, in assertEqualHelper
    self.assertEqual(get_ray_result(cython_func, *args), expected)
  File "test/cython_test.py", line 13, in get_ray_result
    func = ray.remote(cython_func)
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/worker.py", line 2647, in remote
    return make_decorator(worker=worker)(args[0])
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/worker.py", line 2608, in decorator
    num_return_vals, max_calls)
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/remote_function.py", line 74, in __init__
    self._function_id = compute_function_id(self._function).id()
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/remote_function.py", line 33, in compute_function_id
    source = inspect.getsource(function).encode("ascii")
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 968, in getsource
    lines, lnum = getsourcelines(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 955, in getsourcelines
    lines, lnum = findsource(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 768, in findsource
    file = getsourcefile(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 684, in getsourcefile
    filename = getfile(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 666, in getfile
    'function, traceback, frame, or code object'.format(object))
TypeError: <cyfunction simple_func at 0x7fb078fb5270> is not a module, class, method, function, traceback, frame, or code object

----------------------------------------------------------------------
Ran 3 tests in 5.992s

FAILED (errors=2)

travis_time:end:038101ed:start=1526624985818047460,finish=1526624992832056081,duration=7014008621
�[0K
�[31;1mThe command "python test/cython_test.py" exited with 1

AmplabJenkins · 2018-05-18T08:02:36Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5476/
Test PASSed.

* master: (22 commits) [xray] Fix bug in updating actor execution dependencies (ray-project#2064) [DataFrame] Refactor __delitem__ (ray-project#2080) [xray] Better error messaging when pulling from self. (ray-project#2068) Use source code in hash where possible (fix ray-project#2089) (ray-project#2090) Functions for flushing done tasks and evicted objects. (ray-project#2033) Fix compilation error for RAY_USE_NEW_GCS with latest clang. (ray-project#2086) [xray] Corrects Error Handling During Push and Pull. (ray-project#2059) [xray] Sophisticated task dependency management (ray-project#2035) Support calling positional arguments by keyword (fix ray-project#998) (ray-project#2081) [DataFrame] Improve performance of iteration methods (ray-project#2026) [DataFrame] Implement to_csv (ray-project#2014) [xray] Lineage cache only requests notifications about remote parent tasks (ray-project#2066) [rllib] Add magic methods for rollouts (ray-project#2024) [DataFrame] Allows DataFrame constructor to take in another DataFrame (ray-project#2072) Pin Pandas version for Travis to 0.22 (ray-project#2075) Fix python linting (ray-project#2076) [xray] Fix GCS table prefixes (ray-project#2065) Some tests for _submit API. (ray-project#2062) [rllib] Queue lib for python 2.7 (ray-project#2057) [autoscaler] Remove faulty assert that breaks during downscaling, pull configs from env (ray-project#2006) ...

* master: (24 commits) Performance fix (ray-project#2110) Use flake8-comprehensions (ray-project#1976) Improve error message printing and suppression. (ray-project#2104) [rllib] [doc] Broken link in ddpg doc YAPF, take 3 (ray-project#2098) [rllib] rename async -> _async (ray-project#2097) fix unused lambda capture (ray-project#2102) [xray] Use pubsub instead of timeout for ObjectManager Pull. (ray-project#2079) [DataFrame] Update _inherit_docstrings (ray-project#2085) [JavaWorker] Changes to the build system for support java worker (ray-project#2092) [xray] Fix bug in updating actor execution dependencies (ray-project#2064) [DataFrame] Refactor __delitem__ (ray-project#2080) [xray] Better error messaging when pulling from self. (ray-project#2068) Use source code in hash where possible (fix ray-project#2089) (ray-project#2090) Functions for flushing done tasks and evicted objects. (ray-project#2033) Fix compilation error for RAY_USE_NEW_GCS with latest clang. (ray-project#2086) [xray] Corrects Error Handling During Push and Pull. (ray-project#2059) [xray] Sophisticated task dependency management (ray-project#2035) Support calling positional arguments by keyword (fix ray-project#998) (ray-project#2081) [DataFrame] Improve performance of iteration methods (ray-project#2026) ...

* fix-a3c-torch: (37 commits) Add missing channel major Use correct filter size Add TODO Fix shape errors fmt Performance fix (ray-project#2110) Use flake8-comprehensions (ray-project#1976) Improve error message printing and suppression. (ray-project#2104) [rllib] [doc] Broken link in ddpg doc YAPF, take 3 (ray-project#2098) [rllib] rename async -> _async (ray-project#2097) fix unused lambda capture (ray-project#2102) [xray] Use pubsub instead of timeout for ObjectManager Pull. (ray-project#2079) [DataFrame] Update _inherit_docstrings (ray-project#2085) [JavaWorker] Changes to the build system for support java worker (ray-project#2092) [xray] Fix bug in updating actor execution dependencies (ray-project#2064) [DataFrame] Refactor __delitem__ (ray-project#2080) [xray] Better error messaging when pulling from self. (ray-project#2068) Use source code in hash where possible (fix ray-project#2089) (ray-project#2090) Functions for flushing done tasks and evicted objects. (ray-project#2033) ...

Use source code in hash where possible (fix ray-project#2089)

56f7ef1

robertnishihara reviewed May 18, 2018

View reviewed changes

Compatibility with Python<3.3

56a36b6

robertnishihara mentioned this pull request May 18, 2018

Same function assigned different IDs on driver and worker #2089

Closed

robertnishihara approved these changes May 18, 2018

View reviewed changes

Fix for cython functions.

b199f5b

robertnishihara merged commit 57419f9 into ray-project:master May 18, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use source code in hash where possible (fix #2089)#2090

Use source code in hash where possible (fix #2089)#2090
robertnishihara merged 3 commits intoray-project:masterfrom
AdamGleave:funcid-fix

AdamGleave commented May 18, 2018

Uh oh!

AmplabJenkins commented May 18, 2018

Uh oh!

AdamGleave commented May 18, 2018

Uh oh!

AdamGleave commented May 18, 2018

Uh oh!

robertnishihara May 18, 2018

Uh oh!

AdamGleave May 18, 2018

Uh oh!

AmplabJenkins commented May 18, 2018

Uh oh!

AmplabJenkins commented May 18, 2018

Uh oh!

robertnishihara commented May 18, 2018

Uh oh!

AmplabJenkins commented May 18, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

AdamGleave commented May 18, 2018

What do these changes do?

Related issue number

Uh oh!

AmplabJenkins commented May 18, 2018

Uh oh!

AdamGleave commented May 18, 2018

Uh oh!

AdamGleave commented May 18, 2018

Uh oh!

robertnishihara May 18, 2018

Choose a reason for hiding this comment

Uh oh!

AdamGleave May 18, 2018

Choose a reason for hiding this comment

Uh oh!

AmplabJenkins commented May 18, 2018

Uh oh!

AmplabJenkins commented May 18, 2018

Uh oh!

robertnishihara commented May 18, 2018

Uh oh!

AmplabJenkins commented May 18, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants