Skip to content

Update arrow to efficiently serialize more types of numpy arrays.#1889

Merged
pcmoritz merged 1 commit intoray-project:masterfrom
robertnishihara:updatearrownumpy
Apr 13, 2018
Merged

Update arrow to efficiently serialize more types of numpy arrays.#1889
pcmoritz merged 1 commit intoray-project:masterfrom
robertnishihara:updatearrownumpy

Conversation

@robertnishihara
Copy link
Copy Markdown
Collaborator

@robertnishihara robertnishihara commented Apr 13, 2018

This updates arrow primarily to pull in apache/arrow#1887, which allows us to efficiently serialize numpy arrays of bools and strings and things like that.

cc @devin-petersohn

@AmplabJenkins
Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/4842/
Test FAILed.

@robertnishihara
Copy link
Copy Markdown
Collaborator Author

retest this please

@AmplabJenkins
Copy link
Copy Markdown

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/4847/
Test PASSed.

Copy link
Copy Markdown
Member

@devin-petersohn devin-petersohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this!

@pcmoritz pcmoritz merged commit 24c944e into ray-project:master Apr 13, 2018
@pcmoritz pcmoritz deleted the updatearrownumpy branch April 13, 2018 16:41
royf added a commit to royf/ray that referenced this pull request Apr 22, 2018
* master: (56 commits)
  [xray] Turn on flushing to the GCS for the lineage cache (ray-project#1907)
  Single Big Object Parallel Transfer. (ray-project#1827)
  Remove num_threads as a parameter. (ray-project#1891)
  Adds Valgrind tests for multi-threaded object manager. (ray-project#1890)
  Pin cython version in docker base dependencies file. (ray-project#1898)
  Update arrow to efficiently serialize more types of numpy arrays. (ray-project#1889)
  updates (ray-project#1896)
  [DataFrame] Inherit documentation from Pandas (ray-project#1727)
  Update arrow and parquet-cpp. (ray-project#1875)
  raylet command line resource configuration plumbing (ray-project#1882)
  use raylet for remote ray nodes (ray-project#1880)
  [rllib] Propagate dim option to deepmind wrappers (ray-project#1876)
  [RLLib] DDPG (ray-project#1685)
  Lint Python files with Yapf (ray-project#1872)
  [DataFrame] Fixed repr, info, and memory_usage (ray-project#1874)
  Fix getattr compat (ray-project#1871)
  check if arrow build dir exists (ray-project#1863)
  [DataFrame] Encapsulate index and lengths into separate class (ray-project#1849)
  [DataFrame] Implemented __getattr__ (ray-project#1753)
  Add better analytics to docs (ray-project#1854)
  ...

# Conflicts:
#	python/ray/rllib/__init__.py
#	python/setup.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants