Skip to content

ARROW-3336: [JS] Fix IPC writer serializing sliced Utf8Vectors#2742

Closed
trxcllnt wants to merge 1 commit intoapache:masterfrom
trxcllnt:js-fix-writing-sliced-arrays
Closed

ARROW-3336: [JS] Fix IPC writer serializing sliced Utf8Vectors#2742
trxcllnt wants to merge 1 commit intoapache:masterfrom
trxcllnt:js-fix-writing-sliced-arrays

Conversation

@trxcllnt
Copy link
Contributor

I just noticed this commit wasn't included in #2638 :-[. My bad, here's the rest of the fix for ARROW-3336

Copy link
Member

@wesm wesm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Aside: how do you think we can recruit more Typescript developers into this project?

@wesm wesm closed this in 098bd46 Oct 11, 2018
@trxcllnt
Copy link
Contributor Author

trxcllnt commented Oct 11, 2018

@wesm three main things pop out for me: DX (tooling, docs), stabilizing/enhancing the APIs (adding higher-level APIs like making Vectors from JS objects, seamless interop between node/whatwg streams, really nailing the generic type signatures), and then integrations/examples/demos.

Recently Graphistry had the good fortune to hire my buddy @cwharris, a stellar JS/TS and .NET engineer. He and I are going to finish my js-data-refactor branch over the next few weeks as part of his Arrow-related on-boarding. We're extracting and cleaning up a lot of my internal Arrow helper functions, as well as all the lessons and integrations I've done for the IPC streams in both node and the browser.

With regards to DX, yesterday @cwharris did a v1 of an Arrow-file-viewer VSCode plugin, and I expect we'll continue to add similar scripts or tooling as necessary. I've made extensive use of our little arrow2csv utility, and will be adding a few more scripts like it. it already works pretty well with less, and once I fix a few things with the stream handling it'll work better with others like head and tail.

Docs, integrations, and examples will be better to write once we're done with that branch since there's a lot of breaking changes, but we already have a fastify plugin that handles HTTP content-type application/octet-stream and multipart/form Arrow streams, which we'll be publishing soon as well.

@wesm
Copy link
Member

wesm commented Oct 14, 2018

That's great to hear. It might be worth spending an hour sometime jotting own a "wishlist" for the Arrow JS project so we can give people an idea of what's in scope and what you would like to see get built. Essentially to give a "request for patches" in case some people come out of the wood work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants