Make nodes runnable standalone, prompting the user for inputs#1111
Make nodes runnable standalone, prompting the user for inputs#1111
Conversation
Adds a new interactive mode to the Rust node API, which automatically activates when no `DORA_NODE_CONFIG` env variable is set. When run in interactive mode, the user is prompted for inputs on the command line. Input data can be given as string or as JSON. This feature makes it possible to try a node with some inputs and see how it reacts, without the need to set up a full dataflow for it. This can be useful during development.
8f62811 to
b8b5abe
Compare
|
Could you add example and documentation on how to use this? |
We have some Python tests that try to initialize a dora node. We don't want to run in interactive mode in this case because the executable is run by `pytest`. So there is no way to supply interactive input. By checking if stdin is a terminal, we can continue to throw an error in this case.
|
I added some docs with an example in 57237d2. |
|
I also just tried it with the Python template generated by |
|
yeah I meant like markdown based documentation and / or examples within the example folder on how to use this. It's not obvious to me what the input format should look like |
|
A markdown example/documentation is within the docs for
I can put this also somewhere else if you like, e.g. the web docs at dora-rs.ai?
This should basically work with any of the existing nodes, so creating a separate example feels a bit duplicated. Given that it's interactive, it's difficult to test this through an automated example. The only limitation right now is that only string or struct inputs (as JSON) are currently supported. |
|
Yeah we really need an individual markdown file for this on dora-rs.ai at least. Also could you provide an example using json? |
I opened dora-rs/dora-rs.github.io#46 for this.
Added in d2092ee. |
|
Sorry for being annoying but could add a simpler example with just a simple integer array or simple string? I tried to run the example with a simple integer specified array but it seems to be automatically converted into uint8 array: > Input ID 123
> Data [123, 123]
Ignoring unexpected input 123 data: `ArrowData(PrimitiveArray<UInt8>
[
91,
49,
50,
51,
44,
32,
49,
50,
51,
93,
])`Same for string: |
|
No worries, thanks for the examples! Right now only JSON objects are handled as JSON, everything else is interpreted as a string. And it turns out that I used the wrong arrow type for string representation (I stored it as an u8 array). Let me try to push a new version that always go through the json schema inference to see how that works. |
|
@haixuanTao Could you try again with 1de895e? |
haixuanTao
left a comment
There was a problem hiding this comment.
Looks great! Thanks!
Adds a new interactive mode to the Rust node API, which automatically activates when no
DORA_NODE_CONFIGenv variable is set. When run in interactive mode, the user is prompted for inputs on the command line. Input data can be given as string or as JSON.This feature makes it possible to try a node with some inputs and see how it reacts, without the need to set up a full dataflow for it. This can be useful during development.
Partially resolves #1019