Skip to content

Make adding more NodeEvent variants backwards-compatible#1273

Merged
phil-opp merged 11 commits intomainfrom
ignore-unknown-node-events
Dec 23, 2025
Merged

Make adding more NodeEvent variants backwards-compatible#1273
phil-opp merged 11 commits intomainfrom
ignore-unknown-node-events

Conversation

@phil-opp
Copy link
Copy Markdown
Collaborator

@phil-opp phil-opp commented Dec 20, 2025

Ignore unknown event types during deserialization.

@phil-opp
Make adding more NodeEvent variants backwards-compatible
e11016c 
Ignore unknown event types during deserialization.
phil-opp added a commit to guptapratykshh/pg-dora that referenced this pull request Dec 20, 2025
@phil-opp
Bump dora-message version to 0.7.0-alpha
466f384 
We added a new `NodeEvent` variant, which is not backwards compatible (yet, see dora-rs#1273). We also added a new `InterDaemonEvent` variant that is not backwards compatible either.
phil-opp added a commit to guptapratykshh/pg-dora that referenced this pull request Dec 20, 2025
@phil-opp
Bump dora-message version to 0.7.0-alpha
615788b 
We added a new `NodeEvent` variant, which is not backwards compatible (yet, see dora-rs#1273). We also added a new `InterDaemonEvent` variant that is not backwards compatible either.
@phil-opp phil-opp enabled auto-merge (squash) December 21, 2025 10:49
@phil-opp
Copy link
Copy Markdown
Collaborator Author

phil-opp commented Dec 22, 2025

Debug notes:

The CI CLI test on Ubuntu hangs at:

 Running CI Async test
2025-12-22T09:38:01.771305Z  INFO dora_core::descriptor::validate: skipping path check for node with build command
2025-12-22T09:38:01.771324Z  INFO dora_core::descriptor::validate: skipping path check for node with build command
2025-12-22T09:38:01.771958Z  INFO zenoh::net::runtime: Using ZID: dc4b7dbf34b83d1d65cf50943fc1cace
2025-12-22T09:38:01.772600Z  INFO zenoh::net::runtime::orchestrator: Zenoh can be reached at: tcp/[fe80::7e1e:52ff:feda:dd8f]:41501
2025-12-22T09:38:01.772611Z  INFO zenoh::net::runtime::orchestrator: Zenoh can be reached at: tcp/10.1.0.42:41501
2025-12-22T09:38:01.772661Z  INFO zenoh::net::runtime::orchestrator: zenohd listening scout messages on 224.0.0.224:7446
09:38:02 DEBUG   receive_data_with_sleep: daemon::spawner  spawning node
09:38:02 DEBUG   send_data: daemon::spawner  spawning node
09:38:02 INFO    receive_data_with_sleep: spawner  spawning: uv run python -u /home/runner/work/dora/dora/examples/python-async/receive_data.py
09:38:02 INFO    send_data: spawner  spawning: uv run python -u /home/runner/work/dora/dora/examples/python-async/send_data.py
09:38:02 INFO    dora daemon  finished building nodes, spawning...
09:38:02 INFO    receive_data_with_sleep: spawner  spawning `uv` in `/home/runner/work/dora/dora/examples/python-async`
09:38:02 DEBUG   receive_data_with_sleep: spawner  spawned node with pid 41470
09:38:02 INFO    send_data: spawner  spawning `uv` in `/home/runner/work/dora/dora/examples/python-async`
09:38:02 DEBUG   send_data: spawner  spawned node with pid 41471
09:38:02 INFO    receive_data_with_sleep: daemon  node is ready
09:38:02 INFO    opentelemetry  Global meter provider is set. Meters can now be created using global::meter() or global::meter_with_scope().
09:38:02 INFO    opentelemetry  Global meter provider is set. Meters can now be created using global::meter() or global::meter_with_scope().
09:38:02 INFO    send_data: daemon  node is ready
09:38:02 INFO    daemon  all nodes are ready, starting dataflow
09:38:03 stdout  send_data:  
09:38:03 stdout  send_data:  
09:38:03 DEBUG   send_data: daemon  handling node stop with exit status Success
09:38:03 INFO    send_data: daemon  send_data finished successfully

On main, the output is:

Running CI Async test
2025-12-20T14:05:34.083630Z  INFO dora_core::descriptor::validate: skipping path check for node with build command
2025-12-20T14:05:34.083646Z  INFO dora_core::descriptor::validate: skipping path check for node with build command
2025-12-20T14:05:34.084235Z  INFO zenoh::net::runtime: Using ZID: 9add8257a9b832f67a03b8526b17fc58
2025-12-20T14:05:34.084881Z  INFO zenoh::net::runtime::orchestrator: Zenoh can be reached at: tcp/[fe80::20d:3aff:fe33:8e34]:36831
2025-12-20T14:05:34.084892Z  INFO zenoh::net::runtime::orchestrator: Zenoh can be reached at: tcp/10.1.0.167:36831
2025-12-20T14:05:34.084927Z  INFO zenoh::net::runtime::orchestrator: zenohd listening scout messages on 224.0.0.224:7446
14:05:34 DEBUG   receive_data_with_sleep: daemon::spawner  spawning node
14:05:34 DEBUG   send_data: daemon::spawner  spawning node
14:05:34 INFO    receive_data_with_sleep: spawner  spawning: uv run python -u /home/runner/work/dora/dora/examples/python-async/receive_data.py
14:05:34 INFO    send_data: spawner  spawning: uv run python -u /home/runner/work/dora/dora/examples/python-async/send_data.py
14:05:34 INFO    dora daemon  finished building nodes, spawning...
14:05:34 INFO    receive_data_with_sleep: spawner  spawning `uv` in `/home/runner/work/dora/dora/examples/python-async`
14:05:34 DEBUG   receive_data_with_sleep: spawner  spawned node with pid 10480
14:05:34 INFO    send_data: spawner  spawning `uv` in `/home/runner/work/dora/dora/examples/python-async`
14:05:34 DEBUG   send_data: spawner  spawned node with pid 10481
14:05:34 INFO    opentelemetry  Global meter provider is set. Meters can now be created using global::meter() or global::meter_with_scope().
14:05:34 INFO    receive_data_with_sleep: daemon  node is ready
14:05:34 INFO    opentelemetry  Global meter provider is set. Meters can now be created using global::meter() or global::meter_with_scope().
14:05:34 INFO    send_data: daemon  node is ready
14:05:34 INFO    daemon  all nodes are ready, starting dataflow
14:05:35 stdout  receive_data_with_sleep:  done!
14:05:35 stdout  receive_data_with_sleep:  
14:05:35 stdout  receive_data_with_sleep:  
14:05:35 DEBUG   receive_data_with_sleep: daemon  handling node stop with exit status Success
14:05:35 INFO    receive_data_with_sleep: daemon  receive_data_with_sleep finished successfully
14:05:35 stdout  send_data:  
14:05:35 stdout  send_data:  
14:05:35 DEBUG   send_data: daemon  handling node stop with exit status Success
14:05:35 INFO    send_data: daemon  send_data finished successfully
14:05:35 INFO    daemon  dataflow finished on machine `c7838683-2371-485b-a5ef-48131c0366b8`
2025-12-20T14:05:35.837503Z  INFO run_inner: dora_daemon: exiting daemon because all required dataflows are finished self.daemon_id=DaemonId { machine_id: None, uuid: c7838683-2371-485b-a5ef-48131c0366b8 }
2025-12-20T14:05:35.837534Z  INFO run_inner: zenoh::api::session: close session zid=9add8257a9b832f67a03b8526b17fc58 self.daemon_id=DaemonId { machine_id: None, uuid: c7838683-2371-485b-a5ef-48131c0366b8 }

Interestingly, it works on the other OSs. We also see some input events logged there, which are strangely absent on Linux.

@phil-opp
Copy link
Copy Markdown
Collaborator Author

phil-opp commented Dec 22, 2025

Interestingly, it works on the other OSs. We also see some input events logged there, which are strangely absent on Linux.

Correction: We skip the test on Windows, so it only works on macOS. We don't see any additional output there.

phil-opp referenced this pull request Dec 22, 2025
@haixuanTao
adding native logging
40de6a2
@phil-opp
Fill in node_id, dataflow_id, and daemon_id in daemon
fc87244 
Instead of providing it via the Python node API.

This approach has the advantage that it works consistently across languages. For example, the Rust node API was not setting these fields, leading to log messages without a node_id. This commit fixes that.
@phil-opp
Panic/error when trying to recv event after None was returned
02b9acd 
It will block forever.
@phil-opp
Remove default target from Python logs
849af49 
Also: Fall back to fields when `target` is the empty string
@phil-opp
Debug: set RUST_BACKTRACE=1
2c1299d
@phil-opp
Fix closed setting
436df6b
@phil-opp
Debug: Add a lot of logging
e36fd10
@phil-opp
Enable debug logging for Python CLI example
020b8cd
@phil-opp phil-opp force-pushed the ignore-unknown-node-events branch from fdb9b57 to 020b8cd Compare December 23, 2025 06:32
@phil-opp phil-opp merged commit f238150 into main Dec 23, 2025
50 checks passed
@phil-opp phil-opp deleted the ignore-unknown-node-events branch December 23, 2025 08:11
phil-opp added a commit that referenced this pull request Dec 23, 2025
@phil-opp
Revert "Make adding more NodeEvent variants backwards-compatible (#…
aaf01bc 
…1273)"

This reverts commit f238150.
@phil-opp phil-opp mentioned this pull request Dec 23, 2025
Merged
phil-opp added a commit that referenced this pull request Dec 23, 2025
@phil-opp
Revert "Make adding more NodeEvent variants backwards-compatible" (#…
b0d4f7a 
…1277)

Revert "Make adding more `NodeEvent` variants backwards-compatible (#1273)"

This reverts commit f238150.
@phil-opp
Copy link
Copy Markdown
Collaborator Author

phil-opp commented Dec 23, 2025

Oh, automerge was still on, so it was merged as soon as the CI was green, even though there are lots of debug changes now. I reverted this in #1277.

@phil-opp phil-opp mentioned this pull request Dec 23, 2025
Open
@phil-opp
Copy link
Copy Markdown
Collaborator Author

phil-opp commented Dec 23, 2025

New PR in #1278

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@haixuanTao haixuanTao haixuanTao approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@phil-opp @haixuanTao