This PR makes it possible to use both rust and python native logging system and get formatting and filtering within dora run. This also persist the logs within the logging filesystem as json making them searchable through tool like elastic search, Grafana Loki, ...

This improve the current stdout by adding:

• file path information
• line id
• levels
• target
• and fields that can be fully customizable
• usable with send_stdout_as and receive event as text json lines in order to use a custom log aggregator user node.
• Usable with opentelemetry in order to use an external opentelemetry aggregator / collector

To use in rust:

use tracing:warn:

warn!("my message"); 

To use in python

import logging

logging.warn("my message")

To filter log, just as before use RUST_LOG env variable.

The output is going to be formatted as is:

Output

INFO  receive_data_with_sleep: root  info [108171982329125]
DEBUG receive_data_with_sleep: __main__  received tick with data
INFO  receive_data_with_sleep: root  info [108171991792958]
DEBUG receive_data_with_sleep: __main__  received tick with data
INFO  receive_data_with_sleep: root  info [108172002233958]
DEBUG receive_data_with_sleep: __main__  received tick with data
INFO  receive_data_with_sleep: root  info [108172012020041]
DEBUG receive_data_with_sleep: __main__  received tick with data
INFO  receive_data_with_sleep: root  info [108172022420791]
DEBUG receive_data_with_sleep: __main__  received tick with data
INFO  receive_data_with_sleep: root  info [108172032057625]
DEBUG receive_data_with_sleep: __main__  received tick with data

You will find attached example within the PR