System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
• Ray installed from (source or binary):
• Ray version: 0.5X
• Python version:
• Exact command to reproduce:

Describe the problem

Current backend logging level is fixed. If we want to change the logging level, we need to do re-compiling. Our users have the following requirements:

1. The logging level should be changed at runtime without re-compiling.
2. The logging level in the client dynamic lib (e.g. local_scheducler_client, plasma client in both java and python) should also be able to get changed.
3. The logging level for stderr and logging file should be different.

For Requirement 1, what do you think is the best practice?

1. Environment variable: users need to change the environment variables for all machines in a cluster, which is not convenient.
2. Argument: this is not convenient for dynamic libs.
3. Using config files: Java uses this method. Will backend use this also?

Source code / logs