Separate configurations for colorized and json logs in Task SDK / Celery Executor#51082
Merged
amoghrajesh merged 4 commits intoapache:mainfrom Jun 3, 2025
Conversation
…'logging_processors' and 'configure_logging' functions. This allows for customizable console output with or without colors based on user preference or configuration settings.
uranusjr
reviewed
May 27, 2025
uranusjr
reviewed
May 27, 2025
amoghrajesh
reviewed
May 27, 2025
Contributor
|
@highlyavailable would you be able to address review comments so we can target this for Airflow 3.0.2 whose RC cut is on Tuesday? |
Contributor
Author
|
Yes I will get it done this morning |
- Remove conditional imports, use direct \ - Rename parameter from \ to \ to match existing config - Remove unrelated formatting options (show_locals) from color logic - Update tests to reflect the changes
Contributor
Author
@amoghrajesh I believe my most recent commit addresses all of the feedback in this thread. |
Contributor
|
@uranusjr does the PR address your review comments? |
uranusjr
approved these changes
Jun 3, 2025
amoghrajesh
approved these changes
Jun 3, 2025
Contributor
amoghrajesh
left a comment
There was a problem hiding this comment.
One comment, more like a nit, can be addressed now or in a followup.
phanikumv
approved these changes
Jun 3, 2025
|
Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions. |
github-actions bot
pushed a commit
that referenced
this pull request
Jun 3, 2025
…sk SDK / Celery Executor (#51082) * Enhance logging configuration by adding 'enable_colors' parameter to 'logging_processors' and 'configure_logging' functions. This allows for customizable console output with or without colors based on user preference or configuration settings. (cherry picked from commit e25351d) Co-authored-by: Peter Bryant <pwbryant1@gmail.com>
github-actions bot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Jun 3, 2025
…sk SDK / Celery Executor (apache#51082) * Enhance logging configuration by adding 'enable_colors' parameter to 'logging_processors' and 'configure_logging' functions. This allows for customizable console output with or without colors based on user preference or configuration settings. (cherry picked from commit e25351d) Co-authored-by: Peter Bryant <pwbryant1@gmail.com>
kaxil
pushed a commit
that referenced
this pull request
Jun 3, 2025
…sk SDK / Celery Executor (#51082) (#51344) * Enhance logging configuration by adding 'enable_colors' parameter to 'logging_processors' and 'configure_logging' functions. This allows for customizable console output with or without colors based on user preference or configuration settings. (cherry picked from commit e25351d) Co-authored-by: Peter Bryant <pwbryant1@gmail.com>
kaxil
pushed a commit
that referenced
this pull request
Jun 3, 2025
…sk SDK / Celery Executor (#51082) (#51344) * Enhance logging configuration by adding 'enable_colors' parameter to 'logging_processors' and 'configure_logging' functions. This allows for customizable console output with or without colors based on user preference or configuration settings. (cherry picked from commit e25351d) Co-authored-by: Peter Bryant <pwbryant1@gmail.com>
sanederchik
pushed a commit
to sanederchik/airflow
that referenced
this pull request
Jun 7, 2025
…ery Executor (apache#51082) * Enhance logging configuration by adding 'enable_colors' parameter to 'logging_processors' and 'configure_logging' functions. This allows for customizable console output with or without colors based on user preference or configuration settings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #50963
This PR separates the control of log format (console vs JSON) from log styling (colors and rich traces) in the Task SDK logging configuration. Previously, the single
enable_pretty_logboolean controlled both aspects, making it impossible to have console logs without colors and rich traces.Problem
Solution
configure_logging()- uses defaultenable_colors=None→ reads configlogging_processors(enable_pretty_log=True)- uses defaultenable_colors=TrueNew Behavior Matrix:
Testing
All new & existing tests are passing, as well as the pre-commit checks. I wrote some simple unit tests in the test SDK,
test_logging_processors_with_colors(): Verifies colored console renderertest_logging_processors_without_colors(): Verifies non-colored console renderertest_logging_processors_json_format(): Verifies JSON renderer unchangedtest_configure_logging_respects_colored_console_log_config(): Tests configuration readingtest_configure_logging_explicit_enable_colors(): Tests explicit parameter overridetest_configure_logging_no_airflow_config(): Tests fallback behaviorSince I am new to the project, I also wrote a short testing script (attached as a
.txt) and ran it in my local Breeze environment, here were the results:demo_test.txt