Skip to content

BYNT-1274: Configure Logging Behavior#51

Merged
tarekio merged 22 commits intomainfrom
BYNT-1274
Jun 18, 2025
Merged

BYNT-1274: Configure Logging Behavior#51
tarekio merged 22 commits intomainfrom
BYNT-1274

Conversation

@cango91
Copy link
Contributor

@cango91 cango91 commented Jan 3, 2025

Jira Issue

  1. BYNT-1274

Description

[Brief description of changes]

Checklist

  • Tests added/updated
  • Documentation updated (if needed)
  • New strings prepared for translations

API Changes (if applicable)

  • Permissions checked
  • Endpoint tests added

Additional Notes

[Any other relevant information]

@cango91 cango91 self-assigned this Jan 3, 2025
@cango91 cango91 requested a review from tarekio January 3, 2025 15:48
@cango91 cango91 requested a review from tarekio January 14, 2025 10:43
Copy link
Contributor

@tarekio tarekio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is better, but the changes have introduced a new issues. Now there seem to be a lot more log entries added to both the app log and and log file when each is active. These log entries don't seem to respect the log level set. Below a snippet:

{"timestamp": 1736874827.8392801, "level": "DEBUG", "message": "Took 0.000s to subset 'hmtx'", "logger": "fontTools.subset.timer", "pathname": "env/lib/python3.12/site-packages/fontTools/misc/loggingTools.py", "lineno": 357}
{"timestamp": 1736874827.8393404, "level": "INFO", "message": "hmtx subsetted", "logger": "fontTools.subset", "pathname": "env/lib/python3.12/site-packages/fontTools/subset/__init__.py", "lineno": 3527}
{"timestamp": 1736874827.8394225, "level": "DEBUG", "message": "Reading 'GDEF' table from disk", "logger": "fontTools.ttLib.ttFont", "pathname": "env/lib/python3.12/site-packages/fontTools/ttLib/ttFont.py", "lineno": 467}
{"timestamp": 1736874827.8395038, "level": "DEBUG", "message": "Decompiling 'GDEF' table", "logger": "fontTools.ttLib.ttFont", "pathname": "env/lib/python3.12/site-packages/fontTools/ttLib/ttFont.py", "lineno": 476}
{"timestamp": 1736874827.839723, "level": "DEBUG", "message": "Took 0.000s to subset 'GDEF'", "logger": "fontTools.subset.timer", "pathname": "env/lib/python3.12/site-packages/fontTools/misc/loggingTools.py", "lineno": 357}
{"timestamp": 1736874827.839787, "level": "INFO", "message": "GDEF subsetted", "logger": "fontTools.subset", "pathname": "env/lib/python3.12/site-packages/fontTools/subset/__init__.py", "lineno": 3527}
{"timestamp": 1736874827.8398619, "level": "INFO", "message": "name subsetting not needed", "logger": "fontTools.subset", "pathname": "env/lib/python3.12/site-packages/fontTools/subset/__init__.py", "lineno": 3516}
{"timestamp": 1736874827.8399282, "level": "DEBUG", "message": "Took 0.000s to subset GlyphOrder", "logger": "fontTools.subset.timer", "pathname": "env/lib/python3.12/site-packages/fontTools/misc/loggingTools.py", "lineno": 357}
{"timestamp": 1736874827.840055, "level": "DEBUG", "message": "Reading 'head' table from disk", "logger": "fontTools.ttLib.ttFont", "pathname": "env/lib/python3.12/site-packages/fontTools/ttLib/ttFont.py", "lineno": 467}
{"timestamp": 1736874827.8401403, "level": "DEBUG", "message": "Decompiling 'head' table", "logger": "fontTools.ttLib.ttFont", "pathname": "env/lib/python3.12/site-packages/fontTools/ttLib/ttFont.py", "lineno": 476}
{"timestamp": 1736874827.8402147, "level": "DEBUG", "message": "Took 0.000s to prune 'head'", "logger": "fontTools.subset.timer", "pathname": "env/lib/python3.12/site-packages/fontTools/misc/loggingTools.py", "lineno": 357}

You can replicate by approving an export request, guessing this is coming from weasyprint but could be wrong.

@cango91
Copy link
Contributor Author

cango91 commented Jan 15, 2025

This is better, but the changes have introduced a new issues. Now there seem to be a lot more log entries added to both the app log and and log file when each is active. These log entries don't seem to respect the log level set. Below a snippet:

{"timestamp": 1736874827.8392801, "level": "DEBUG", "message": "Took 0.000s to subset 'hmtx'", "logger": "fontTools.subset.timer", "pathname": "env/lib/python3.12/site-packages/fontTools/misc/loggingTools.py", "lineno": 357}
{"timestamp": 1736874827.8393404, "level": "INFO", "message": "hmtx subsetted", "logger": "fontTools.subset", "pathname": "env/lib/python3.12/site-packages/fontTools/subset/__init__.py", "lineno": 3527}
{"timestamp": 1736874827.8394225, "level": "DEBUG", "message": "Reading 'GDEF' table from disk", "logger": "fontTools.ttLib.ttFont", "pathname": "env/lib/python3.12/site-packages/fontTools/ttLib/ttFont.py", "lineno": 467}
{"timestamp": 1736874827.8395038, "level": "DEBUG", "message": "Decompiling 'GDEF' table", "logger": "fontTools.ttLib.ttFont", "pathname": "env/lib/python3.12/site-packages/fontTools/ttLib/ttFont.py", "lineno": 476}
{"timestamp": 1736874827.839723, "level": "DEBUG", "message": "Took 0.000s to subset 'GDEF'", "logger": "fontTools.subset.timer", "pathname": "env/lib/python3.12/site-packages/fontTools/misc/loggingTools.py", "lineno": 357}
{"timestamp": 1736874827.839787, "level": "INFO", "message": "GDEF subsetted", "logger": "fontTools.subset", "pathname": "env/lib/python3.12/site-packages/fontTools/subset/__init__.py", "lineno": 3527}
{"timestamp": 1736874827.8398619, "level": "INFO", "message": "name subsetting not needed", "logger": "fontTools.subset", "pathname": "env/lib/python3.12/site-packages/fontTools/subset/__init__.py", "lineno": 3516}
{"timestamp": 1736874827.8399282, "level": "DEBUG", "message": "Took 0.000s to subset GlyphOrder", "logger": "fontTools.subset.timer", "pathname": "env/lib/python3.12/site-packages/fontTools/misc/loggingTools.py", "lineno": 357}
{"timestamp": 1736874827.840055, "level": "DEBUG", "message": "Reading 'head' table from disk", "logger": "fontTools.ttLib.ttFont", "pathname": "env/lib/python3.12/site-packages/fontTools/ttLib/ttFont.py", "lineno": 467}
{"timestamp": 1736874827.8401403, "level": "DEBUG", "message": "Decompiling 'head' table", "logger": "fontTools.ttLib.ttFont", "pathname": "env/lib/python3.12/site-packages/fontTools/ttLib/ttFont.py", "lineno": 476}
{"timestamp": 1736874827.8402147, "level": "DEBUG", "message": "Took 0.000s to prune 'head'", "logger": "fontTools.subset.timer", "pathname": "env/lib/python3.12/site-packages/fontTools/misc/loggingTools.py", "lineno": 357}

You can replicate by approving an export request, guessing this is coming from weasyprint but could be wrong.

Couldn't recreate. Restarting app + celery after changing log level in the .env file, both respect the log level on my end.

Copy link
Contributor

@tarekio tarekio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect now. @level09 please review and approve.

@tarekio tarekio requested a review from level09 January 15, 2025 18:47
Copy link
Collaborator

@level09 level09 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried setting APP_LOG_ENABLED to True and False as well as CELERY_LOG_ENABLED but couldn't see any difference. logs are always channeled to the log handler and never show in the stdout.

Did I miss something ? I think the names of the variables is also confusing. unless they suppress the logs when turned to False.

@cango91
Copy link
Contributor Author

cango91 commented Jan 21, 2025

If APP_LOG_ENABLED is False, neither our app logs nor stdout should get any logging (so it suppresses all logging from within the app). When CELERY_LOG_ENABLED is False, our app logs don't capture any Celery logs, but Celery's normal logging behavior should continue to function, however respecting our app's LOG_LEVEL parameter instead of Celery's defaults.

@tarekio tarekio requested a review from level09 June 10, 2025 19:06
Copy link
Collaborator

@level09 level09 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good overall. just found a small issue that could cause race condition /inefficiency


# Only add handler if it doesn't exist to prevent duplicate logging
if not logger.handlers:
if cfg.APP_LOG_ENABLED:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If two threads call the get_logger method they could overwrite the handlers and cause issues. I suggest this simple check to fix it:

 if not logger.handlers:
        if cfg.APP_LOG_ENABLED:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added in a5cae4f

LOG_LEVEL = os.environ.get("LOG_LEVEL", "INFO")
APP_LOG_ENABLED = os.environ.get("APP_LOG_ENABLED", "True").lower() == "true"
CELERY_LOG_ENABLED = os.environ.get("CELERY_LOG_ENABLED", "True").lower() == "true"
LOG_LEVEL = os.environ.get("LOG_LEVEL", "ERROR")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we decided to have info let's default to INFO here for consistency

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in a5cae4f

cango91 added 2 commits June 17, 2025 13:25
- Added if not logger.handlers: check before adding handlers to prevent duplicate logging if multiple threads call get_logger simultaneously.
- Changed the default from "ERROR" to "INFO" for consistency as mentioned in the PR discussion.
@tarekio tarekio requested a review from level09 June 18, 2025 14:37
Copy link
Collaborator

@level09 level09 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All Good !

@tarekio tarekio merged commit e12a912 into main Jun 18, 2025
6 of 7 checks passed
@tarekio tarekio deleted the BYNT-1274 branch June 18, 2025 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants