Skip to content
This repository was archived by the owner on Mar 16, 2026. It is now read-only.

Replace custom logger with stdlib logging#17

Merged
Wenzel merged 9 commits intoIntelLabs:masterfrom
Wenzel:use_stdlib_logging
Oct 14, 2022
Merged

Replace custom logger with stdlib logging#17
Wenzel merged 9 commits intoIntelLabs:masterfrom
Wenzel:use_stdlib_logging

Conversation

@Wenzel
Copy link
Copy Markdown
Contributor

@Wenzel Wenzel commented Oct 4, 2022

This PR replaces the custom logger.py with a python stdlib logging library implementation.

  • the main logging configuration is defined in kafl_fuzzer/logging.yaml (file is installed with the kafl_fuzzer package)
  • the logging library is configured with dictConfig
  • Workers are using a logging.LoggerAdapter to prefix their own messages with os.getpid() and their own fuzzer pid as well
    • as a consequence, there is no need anymore to prefix the logging messages with logging.info('%s ....', % self)
  • log file output has renamed from debug.log to fuzzer.log in the work dir

Example output for workers:
image

Help to fix #1

Opened Items

  • initialize the logger in 2 phases, first allowing console logging, and then adding the file logging handler when the workdir is ready (and has been cleaned up if necessary)
  • handle concurrent logging from multiple processes into the same file. An approach is proposed on the Python documentation based on a SocketHandler

@Wenzel Wenzel force-pushed the use_stdlib_logging branch 4 times, most recently from e1799bc to 34bc4be Compare October 5, 2022 11:55
@Wenzel Wenzel marked this pull request as ready for review October 5, 2022 12:00
@il-steffen
Copy link
Copy Markdown
Contributor

il-steffen commented Oct 11, 2022

Thanks a lot!

Comparing some old/new log output:
0:00:39.750624 Worker-12 exit code: None => INFO:kafl_fuzzer.worker.qemu:exit code: None

  • the file log lost the full time stamp (time since fuzzer start) - I think we should keep this since one often considers fuzzer activity over longer time
  • qemu still generates its own Worker-N prefix - maybe we hand it the logger created by Worker?
  • the "worker-N" prefix got lost when changing the logger.file_log() in qemu.shutdown()
  • lets rename fuzzer.log to kafl_fuzzer.log? "fuzzer" is too generic, "kafl_fuzzer" maps to the python name.

The level+module prefix is nice but unfortunately destroys alignment, making it really hard to find something useful in the log:

DEBUG:kafl_fuzzer.manager.manager:Received new input (exit=regular): b'\xe4Gh)q]\x80]d\x13FU\xee\xa3\xc9\xfb\x9f\xe6\xd1\xccQ\x9c\x0e:'
DEBUG:kafl_fuzzer.worker.worker:767061:Worker-34:Stage initial found alternative payload for node 24
DEBUG:kafl_fuzzer.worker.worker:767061:Worker-34:Stage initial found alternative payload for node 56
DEBUG:kafl_fuzzer.technique.trim:stream_extend: detected 65567 padding bytes
DEBUG:kafl_fuzzer.technique.trim:stream_extend: dropped funky NUL padding (len=65567, other finds=0)
DEBUG:kafl_fuzzer.technique.trim:stream_extend: detected 64831 padding bytes
DEBUG:kafl_fuzzer.manager.manager:Received new input (exit=timeout): b'\xcd\xa2\xb1\x05\xe6o\xbb5\xcex\xcf\x0c\xc4=\xd2\xd8\xe1=\xffH\xe4\xde\xb4A'
DEBUG:kafl_fuzzer.manager.manager:Received new input (exit=regular): b'\xcd\xa2\xb1\x05\xe6o\xbb5\xcex\xcf\x0c\xc4=\xd2\xd8\xe1=\xffH\xe4\xde\xb4A'
0:00:57.640407 Received new input (exit=regular): b'\xc3\xfb\xb0,26\x1a\x95c\xf1\xaeNi\x8f\xbd\xde\x8c\x8d\x081\xdcx\x04('
0:00:57.640693 Received duplicate payload with exit=regular, discarding.
0:00:57.665143 stream_extend: detected 65567 padding bytes
0:00:57.705712 Received new input (exit=regular): b'\xc3\xfb\xb0,26\x1a\x95c\xf1\xaeNi\x8f\xbd\xde\x8c\x8d\x081\xdcx\x04('
0:00:57.718928 Worker-03 HAVOC times: afl: 7.8, splice: 19.2, grim: 0.0, rdmsa: 0.0
0:00:57.721715 Worker-62 HAVOC times: afl: 7.4, splice: 18.6, grim: 0.0, rdmsa: 0.0
0:00:57.730822 Worker-62 RedQ-Dict: Have performed 0 iters
0:00:57.733465 Worker-01 HAVOC times: afl: 7.9, splice: 19.5, grim: 0.0, rdmsa: 0.0
0:00:57.766072 Worker-01 Stage initial found alternative payload for node 39

I think we want to use names based on subsystems, not module hierarchy, and append them to the Worker-N notation for alignment. Similar to shown in the last few lines here. This is not totally consistent in the code right now, but we can fix it some other time. (Some modules need to be turned into a class first so that we can properly hand down a logger module to them.)
[Edit: To clarify: lets have Worker-N as automated prefix and hand down that object to use by other modules whenever currently possible. The sub-components should add a string if desired, eg. logger.info("RedQueen: starting redqueen mutations..") ]

One more thing - maybe we want to restore console output to normal verbosity once --log is provided?

@Wenzel
Copy link
Copy Markdown
Contributor Author

Wenzel commented Oct 11, 2022

Thanks for the review !

TODO overview

  • rebase PR after Basic CI/CD #16 merge
  • rename fuzzer.log to kafl_fuzzer.log
  • add timestamp prefix in file log
  • qemu.shutdown(): worker-N prefix git lost
  • qemu still generates its own Worker-N prefix - maybe we hand it the logger created by Worker?
  • file log format breaks alignment and makes it hard to follow. Rethink file log output
  • restore console output to normal if --log

@Wenzel Wenzel force-pushed the use_stdlib_logging branch from 34bc4be to e1d5e67 Compare October 11, 2022 14:07
@Wenzel
Copy link
Copy Markdown
Contributor Author

Wenzel commented Oct 11, 2022

@il-steffen regarding the "Worker-N" prefix, I moved the WorkerLogAdapter into common/logger.py, and used it in worker.py and qemu.py this time.
(before your comment, I only updated worker.py.

Is this output the one you expect ? (in terms of Worker-xx prefix)
image

@il-steffen
Copy link
Copy Markdown
Contributor

Yes that is how it should look. I had the same exit code line above from the old logger out:
0:00:39.750624 Worker-12 exit code: None

I would avoid the extra colon separators as well, or put a space after colon.

@Wenzel Wenzel force-pushed the use_stdlib_logging branch from a35dfb4 to 681eeb8 Compare October 14, 2022 12:10
@Wenzel Wenzel force-pushed the use_stdlib_logging branch from 681eeb8 to 877440b Compare October 14, 2022 12:13
@Wenzel Wenzel requested a review from il-steffen October 14, 2022 12:14
@Wenzel Wenzel merged commit 4a8bc1d into IntelLabs:master Oct 14, 2022
@Wenzel Wenzel deleted the use_stdlib_logging branch October 14, 2022 12:19
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

logger can cause worker to crash with print

2 participants