System Info

Platform : Windows 11

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder
• My own task or dataset (give details below)

Reproduction

To reproduce the issue, follow these steps:

• Run the script with a set of log files.
• Notice the increased processing time when dealing with a large number of logs.
• Observe the redundancy in file operations and data handling.

Expected behavior

The script should handle large sets of log files more efficiently, avoiding redundant operations and optimizing the processing loop.
The current implementation could benefit from batch processing techniques, improved file handling, and more efficient data parsing mechanisms.