This pull request introduces significant improvements to the gowarc project by restructuring the CLI tools, enhancing usability, and improving code organization and documentation. The main changes include modularizing command implementations, introducing a robust logging and flag system, and providing comprehensive documentation for CLI usage, especially for the new mend command.

CLI mend command:

This command replicates the mend feature of warctool, allowing to properly truncate and close/rename .open WARC files left by gowarc in case of crashes.

CLI and Documentation Enhancements:

• Added a detailed "CLI Tools" section to README.md, documenting installation, available commands (extract, mend, verify, completion), usage examples, global flags, and linking to further documentation for mend.
• Introduced a new cmd/mend/README.md with extensive documentation on the mend command, outlining features, usage, safety, and example outputs.

Codebase Refactoring and Modularization:

• Refactored the extract command implementation:
  • Moved from cmd/extract.go to cmd/extract/extract.go for better modularity and maintainability.
  • Updated imports to use a new utils package for shared functionality.
  • Centralized flag parsing and record filtering logic into utility functions, improving code reuse and clarity.
  • Replaced hardcoded constants (e.g., permissions, filename lengths) with named constants from utils for better maintainability [1] [2] [3] [4] [5].
  • Added a summary report function after extraction completes.

CLI Framework and Logging Improvements:

• Updated cmd/main.go to:
  • Register subcommands (extract, mend, verify) using their modularized Command objects instead of monolithic command definitions [1] [2].
  • Add global flags for JSON output and verbose logging, and configure a global logger using slog based on these flags.
  • Remove duplicate or now-unnecessary flag definitions from the main file, delegating them to each command module.

These changes collectively improve the usability, maintainability, and extensibility of the gowarc CLI, making it easier for users to interact with WARC files and for developers to add new features in the future.

References:

• [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]