Skip to content

main to 3.1.0#55

Merged
jaimetur merged 4 commits into3.1.0from
main
Mar 17, 2025
Merged

main to 3.1.0#55
jaimetur merged 4 commits into3.1.0from
main

Conversation

@jaimetur
Copy link
Copy Markdown
Owner

No description provided.

@jaimetur jaimetur merged commit 868f7d9 into 3.1.0 Mar 17, 2025
jaimetur added a commit that referenced this pull request Jul 30, 2025
- ### Main Changes:
  - #### 🚀 Enhancements:
    - [x] Updated GPTH to version `4.1.0` (by @Xentraxx) which includes several new features, improvements and bugs fixing extracting metadata info from Google Takeouts. 

      - #### ✨ **New Features**
          - **Partner Sharing Support** - Added `--divide-partner-shared` flag to separate partner shared media from personal uploads into dedicated `PARTNER_SHARED` folder (Issue #56)
            - Automatically detects partner shared photos from JSON metadata (`googlePhotoOrigin.fromPartnerSharing`)
            - Creates separate folder structure while maintaining date division and album organization
            - Works with all album handling modes (shortcut, duplicate-copy, reverse-shortcut, json, nothing)
            - Preserves album relationships for partner shared media
          - **Added folder year date extraction strategy** - New fallback date extractor that extracts year from parent folder names like "Photos from 2005" when other extraction methods fail (Issue #28)
          - **Centralized concurrency management** - Introduced `ConcurrencyManager` for consistent concurrency calculations across all services, eliminating hardcoded multipliers scattered throughout the codebase
          - **Displaying version of Exiftool when found** - Instead of just displaying that Exif tool was found, we display the version now as well.
      - #### 🚀 **Performance Improvements**
      - **EXIF processing optimization** - Native `exif_reader` library integration for 15-40% performance improvement in EXIF data extraction
        - Uses fast native library for supported formats (JPEG, TIFF, HEIC, PNG, WebP, AVIF, JXL, CR3, RAF, ARW, DNG, CRW, NEF, NRW)
        - Automatic fallback to ExifTool for unsupported formats or when native extraction fails
        - Centralized MIME type constants verified against actual library source code
        - Improved error logging with GitHub issue reporting guidance when native extraction fails
      - **GPS coordinate extraction optimization** - Dedicated coordinate extraction service with native library support
        - 15-40% performance improvement for GPS-heavy photo collections
        - Clean architectural separation between date and coordinate extraction
        - Centralized MIME type support across all EXIF processing operations
      - **Significantly increased parallelization** - Changed CPU concurrency multiplier from ×2 to ×8 for most operations, dramatically improving performance on multi-core systems
      - **Removed concurrency caps** - Eliminated `.clamp()` limits that were artificially restricting parallelization on high-core systems
      - **Platform-optimized concurrency**:
        - **Linux**: Improved from `CPU cores + 1` to `CPU cores × 8` (massive improvement for Linux users)
        - **macOS**: Improved from `CPU cores + 1` to `CPU cores × 6` 
        - **Windows**: Maintained at `CPU cores × 8` (already optimized)
      - **Operation-specific concurrency tuning**:
        - **Hash operations**: `CPU cores × 4` (balanced for CPU + I/O workload)
        - **EXIF/Metadata**: `CPU cores × 6` (I/O optimized for modern SSDs)
        - **Duplicate detection**: `CPU cores × 6` (memory intensive, conservative)
        - **Network operations**: `CPU cores × 16` (high for I/O waiting)
      - **Adaptive concurrency scaling** - Dynamic performance-based concurrency adjustment that scales up to ×24 for high-performance scenarios
    - #### 🐛 **Bug Fixes**
      - **Fixed memory exhaustion during ZIP extraction** - Implemented streaming extraction to handle large ZIP files without running out of memory
      - **Fixed atomic file operations** - Changed to atomic file rename operations to resolve situations where only the json was renamed in file extension correction (Issue #60)
      - **Fixed album relationship processing** - Improved album relationship service to handle edge cases properly (Issue #61)
      - **Fixed interactive presenter display** - Corrected display issue in interactive mode (Issue #62)
      - **Fixed date division behavior for albums** - The `--divide-to-dates` flag now only applies to ALL_PHOTOS folder, leaving album folders flattened without date subfolders (Issue #55)
      - **Reaorganised ReadMe for a more intuitive structure** - First Installation, then prerequisites and then the quickstart.
      - **Step 8 now also uses a progress bar instead of simple print statements**
      - **Supressed some unnecessary ouput**

  - #### 📚 Documentation: 
    - [x] Updated documentation with all changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant