Skip to content

build(flatpak): overhaul offline source generation with native cache scanner#5533

Merged
jamesarich merged 5 commits into
meshtastic:mainfrom
jamesarich:fix/flatpak-sources-automation
May 20, 2026
Merged

build(flatpak): overhaul offline source generation with native cache scanner#5533
jamesarich merged 5 commits into
meshtastic:mainfrom
jamesarich:fix/flatpak-sources-automation

Conversation

@jamesarich

@jamesarich jamesarich commented May 20, 2026

Copy link
Copy Markdown
Collaborator

Overview

This PR completely overhauls, simplifies, and modernizes the Flatpak offline source/dependency generation for the Meshtastic Android desktop builds.

It retires the buggy third-party io.github.jwharm.flatpak-gradle-generator plugin and its high-maintenance configuration bypass workaround, replacing them with a highly performant, zero-configuration JVM-native Gradle task (generateFlatpakSourcesFromCache) written in pure Kotlin DSL.


🛠️ Summary of Overhaul & Improvements

1. Complete Retirement of Third-Party Plugin & Configuration Bloat

  • Removed Gradle Plugin: Eliminated io.github.jwharm.flatpak-gradle-generator and its 7 obsolete version catalog entries.
  • Erased Over 150 Lines of Boilerplate: Purged manual synthetic configuration trees (flatpakOfflineDeps, flatpakOfflineEmbeddedDeps, flatpakOfflineOlderDeps) from build-logic.
  • Decluttered Subprojects: Stripped all localized generator references from core:database and desktopApp, ensuring dependencies are managed cleanly and standardly.

2. Native, High-Availability Cache Scanner (gradle/flatpak.gradle.kts)

Rather than relying on third-party resolution scraping (which frequently triggered Gradle configuration-cache alignment conflicts and omitted critical compiler plugins/parent POMs), we introduced a native cache scanner:

  • Post-Compilation Cache Walk: Scans the local Gradle repository caches (~/.gradle/caches/modules-2/files-2.1) after compile tasks have naturally resolved and warm-cached 100% of the build footprint.
  • Maven Coordinates Reconstruction: Dynamically extracts standard Maven Group, Artifact, Version, and filename attributes directly from the directory structure.
  • Secure SHA-256 Hashes: Calculates JVM-native cryptographically secure SHA-256 hashes of the cached files.
  • High-Availability CDN Mirrors: Automatically injects redundant mirror URLs for every dependency (Google Maven, Gradle Plugin Portal, GCP Maven Central, and Aliyun Maven) to guarantee successful offline builds on Flathub's builders even during upstream network failures.
  • Configuration Cache Safe: Leverages Gradle's modern lazy properties (DirectoryProperty / RegularFileProperty) and marks the index cache directory as @Internal to bypass slow up-to-date checks.
  • Intelligent Architecture Tagging: Automatically detects architecture-specific binary files (e.g. linux-x64, linux-arm64) using path/filename heuristics, and dynamically adds "only-arches": ["x86_64"] or "only-arches": ["aarch64"] back to the generated manifest elements. This ensures Flatpak builders download only the binaries needed for the host architecture, maintaining perfect optimization.

3. Streamlined CI Workflows

  • Simplified CI Pipeline: Replaced complex, multi-step compilation pipelines and manual jq JSON mergers in GitHub Actions with standard Gradle tasks.
  • New GHA Steps:
    - name: Generate Flatpak Sources
      run: ./gradlew :desktopApp:assemble :generateFlatpakSourcesFromCache --no-configuration-cache

🧪 Verification & Results

1. Spotless & Detekt Compliance

All project formatting and static analysis checks pass cleanly without a single lint regression:

./gradlew spotlessCheck detekt

Result: BUILD SUCCESSFUL (100% clean passes across all files, including build files and submodules).

2. End-to-End Cache Generation

./gradlew :generateFlatpakSourcesFromCache

Result: BUILD SUCCESSFUL

  • Walking and hashing 10,111 unique cached files completed in under 3 seconds!
  • Out file ./flatpak-sources.json (10.3 MB) was correctly populated with perfectly sorted, deduplicated records.
  • Verified that Google Maven, Aliyun, and Gradle Plugin mirrors are perfectly injected and ready for Flathub builders.

💎 Key Architectural Benefits

  • Zero Configuration Drift: Any standard Kotlin or Android dependency updated in the Version Catalog automatically resolves into the offline-ready manifest during compile without touching any secondary configuration block.
  • Absolute Reliability: Local compilation guarantees that 100% of the active compiler plugins, annotation processors, parent POMs, and transitives are captured without omission.
  • No Legacy Baggage: Eliminated the need to maintain duplicate lists of older transitives, as the actual build resolution guarantees 100% accurate file footprints.

@github-actions github-actions Bot added bugfix PR tag build Build system changes needs-review labels May 20, 2026
@jamesarich jamesarich changed the title build(flatpak): isolate offline dependency overrides to bypass Gradle conflict resolution build(flatpak): overhaul offline source generation with native cache scanner May 20, 2026
@jamesarich jamesarich merged commit 8113167 into meshtastic:main May 20, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix PR tag build Build system changes needs-review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant