Fix the failing GitHub Actions job for the Gallery Agent workflow in mudler/LocalAI.

Context:

• Failing workflow run: 27027765104
• Failing job: 79772009990
• Workflow file: .github/workflows/gallery-agent.yaml
• Commit where the failure was observed: e837921c2cd49ed91359e7b60dd277bc587148ec
• The failure occurs in the Run gallery agent step, which runs go run ./.github/gallery-agent with GALLERY_INDEX_PATH=$PWD/gallery/index.yaml.

Observed logs:

• The decisive failure is:
  • Loaded 1427 existing gallery entries
  • Searching for trending models on HuggingFace...
  • Error fetching models: failed to fetch models. Status code: 429
  • exit status 1
• There is also an earlier cache restore warning from /usr/bin/tar reporting many Cannot open: File exists messages while restoring the Go toolchain cache, but the job continues past that, prints go version go1.26.0 linux/amd64, and only fails later in the gallery-agent execution. Treat the 429 from Hugging Face as the root cause of the job failure.

Goal:
Make the gallery agent resilient to transient Hugging Face rate limiting so scheduled runs do not fail just because the upstream API returns HTTP 429.

Requirements:

1. Find the gallery agent code responsible for fetching trending models from Hugging Face.
2. Implement retry logic for transient HTTP failures, especially 429 responses.
3. Honor Retry-After when present.
4. Use exponential backoff with a sensible cap for retries.
5. Ensure non-retryable 4xx errors still fail fast.
6. Prefer a solution that keeps the workflow green for temporary upstream throttling during scheduled runs. If appropriate based on the code structure, treat persistent 429/rate-limit conditions as a no-op/skip rather than a hard failure, but do not hide genuine logic bugs.
7. Update or add tests if the repository has an appropriate test location/pattern for this code.
8. Keep the change focused; avoid unrelated workflow churn.

Acceptance criteria:

• The gallery agent no longer fails immediately on a single HTTP 429 from Hugging Face.
• Retries/backoff are implemented in code rather than by blindly rerunning the whole workflow.
• The resulting PR explains the root cause and the chosen mitigation.

Please open a pull request with the fix against master.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: Please find a solution for this failing job. Use the logs, job definition, and any referenced files where the failure occurred. Keep your response focused on the solution and include code suggestions when appropriate.

Assistant: The failure is in the Run gallery agent step of .github/workflows/gallery-agent.yaml at gallery-agent.yaml#L126-L136, where the workflow executes:

- name: Run gallery agent
  env:
    SEARCH_TERM: ${{ github.event.inputs.search_term || 'GGUF' }}
    LIMIT: ${{ github.event.inputs.limit || '15' }}
    QUANTIZATION: ${{ github.event.inputs.quantization || 'Q4_K_M' }}
    MAX_MODELS: ${{ github.event.inputs.max_models || '1' }}
    EXTRA_SKIP_URLS: ${{ steps.open_prs.outputs.urls }}
  run: |
    export GALLERY_INDEX_PATH=$PWD/gallery/index.yaml
    go run ./.github/gallery-agent

Root cause

The actual error in the logs is:

Loaded 1427 existing gallery entries
Searching for trending models on HuggingFace...
Error fetching models: failed to fetch models. Status code: 429
exit status 1

A 429 means the Hugging Face API rate-limited the job.
So this is not a codegen/protobuf/setup-go issue, and not the tar cache warning either — that restore warning is noisy but non-fatal. The job fails because the gallery agent exits immediately on a temporary upstream rate limit.

Solution

Make the gallery agent resilient to transient 429s by adding:

1. retry with exponential backoff
2. special handling for Retry-After
3. non-fatal behavior when no data can be fetched after retries for scheduled runs

Recommended code change

In the code that fetches Hugging Face models, wrap the HTTP call with retries. The exact file wasn’t included in the retrieved context, but the failing code is the Hugging Face model fetch path used by go run ./.github/gallery-agent.

Use a pattern like this:

package main

import (
	"errors"
	"fmt"
	"io"
	"math"
	"net/http"
	"strconv"
	"time"
)

func fetchWithRetry(url string) ([]byte, error) {
	client := &http.Client{
		Timeout: 30 * time.Second,
	}

	var lastErr error
	maxAttempts := 6

	for attempt := 1; attempt <= maxAttempts; attempt++ {
		resp, err := client.Get(url)
		if err != nil {
			lastErr = err
		} else {
			defer resp.Body.Close()

			if resp.StatusCode >= 200 && resp.StatusCode < 300 {
				return io.ReadAll(resp.Body)
			}

			if resp.StatusCode == http.StatusTooManyRequests {
				wait := retryDelay(res...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>