Skip to content

Mirror web/browser plugin migration for FAL image generation backend #26241

@kshitijk4poor

Description

@kshitijk4poor

Summary

Migrate the FAL backend out of tools/image_generation_tool.py and into plugins/image_gen/fal/, matching the architecture established by PR #25182 (web), PR #25214 (browser), and the existing image_gen plugins (OpenAI, xAI, openai-codex). After this lands, TOOL_CATEGORIES["image_gen"]["providers"] drops to zero hardcoded rows — same shape as video_gen.

This is the explicit follow-up referenced in hermes_cli/tools_config.py::_plugin_image_gen_providers:

"FAL is skipped — it's already exposed by the hardcoded TOOL_CATEGORIES["image_gen"] entries. When FAL gets ported to a plugin in a follow-up PR, the hardcoded entries go away and this function surfaces it alongside OpenAI automatically."

Current state (what to migrate)

Backend code — all in tools/image_generation_tool.py (~700 LoC of FAL-specific surface):

  • 18-model catalog (_MODELS dict at line ~140-323): fal-ai/flux-2/klein/9b (default), fal-ai/flux-pro/v1.1, fal-ai/recraft-v3, fal-ai/imagen4/preview, etc. — each with per-model default_aspect_ratio, default_quality, default_steps, etc.
  • DEFAULT_MODEL = "fal-ai/flux-2/klein/9b" (line 326)
  • Managed Nous gateway integration (lines ~352-471):
    • _resolve_managed_fal_gateway() — consults resolve_managed_tool_gateway("fal-queue")
    • _ManagedFalSyncClient — wraps fal_client.SyncClient for Nous-subscription users with idempotency-key handling
    • _get_managed_fal_client() + _managed_fal_client + _managed_fal_client_config + lock
  • Direct FAL SDK path_submit_fal_request() (line ~472) dispatches between managed + direct
  • Payload builder_build_fal_payload() (line ~561) — model-specific knob mapping
  • Built-in upscaler_upscale_image() via fal-ai/clarity-upscaler (line ~607); fires after image_generate_tool succeeds when the per-model spec says upscale: True
  • Resolution helpers_resolve_fal_model(), _read_configured_image_model(), _extract_http_status()
  • check_fal_api_key() — reaches into both gateway state + FAL_KEY env var

Dispatcher_dispatch_to_plugin_provider() at image_generation_tool.py:993 explicitly skips FAL:

if not configured or configured == "fal":
    return None  # fall through to in-tree FAL path

Picker_plugin_image_gen_providers() in hermes_cli/tools_config.py:1467-1470 skips FAL because it's surfaced via the hardcoded TOOL_CATEGORIES["image_gen"]["providers"] entries.

Why this is bigger than #25214 looked

Three things that don't apply to the browser migration:

  1. FAL is the default backend. Browser providers are all opt-in via explicit browser.cloud_provider. FAL fires whenever image_gen.provider is unset OR explicitly set to "fal". The plugin needs to participate cleanly in both code paths — including the no-config fall-through that _dispatch_to_plugin_provider currently skips.

  2. Managed Nous gateway plumbing. Browser providers manage their own credentials end-to-end. FAL has a gateway-vs-direct selection inside the provider AND check_image_generation_requirements() consults the gateway state for the requirements check. The plugin needs to either inherit the gateway code or call back into a shared helper.

  3. video_gen FAL plugin already exists at plugins/video_gen/fal/__init__.py with its own _load_fal_client lazy-import + direct fal_client.subscribe() calls. It does NOT have managed-gateway support today — it's direct-only. Image-gen FAL should likely share the fal_client plumbing with video_gen rather than each plugin re-implementing it. Right ordering is:

    a. Factor out tools/fal_common.py (lazy fal_client load, managed-gateway resolution, _ManagedFalSyncClient, _submit_fal_request).
    b. Migrate image_gen FAL → plugins/image_gen/fal/ consuming tools.fal_common.
    c. Refactor plugins/video_gen/fal/__init__.py to consume the same tools.fal_common (drops its inline _load_fal_client duplicate).
    d. (Optional) Add managed-gateway support to video_gen FAL while we're in there.

What "applying the #25214 template" means concretely

Step 1 — factor shared FAL plumbing into tools/fal_common.py

Move these out of tools/image_generation_tool.py (keep as re-export shims for backward compat with patched tests):

  • _load_fal_client() + fal_client placeholder
  • _resolve_managed_fal_gateway()
  • _normalize_fal_queue_url_format()
  • _ManagedFalSyncClient
  • _get_managed_fal_client() + _managed_fal_client global + lock
  • _submit_fal_request()
  • _extract_http_status()

tests/tools/test_managed_media_gateways.py (~298 LoC) already patches these names — preserve them via re-exports per the in-tree-to-plugin-migration skill's "re-export shims" pattern.

Step 2 — refactor plugins/video_gen/fal/__init__.py to use tools.fal_common

Drops the inline _load_fal_client + _fal_client global. Behaves identically. (Bonus: opens the door to managed-gateway support in a future PR.)

Step 3 — new plugins/image_gen/fal/{plugin.yaml,__init__.py}

Mirrors plugins/image_gen/openai/__init__.py structure (single-file ABC impl + register()):

class FalImageGenProvider(ImageGenProvider):
    @property
    def name(self) -> str: return "fal"

    @property
    def display_name(self) -> str: return "FAL"

    def is_available(self) -> bool:
        # FAL_KEY direct OR Nous managed gateway both count
        return check_fal_api_key()

    def list_models(self) -> List[Dict[str, Any]]:
        # 18-entry catalog from _MODELS dict

    def default_model(self) -> Optional[str]:
        return "fal-ai/flux-2/klein/9b"

    def get_setup_schema(self) -> Dict[str, Any]:
        # Matches the existing hardcoded TOOL_CATEGORIES["image_gen"] FAL entry
        # — env_vars for FAL_KEY + Nous Subscription gating

    def generate(self, prompt, aspect_ratio="landscape", **kwargs) -> Dict[str, Any]:
        # Wraps tools.fal_common._submit_fal_request + _build_fal_payload

The _MODELS catalog moves into the plugin (it's purely FAL-specific). _build_fal_payload, _resolve_fal_model likewise — these have zero callers outside image_generate_tool.

Step 4 — dispatcher cutover

Remove the configured == "fal" skip in _dispatch_to_plugin_provider(). Have image_generate_tool()'s no-config path also route through the registry (fetching the "fal" provider explicitly when nothing is configured), so the plugin handles every code path.

Step 5 — drop the FAL skip in _plugin_image_gen_providers() + delete hardcoded FAL row

After Step 4, TOOL_CATEGORIES["image_gen"]["providers"] can drop to zero hardcoded rows — matches video_gen. The "Nous Subscription (Managed FAL)" UX row stays as a non-provider setup-flow entry, same shape as the browser migration kept "Nous Subscription (Browser Use cloud)".

Step 6 — keep _upscale_image() in tools/image_generation_tool.py

The upscaler currently fires after image_generate_tool succeeds when the model spec has upscale: True. It's a wrapper concern, not a backend concern. Stays in the tool wrapper, calls back into FAL via the plugin (or tools.fal_common._submit_fal_request directly — same difference).

Step 7 — tests

  • Add tests/plugins/image_gen/test_fal_provider.py (~150 LoC mirroring test_openai_provider.py / test_xai_provider.py)
  • Existing tests/tools/test_image_generation*.py tests keep working unchanged because the patched names are still importable from tools.image_generation_tool (re-export shims)
  • Behavior-parity subprocess sweep mirroring tests/plugins/browser/check_parity_vs_main.py:
    • image_gen.provider unset → resolves to FAL (default)
    • image_gen.provider: fal explicit → resolves to FAL (via registry)
    • image_gen.provider: openai → resolves to OpenAI plugin (no regression)
    • image_gen.provider: <typo> → falls back to FAL default (preserves legacy)
    • With Nous managed gateway configured + no FAL_KEY → still resolves
    • With FAL_KEY + tool_gateway.image_gen: gateway → managed wins

Out of scope

  • _upscale_image becoming pluggable. It's coupled to the model catalog (per-model upscale: bool knob). Could be its own plugin category in the future but not in scope here.
  • video_gen FAL gaining managed-gateway support. Could land in the same PR if it's clean, but file as a separate issue if it adds complexity.
  • Migrating the 18-model catalog format. The catalog stays as a dict literal in the plugin; restructuring into ImageGenProvider.list_models() entries with first-class typing is out of scope.

Architectural parity payoff

After this, all four pluggable-backend subsystems work identically:

  • plugins/web/<vendor>/plugins/browser/<vendor>/plugins/image_gen/<vendor>/plugins/video_gen/<vendor>/

And both FAL-using plugins share tools/fal_common.py instead of each re-implementing the fal_client lazy-load.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — cosmetic, nice to havecomp/pluginsPlugin system and bundled pluginstool/visionVision analysis and image generationtype/refactorCode restructuring, no behavior change

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions