webui: add cancel button for in-progress model loading by marksverdhei · Pull Request #8 · heiervang-technologies/ht-llama.cpp

marksverdhei · 2026-03-07T07:56:40Z

Summary

Adds a cancel (X) button next to the loading spinner when a model is being loaded in router mode
Cancel button appears in both the model selector trigger (dropdown/sheet) and individual model option rows
Uses the existing /models/unload endpoint which already supports unloading models in LOADING state
Frontend polling loop is cleanly interrupted via AbortController to prevent stale error toasts

Motivation

When a model gets stuck loading (e.g. large models on ROCm, slow downloads, or misconfigured models), users currently have no way to cancel the operation from the frontend. They must kill the server process entirely. This PR adds a simple cancel mechanism that leverages the existing server-side unload capability.

Related upstream issues:

ggml-org/llama.cpp#4462 - C API cancel via progress_callback
ggml-org/llama.cpp#4551 - Feature request for abort model loading
ggml-org/llama.cpp#19482 - ROCm model loading hangs

Changes

models.svelte.ts: Added cancelLoadModel() method with AbortController support for clean polling interruption
ModelsSelectorOption.svelte: Cancel button next to spinner in model option rows
ModelsSelector.svelte: Cancel button next to spinner in dropdown trigger
ModelsSelectorSheet.svelte: Cancel button next to spinner in sheet trigger

Test plan

Start loading a model in router mode
Verify the X cancel button appears next to the loading spinner
Click cancel and verify the model loading stops and status returns to unloaded
Verify no error toasts appear after cancellation
Verify normal model loading still works after a cancel
Test in both desktop (dropdown) and mobile (sheet) views

🤖 Generated with Claude Code

marksverdhei

PR Review

The cancel functionality is well-implemented and addresses a real pain point. A few things to address:

Issues

1. Duplicated logic between ModelsSelector.svelte and ModelsSelectorSheet.svelte

handleCancelLoad(), loadingModelId, and isCancellingModel are copy-pasted between these two components. If cancel behavior changes later, both need updating. Consider extracting into the store or a shared helper.

2. Race condition in handleCancelLoad()

isLoadingModel = false;  // UI shows chevron briefly
await modelsStore.cancelLoadModel(cancelId);  // then cancelling spinner appears

Setting isLoadingModel = false before the cancel completes creates a brief flash where the trigger shows neither a loading nor cancelling spinner.

3. No timeout on cancel polling loop

In cancelLoadModel(), the while(true) loop has no bail-out. If the server never transitions out of LOADING, this polls forever. The regular pollForModelStatus has attempt counting and unexpected-state checks, but the cancel poll doesn't.

4. Cancel button should be a <button>, not a <div>

The a11y_no_static_element_interactions and a11y_click_events_have_key_events suppressions in ModelsSelector.svelte and ModelsSelectorSheet.svelte hide real accessibility issues. The cancel target should be a <button> element so it's keyboard-focusable and announced correctly by screen readers.

5. Theme/rebrand change bundled with cancel feature

The turquoise/purple theme and "ht-llama.cpp" renaming is a significant visual change in a PR titled "add cancel button." These should be separate PRs to keep review and potential reverts clean. The commits are separate which helps, but they land together.

Positives

Clean AbortController usage to stop polling without stale toasts
Color coding (green=loading, orange=cancelling, red=unloading) is intuitive
animate-spin-reverse reusing the existing spin keyframe is nice
Good separation of cancelling vs unloading states in the store

🤖 Generated with Claude Code

Allow users to cancel a model that is stuck loading or taking too long in the router mode model selector. The cancel button appears next to the loading spinner in both the model selector dropdown/sheet trigger and within individual model option rows. Uses the existing /models/unload endpoint which already supports unloading models in LOADING state. The frontend polling loop is interrupted via AbortController to prevent stale error toasts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Show orange "Cancelling" indicator with spinner while cancel is in progress - Poll until server confirms model is no longer in LOADING state before clearing the cancelling indicator - Guard against redundant unload calls on already-unloaded models - Keep loadingModelId alive during cancel so selector trigger shows the cancelling state correctly Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Loading: green spinner, clockwise - Unloading: red spinner, reverse direction with "Unloading" label - Cancelling: orange spinner, reverse direction - Track unloading state separately in models store Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Remove duplicated cancel logic from ModelsSelector and ModelsSelectorSheet by deriving loading/cancelling state from the store (issue #1) - Fix race condition: no longer set isLoadingModel=false before cancel completes, preventing brief UI flash (issue #2) - Add MAX_CANCEL_POLL_ATTEMPTS (60) timeout to cancel polling loop to prevent infinite polling if server never transitions (issue #3) - Replace div cancel buttons with proper <button> elements for keyboard accessibility and screen reader support (issue #4) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

marksverdhei · 2026-03-07T10:44:43Z

Addressed all review feedback:

Duplicated logic — Replaced local isLoadingModel/loadingModelId/handleCancelLoad in both ModelsSelector and ModelsSelectorSheet with store-derived state using triggerModelId + modelsStore.isModelOperationInProgress() / isModelCancelling().
Race condition — No longer set isLoadingModel = false before cancel completes. The loading/cancelling states are now derived from the store, which manages the lifecycle correctly.
No timeout on cancel polling — Added MAX_CANCEL_POLL_ATTEMPTS = 60 (30 seconds at 500ms interval) with a warning toast if it times out.
Cancel button a11y — Replaced <div> cancel targets with <button type="button" aria-label="Cancel loading"> in all three components. Removed a11y_no_static_element_interactions suppressions.
Theme bundled with cancel — Split theme/rebrand into separate PR feat(webui): rebrand to ht-llama.cpp and add turquoise/purple theme #10.

marksverdhei

Re-review: All feedback addressed

All four issues from the initial review have been fixed in 5d24d08:

Duplicated logic — isLoadingModel and isCancellingModel are now $derived from the store via triggerModelId. Clean.
Race condition — No more manual isLoadingModel = false before cancel completes. State is derived from the store's operation tracking, so no UI flash.
Cancel polling timeout — Now bounded by MAX_CANCEL_POLL_ATTEMPTS = 60 (~30s) with a warning toast on timeout.
Accessibility — Cancel buttons are proper <button> elements with type="button" and aria-label. No more svelte-ignore suppressions.
Theme/rebrand removed — Correctly split out of this PR.

Minor nit (non-blocking): In ModelsSelectorOption.svelte:103, the cancel <button> wraps an ActionIcon which likely renders its own button — nested interactive elements. Works functionally but worth cleaning up later.

LGTM, ready to merge.

🤖 Generated with Claude Code

* webui: add cancel button for in-progress model loading Allow users to cancel a model that is stuck loading or taking too long in the router mode model selector. The cancel button appears next to the loading spinner in both the model selector dropdown/sheet trigger and within individual model option rows. Uses the existing /models/unload endpoint which already supports unloading models in LOADING state. The frontend polling loop is interrupted via AbortController to prevent stale error toasts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * webui: add cancelling state indicator and fix cancel polling - Show orange "Cancelling" indicator with spinner while cancel is in progress - Poll until server confirms model is no longer in LOADING state before clearing the cancelling indicator - Guard against redundant unload calls on already-unloaded models - Keep loadingModelId alive during cancel so selector trigger shows the cancelling state correctly Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(webui): color-coded spinners for model load/unload/cancel states - Loading: green spinner, clockwise - Unloading: red spinner, reverse direction with "Unloading" label - Cancelling: orange spinner, reverse direction - Track unloading state separately in models store Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(webui): address PR review feedback for cancel model loading - Remove duplicated cancel logic from ModelsSelector and ModelsSelectorSheet by deriving loading/cancelling state from the store (issue #1) - Fix race condition: no longer set isLoadingModel=false before cancel completes, preventing brief UI flash (issue #2) - Add MAX_CANCEL_POLL_ATTEMPTS (60) timeout to cancel polling loop to prevent infinite polling if server never transitions (issue #3) - Replace div cancel buttons with proper <button> elements for keyboard accessibility and screen reader support (issue #4) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

) * ggml: backend-agnostic tensor parallelism * support for GPT-OSS, Qwen 3 MoE * partial Vulkan fix * add support for 4/8 GPUs * unconditional peer access * re-use buffers + ggml contexts * fix output pattern * NCCL support * GGML: HIP: add RCCL support * Remove shfl and AllReduce from backend interface * move allocation workaround out of ggml-alloc.c * 2d tensor set/get support * Fix the seg fault without NCCL * Apply suggestion from JohannesGaessler * support for tensor dims % n_devs != 0 * fix view_offs scaling * arbitrary num. of GPUs/tensor split * fix compilation * better granularity estimate * Support device-specific host buffer types if all underlying backends expose the same type. This allows using pinned memory instead of pageable memory for CUDA. Fix compilation errors. * partial Qwen 3 Next support * Fix qwen3 30b (#8) * Fix crash with Qwen-30B-A3B Q4_0 Qwen-30B-A3B Q4_0 has an intermediate dimension of 768. Using a granularity of 256 forces an uneven split between GPUs, which is not supported by the current implementation. * Decide block size based on tensor quantization type * Fix crashes due to KV cache serialization (#9) KV cache serialization requires non-zero offsets on the tensor. Add support in the meta backend to set/get a tensor with a non-zero offset. * metal : fix build (#7) * static memory allocations, fix usage count * fix tensor granularity * more even memory distribution * use BF16 for allreduce * rebase fixup * better error message for unsupported architectures * Fix device mismatch during scatter of allReduce. (#11) There is a mismatch between the dst buffer device and the backend device, causing the use of sync copies * Enable the previous allreduce implementation. It is better in both perf and stability (#12) * delay AllReduce for Moe for less I/O * build : clean-up compile warnings * backend : move most of the meta backend API to ggml-backend-impl.h * cont : hide unused public API in the implementation * llama : use llama_device + remove ggml_backend_dev_is_meta() * ggml-backend : remove unused alloc include * minor : remove regex include * ggml : introduce ggml-ext.h for staging new APIs * rebase fixup * fix tests * llama : more robust logic for determining Meta devices (#16) * llama : more robust logic for determining Meta devices * cont : fix devs size check Co-authored-by: Johannes Gäßler <johannesg@5d6.de> * cont : fix log type Co-authored-by: Johannes Gäßler <johannesg@5d6.de> --------- Co-authored-by: Johannes Gäßler <johannesg@5d6.de> * disable roundtrip for meta backend * fix arch selection * Qwen 3.5 support * fix Gemma 4 MoE * fix OpenVino, SYCL * fix test-llama-archs for CPU-only builds * Fix Qwen 3.5 MoE * disable meta backend tests for WebGPU * tests : filter CPU-based devices from the Meta backend tests (#17) * meta : formatting, naming, indentation (#18) * formatting : llama-model.cpp * formatting : ggml-ext.h * formatting : ggml-backend-meta.cpp * meta : add TODO * add documentation * better error messages * fix GPT-OSS --------- Co-authored-by: Carl Philipp Klemm <carl@uvos.xyz> Co-authored-by: Gaurav Garg <gaugarg@nvidia.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

marksverdhei commented Mar 7, 2026

View reviewed changes

marksverdhei mentioned this pull request Mar 7, 2026

feat(webui): rebrand to ht-llama.cpp and add turquoise/purple theme #10

Merged

3 tasks

marksverdhei and others added 4 commits March 7, 2026 11:43

marksverdhei force-pushed the feat/cancel-model-loading branch from f4ab0ee to 5d24d08 Compare March 7, 2026 10:44

marksverdhei commented Mar 7, 2026

View reviewed changes

marksverdhei merged commit eec820b into ht Mar 7, 2026

marksverdhei deleted the feat/cancel-model-loading branch March 7, 2026 11:09

marksverdhei mentioned this pull request May 12, 2026

Runbook: rebase ht onto upstream/master (parked 2026-05-12) #38

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

webui: add cancel button for in-progress model loading#8

webui: add cancel button for in-progress model loading#8
marksverdhei merged 4 commits into
htfrom
feat/cancel-model-loading

marksverdhei commented Mar 7, 2026

Uh oh!

marksverdhei left a comment

Uh oh!

marksverdhei commented Mar 7, 2026

Uh oh!

marksverdhei left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

marksverdhei commented Mar 7, 2026

Summary

Motivation

Changes

Test plan

Uh oh!

marksverdhei left a comment

Choose a reason for hiding this comment

PR Review

Issues

Positives

Uh oh!

marksverdhei commented Mar 7, 2026

Uh oh!

marksverdhei left a comment

Choose a reason for hiding this comment

Re-review: All feedback addressed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant