Closed five-role enum covering the metadata files the frontend
consumes today (config, tokenizer, tokenizer_config, chat_template,
generation_config) plus a struct describing one entry in the upcoming
files vector on ModelDeploymentCard: filename, opaque uri, blake3
checksum, byte size, and role.

Types only; no call sites yet. Refs #8749.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

Removes three methods with no production callers and the two tests
that depended on them:

  - unregister_model: no detach path calls it; speculative addition.
    Worker process exit clears the whole map already. When we want
    per-model cleanup, ~10 LOC re-adds it.
  - len, is_empty: only used by tests. Tests now assert correctness
    via get() — the actual contract — instead of the count.

Also drops the `clone_shares_state` test, which only exercised the
Arc-share semantic that #[derive(Clone)] documents at the type level.

Module shrinks from ~125 to ~85 LOC; production API is now
new / register / get. All four remaining tests pass. Refs #8749.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

…pty (gh-8749)

Effectively reverts 0d4ab6d.

Restoring the methods on second look:
  - register / unregister: parallel-construction symmetry. Even though
    no production caller invokes unregister yet, the pair reads as a
    natural domain API. Detach paths will need it; having it ready
    keeps that change a one-liner.
  - len / is_empty: standard collection accessors. Lightweight
    (one read-lock each) and useful for observability/diagnostics
    beyond the test suite.

The clone_shares_state test also comes back — small but explicit
coverage of the Arc-share semantic, which is load-bearing for the
DistributedRuntime → SystemStatusServer handoff.

Module returns to its pre-simplification shape (~125 LOC, 6 tests).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

…es (gh-8749)

Two renames in service of clearer naming and parallel construction:

  Worker side (local_model.rs):
    wire_self_hosted_artifacts -> move_to_self_host

  Mirrors the existing ModelDeploymentCard::move_to_url pair (same
  shape: walks every typed-enum CheckedFile, rewrites URIs in place;
  opposite of update_dir). "wire" was an outlier — no other lib/
  function uses it.

  Frontend side (model_card.rs):
    try_self_host_download -> try_download_files
    self_host_cache_dir    -> files_cache_dir
    .cache/dynamo/self-host-metadata/  -> .cache/dynamo/model-metadata/

  The frontend consumer is scheme-agnostic: it walks `files`, picks
  http(s):// entries, fetches them, and verifies blake3. It doesn't
  know or care whether the URIs point at the originating worker, a
  peer holder, modelexpress, or some other front-cache. Naming the
  function "self_host" baked in a worker-side concern that has no
  business in the consumer.

Refs #8749.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

…h-8749)

Match the legacy `download_config` posture: it doesn't own a disk
cache, it delegates to `hub::from_hf` which manages HF Hub's cache.
The new branch was creating its own persistent directory under
~/.cache/dynamo/model-metadata/, which is asymmetric and accumulates
state in $HOME with no cleanup.

Switch to a per-process scratch directory under `std::env::temp_dir()`
keyed by PID + slug:
  /tmp/dynamo-mdc-<pid>/<slug>/<filename>

Process restart re-fetches (~10 MiB total per model, sub-second on a
DC network — same cost the legacy path pays when HF Hub's cache
misses). The OS reaps tempdir on reboot. No long-running disk
artifact in $HOME, no cleanup logic.

Renames `files_cache_dir` -> `files_download_dir` to reflect that
this is transient scratch, not a cache.

Refs #8749.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

)

Replaces two ad-hoc helpers with a scheme-uniform resolver and an
HF-Hub-style content-addressed cache.

resolve_uri(client, uri, expected_checksum, dest) dispatches by URI
scheme, so the consumer side is genuinely transport-agnostic:
  - http/https — reqwest GET, body verified
  - file       — tokio::fs::read of the local path, verified
  - hf://repo[@rev]/filename — hub::from_hf, copy from snapshot,
                                verified
All schemes write atomically (tmp + rename); checksum mismatch is a
hard error in every branch.

Cache layout mirrors HF Hub:
  ~/.cache/dynamo/mdc/blobs/<blake3-hex>           — bytes
  ~/.cache/dynamo/mdc/by-slug/<slug>/<filename>    — symlink → blob
Multi-worker dedup is automatic: same blake3 → same blob path → the
existence check skips the fetch. Frontend restart hits the warm
cache. Cleanup is user-managed, same posture as HF Hub.

try_download_files becomes download_files, walks every entry in the
files vector through resolve_uri, and update_dirs the typed
CheckedFiles at the per-slug symlink directory. Returns Ok(false)
only when files is empty so the caller falls through to the legacy
hub::from_hf path; non-empty files but a single resolution failure
is loud.

move_to_url now also walks the files vector, rewriting `file://`
entries to the same `hf://repo/filename` URLs it produces for the
typed enums. Self-host (`http://`) entries are preserved — the
file://-prefix gate keeps move_to_url from clobbering them when it
runs after move_to_self_host. Shared-volume deployments
(source_path.exists() on the worker) skip move_to_url entirely, so
their `file://` URIs are preserved across both surfaces.

Updates the in-source mockito test to call resolve_uri instead of
the removed fetch_and_verify, plus a parse_hf_uri unit test.

Refs #8749.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

…h-8749)

Five new tests in lib/llm/tests/model_card.rs alongside the existing
TinyLlama-fixture tests:

  - test_files_vector_populated_from_local_repo — load TinyLlama,
    assert mdc.files has entries with file:// URIs, blake3 checksums,
    sizes, and roles drawn from the closed five-role enum.
  - test_download_files_resolves_local_file_scheme — round-trip
    file:// resolution into the content-addressed cache, verify blob +
    by-slug symlink layout, assert the tokenizer loads via the
    resolved path.
  - test_download_files_dedupes_by_blake3 — call download_config
    twice, assert blob mtimes are unchanged on the second call (the
    blake3-keyed exists() check skips the fetch).
  - test_download_files_rejects_blake3_mismatch — tamper with one
    ArtifactRef.checksum, assert download_config errors with a clear
    "checksum mismatch" message and leaves no blob behind.
  - test_download_files_resolves_hf_scheme_with_token — HF_TOKEN
    runtime-skip pattern (matches preprocessor.rs); same Llama-3.1
    fixture the existing preprocessor tests use; exercises the
    resolve_uri hf:// branch end-to-end.

Tests are serialized via serial_test::serial because they share the
$HOME-rooted MDC cache. Each test installs its own tempdir-backed
$HOME so it doesn't pollute the user's real ~/.cache/dynamo/.

Bug fix: artifact_ref_from_checked_file canonicalizes the local path
before constructing the file:// URI. url::Url::from_file_path
requires an absolute path; relative paths returned None and the
files vector came out empty for any caller that loaded the MDC from
a relative path (production goes through LocalModelBuilder::build
which already canonicalizes; the bug only manifested in tests and
direct load_from_disk callers). Refs #8749.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

…lt (gh-8749)

Lets deployments turn self-hosting on without modifying every worker
recipe. When DYN_SELF_HOST_METADATA is set to a truthy value (1,
true, yes, on; case-insensitive), `LocalModelBuilder::default()`
initializes `self_host_metadata` to true. Explicit setter calls
(`.self_host_metadata(...)`) and explicit PyO3 kwargs always win
over the env-var default.

Also tightens the PyO3 binding: the builder method is now only
called when the kwarg is `Some(_)`. An unset kwarg falls through to
the builder's default, so the env var is honored end-to-end.

Two unit tests cover the truthy/falsy parsing, gated by
serial_test::serial because they share a process-global env var.

Refs #8749.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

gh-8749)

Tightens doc comments and inline comments across the new self-host
code to match the surrounding files' style: brief one-line docs on
public APIs, no doc on simple private helpers, longer prose only
where the WHY is non-obvious. Removes design rationale, motivation,
and "mirror existing pattern" prose — the type signatures and
existing tests carry that load.

Also adds tests/frontend/test_vllm_self_host_metadata.py — a local
e2e clone of test_vllm.py with the only diff being
DYN_SELF_HOST_METADATA=true in the worker process env. Verifies the
full chain by asserting the model lands on /v1/models. Uses QWEN
(Qwen3-0.6B). Marked vllm/gpu_1/e2e/post_merge so it slots into the
existing vLLM e2e CI gate.

Refs #8749.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

…h-8749)

Strengthens the Python e2e to actually validate that the frontend
fetched via the self-host http path: the frontend process now runs
under an isolated HOME=<tmp_path> so any cache writes are scoped
to the test, and the test asserts at least one blake3-keyed blob
appears under <HOME>/.cache/dynamo/mdc/blobs/. That directory is
only ever written by ModelDeploymentCard::download_files, so its
contents prove the http resolver ran end-to-end (the legacy
hub::from_hf fallback writes to the HF Hub cache, not to this
directory).

Adds a DRT-level e2e in lib/llm/tests/model_card.rs under
#[cfg(feature = "integration")] mod integration_tests, modeled on
lib/llm/tests/http_metrics.rs. Real DistributedRuntime with
DYN_SYSTEM_PORT=0, real worker LocalModel::attach -> ModelWatcher
-> download_config -> cache populated. Asserts blob count matches
the artifact list and the tokenizer loads. Gated #[ignore] because
it requires etcd; run with
cargo test -p dynamo-llm --test model_card --features integration -- --ignored.

Parallelism:
  - Within a binary: #[serial_test::serial] because HOME and
    DYN_SYSTEM_PORT are process-global.
  - Across cargo test processes: each gets its own tempdir-rooted
    HOME, its own random port (DYN_SYSTEM_PORT=0), and a unique
    etcd namespace (self-host-it-<pid>-<uuid>); the watcher uses
    NamespaceFilter::Exact so two parallel runs don't see each
    other's MDCs.

Replaces the bare tempfile::TempDir isolated_home() helper with a
true RAII guard (IsolatedHome) that restores the previous HOME on
drop, so subsequent tests aren't left pointing at a deleted
tempdir path.

Refs #8749.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

Apply the same lessons from grahamking's review across the rest of
the PR's new code:

- ENV_SELF_HOST_METADATA constant doc trimmed from 5 lines of
  parsing rules to a one-liner, matching the existing convention
  in lib/runtime/src/config/environment_names.rs
  (e.g. DYN_SYSTEM_HOST/PORT/etc).
- LocalModelBuilder::self_host_metadata setter doc rewritten to
  match the runtime config field convention from
  lib/runtime/src/config.rs ("Set this at runtime with environment
  variable DYN_FOO").
- model_card.rs: demote the "resolved model metadata files" log
  from info! to debug! — once-per-registration is too quiet for
  info, same pattern we just demoted in move_to_self_host.
- model_card.rs::stream_to_tmp: cap.saturating_add(1) -> cap + 1
  (cap bounded by ABSOLUTE_MAX_METADATA_BYTES = 1 GiB, no overflow).
- model_card.rs::stream_to_tmp: surface flush errors via
  with_context()? instead of silent .ok(); a flush failure right
  before atomic-rename would mean the staged bytes weren't fully
  on disk.
- local_model.rs::move_to_self_host: use fs::canonicalize (the
  module is already imported via `use std::fs;` at top of file).

Signed-off-by: nnshah1 <neelays@nvidia.com>

…ymlink (gh-8749)

`move_to_self_host` was calling `fs::canonicalize()` on the local
path and then taking `file_name()` from the canonical result. For
workers backed by the HuggingFace cache, snapshot/<rev>/<filename>
is itself a symlink into blobs/<git-sha1>; canonicalize follows it
and returns the SHA1 as the file_name, so:

  * the http URL becomes /v1/metadata/<slug>/<sha1>
  * the per-(slug, mdcsum) symlink in the frontend's cache is
    placed at <slug>/<mdcsum>/<sha1>
  * downstream code that opens files by literal name (e.g.
    `HFConfig::from_json_file` doing
    `parent.join("generation_config.json")` to fall back to
    generation_config.json for `eos_token_id`) fails with
    "Failed to open file" — the file *is* in the cache, just
    under a different name.

This bit the vllm Multi-GPU `mm_e_pd_llava-1.5-7b` test in CI
because llava-1.5-7b's `eos_token_id` lives only in
generation_config.json (Qwen3-0.6B has it directly in
config.json, which is why our mocker tests didn't catch it).

Fix: derive `filename` from the original (pre-canonicalize)
path so the URL, the per-slug symlink, and the registry key all
use the human-readable name. Keep canonicalize for the
*registry value* — the http handler still needs an absolute
path to the actual blob to serve bytes.

Local repro added (and passes after fix):

  pytest -xvs tests/frontend/test_llava_repro.py
  ========================= 1 passed in 13.63s =====================

Existing mocker tests still pass:

  pytest -xvs tests/frontend/test_mocker_self_host_multi_replica.py
  ========================= 4 passed in 69.12s =====================

Signed-off-by: nnshah1 <neelays@nvidia.com>

)

`ModelDeploymentCard::update_dir` deliberately skips the
chat_template_file slot when `is_custom == true` — the legacy
`hub::from_hf` fall-through doesn't fetch user-supplied custom
templates (they aren't on HF), so leaving the path untouched kept
the original local one for the legacy code path.

In the new-format path we *do* fetch every populated slot
(including custom templates served via the worker's self-host
route), so the URL-skip exception no longer applies. The previous
exception left `chat_template_file.path()` as a URL post-resolve,
and `PromptFormatter::from_mdc` would bail with:

  HfChatTemplateJinja for <model> is a URL, cannot load

Caught by the trtllm Multi-GPU `raw_embeddings_epd-2` test
(which uses `--custom-jinja-template` via `agg_raw_embeddings_llava.sh`).

Fix: in `resolve_metadata_files`, after fetching every slot via
`iter_metadata_files`, walk `iter_metadata_files_mut()` and call
`CheckedFile::update_dir(&slug_dir)` on each — same enumeration
as the fetch loop, so we cover exactly the files we just fetched.
This bypasses the legacy `is_custom` exclusion that's still load-
bearing for the legacy `hub::from_hf` fall-through.

Verified locally: existing mocker suite (3 schemes + multi-frontend)
+ llava repro all pass: 5 passed in 82.63s.

Signed-off-by: nnshah1 <neelays@nvidia.com>

…h-8749)

Comment-only follow-up to 262f314 + 1257fe0. Trim two
WHY blocks (7-line and 6-line) down to 3-4 lines each while
keeping the load-bearing reason for each non-obvious choice
(canonicalize-vs-original-filename, and skip-self.update_dir
because of legacy is_custom exclusion).

Signed-off-by: nnshah1 <neelays@nvidia.com>

…h-8749)

- `Checksum::hash()` / `algorithm()` accessors so callers can read the
  blake3 hex without round-tripping through `Display` + `strip_prefix`.
- Drop `blake3_hex_of` helper in `model_card.rs`; callers use
  `cf.checksum().hash()` directly.
- Swap `std::sync::RwLock` for `parking_lot::RwLock` in
  `metadata_registry`; drops `.unwrap()` calls and matches the rest of
  the codebase.

Per Graham's review on PR #8754.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>

…h-8749)

Drop comments that just restate function names / short signatures:
- `symlink_force` doc reduced to the concurrency note.
- `filename_from_checked_file`, `pf_checked_file`, `copy_to_tmp` —
  function names + signatures already say what they do.
- `move_to_self_host` 6-line preamble collapsed to 3 lines that capture
  the URL-vs-local dispatch (the non-obvious part).
- `clear_self_hosted_artifacts` 8-line block collapsed to 2.
- `self_host_base_url` host-selection paragraph removed; the IPv4/v6
  resolver behavior lives in `ip_resolver`, not here.

Cache/lock/concurrency rationale (BlobLock, stage_and_rename,
unique_tmp_path, resolve_uri scheme list) kept verbatim.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: nnshah1 <neelays@nvidia.com>