Skip to content

Fix failing API tests due to outdated model metadata expectations#124

Merged
Narsil merged 2 commits into
huggingface:mainfrom
bmqube:test-fixes
Sep 27, 2025
Merged

Fix failing API tests due to outdated model metadata expectations#124
Narsil merged 2 commits into
huggingface:mainfrom
bmqube:test-fixes

Conversation

@bmqube

@bmqube bmqube commented Sep 20, 2025

Copy link
Copy Markdown
Contributor

Fix failing API tests due to outdated model metadata expectations

Problem

Two tests in the test suite were failing (2/26 tests):

  • detailed_info test in src/api/sync.rs
  • info_request test in src/api/tokio.rs

Both tests were failing because they contained hardcoded assertions against live API responses from the mcpotato/42-eicar-street model on Hugging Face Hub. The model's metadata had naturally evolved over time, causing the tests to expect outdated values.

Root Cause

The tests were performing exact JSON comparisons against live API responses, which is inherently brittle since:

  1. Model metadata can change over time (likes, spaces, etc.)
  2. The Hugging Face Hub is a live platform where user interactions modify model statistics
  3. Models can be added to new spaces or receive more likes from the community

Changes Made

Updated the hardcoded test expectations in both sync and async API implementations to match the current state of the model metadata:

sync.rs changes:

  • Updated "likes": 3"likes": 4 in the detailed_info test
  • Added "szk2024/est" to the spaces array

tokio.rs changes:

  • Updated "likes": 3"likes": 4 in the info_request test
  • Added "szk2024/est" to the spaces array

Impact

  • ✅ All tests now pass (26/26)
  • ✅ No functional code changes - only test expectation updates
  • ✅ Both sync and async API implementations remain fully functional
  • ✅ Test coverage maintained without compromising test validity

Future Considerations

These tests validate the exact API response format but are vulnerable to live data changes. Consider:

  • Using mock responses for exact JSON structure validation
  • Testing against more stable models with predictable metadata
  • Focusing assertions on response structure rather than exact values for live API tests

Testing

cargo test
# All 26 tests pass

Technical Details

Files Changed

  • src/api/sync.rs - Updated model metadata expectations in sync API test
  • src/api/tokio.rs - Updated model metadata expectations in async API test

Commit History

33cfedd test: add szk2024/est to spaces array in async API model metadata test
f618dfa test: update mcpotato model likes count from 3 to 4 in sync API test

Branch Information

  • Source branch: test-fixes
  • Target branch: main
  • Type: Bug fix (test maintenance)

Validation

Before Fix

test result: FAILED. 24 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out

After Fix

test result: ok. 26 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

@Narsil Narsil merged commit 1d3037a into huggingface:main Sep 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants