2026-06-18 14:37:44,123 - INFO - Starting to create PR results comment...
2026-06-18 14:37:44,123 - INFO - Using reference models: intfloat/multilingual-e5-large, google/gemini-embedding-001
Traceback (most recent call last):
File "/home/runner/work/results/results/scripts/create_pr_results_comment.py", line 350, in <module>
main(args.reference_models, args.output)
File "/home/runner/work/results/results/scripts/create_pr_results_comment.py", line 342, in main
markdown = generate_markdown_content(model_tasks, reference_models)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/results/results/scripts/create_pr_results_comment.py", line 295, in generate_markdown_content
df, high_model_performance_tasks, all_training_datasets = create_comparison_table(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/results/results/scripts/create_pr_results_comment.py", line 117, in create_comparison_table
results = cache.load_results(models=models, tasks=tasks, validate_and_filter=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/results/results/.venv/lib/python3.12/site-packages/mteb/cache/result_cache.py", line 1389, in load_results
task_result = TaskResult.from_disk(path)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/results/results/.venv/lib/python3.12/site-packages/mteb/results/task_result.py", line 410, in from_disk
and Version(data["mteb_version"]) < Version("1.11.0")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/results/results/.venv/lib/python3.12/site-packages/packaging/version.py", line 438, in __init__
raise InvalidVersion(f"Invalid version: {version!r}")
packaging.version.InvalidVersion: Invalid version: '2.12.16-2.15.4'
Describe the bug
To reproduce
https://github.com/embeddings-benchmark/results/actions/runs/27767067343/job/82156582774?pr=574
Additional information
No response
Are you interested to contribute a fix for this bug?
Yes