Skip to content

fix: Update lock and remove python limit fo pylate and colbert_engine#4783

Merged
Samoed merged 7 commits into
mainfrom
update_lock
Jun 8, 2026
Merged

fix: Update lock and remove python limit fo pylate and colbert_engine#4783
Samoed merged 7 commits into
mainfrom
update_lock

Conversation

@Samoed

@Samoed Samoed commented Jun 8, 2026

Copy link
Copy Markdown
Member

Remove python_full_version < '3.13' for pylate and colbert_engine. They're supporting higher python verisions. I updated uv.lock

@Samoed Samoed requested a review from KennethEnevoldsen June 8, 2026 15:18
Comment thread tests/test_evaluators/test_ClusteringEvaluator.py
@Samoed

Samoed commented Jun 8, 2026

Copy link
Copy Markdown
Member Author

Fix metrics.accuracy_score, metrics.hamming_loss metrics.zero_one_loss, metrics.matthews_corrcoef and metrics.confusion_matrix (when labels is not None) now raise an error when y_true is string and y_pred is numeric, for all array-like inputs. Previously, lists and numpy arrays not of object dtype did not raise an error for this mixed input case. The above metrics will also raise an error for label indicator matrix inputs of inconsistent size, except for metrics.confusion_matrix which does not accept label indicator matrix inputs. By Lucy Liu. #33086 https://scikit-learn.org/stable/whats_new/v1.9.html#sklearn-metrics

Breaks our zero-shot tasks. I always wanted to fix it to support more metrics, but it seems this is really necessary now.

@KennethEnevoldsen KennethEnevoldsen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for now should we set an upper limit for sklearn until we have fixes the issues?

Otherwise good

@KennethEnevoldsen KennethEnevoldsen changed the title Update lock fix: Update lock and remove python limit fo pylate and colbert_engine Jun 8, 2026
@Samoed

Samoed commented Jun 8, 2026

Copy link
Copy Markdown
Member Author

Created an issue #4784

@Samoed Samoed enabled auto-merge (squash) June 8, 2026 21:48
@Samoed Samoed merged commit d93a2c0 into main Jun 8, 2026
13 of 14 checks passed
@Samoed Samoed deleted the update_lock branch June 8, 2026 21:56
KennethEnevoldsen added a commit that referenced this pull request Jun 10, 2026
scikit-learn 1.9 raises "ValueError: Mix of label input types" when
classification metrics receive string y_true with numeric y_pred.
Zeroshot predictions are always integer indices into the candidate
labels, so string dataset labels are now mapped to their candidate
index before scoring. Unmappable string labels raise a clear error
instead of silently scoring 0.0, which is what scikit-learn < 1.9 did.

Removes the <1.9.0 pin introduced as a stopgap in #4783.

Fixes #4784

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants