Skip to content

TST Add array API continuous metric common tests#32793

Draft
lucyleeow wants to merge 3 commits intoscikit-learn:mainfrom
lucyleeow:tst_continuous_bin_aapi
Draft

TST Add array API continuous metric common tests#32793
lucyleeow wants to merge 3 commits intoscikit-learn:mainfrom
lucyleeow:tst_continuous_bin_aapi

Conversation

@lucyleeow
Copy link
Copy Markdown
Member

Reference Issues/PRs

Ref: https://github.com/scikit-learn/scikit-learn/pull/32422/files#r2548690179

What does this implement/fix? Explain your changes.

Moves the array API tests for continuous metrics added in #32422 to test_common.py, so they can be re-used for other continuous metrics (e.g., #32626)

Any other comments?

cc @OmarManzoor

@github-actions
Copy link
Copy Markdown

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: f47ace0. Link to the linter CI: here

Copy link
Copy Markdown
Contributor

@OmarManzoor OmarManzoor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @lucyleeow

if "brier" in metric.__name__:
# `brier_score_loss` and `d2_brier_score` require specifying the
# `pos_label`
metric_kwargs["pos_label"] = "yes"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not needed for the multi class case.

@lucyleeow
Copy link
Copy Markdown
Member Author

@OmarManzoor I've just realised the string y_true tests are to check for mixed array inputs.

I am wondering whether we separate testing of:

  • support for mixed array input, and
  • support for array API - i.e. result is the same when using numpy vs xp

I've actually have a PR to add a common test to check for mixed array input support: #32755. It means that:

  • we can check for the specific array conversion combinations we wish to
    • it also allows us to check for non-numpy to numpy conversion, which this particular set up does not
  • avoids us amending current common metric tests to add mixed array input tests
    • the separation also makes it also easier to test, as not all metrics will support mixed input during the process of adding support

cc @ogrisel for your thoughts too.

@OmarManzoor
Copy link
Copy Markdown
Contributor

@OmarManzoor I've just realised the string y_true tests are to check for mixed array inputs.

I am wondering whether we separate testing of:

  • support for mixed array input, and
  • support for array API - i.e. result is the same when using numpy vs xp

I've actually have a PR to add a common test to check for mixed array input support: #32755. It means that:

  • we can check for the specific array conversion combinations we wish to

    • it also allows us to check for non-numpy to numpy conversion, which this particular set up does not
  • avoids us amending current common metric tests to add mixed array input tests

Yes that should be good I think. The reason the tests for y_true being strings were added in the same test was because we wanted to support that for these metrics. I think you can separate them out now considering the other PR related to mixed types.

@lucyleeow
Copy link
Copy Markdown
Member Author

Thanks @OmarManzoor ! I think I need to push #32755 first, before this can go in.

@lucyleeow lucyleeow marked this pull request as draft November 27, 2025 01:23
@lucyleeow lucyleeow moved this to In Progress in Array API Nov 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

2 participants