Skip to content

stats: statistical test result object? #13118

@mdhaber

Description

@mdhaber

There are several recent PRs featuring new functions for statistical tests:

The new make_tuple_bunch (gh-12323) is for adding new outputs to existing tests in a backwards compatible way, but what should the new functions return?

To finish gh-11119 without needing to make this decision, cramervonmises returns a bare-bones CramerVonMisesResult:

class CramerVonMisesResult:
    def __init__(self, statistic, pvalue):
        self.statistic = statistic
        self.pvalue = pvalue

    def __repr__(self):
        return (f"{self.__class__.__name__}(statistic={self.statistic}, "
                f"pvalue={self.pvalue})")

I'm not sure the decision about supporting Python 3.6 was made yet then (is it official?), but the same behavior could have been achieved with make_dataclass.

This suggests a few questions.

  1. Should there be a class like OptimizeResult, say, StatsTestResult, that takes care of some of the common behavior, or should we use make_dataclass? Or should each test make it's own unrelated custom class? What behavior would we want this class to take care of? For instance, OptimizeResult is a subclass of dict but includes a __getattr__ that allows accessing elements as if they were fields.
  2. Should fields statistic and pvalue be standard names of the fields (where applicable) or should the name of the statistic be more specific?

The new BinomTestResult currently returned by binomtest (gh-12603) brings a few other questions to mind.

  1. A relatively expensive calculation that is related but not required by the binomial test is calculation of a confidence interval of the proportion of successes. Currently, a method for performing that test is included in the BinomTestResult so that it is easy for the user to follow up a binomtest with calculation of the confidence interval, but it does not take time if it is not required. Do we agree that this is a good compromise between performing the CI calculation within binomtest and making the CI calculation a totally separate function?
  2. How much information about the test that was performed should be returned in the result? Currently, BinomTestResult includes (beside pvalue) much of the user's input, including k, n, and alternative. Should it include more of what the user entered (e.g. the proportion of successes of the null hypothesis), or only what it needs to retain for performing follow-up calculations (e.g. confidence interval).

@WarrenWeckesser please feel free to edit this directly

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions