Skip to content

ROCFunctions

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

ROCFunctions

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
ROCFunctions
ROCFunctions
 
 
docs
docs
 
 
examples
examples
 
 
tests
tests
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
pyproject.toml
pyproject.toml
 
 
setup.py
setup.py
 
 

README.md

ROCFunctions basic usage

This repository has the code of a Python package for Receiver Operating Characteristic (ROC) functions.

The ROC framework is used for analysis and tuning of binary classifiers, [Wk1]. (The classifiers are assumed to classify into a positive/true label or a negative/false label. )

For computational introduction to ROC utilization (in Mathematica) see the article "Basic example of using ROC with Linear regression" , [AA1].

The examples below use the package "RandomDataGenerators", [AA2].

Installation

From PyPI.org:

python3 -m pip install ROCFunctions

Usage examples

Properties

Here are some retrieval functions:

import pandas
from ROCFunctions import *
print(roc_functions("properties"))
['FunctionInterpretations', 'FunctionNames', 'Functions', 'Methods', 'Properties']

print(roc_functions("FunctionInterpretations"))
{'TPR': 'true positive rate', 'TNR': 'true negative rate', 'SPC': 'specificity', 'PPV': 'positive predictive value', 'NPV': 'negative predictive value', 'FPR': 'false positive rate', 'FDR': 'false discovery rate', 'FNR': 'false negative rate', 'ACC': 'accuracy', 'AUROC': 'area under the ROC curve', 'FOR': 'false omission rate', 'F1': 'F1 score', 'MCC': 'Matthews correlation coefficient', 'Recall': 'same as TPR', 'Precision': 'same as PPV', 'Accuracy': 'same as ACC', 'Sensitivity': 'same as TPR'}

print(roc_functions("FPR"))
<function FPR at 0x7f7612f48050>

Single ROC record

Definition: A ROC record (ROC-dictionary, or ROC-hash, or ROC-hash-map) is an associative object that has the keys: "FalseNegative", "FalsePositive", "TrueNegative", "TruePositive".Here is an example:

{"FalseNegative": 50, "FalsePositive": 51, "TrueNegative": 60, "TruePositive": 39}
{'FalseNegative': 50,
 'FalsePositive': 51,
 'TrueNegative': 60,
 'TruePositive': 39}

Here we generate a random "dataset" with columns "Actual" and "Predicted" that have the values "true" and "false"and show the summary:

from RandomDataGenerators import *

dfRandomLabels = random_data_frame(200, ["Actual", "Predicted"],
                                   generators={"Actual": ["true", "false"],
                                               "Predicted": ["true", "false"]})
dfRandomLabels.shape
(200, 2)

Here is a sample of the dataset:

print(dfRandomLabels[:4])
  Actual Predicted
0  false     false
1  false     false
2  false     false
3   true     false

Here we make the corresponding ROC dictionary:

to_roc_dict('true', 'false',
            list(dfRandomLabels.Actual.values),
            list(dfRandomLabels.Predicted.values))
{'TruePositive': 52,
 'FalsePositive': 48,
 'TrueNegative': 50,
 'FalseNegative': 50}

Multiple ROC records

Here we make random dataset with entries that associated with a certain threshold parameter with three unique values:

dfRandomLabels2 = random_data_frame(200, ["Threshold", "Actual", "Predicted"],
                                    generators={"Threshold": [0.2, 0.4, 0.6],
                                                "Actual": ["true", "false"],
                                                "Predicted": ["true", "false"]})

Remark: Threshold parameters are typically used while tuning Machine Learning (ML) classifiers. Here we find and print the ROC records(dictionaries) for each unique threshold value:

thresholds = list(dfRandomLabels2.Threshold.drop_duplicates())

rocGroups = {}
for x in thresholds:
    dfLocal = dfRandomLabels2[dfRandomLabels2["Threshold"] == x]
    rocGroups[x] = to_roc_dict('true', 'false',
                        list(dfLocal.Actual.values),
                        list(dfLocal.Predicted.values))

rocGroups
{0.4: {'TruePositive': 13,
  'FalsePositive': 23,
  'TrueNegative': 24,
  'FalseNegative': 12},
 0.2: {'TruePositive': 18,
  'FalsePositive': 11,
  'TrueNegative': 19,
  'FalseNegative': 18},
 0.6: {'TruePositive': 23,
  'FalsePositive': 9,
  'TrueNegative': 16,
  'FalseNegative': 14}}

Application of ROC functions

Here we define a list of ROC functions:

funcs = ["PPV", "NPV", "TPR", "ACC", "SPC", "MCC"]

Here we apply each ROC function to each of the ROC records obtained above:

import pandas
rocRes = { k : {f: roc_functions(f)(v) for f in funcs} for (k, v) in rocGroups.items()}

print(pandas.DataFrame(rocRes))
          0.4       0.2       0.6
PPV  0.361111  0.620690  0.718750
NPV  0.666667  0.513514  0.533333
TPR  0.520000  0.500000  0.621622
ACC  0.513889  0.560606  0.629032
SPC  0.510638  0.633333  0.640000
MCC  0.030640  0.134535  0.261666

References

Articles

[Wk1] Wikipedia entry, "Receiver operating characteristic".

[AA1] Anton Antonov, "Basic example of using ROC with Linear regression" , (2016), MathematicaForPrediction at WordPress.

[AA2] Anton Antonov, "Introduction to data wrangling with Raku" , (2021), RakuForPrediction at WordPress.

Packages

[AAp1] Anton Antonov, ROCFunctions Mathematica package, (2016-2022), MathematicaForPrediction at GitHub/antononcube.

[AAp2] Anton Antonov, ROCFunctions R package, (2021), R-packages at GitHub/antononcube.

[AAp3] Anton Antonov, ML::ROCFunctions Raku package, (2022), GitHub/antononcube.