Incorrect argument definitions?

[eremingt]

Thanks for implementing this!

In both delong_roc_variance() and delong_roc_test(), predictions and predictions_one/predictions_two are defined as "probability of being class 1". Are these arguments necessarily arrays of probability values? I think ROC curves can be built off of any kind of numerical 'score', not only probabilities.

Your example code (https://github.com/yandexdataschool/roc_comparison/blob/master/example.py) passes normal random variables, rather than probabilities (variable is even called scores). I also found that scaling the inputs has no effect on the p value returned from delong_roc_test, and there are not errors returned for input values outside [0, 1].

So would it make sense to revise the parameter descriptions to call them 'scores' rather than 'probabilities'? It might even make sense to rename the variable from predictions to scores.

[google-labs-jules]

Jules is on it. When finished, you will see another comment and be able to review a PR.

[google-labs-jules]

Ready for a review! A PR has been created.