auc-issue.py

import numpy as np
from modshogun import *

roc = ROCEvaluation()
roc.evaluate(y_pred, y_test)

auc = roc.get_auROC()
print(auc) # 0.845606993532

roc = ROCEvaluation()
roc.evaluate(BinaryLabels(y_pred.get_labels()), BinaryLabels(y_test.get_labels()))

auc = roc.get_auROC()
print(auc) # 0.563780270879

Try get_values rather than get_labels

The problem here is that the API doesnt nicely distinguish between the binary valued class labels, and the scores associated with them (those are used for the ROC computation). If you create a new labels object with the binary version get_labels(), then the AUC is different when you replicate the scores themselves.

Actually another candidate for API issues of SHogun :D

get_values instead of get_labels fixed the issue -- the AUC both print 0.845.