aangelopoulos / conformal_classification Goto Github PK

Hi, I notice that this project provides api for classification model, which may not need to be a CNN. I wonder if it is possible to extend this work in other area, for example, the application in text classification, etc.

Hi Anastasios,

thanks for providing the code for your conformal classification work.

When applying the code to a different dataset and validating it on some test data, I found that in some cases the predictive set is empty. This seems to be caused by line 170 in the function gcq in the file conformal.py. When sizes_base is 1 and np.random.random(V.shape) is larger or equal than V, this results in values of 0.
I'm not entirely sure whether the gcp function is called only during validation or whether it also has an influence on finding the parameters used for the classification, so I wanted to make you aware of this issue.

Best,
Linda

At http://people.eecs.berkeley.edu/~angelopoulos/blog/posts/conformal-classification/ it computes \lambda 1{j > k_{reg}}. Should j be i instead? If not, why does it make sense to compare an image index to the size of prediction set? (j appears referred to as image index) Thanks!