CNN or NME? about pycil HOT 4 CLOSED

Hi,

For those mentioned "we use NME classifier" in the paper (e.g., [1][2]), we report the NME accuracy.

For others, we report CNN accuracy.

I am not quite sure about your bug, maybe you should attach your json setting for bic.

[1] iCaRL: Incremental Classifier and Representation Learning
[2] Co-Transport for Class-Incremental Learning

from pycil.

Hi @zhoudw-zdw ,

Thanks for your reply. Here is json setting that I've used for bic experiment

{
"prefix": "reproduce",
"dataset": "cifar100",
"memory_size": 2000,
"memory_per_class": 20,
"fixed_memory": false,
"shuffle": true,
"init_cls": 10,
"increment": 10,
"model_name": "bic",
"convnet_type": "resnet32",
"device": ["2"],
"seed": [1993]
}

from pycil.

It seems correct, and the training results are somehow normal (except its results are lower than icarl).

I did not find the "score of bic at task '60-69' is almost 0", where did it happen?

BTW, you should check the algorithm-specific params in bic and make it cosistent with this repo.

from pycil.

Hi @zhoudw-zdw,

Thanks for your instant reply. Yes, I'm pretty sure that the algorithm-specific params in bic is the same as the original code of this repo. About the score of bic at task '60-69', you can see in the results that I showed above

bic:
2023-05-23 01:51:11,906 [trainer.py] => CNN: {'total': 39.06, '00-09': 53.9, '10-19': 36.5, '20-29': 47.1, '30-39': 39.9, '40-49': 51.6, '50-59': 42.1, '60-69': 0.0, '70-79': 11.5, '80-89': 46.4, '90-99': 61.6, 'old': 36.56, 'new': 61.6}

from pycil.