g-u-n / pycil Goto Github PK
View Code? Open in Web Editor NEWPyCIL: A Python Toolbox for Class-Incremental Learning
License: Other
pycil's Issues
about the ucir implementation
Hi, thanks for your great repo!
I found you provide "ucir_resnet.py" in your code but I don't find the implementation of the ucir method.
problems about dataset
Thanks for your excellent work!
Could you please offer some instructions about preparing ImageNet-100/ImageNet-1000? Are ImageNet-Subset and ImageNet-100 the same dataset?
Missing configs for Imagenet-Subset
Hey, thank you for this benchmark. I'm trying to reproduce your results for Imagenet-Subset dataset with 50+5x10 split. I am unable to achieve your accuracy (85%) after first task, I get 80%. Can you share experiment configs?
How to display the confusion matrix in the pycil framework, taking the wa method as an example
Inquiry about Pre-trained Model & Parameter Setup
Many thanks for this wonderful framework! It really helps our work a lot!
I have some questions about your experiment setup.
- I have reproduced your 10-stage CIFAR-100 experiments on my own PC (3*3090). The results are as followed:
I followed all the bash files and parameters you have set up, but the results seem to be much lower than yours. Is that because you use an ImageNet pre-trained ResNet? Thanks!
- I found you set different model optimization parameters (e.g. learning rate, epoch, milestone, etc.) for each continual learning approach (instead of setting the same hyperparameter policy for all the continual learning approaches). I was wondering whether this kind of parameter setup could be considered a "fair" comparison?
Thanks in advance for your answer!
Add Arxiv link to readme
Thanks for this package. I see the following Arxiv link exists, and can probably be documented in the readme:
https://arxiv.org/abs/2112.12533
PyCIL: A Python Toolbox for Class-Incremental Learning
why FeTrIL acc is so bad
config:
{
"prefix": "train",
"dataset": "cifar100",
"memory_size": 0,
"shuffle": true,
"init_cls": 50,
"increment": 10,
"model_name": "fetril",
"convnet_type": "resnet32",
"device": ["0"],
"seed": [1993],
"init_epochs": 200,
"init_lr" : 0.1,
"init_weight_decay" : 0,
"epochs" : 50,
"lr" : 0.05,
"batch_size" : 128,
"weight_decay" : 0,
"num_workers" : 8,
"T" : 2
}
final result:
2022-12-16 23:22:29,436 [fetril.py] => svm train: acc: 10.01
2022-12-16 23:22:29,451 [fetril.py] => svm evaluation: acc_list: [15.2, 12.47, 10.84, 9.78, 9.06, 8.13]
2022-12-16 23:22:31,440 [trainer.py] => No NME accuracy.
2022-12-16 23:22:31,441 [trainer.py] => CNN: {'total': 13.63, '00-09': 19.1, '10-19': 18.3, '20-29': 22.5, '30-39': 17.3, '40-49': 30.1, '50-59': 3.5, '60-69': 11.4, '70-79': 3.1, '80-89': 6.1, '90-99': 4.9, 'old': 14.6, 'new': 4.9}
2022-12-16 23:22:31,441 [trainer.py] => CNN top1 curve: [28.94, 21.87, 18.54, 16.76, 15.1, 13.63]
2022-12-16 23:22:31,441 [trainer.py] => CNN top5 curve: [53.28, 47.07, 43.17, 39.08, 36.41, 33.98]
I can provide log if you need.
coil fixed memory
Amazing toolbox!!!
I got a question about ur results of coil.
In your work. Section 5.2
Since all compared methods are exemplar-based, we fix an equal number of exemplars for every method, i.e., 2,000 exemplars for CIFAR-100 and ImageNet100, 20,000 for ImageNet-1000. As a result, the picked exemplars per class is 20, which is abundant for every class.
I just wanna check the replay size with fixed memory of 2,000 in totoal over training process, which means that the "fixed_memory" in json file is set false, as shown in this link. I'm a little bit confused about this setting due to there are different protocols in recent community.
Line 6 in 6d2c128
The reason why I came corss this issues is:
As shown in this table, the icarl results of 10 steps is reported about 61.74, which is lower than that in the original paper of about 64.
Hope to get ur replay early. THX in advance.
Results of the GEM
Hi! Thanks for your contribution to this excellent job!
I have a question regarding the acc of GEM.
I ran the main.py script with the configuration file ./exps/gem.json. However, I noticed that the accuracy decreases rapidly, and the final accuracy is as follows:
- CNN top1 curve: [89.7, 41.2, 31.17, 22.5, 20.52, 15.83, 14.87, 13.25, 11.96, 10.09]
Have you also run the code? I'm curious to know about your results.
The log file is as follows:
2023-07-10 15:20:22,349 [trainer.py] => config: ./exps/gem.json
2023-07-10 15:20:22,400 [trainer.py] => prefix: reproduce
2023-07-10 15:20:22,401 [trainer.py] => dataset: cifar100
2023-07-10 15:20:22,401 [trainer.py] => memory_size: 2000
2023-07-10 15:20:22,402 [trainer.py] => memory_per_class: 20
2023-07-10 15:20:22,402 [trainer.py] => fixed_memory: False
2023-07-10 15:20:22,403 [trainer.py] => shuffle: True
2023-07-10 15:20:22,403 [trainer.py] => init_cls: 10
2023-07-10 15:20:22,403 [trainer.py] => increment: 10
2023-07-10 15:20:22,403 [trainer.py] => model_name: gem
2023-07-10 15:20:22,404 [trainer.py] => convnet_type: resnet32
2023-07-10 15:20:22,404 [trainer.py] => device: [device(type='cuda', index=0), device(type='cuda', index=1), device(type='cuda', index=2), device(type='cuda', index=3)]
2023-07-10 15:20:22,404 [trainer.py] => seed: 1993
2023-07-10 15:20:25,708 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-07-10 15:20:26,016 [trainer.py] => All params: 464154
2023-07-10 15:20:26,017 [trainer.py] => Trainable params: 464154
2023-07-10 15:20:26,017 [gem.py] => Learning on 0-10
2023-07-10 15:37:02,190 [gem.py] => Task 0, Epoch 200/200 => Loss 0.050, Train_accy 99.32
2023-07-10 15:37:02,197 [base.py] => Reducing exemplars...(200 per classes)
2023-07-10 15:37:02,198 [base.py] => Constructing exemplars...(200 per classes)
2023-07-10 15:37:10,648 [gem.py] => Exemplar size: 2000
2023-07-10 15:37:10,649 [trainer.py] => CNN: {'total': 89.7, '00-09': 89.7, 'old': 0, 'new': 89.7}
2023-07-10 15:37:10,649 [trainer.py] => NME: {'total': 89.2, '00-09': 89.2, 'old': 0, 'new': 89.2}
2023-07-10 15:37:10,650 [trainer.py] => CNN top1 curve: [89.7]
2023-07-10 15:37:10,650 [trainer.py] => CNN top5 curve: [99.1]
2023-07-10 15:37:10,650 [trainer.py] => NME top1 curve: [89.2]
2023-07-10 15:37:10,651 [trainer.py] => NME top5 curve: [99.2]
2023-07-10 15:37:10,652 [trainer.py] => All params: 464804
2023-07-10 15:37:10,653 [trainer.py] => Trainable params: 464804
2023-07-10 15:37:10,654 [gem.py] => Learning on 10-20
2023-07-10 15:55:21,477 [gem.py] => Task 1, Epoch 100/100 => Loss 0.115, Train_accy 97.28
2023-07-10 15:55:21,490 [base.py] => Reducing exemplars...(100 per classes)
2023-07-10 15:55:24,805 [base.py] => Constructing exemplars...(100 per classes)
2023-07-10 15:55:33,788 [gem.py] => Exemplar size: 2000
2023-07-10 15:55:33,789 [trainer.py] => CNN: {'total': 41.2, '00-09': 2.5, '10-19': 79.9, 'old': 2.5, 'new': 79.9}
2023-07-10 15:55:33,789 [trainer.py] => NME: {'total': 54.1, '00-09': 38.8, '10-19': 69.4, 'old': 38.8, 'new': 69.4}
2023-07-10 15:55:33,790 [trainer.py] => CNN top1 curve: [89.7, 41.2]
2023-07-10 15:55:33,790 [trainer.py] => CNN top5 curve: [99.1, 86.3]
2023-07-10 15:55:33,791 [trainer.py] => NME top1 curve: [89.2, 54.1]
2023-07-10 15:55:33,791 [trainer.py] => NME top5 curve: [99.2, 88.95]
2023-07-10 15:55:33,792 [trainer.py] => All params: 465454
2023-07-10 15:55:33,793 [trainer.py] => Trainable params: 465454
2023-07-10 15:55:33,794 [gem.py] => Learning on 20-30
2023-07-10 16:17:18,557 [gem.py] => Task 2, Epoch 100/100 => Loss 0.027, Train_accy 99.40
2023-07-10 16:17:18,565 [base.py] => Reducing exemplars...(66 per classes)
2023-07-10 16:17:24,680 [base.py] => Constructing exemplars...(66 per classes)
2023-07-10 16:17:33,541 [gem.py] => Exemplar size: 1980
2023-07-10 16:17:33,541 [trainer.py] => CNN: {'total': 31.17, '00-09': 0.2, '10-19': 2.9, '20-29': 90.4, 'old': 1.55, 'new': 90.4}
2023-07-10 16:17:33,541 [trainer.py] => NME: {'total': 50.6, '00-09': 39.0, '10-19': 36.1, '20-29': 76.7, 'old': 37.55, 'new': 76.7}
2023-07-10 16:17:33,541 [trainer.py] => CNN top1 curve: [89.7, 41.2, 31.17]
2023-07-10 16:17:33,541 [trainer.py] => CNN top5 curve: [99.1, 86.3, 70.37]
2023-07-10 16:17:33,541 [trainer.py] => NME top1 curve: [89.2, 54.1, 50.6]
2023-07-10 16:17:33,541 [trainer.py] => NME top5 curve: [99.2, 88.95, 85.07]
2023-07-10 16:17:33,542 [trainer.py] => All params: 466104
2023-07-10 16:17:33,542 [trainer.py] => Trainable params: 466104
2023-07-10 16:17:33,543 [gem.py] => Learning on 30-40
2023-07-10 16:44:41,096 [gem.py] => Task 3, Epoch 100/100 => Loss 0.053, Train_accy 98.58
2023-07-10 16:44:41,097 [base.py] => Reducing exemplars...(50 per classes)
2023-07-10 16:44:49,552 [base.py] => Constructing exemplars...(50 per classes)
2023-07-10 16:44:58,090 [gem.py] => Exemplar size: 2000
2023-07-10 16:44:58,090 [trainer.py] => CNN: {'total': 22.5, '00-09': 0.1, '10-19': 0.0, '20-29': 3.8, '30-39': 86.1, 'old': 1.3, 'new': 86.1}
2023-07-10 16:44:58,090 [trainer.py] => NME: {'total': 41.82, '00-09': 35.5, '10-19': 26.2, '20-29': 41.0, '30-39': 64.6, 'old': 34.23, 'new': 64.6}
2023-07-10 16:44:58,090 [trainer.py] => CNN top1 curve: [89.7, 41.2, 31.17, 22.5]
2023-07-10 16:44:58,091 [trainer.py] => CNN top5 curve: [99.1, 86.3, 70.37, 59.52]
2023-07-10 16:44:58,091 [trainer.py] => NME top1 curve: [89.2, 54.1, 50.6, 41.82]
2023-07-10 16:44:58,091 [trainer.py] => NME top5 curve: [99.2, 88.95, 85.07, 78.03]
2023-07-10 16:44:58,091 [trainer.py] => All params: 466754
2023-07-10 16:44:58,092 [trainer.py] => Trainable params: 466754
2023-07-10 16:44:58,093 [gem.py] => Learning on 40-50
2023-07-10 17:18:36,563 [gem.py] => Task 4, Epoch 100/100 => Loss 0.043, Train_accy 99.44
2023-07-10 17:18:36,564 [base.py] => Reducing exemplars...(40 per classes)
2023-07-10 17:18:48,883 [base.py] => Constructing exemplars...(40 per classes)
2023-07-10 17:18:57,582 [gem.py] => Exemplar size: 2000
2023-07-10 17:18:57,582 [trainer.py] => CNN: {'total': 20.52, '00-09': 0.1, '10-19': 0.0, '20-29': 0.1, '30-39': 10.7, '40-49': 91.7, 'old': 2.72, 'new': 91.7}
2023-07-10 17:18:57,582 [trainer.py] => NME: {'total': 38.58, '00-09': 32.9, '10-19': 24.0, '20-29': 33.1, '30-39': 28.5, '40-49': 74.4, 'old': 29.62, 'new': 74.4}
2023-07-10 17:18:57,582 [trainer.py] => CNN top1 curve: [89.7, 41.2, 31.17, 22.5, 20.52]
2023-07-10 17:18:57,583 [trainer.py] => CNN top5 curve: [99.1, 86.3, 70.37, 59.52, 54.56]
2023-07-10 17:18:57,583 [trainer.py] => NME top1 curve: [89.2, 54.1, 50.6, 41.82, 38.58]
2023-07-10 17:18:57,583 [trainer.py] => NME top5 curve: [99.2, 88.95, 85.07, 78.03, 75.4]
2023-07-10 17:18:57,583 [trainer.py] => All params: 467404
2023-07-10 17:18:57,584 [trainer.py] => Trainable params: 467404
2023-07-10 17:18:57,584 [gem.py] => Learning on 50-60
2023-07-10 17:57:50,484 [gem.py] => Task 5, Epoch 100/100 => Loss 0.038, Train_accy 99.12
2023-07-10 17:57:50,486 [base.py] => Reducing exemplars...(33 per classes)
2023-07-10 17:58:08,334 [base.py] => Constructing exemplars...(33 per classes)
2023-07-10 17:58:17,711 [gem.py] => Exemplar size: 1980
2023-07-10 17:58:17,711 [trainer.py] => CNN: {'total': 15.83, '00-09': 0.2, '10-19': 0.0, '20-29': 0.0, '30-39': 0.6, '40-49': 7.6, '50-59': 86.6, 'old': 1.68, 'new': 86.6}
2023-07-10 17:58:17,711 [trainer.py] => NME: {'total': 32.87, '00-09': 26.8, '10-19': 20.6, '20-29': 24.8, '30-39': 19.6, '40-49': 38.7, '50-59': 66.7, 'old': 26.1, 'new': 66.7}
2023-07-10 17:58:17,711 [trainer.py] => CNN top1 curve: [89.7, 41.2, 31.17, 22.5, 20.52, 15.83]
2023-07-10 17:58:17,711 [trainer.py] => CNN top5 curve: [99.1, 86.3, 70.37, 59.52, 54.56, 46.77]
2023-07-10 17:58:17,711 [trainer.py] => NME top1 curve: [89.2, 54.1, 50.6, 41.82, 38.58, 32.87]
2023-07-10 17:58:17,711 [trainer.py] => NME top5 curve: [99.2, 88.95, 85.07, 78.03, 75.4, 68.92]
2023-07-10 17:58:17,712 [trainer.py] => All params: 468054
2023-07-10 17:58:17,712 [trainer.py] => Trainable params: 468054
2023-07-10 17:58:17,713 [gem.py] => Learning on 60-70
2023-07-10 18:43:33,590 [gem.py] => Task 6, Epoch 100/100 => Loss 0.031, Train_accy 99.26
2023-07-10 18:43:33,592 [base.py] => Reducing exemplars...(28 per classes)
2023-07-10 18:43:51,437 [base.py] => Constructing exemplars...(28 per classes)
2023-07-10 18:44:00,666 [gem.py] => Exemplar size: 1960
2023-07-10 18:44:00,667 [trainer.py] => CNN: {'total': 14.87, '00-09': 0.0, '10-19': 0.0, '20-29': 0.0, '30-39': 0.0, '40-49': 0.2, '50-59': 13.6, '60-69': 90.3, 'old': 2.3, 'new': 90.3}
2023-07-10 18:44:00,667 [trainer.py] => NME: {'total': 30.54, '00-09': 22.5, '10-19': 12.6, '20-29': 24.2, '30-39': 22.4, '40-49': 28.0, '50-59': 33.8, '60-69': 70.3, 'old': 23.92, 'new': 70.3}
2023-07-10 18:44:00,667 [trainer.py] => CNN top1 curve: [89.7, 41.2, 31.17, 22.5, 20.52, 15.83, 14.87]
2023-07-10 18:44:00,667 [trainer.py] => CNN top5 curve: [99.1, 86.3, 70.37, 59.52, 54.56, 46.77, 40.73]
2023-07-10 18:44:00,667 [trainer.py] => NME top1 curve: [89.2, 54.1, 50.6, 41.82, 38.58, 32.87, 30.54]
2023-07-10 18:44:00,667 [trainer.py] => NME top5 curve: [99.2, 88.95, 85.07, 78.03, 75.4, 68.92, 63.96]
2023-07-10 18:44:00,667 [trainer.py] => All params: 468704
2023-07-10 18:44:00,668 [trainer.py] => Trainable params: 468704
2023-07-10 18:44:00,669 [gem.py] => Learning on 70-80
2023-07-10 19:35:29,475 [gem.py] => Task 7, Epoch 100/100 => Loss 0.049, Train_accy 99.36
2023-07-10 19:35:29,483 [base.py] => Reducing exemplars...(25 per classes)
2023-07-10 19:35:49,603 [base.py] => Constructing exemplars...(25 per classes)
2023-07-10 19:35:58,735 [gem.py] => Exemplar size: 2000
2023-07-10 19:35:58,735 [trainer.py] => CNN: {'total': 13.25, '00-09': 0.0, '10-19': 0.0, '20-29': 0.0, '30-39': 0.0, '40-49': 0.1, '50-59': 3.8, '60-69': 15.7, '70-79': 86.4, 'old': 2.8, 'new': 86.4}
2023-07-10 19:35:58,735 [trainer.py] => NME: {'total': 30.31, '00-09': 29.2, '10-19': 16.5, '20-29': 26.4, '30-39': 20.0, '40-49': 28.9, '50-59': 22.8, '60-69': 37.9, '70-79': 60.8, 'old': 25.96, 'new': 60.8}
2023-07-10 19:35:58,735 [trainer.py] => CNN top1 curve: [89.7, 41.2, 31.17, 22.5, 20.52, 15.83, 14.87, 13.25]
2023-07-10 19:35:58,735 [trainer.py] => CNN top5 curve: [99.1, 86.3, 70.37, 59.52, 54.56, 46.77, 40.73, 40.42]
2023-07-10 19:35:58,735 [trainer.py] => NME top1 curve: [89.2, 54.1, 50.6, 41.82, 38.58, 32.87, 30.54, 30.31]
2023-07-10 19:35:58,735 [trainer.py] => NME top5 curve: [99.2, 88.95, 85.07, 78.03, 75.4, 68.92, 63.96, 64.45]
2023-07-10 19:35:58,736 [trainer.py] => All params: 469354
2023-07-10 19:35:58,736 [trainer.py] => Trainable params: 469354
2023-07-10 19:35:58,737 [gem.py] => Learning on 80-90
2023-07-10 20:37:40,337 [gem.py] => Task 8, Epoch 100/100 => Loss 0.062, Train_accy 99.18
2023-07-10 20:37:40,338 [base.py] => Reducing exemplars...(22 per classes)
2023-07-10 20:38:05,180 [base.py] => Constructing exemplars...(22 per classes)
2023-07-10 20:38:14,743 [gem.py] => Exemplar size: 1980
2023-07-10 20:38:14,743 [trainer.py] => CNN: {'total': 11.96, '00-09': 0.0, '10-19': 0.0, '20-29': 0.0, '30-39': 0.0, '40-49': 0.0, '50-59': 0.1, '60-69': 0.7, '70-79': 17.5, '80-89': 89.3, 'old': 2.29, 'new': 89.3}
2023-07-10 20:38:14,743 [trainer.py] => NME: {'total': 27.93, '00-09': 23.6, '10-19': 14.9, '20-29': 23.5, '30-39': 17.1, '40-49': 27.5, '50-59': 22.5, '60-69': 28.8, '70-79': 33.2, '80-89': 60.3, 'old': 23.89, 'new': 60.3}
2023-07-10 20:38:14,743 [trainer.py] => CNN top1 curve: [89.7, 41.2, 31.17, 22.5, 20.52, 15.83, 14.87, 13.25, 11.96]
2023-07-10 20:38:14,743 [trainer.py] => CNN top5 curve: [99.1, 86.3, 70.37, 59.52, 54.56, 46.77, 40.73, 40.42, 35.9]
2023-07-10 20:38:14,743 [trainer.py] => NME top1 curve: [89.2, 54.1, 50.6, 41.82, 38.58, 32.87, 30.54, 30.31, 27.93]
2023-07-10 20:38:14,743 [trainer.py] => NME top5 curve: [99.2, 88.95, 85.07, 78.03, 75.4, 68.92, 63.96, 64.45, 62.21]
2023-07-10 20:38:14,744 [trainer.py] => All params: 470004
2023-07-10 20:38:14,744 [trainer.py] => Trainable params: 470004
2023-07-10 20:38:14,745 [gem.py] => Learning on 90-100
2023-07-10 21:45:46,207 [gem.py] => Task 9, Epoch 100/100 => Loss 0.075, Train_accy 98.40
2023-07-10 21:45:46,214 [base.py] => Reducing exemplars...(20 per classes)
2023-07-10 21:46:12,361 [base.py] => Constructing exemplars...(20 per classes)
2023-07-10 21:46:22,382 [gem.py] => Exemplar size: 2000
2023-07-10 21:46:22,382 [trainer.py] => CNN: {'total': 10.09, '00-09': 0.0, '10-19': 0.0, '20-29': 0.0, '30-39': 0.0, '40-49': 0.0, '50-59': 0.1, '60-69': 0.0, '70-79': 1.4, '80-89': 14.3, '90-99': 85.1, 'old': 1.76, 'new': 85.1}
2023-07-10 21:46:22,382 [trainer.py] => NME: {'total': 26.77, '00-09': 21.9, '10-19': 16.6, '20-29': 23.1, '30-39': 17.1, '40-49': 25.5, '50-59': 16.3, '60-69': 30.0, '70-79': 28.7, '80-89': 28.6, '90-99': 59.9, 'old': 23.09, 'new': 59.9}
2023-07-10 21:46:22,382 [trainer.py] => CNN top1 curve: [89.7, 41.2, 31.17, 22.5, 20.52, 15.83, 14.87, 13.25, 11.96, 10.09]
2023-07-10 21:46:22,382 [trainer.py] => CNN top5 curve: [99.1, 86.3, 70.37, 59.52, 54.56, 46.77, 40.73, 40.42, 35.9, 30.94]
2023-07-10 21:46:22,382 [trainer.py] => NME top1 curve: [89.2, 54.1, 50.6, 41.82, 38.58, 32.87, 30.54, 30.31, 27.93, 26.77]
2023-07-10 21:46:22,382 [trainer.py] => NME top5 curve: [99.2, 88.95, 85.07, 78.03, 75.4, 68.92, 63.96, 64.45, 62.21, 57.29]
Looking forward to your response. Thank you sincerely!
Fetril gets error on cifar10
Hi! I saw the same issue that was closed. However, I got it now. How can I resolve it?
Config:
{
"prefix": "train",
"dataset": "clipart",
"memory_size": 0,
"shuffle": true,
"init_cls": 40,
"increment": 1,
"model_name": "fetril",
"convnet_type": "resnet32",
"device": ["0"],
"seed": [1993],
"init_epochs": 200,
"init_lr" : 0.1,
"init_weight_decay" : 5e-4,
"epochs" : 50,
"lr" : 0.05,
"batch_size" : 128,
"weight_decay" : 5e-4,
"num_workers" : 8,
"T" : 2
}
The error:
0%| | 0/200 [00:00<?, ?it/s]
Error in sys.excepthook:
Traceback (most recent call last):
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/humbug/report.py", line 525, in _hook
self.error_report(error=exception_instance, tags=tags, publish=publish)
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/humbug/report.py", line 267, in error_report
traceback.format_exception(
TypeError: format_exception() got an unexpected keyword argument 'etype'
Original exception was:
Traceback (most recent call last):
File "/home/khan_lera/pycil/main.py", line 31, in
main()
File "/home/khan_lera/pycil/main.py", line 12, in main
train(args)
File "/home/khan_lera/pycil/trainer.py", line 18, in train
_train(args)
File "/home/khan_lera/pycil/trainer.py", line 65, in _train
model.incremental_train(data_manager)
File "/home/khan_lera/pycil/models/fetril.py", line 86, in incremental_train
self._train(self.train_loader, self.test_loader)
File "/home/khan_lera/pycil/models/fetril.py", line 102, in _train
self._train_function(train_loader, test_loader, optimizer, scheduler)
File "/home/khan_lera/pycil/models/fetril.py", line 175, in train_function
for i, (, inputs, targets) in enumerate(train_loader):
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 681, in next
data = self._next_data()
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data
return self._process_data(data)
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data
data.reraise()
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/torch/_utils.py", line 461, in reraise
raise exception
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/khan_lera/pycil/utils/data_manager.py", line 202, in getitem
image = self.trsf(pil_loader(self.images[idx]))
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/torchvision/transforms/transforms.py", line 94, in call
img = t(img)
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/torchvision/transforms/transforms.py", line 134, in call
return F.to_tensor(pic)
File "/home/khan_lera/miniconda3/envs/contle/lib/python3.10/site-packages/torchvision/transforms/functional.py", line 138, in to_tensor
raise TypeError(f"pic should be PIL Image or ndarray. Got {type(pic)}")
TypeError: pic should be PIL Image or ndarray. Got <class 'torch.Tensor'>
Single GPU training error in DER
Hi,
Thank you for this wonderful code base!
I noticed the current version doesn't support single GPU training. Could you please add this feature?
Thank you!
CNN or NME?
Hi,
Thanks for your interesting work! I would love to know if the reproduced score demonstrated in your paper is CNN top 1 curve or NME top1 curve in the provided code? I tried to run your code without changing anything and got the following results for three CIL method: bic, icarl, and foster. The experiments are run on CIFAR100 dataset with 10 incremental classes for each task
bic:
2023-05-23 01:51:11,906 [trainer.py] => CNN: {'total': 39.06, '00-09': 53.9, '10-19': 36.5, '20-29': 47.1, '30-39': 39.9, '40-49': 51.6, '50-59': 42.1, '60-69': 0.0, '70-79': 11.5, '80-89': 46.4, '90-99': 61.6, 'old': 36.56, 'new': 61.6}
2023-05-23 01:51:11,906 [trainer.py] => NME: {'total': 46.11, '00-09': 49.6, '10-19': 32.3, '20-29': 50.6, '30-39': 38.5, '40-49': 50.4, '50-59': 38.4, '60-69': 46.0, '70-79': 45.5, '80-89': 52.6, '90-99': 57.2, 'old': 44.88, 'new': 57.2}
2023-05-23 01:51:11,906 [trainer.py] => CNN top1 curve: [86.5, 71.3, 68.4, 62.35, 61.42, 58.17, 48.27, 42.31, 40.74, 39.06]
2023-05-23 01:51:11,906 [trainer.py] => CNN top5 curve: [99.0, 94.25, 91.77, 88.98, 87.98, 86.15, 72.44, 70.03, 68.52, 66.57]
2023-05-23 01:51:11,906 [trainer.py] => NME top1 curve: [86.3, 71.6, 68.73, 62.7, 61.72, 57.62, 55.69, 50.46, 48.24, 46.11]
2023-05-23 01:51:11,906 [trainer.py] => NME top5 curve: [98.7, 94.0, 91.43, 88.88, 87.56, 84.62, 82.54, 79.0, 77.08, 75.35]
icarl:
2023-05-23 00:26:06,251 [trainer.py] => CNN: {'total': 42.06, '00-09': 35.4, '10-19': 23.3, '20-29': 34.9, '30-39': 29.6, '40-49': 38.7, '50-59': 29.2, '60-69': 43.1, '70-79': 44.5, '80-89': 60.9, '90-99': 81.0, 'old': 37.73, 'new': 81.0}
2023-05-23 00:26:06,251 [trainer.py] => NME: {'total': 49.4, '00-09': 47.4, '10-19': 32.0, '20-29': 48.5, '30-39': 40.2, '40-49': 51.3, '50-59': 42.1, '60-69': 55.2, '70-79': 53.2, '80-89': 59.3, '90-99': 64.8, 'old': 47.69, 'new': 64.8}
2023-05-23 00:26:06,251 [trainer.py] => CNN top1 curve: [89.8, 77.95, 72.6, 63.72, 58.54, 54.17, 52.29, 47.32, 44.53, 42.06]
2023-05-23 00:26:06,251 [trainer.py] => CNN top5 curve: [99.5, 96.25, 93.77, 90.4, 87.38, 83.02, 80.8, 77.85, 75.43, 72.11]
2023-05-23 00:26:06,251 [trainer.py] => NME top1 curve: [89.8, 79.1, 74.27, 68.75, 64.44, 60.77, 58.09, 54.55, 51.34, 49.4]
2023-05-23 00:26:06,251 [trainer.py] => NME top5 curve: [99.4, 96.0, 94.37, 91.35, 89.32, 85.45, 83.87, 82.19, 79.02, 77.53]
foster:
2023-05-23 01:18:52,589 [trainer.py] => CNN: {'total': 54.55, '00-09': 48.8, '10-19': 33.5, '20-29': 51.3, '30-39': 46.8, '40-49': 59.0, '50-59': 52.7, '60-69': 66.9, '70-79': 60.5, '80-89': 64.0, '90-99': 62.0, 'old': 53.72, 'new': 62.0}
2023-05-23 01:18:52,589 [trainer.py] => NME: {'total': 48.13, '00-09': 46.9, '10-19': 32.9, '20-29': 49.5, '30-39': 42.9, '40-49': 52.0, '50-59': 42.4, '60-69': 50.2, '70-79': 43.1, '80-89': 45.6, '90-99': 75.8, 'old': 45.06, 'new': 75.8}
2023-05-23 01:18:52,589 [trainer.py] => CNN top1 curve: [90.8, 81.2, 76.27, 67.72, 63.86, 60.65, 59.81, 56.8, 55.58, 54.55]
2023-05-23 01:18:52,589 [trainer.py] => CNN top5 curve: [99.2, 95.75, 93.83, 90.25, 87.82, 85.35, 84.29, 83.19, 82.04, 81.68]
2023-05-23 01:18:52,589 [trainer.py] => NME top1 curve: [90.2, 82.35, 77.07, 68.8, 64.28, 59.82, 57.11, 51.94, 51.09, 48.13]
2023-05-23 01:18:52,589 [trainer.py] => NME top5 curve: [99.2, 96.85, 95.03, 91.02, 88.38, 85.8, 83.57, 81.66, 79.91, 76.7]
In addition, I found that score of bic at task '60-69' is almost 0 which indicate that the learning algorithm lost its capability in predicting for that task. I don't know if this is normal for bic method or I have encountered some issues when training from my side. Thanks for your help!
ImageNet100各方法精度
您好，我没找到利用ResNet18作为backbone的ImageNet100的top1精度，看到您这里有，所以想咨询下方便给出以ResNet18而不是32 为backbone的WA，BiC，iCaRL，DER，DER w/o p的每个session的top1精度和最终avg精度吗？谢谢了
Some questions when run gem
File "/home/doublezore/Code/Continual learning/PyCIL/models/gem.py", line 184, in update_representation
G[stpt:endpt, k].data.copy(params.grad.data.view(-1))
AttributeError: 'NoneType' object has no attribute 'data'
After investigation, I found that in the last fc layer of the model, there is no forward propagation. I would like to ask you how to solve this problem?
foster error: TypeError: string indices must be integers
after train the second task and before compress:
Task 1, Epoch 167/170 => Loss 0.752, Loss_clf 0.000, Loss_fe 0.001, Loss_kd 0.375, Train_accy 99.00: 98%|█████████▊| 167/170 [09:02<00:10, 3.35s/it]
Task 1, Epoch 168/170 => Loss 0.746, Loss_clf 0.001, Loss_fe 0.001, Loss_kd 0.372, Train_accy 98.75: 98%|█████████▊| 167/170 [09:04<00:10, 3.35s/it]
Task 1, Epoch 168/170 => Loss 0.746, Loss_clf 0.001, Loss_fe 0.001, Loss_kd 0.372, Train_accy 98.75: 99%|█████████▉| 168/170 [09:04<00:06, 3.20s/it]
Task 1, Epoch 169/170 => Loss 0.744, Loss_clf 0.001, Loss_fe 0.001, Loss_kd 0.371, Train_accy 98.81: 99%|█████████▉| 168/170 [09:07<00:06, 3.20s/it]
Task 1, Epoch 169/170 => Loss 0.744, Loss_clf 0.001, Loss_fe 0.001, Loss_kd 0.371, Train_accy 98.81: 99%|█████████▉| 169/170 [09:07<00:03, 3.10s/it]
Task 1, Epoch 170/170 => Loss 0.746, Loss_clf 0.001, Loss_fe 0.001, Loss_kd 0.372, Train_accy 98.67: 99%|█████████▉| 169/170 [09:10<00:03, 3.10s/it]
Task 1, Epoch 170/170 => Loss 0.746, Loss_clf 0.001, Loss_fe 0.001, Loss_kd 0.372, Train_accy 98.67: 100%|██████████| 170/170 [09:10<00:00, 3.01s/it]
Task 1, Epoch 170/170 => Loss 0.746, Loss_clf 0.001, Loss_fe 0.001, Loss_kd 0.372, Train_accy 98.67: 100%|██████████| 170/170 [09:10<00:00, 3.24s/it]
Traceback (most recent call last):
File "foster.py", line 31, in <module>
main()
File "foster.py", line 12, in main
train(args)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/trainer.py", line 18, in train
_train(args)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/trainer.py", line 67, in _train
model.incremental_train(data_manager)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/models/foster.py", line 89, in incremental_train
self._train(self.train_loader, self.test_loader)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/models/foster.py", line 169, in _train
self._feature_compression(train_loader, test_loader)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/models/foster.py", line 289, in _feature_compression
self._snet = FOSTERNet(self.args["convnet_type"], False)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/utils/inc_net.py", line 402, in __init__
self.convnet_type = args["convnet_type"]
TypeError: string indices must be integers
Dynamical Expansion of DER
Hello, you did an excellent job and thanks for your contribution.
But at the DER learner, I can't find the code of Dynamical Expansion, did you implement this part of DER?
how to use tiny imagenet?
how to download it?
how to resume training
How can we resume training if PC shuts down or hangs due to some issue.
Question of DER: freeze the old module is left out in the file der.py
Why "freezing old network" of after_task is left out in file der.py, while other methods, such as podnet and WA execute this step as follows:
self._old_network = self._network.copy().freeze()?
where's the trained model? where can I see the classification result?
I am eager to apply incremental learning not only in academic papers, for example, just evaluating model accuracy results; but also in practical scenarios, such as enabling continuous evolution of the model. However, after studying some incremental learning code, I have not yet found any that involve saving the trained model for further retraining, or simply saving the incrementally trained model for future classification tasks. I wonder what your thoughts are on this? If there are any difficulties or complexities in such an implementation, could you please provide me with some guidance? I would greatly appreciate your help in resolving my doubts!
the memory of rmm
Hello!
First of all, thank you very much for your work.When I read the rmm code, I found that memory is not 2000. Instead, only 2000 pieces of data were used for the new task. Is rmm originally like this?
I add the following code to rmm.py to see the number of memories.
Memory size requiremnt for BiC
I'm applying BiC for a dataset of 18 classes each has more than 100 samples. Could you please explain the memory size requirement?
I tested it on 6 classes increment by 2 (100 samples) with memory size 120, and it worked. However, it issues an error when I used the whole dataset (ValueError: attempt to get argmin of an empty sequence).
关于FOSTER的数据增强
FOSTER很令我感兴趣的工作。
我尝试了cifar100的benchmark的设置。
我发现pycil中的实现并没有数据增强，如：https://github.com/G-U-N/ECCV22-FOSTER/blob/340fbd1a16ca6a787e522b9dc349a5e742f00c30/utils/data.py#L73
因此，pycil的实现结果要差一点。请问还有其他的一些细节上的不同吗？
Why does der use softmax for training and NME for testing? The original classifiers are all softmax?
Why does der use softmax for training and NME for testing? The original classifiers are all softmax?
A problem regarding the replication of PASS on other datasets.
Thank you for the excellent work.
Regarding the self-supervision of PASS, the image size was fixed to 32 when the augmentation was performed (around line 108 of pa2s.py). The image size should be set according to the dataset, otherwise it will report an error on datasets other than CIFAR.
What is the purpose of the reduce Exemplar function
I'd like to know the role of the Reduce examplar functions, construct examplar functions, and construct Exemplar unified functions in your base.py file. Is there a reference for how they work?
why loss_clf = F.cross_entropy(logits[:, self._known_classes:], fake_targets)?
Thank you very much for your excellent work.
In models/finetune.py ， line 117：
fake_targets=targets-self._known_classes
loss_clf = F.cross_entropy(logits[:,self._known_classes:], fake_targets)
loss=loss_clf
why not :
loss_clf = F.cross_entropy(logits, targets)
loss=loss_clf
Does the code have Settings for model saving
memory_size，memory_per_class
Hello, thank you very much for the code you provided. Can you tell me what the parameter memory_size and memory_per_class represents？
Reproducibility
Hi,
Thanks so much for the cool project. I tried to reproduce the results reported in your paper and failed to reproduce it though I did not change anything. Here is the result of icarl training on cifar100 base 0:
It would be great if you could run the same experiment from your side and compare with what I have here. Thanks and really appreciate your help.
2023-05-22 17:20:04,654 [trainer.py] => prefix: reproduce
2023-05-22 17:20:04,654 [trainer.py] => dataset: cifar100
2023-05-22 17:20:04,654 [trainer.py] => memory_size: 2000
2023-05-22 17:20:04,654 [trainer.py] => memory_per_class: 20
2023-05-22 17:20:04,654 [trainer.py] => fixed_memory: False
2023-05-22 17:20:04,654 [trainer.py] => shuffle: True
2023-05-22 17:20:04,654 [trainer.py] => init_cls: 10
2023-05-22 17:20:04,654 [trainer.py] => increment: 10
2023-05-22 17:20:04,654 [trainer.py] => model_name: icarl
2023-05-22 17:20:04,654 [trainer.py] => convnet_type: resnet32
2023-05-22 17:20:04,654 [trainer.py] => device: [device(type='cuda', index=0), device(type='cuda', index=1), device(type='cuda', index=2), device(type='cuda', index=3)]
2023-05-22 17:20:04,654 [trainer.py] => seed: 1993
2023-05-22 17:20:06,016 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-05-22 17:20:06,132 [trainer.py] => All params: 464154
2023-05-22 17:20:06,132 [trainer.py] => Trainable params: 464154
2023-05-22 17:20:06,132 [icarl.py] => Learning on 0-10
2023-05-22 18:17:16,663 [icarl.py] => Task 0, Epoch 200/200 => Loss 0.038, Train_accy 99.44
2023-05-22 18:17:16,663 [base.py] => Reducing exemplars...(200 per classes)
2023-05-22 18:17:16,663 [base.py] => Constructing exemplars...(200 per classes)
2023-05-22 18:17:29,541 [icarl.py] => Exemplar size: 2000
2023-05-22 18:17:29,541 [trainer.py] => CNN: {'total': 91.2, '00-09': 91.2, 'old': 0, 'new': 91.2}
2023-05-22 18:17:29,541 [trainer.py] => NME: {'total': 91.6, '00-09': 91.6, 'old': 0, 'new': 91.6}
2023-05-22 18:17:29,541 [trainer.py] => CNN top1 curve: [91.2]
2023-05-22 18:17:29,541 [trainer.py] => CNN top5 curve: [99.3]
2023-05-22 18:17:29,541 [trainer.py] => NME top1 curve: [91.6]
2023-05-22 18:17:29,541 [trainer.py] => NME top5 curve: [99.8]
2023-05-22 18:17:29,541 [trainer.py] => All params: 464804
2023-05-22 18:17:29,541 [trainer.py] => Trainable params: 464804
2023-05-22 18:17:29,545 [icarl.py] => Learning on 10-20
2023-05-22 19:24:28,935 [icarl.py] => Task 1, Epoch 170/170 => Loss 1.043, Train_accy 99.64
2023-05-22 19:24:28,935 [base.py] => Reducing exemplars...(100 per classes)
2023-05-22 19:24:33,625 [base.py] => Constructing exemplars...(100 per classes)
2023-05-22 19:24:45,992 [icarl.py] => Exemplar size: 2000
2023-05-22 19:24:45,992 [trainer.py] => CNN: {'total': 79.6, '00-09': 79.6, '10-19': 79.6, 'old': 79.6, 'new': 79.6}
2023-05-22 19:24:45,992 [trainer.py] => NME: {'total': 79.1, '00-09': 81.8, '10-19': 76.4, 'old': 81.8, 'new': 76.4}
2023-05-22 19:24:45,992 [trainer.py] => CNN top1 curve: [91.2, 79.6]
2023-05-22 19:24:45,992 [trainer.py] => CNN top5 curve: [99.3, 96.5]
2023-05-22 19:24:45,992 [trainer.py] => NME top1 curve: [91.6, 79.1]
2023-05-22 19:24:45,992 [trainer.py] => NME top5 curve: [99.8, 96.3]
2023-05-22 19:24:45,992 [trainer.py] => All params: 465454
2023-05-22 19:24:45,992 [trainer.py] => Trainable params: 465454
2023-05-22 19:24:45,999 [icarl.py] => Learning on 20-30
2023-05-22 20:31:37,699 [icarl.py] => Task 2, Epoch 170/170 => Loss 1.192, Train_accy 99.81
2023-05-22 20:31:37,700 [base.py] => Reducing exemplars...(66 per classes)
2023-05-22 20:31:47,398 [base.py] => Constructing exemplars...(66 per classes)
2023-05-22 20:31:59,640 [icarl.py] => Exemplar size: 1980
2023-05-22 20:31:59,640 [trainer.py] => CNN: {'total': 73.17, '00-09': 71.6, '10-19': 58.9, '20-29': 89.0, 'old': 65.25, 'new': 89.0}
2023-05-22 20:31:59,640 [trainer.py] => NME: {'total': 75.4, '00-09': 76.7, '10-19': 65.1, '20-29': 84.4, 'old': 70.9, 'new': 84.4}
2023-05-22 20:31:59,640 [trainer.py] => CNN top1 curve: [91.2, 79.6, 73.17]
2023-05-22 20:31:59,641 [trainer.py] => CNN top5 curve: [99.3, 96.5, 94.27]
2023-05-22 20:31:59,641 [trainer.py] => NME top1 curve: [91.6, 79.1, 75.4]
2023-05-22 20:31:59,641 [trainer.py] => NME top5 curve: [99.8, 96.3, 94.27]
2023-05-22 20:31:59,641 [trainer.py] => All params: 466104
2023-05-22 20:31:59,641 [trainer.py] => Trainable params: 466104
2023-05-22 20:31:59,648 [icarl.py] => Learning on 30-40
2023-05-22 21:35:22,460 [icarl.py] => Task 3, Epoch 170/170 => Loss 1.494, Train_accy 99.89
2023-05-22 21:35:22,461 [base.py] => Reducing exemplars...(50 per classes)
2023-05-22 21:35:36,634 [base.py] => Constructing exemplars...(50 per classes)
2023-05-22 21:35:48,986 [icarl.py] => Exemplar size: 2000
2023-05-22 21:35:48,987 [trainer.py] => CNN: {'total': 64.18, '00-09': 62.3, '10-19': 46.0, '20-29': 65.7, '30-39': 82.7, 'old': 58.0, 'new': 82.7}
2023-05-22 21:35:48,987 [trainer.py] => NME: {'total': 69.0, '00-09': 71.2, '10-19': 55.4, '20-29': 71.8, '30-39': 77.6, 'old': 66.13, 'new': 77.6}
2023-05-22 21:35:48,987 [trainer.py] => CNN top1 curve: [91.2, 79.6, 73.17, 64.18]
2023-05-22 21:35:48,987 [trainer.py] => CNN top5 curve: [99.3, 96.5, 94.27, 91.05]
2023-05-22 21:35:48,987 [trainer.py] => NME top1 curve: [91.6, 79.1, 75.4, 69.0]
2023-05-22 21:35:48,987 [trainer.py] => NME top5 curve: [99.8, 96.3, 94.27, 91.32]
2023-05-22 21:35:48,987 [trainer.py] => All params: 466754
2023-05-22 21:35:48,987 [trainer.py] => Trainable params: 466754
2023-05-22 21:35:48,991 [icarl.py] => Learning on 40-50
2023-05-22 23:03:43,196 [trainer.py] => config: ./exps/icarl.json
2023-05-22 23:03:43,196 [trainer.py] => prefix: reproduce
2023-05-22 23:03:43,196 [trainer.py] => dataset: cifar100
2023-05-22 23:03:43,196 [trainer.py] => memory_size: 2000
2023-05-22 23:03:43,196 [trainer.py] => memory_per_class: 20
2023-05-22 23:03:43,196 [trainer.py] => fixed_memory: False
2023-05-22 23:03:43,196 [trainer.py] => shuffle: True
2023-05-22 23:03:43,196 [trainer.py] => init_cls: 10
2023-05-22 23:03:43,196 [trainer.py] => increment: 10
2023-05-22 23:03:43,196 [trainer.py] => model_name: icarl
2023-05-22 23:03:43,196 [trainer.py] => convnet_type: resnet32
2023-05-22 23:03:43,197 [trainer.py] => device: [device(type='cuda', index=0), device(type='cuda', index=1), device(type='cuda', index=2), device(type='cuda', index=3)]
2023-05-22 23:03:43,197 [trainer.py] => seed: 1993
2023-05-22 23:03:44,415 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-05-22 23:03:44,523 [trainer.py] => All params: 464154
2023-05-22 23:03:44,524 [trainer.py] => Trainable params: 464154
2023-05-22 23:03:44,524 [icarl.py] => Learning on 0-10
2023-05-22 23:05:02,666 [trainer.py] => config: ./exps/icarl.json
2023-05-22 23:05:02,666 [trainer.py] => prefix: reproduce
2023-05-22 23:05:02,666 [trainer.py] => dataset: cifar100
2023-05-22 23:05:02,666 [trainer.py] => memory_size: 2000
2023-05-22 23:05:02,666 [trainer.py] => memory_per_class: 20
2023-05-22 23:05:02,666 [trainer.py] => fixed_memory: False
2023-05-22 23:05:02,666 [trainer.py] => shuffle: True
2023-05-22 23:05:02,666 [trainer.py] => init_cls: 10
2023-05-22 23:05:02,666 [trainer.py] => increment: 10
2023-05-22 23:05:02,666 [trainer.py] => model_name: icarl
2023-05-22 23:05:02,666 [trainer.py] => convnet_type: resnet32
2023-05-22 23:05:02,666 [trainer.py] => device: [device(type='cuda', index=0), device(type='cuda', index=1), device(type='cuda', index=2), device(type='cuda', index=3)]
2023-05-22 23:05:02,667 [trainer.py] => seed: 1993
2023-05-22 23:05:03,899 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-05-22 23:05:04,014 [trainer.py] => All params: 464154
2023-05-22 23:05:04,014 [trainer.py] => Trainable params: 464154
2023-05-22 23:05:04,014 [icarl.py] => Learning on 0-10
2023-05-22 23:10:46,408 [trainer.py] => config: ./exps/icarl.json
2023-05-22 23:10:46,408 [trainer.py] => prefix: reproduce
2023-05-22 23:10:46,408 [trainer.py] => dataset: cifar100
2023-05-22 23:10:46,408 [trainer.py] => memory_size: 2000
2023-05-22 23:10:46,408 [trainer.py] => memory_per_class: 20
2023-05-22 23:10:46,408 [trainer.py] => fixed_memory: False
2023-05-22 23:10:46,408 [trainer.py] => shuffle: True
2023-05-22 23:10:46,408 [trainer.py] => init_cls: 10
2023-05-22 23:10:46,408 [trainer.py] => increment: 10
2023-05-22 23:10:46,408 [trainer.py] => model_name: icarl
2023-05-22 23:10:46,409 [trainer.py] => convnet_type: resnet32
2023-05-22 23:10:46,409 [trainer.py] => device: [device(type='cuda', index=0), device(type='cuda', index=1), device(type='cuda', index=2), device(type='cuda', index=3)]
2023-05-22 23:10:46,409 [trainer.py] => seed: 1993
2023-05-22 23:10:47,635 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-05-22 23:10:47,750 [trainer.py] => All params: 464154
2023-05-22 23:10:47,751 [trainer.py] => Trainable params: 464154
2023-05-22 23:10:47,751 [icarl.py] => Learning on 0-10
2023-05-22 23:12:14,130 [trainer.py] => config: ./exps/icarl.json
2023-05-22 23:12:14,130 [trainer.py] => prefix: reproduce
2023-05-22 23:12:14,130 [trainer.py] => dataset: cifar100
2023-05-22 23:12:14,130 [trainer.py] => memory_size: 2000
2023-05-22 23:12:14,130 [trainer.py] => memory_per_class: 20
2023-05-22 23:12:14,130 [trainer.py] => fixed_memory: False
2023-05-22 23:12:14,130 [trainer.py] => shuffle: True
2023-05-22 23:12:14,130 [trainer.py] => init_cls: 10
2023-05-22 23:12:14,130 [trainer.py] => increment: 10
2023-05-22 23:12:14,130 [trainer.py] => model_name: icarl
2023-05-22 23:12:14,131 [trainer.py] => convnet_type: resnet32
2023-05-22 23:12:14,131 [trainer.py] => device: [device(type='cuda', index=0), device(type='cuda', index=1), device(type='cuda', index=2), device(type='cuda', index=3)]
2023-05-22 23:12:14,131 [trainer.py] => seed: 1993
2023-05-22 23:12:15,363 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-05-22 23:12:15,473 [trainer.py] => All params: 464154
2023-05-22 23:12:15,474 [trainer.py] => Trainable params: 464154
2023-05-22 23:12:15,474 [icarl.py] => Learning on 0-10
2023-05-22 23:13:36,958 [trainer.py] => config: ./exps/icarl.json
2023-05-22 23:13:36,958 [trainer.py] => prefix: reproduce
2023-05-22 23:13:36,958 [trainer.py] => dataset: cifar100
2023-05-22 23:13:36,958 [trainer.py] => memory_size: 2000
2023-05-22 23:13:36,958 [trainer.py] => memory_per_class: 20
2023-05-22 23:13:36,958 [trainer.py] => fixed_memory: False
2023-05-22 23:13:36,958 [trainer.py] => shuffle: True
2023-05-22 23:13:36,958 [trainer.py] => init_cls: 10
2023-05-22 23:13:36,958 [trainer.py] => increment: 10
2023-05-22 23:13:36,958 [trainer.py] => model_name: icarl
2023-05-22 23:13:36,958 [trainer.py] => convnet_type: resnet32
2023-05-22 23:13:36,958 [trainer.py] => device: [device(type='cuda', index=0), device(type='cuda', index=1), device(type='cuda', index=2), device(type='cuda', index=3)]
2023-05-22 23:13:36,958 [trainer.py] => seed: 1993
2023-05-22 23:13:38,185 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-05-22 23:13:38,297 [trainer.py] => All params: 464154
2023-05-22 23:13:38,298 [trainer.py] => Trainable params: 464154
2023-05-22 23:13:38,298 [icarl.py] => Learning on 0-10
2023-05-22 23:13:51,633 [trainer.py] => config: ./exps/icarl.json
2023-05-22 23:13:51,633 [trainer.py] => prefix: reproduce
2023-05-22 23:13:51,633 [trainer.py] => dataset: cifar100
2023-05-22 23:13:51,633 [trainer.py] => memory_size: 2000
2023-05-22 23:13:51,633 [trainer.py] => memory_per_class: 20
2023-05-22 23:13:51,633 [trainer.py] => fixed_memory: False
2023-05-22 23:13:51,633 [trainer.py] => shuffle: True
2023-05-22 23:13:51,633 [trainer.py] => init_cls: 10
2023-05-22 23:13:51,633 [trainer.py] => increment: 10
2023-05-22 23:13:51,633 [trainer.py] => model_name: icarl
2023-05-22 23:13:51,633 [trainer.py] => convnet_type: resnet32
2023-05-22 23:13:51,633 [trainer.py] => device: [device(type='cuda', index=0), device(type='cuda', index=1), device(type='cuda', index=2), device(type='cuda', index=3)]
2023-05-22 23:13:51,633 [trainer.py] => seed: 1993
2023-05-22 23:13:52,851 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-05-22 23:13:52,960 [trainer.py] => All params: 464154
2023-05-22 23:13:52,961 [trainer.py] => Trainable params: 464154
2023-05-22 23:13:52,961 [icarl.py] => Learning on 0-10
2023-05-22 23:17:00,922 [trainer.py] => config: ./exps/icarl.json
2023-05-22 23:17:00,922 [trainer.py] => prefix: reproduce
2023-05-22 23:17:00,922 [trainer.py] => dataset: cifar100
2023-05-22 23:17:00,922 [trainer.py] => memory_size: 2000
2023-05-22 23:17:00,922 [trainer.py] => memory_per_class: 20
2023-05-22 23:17:00,922 [trainer.py] => fixed_memory: False
2023-05-22 23:17:00,922 [trainer.py] => shuffle: True
2023-05-22 23:17:00,922 [trainer.py] => init_cls: 10
2023-05-22 23:17:00,922 [trainer.py] => increment: 10
2023-05-22 23:17:00,922 [trainer.py] => model_name: icarl
2023-05-22 23:17:00,922 [trainer.py] => convnet_type: resnet32
2023-05-22 23:17:00,922 [trainer.py] => device: [device(type='cuda', index=1)]
2023-05-22 23:17:00,922 [trainer.py] => seed: 1993
2023-05-22 23:17:02,143 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-05-22 23:17:02,258 [trainer.py] => All params: 464154
2023-05-22 23:17:02,259 [trainer.py] => Trainable params: 464154
2023-05-22 23:17:02,259 [icarl.py] => Learning on 0-10
2023-05-22 23:17:17,629 [trainer.py] => config: ./exps/icarl.json
2023-05-22 23:17:17,629 [trainer.py] => prefix: reproduce
2023-05-22 23:17:17,629 [trainer.py] => dataset: cifar100
2023-05-22 23:17:17,629 [trainer.py] => memory_size: 2000
2023-05-22 23:17:17,629 [trainer.py] => memory_per_class: 20
2023-05-22 23:17:17,629 [trainer.py] => fixed_memory: False
2023-05-22 23:17:17,629 [trainer.py] => shuffle: True
2023-05-22 23:17:17,629 [trainer.py] => init_cls: 10
2023-05-22 23:17:17,630 [trainer.py] => increment: 10
2023-05-22 23:17:17,630 [trainer.py] => model_name: icarl
2023-05-22 23:17:17,630 [trainer.py] => convnet_type: resnet32
2023-05-22 23:17:17,630 [trainer.py] => device: [device(type='cuda', index=1)]
2023-05-22 23:17:17,630 [trainer.py] => seed: 1993
2023-05-22 23:17:18,849 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-05-22 23:17:18,961 [trainer.py] => All params: 464154
2023-05-22 23:17:18,962 [trainer.py] => Trainable params: 464154
2023-05-22 23:17:18,962 [icarl.py] => Learning on 0-10
2023-05-22 23:22:28,264 [icarl.py] => Task 0, Epoch 200/200 => Loss 0.031, Train_accy 99.38
2023-05-22 23:22:28,264 [base.py] => Reducing exemplars...(200 per classes)
2023-05-22 23:22:28,264 [base.py] => Constructing exemplars...(200 per classes)
2023-05-22 23:22:36,462 [icarl.py] => Exemplar size: 2000
2023-05-22 23:22:36,463 [trainer.py] => CNN: {'total': 89.8, '00-09': 89.8, 'old': 0, 'new': 89.8}
2023-05-22 23:22:36,463 [trainer.py] => NME: {'total': 89.8, '00-09': 89.8, 'old': 0, 'new': 89.8}
2023-05-22 23:22:36,463 [trainer.py] => CNN top1 curve: [89.8]
2023-05-22 23:22:36,463 [trainer.py] => CNN top5 curve: [99.5]
2023-05-22 23:22:36,463 [trainer.py] => NME top1 curve: [89.8]
2023-05-22 23:22:36,463 [trainer.py] => NME top5 curve: [99.4]
2023-05-22 23:22:36,463 [trainer.py] => All params: 464804
2023-05-22 23:22:36,464 [trainer.py] => Trainable params: 464804
2023-05-22 23:22:36,464 [icarl.py] => Learning on 10-20
2023-05-22 23:29:08,574 [icarl.py] => Task 1, Epoch 170/170 => Loss 1.007, Train_accy 99.89
2023-05-22 23:29:08,574 [base.py] => Reducing exemplars...(100 per classes)
2023-05-22 23:29:11,659 [base.py] => Constructing exemplars...(100 per classes)
2023-05-22 23:29:19,328 [icarl.py] => Exemplar size: 2000
2023-05-22 23:29:19,328 [trainer.py] => CNN: {'total': 77.95, '00-09': 78.3, '10-19': 77.6, 'old': 78.3, 'new': 77.6}
2023-05-22 23:29:19,328 [trainer.py] => NME: {'total': 79.1, '00-09': 81.8, '10-19': 76.4, 'old': 81.8, 'new': 76.4}
2023-05-22 23:29:19,328 [trainer.py] => CNN top1 curve: [89.8, 77.95]
2023-05-22 23:29:19,328 [trainer.py] => CNN top5 curve: [99.5, 96.25]
2023-05-22 23:29:19,328 [trainer.py] => NME top1 curve: [89.8, 79.1]
2023-05-22 23:29:19,328 [trainer.py] => NME top5 curve: [99.4, 96.0]
2023-05-22 23:29:19,328 [trainer.py] => All params: 465454
2023-05-22 23:29:19,329 [trainer.py] => Trainable params: 465454
2023-05-22 23:29:19,329 [icarl.py] => Learning on 20-30
2023-05-22 23:35:51,984 [icarl.py] => Task 2, Epoch 170/170 => Loss 1.163, Train_accy 99.91
2023-05-22 23:35:51,984 [base.py] => Reducing exemplars...(66 per classes)
2023-05-22 23:35:58,094 [base.py] => Constructing exemplars...(66 per classes)
2023-05-22 23:36:06,453 [icarl.py] => Exemplar size: 1980
2023-05-22 23:36:06,453 [trainer.py] => CNN: {'total': 72.6, '00-09': 69.9, '10-19': 60.4, '20-29': 87.5, 'old': 65.15, 'new': 87.5}
2023-05-22 23:36:06,453 [trainer.py] => NME: {'total': 74.27, '00-09': 76.2, '10-19': 65.1, '20-29': 81.5, 'old': 70.65, 'new': 81.5}
2023-05-22 23:36:06,453 [trainer.py] => CNN top1 curve: [89.8, 77.95, 72.6]
2023-05-22 23:36:06,454 [trainer.py] => CNN top5 curve: [99.5, 96.25, 93.77]
2023-05-22 23:36:06,454 [trainer.py] => NME top1 curve: [89.8, 79.1, 74.27]
2023-05-22 23:36:06,454 [trainer.py] => NME top5 curve: [99.4, 96.0, 94.37]
2023-05-22 23:36:06,454 [trainer.py] => All params: 466104
2023-05-22 23:36:06,454 [trainer.py] => Trainable params: 466104
2023-05-22 23:36:06,455 [icarl.py] => Learning on 30-40
2023-05-22 23:42:45,877 [icarl.py] => Task 3, Epoch 170/170 => Loss 1.461, Train_accy 99.90
2023-05-22 23:42:45,877 [base.py] => Reducing exemplars...(50 per classes)
2023-05-22 23:42:53,898 [base.py] => Constructing exemplars...(50 per classes)
2023-05-22 23:43:01,767 [icarl.py] => Exemplar size: 2000
2023-05-22 23:43:01,767 [trainer.py] => CNN: {'total': 63.72, '00-09': 59.4, '10-19': 46.3, '20-29': 67.2, '30-39': 82.0, 'old': 57.63, 'new': 82.0}
2023-05-22 23:43:01,767 [trainer.py] => NME: {'total': 68.75, '00-09': 70.2, '10-19': 58.3, '20-29': 72.9, '30-39': 73.6, 'old': 67.13, 'new': 73.6}
2023-05-22 23:43:01,767 [trainer.py] => CNN top1 curve: [89.8, 77.95, 72.6, 63.72]
2023-05-22 23:43:01,767 [trainer.py] => CNN top5 curve: [99.5, 96.25, 93.77, 90.4]
2023-05-22 23:43:01,767 [trainer.py] => NME top1 curve: [89.8, 79.1, 74.27, 68.75]
2023-05-22 23:43:01,767 [trainer.py] => NME top5 curve: [99.4, 96.0, 94.37, 91.35]
2023-05-22 23:43:01,767 [trainer.py] => All params: 466754
2023-05-22 23:43:01,767 [trainer.py] => Trainable params: 466754
2023-05-22 23:43:01,768 [icarl.py] => Learning on 40-50
2023-05-22 23:49:38,147 [icarl.py] => Task 4, Epoch 170/170 => Loss 1.550, Train_accy 99.97
2023-05-22 23:49:38,148 [base.py] => Reducing exemplars...(40 per classes)
2023-05-22 23:49:48,599 [base.py] => Constructing exemplars...(40 per classes)
2023-05-22 23:49:56,533 [icarl.py] => Exemplar size: 2000
2023-05-22 23:49:56,534 [trainer.py] => CNN: {'total': 58.54, '00-09': 51.8, '10-19': 38.1, '20-29': 54.8, '30-39': 60.0, '40-49': 88.0, 'old': 51.18, 'new': 88.0}
2023-05-22 23:49:56,534 [trainer.py] => NME: {'total': 64.44, '00-09': 66.1, '10-19': 48.4, '20-29': 66.4, '30-39': 63.6, '40-49': 77.7, 'old': 61.12, 'new': 77.7}
2023-05-22 23:49:56,534 [trainer.py] => CNN top1 curve: [89.8, 77.95, 72.6, 63.72, 58.54]
2023-05-22 23:49:56,535 [trainer.py] => CNN top5 curve: [99.5, 96.25, 93.77, 90.4, 87.38]
2023-05-22 23:49:56,535 [trainer.py] => NME top1 curve: [89.8, 79.1, 74.27, 68.75, 64.44]
2023-05-22 23:49:56,535 [trainer.py] => NME top5 curve: [99.4, 96.0, 94.37, 91.35, 89.32]
2023-05-22 23:49:56,536 [trainer.py] => All params: 467404
2023-05-22 23:49:56,536 [trainer.py] => Trainable params: 467404
2023-05-22 23:49:56,537 [icarl.py] => Learning on 50-60
2023-05-22 23:56:42,554 [icarl.py] => Task 5, Epoch 170/170 => Loss 1.676, Train_accy 99.96
2023-05-22 23:56:42,555 [base.py] => Reducing exemplars...(33 per classes)
2023-05-22 23:56:55,700 [base.py] => Constructing exemplars...(33 per classes)
2023-05-22 23:57:03,836 [icarl.py] => Exemplar size: 1980
2023-05-22 23:57:03,837 [trainer.py] => CNN: {'total': 54.17, '00-09': 46.4, '10-19': 34.5, '20-29': 50.0, '30-39': 47.5, '40-49': 64.3, '50-59': 82.3, 'old': 48.54, 'new': 82.3}
2023-05-22 23:57:03,837 [trainer.py] => NME: {'total': 60.77, '00-09': 59.6, '10-19': 44.4, '20-29': 61.7, '30-39': 56.3, '40-49': 70.0, '50-59': 72.6, 'old': 58.4, 'new': 72.6}
2023-05-22 23:57:03,837 [trainer.py] => CNN top1 curve: [89.8, 77.95, 72.6, 63.72, 58.54, 54.17]
2023-05-22 23:57:03,837 [trainer.py] => CNN top5 curve: [99.5, 96.25, 93.77, 90.4, 87.38, 83.02]
2023-05-22 23:57:03,837 [trainer.py] => NME top1 curve: [89.8, 79.1, 74.27, 68.75, 64.44, 60.77]
2023-05-22 23:57:03,837 [trainer.py] => NME top5 curve: [99.4, 96.0, 94.37, 91.35, 89.32, 85.45]
2023-05-22 23:57:03,837 [trainer.py] => All params: 468054
2023-05-22 23:57:03,837 [trainer.py] => Trainable params: 468054
2023-05-22 23:57:03,838 [icarl.py] => Learning on 60-70
2023-05-23 00:03:44,924 [icarl.py] => Task 6, Epoch 170/170 => Loss 1.724, Train_accy 99.87
2023-05-23 00:03:44,924 [base.py] => Reducing exemplars...(28 per classes)
2023-05-23 00:04:00,661 [base.py] => Constructing exemplars...(28 per classes)
2023-05-23 00:04:08,875 [icarl.py] => Exemplar size: 1960
2023-05-23 00:04:08,875 [trainer.py] => CNN: {'total': 52.29, '00-09': 45.2, '10-19': 31.0, '20-29': 48.7, '30-39': 42.1, '40-49': 54.7, '50-59': 55.3, '60-69': 89.0, 'old': 46.17, 'new': 89.0}
2023-05-23 00:04:08,875 [trainer.py] => NME: {'total': 58.09, '00-09': 56.7, '10-19': 42.5, '20-29': 59.5, '30-39': 49.7, '40-49': 64.5, '50-59': 58.6, '60-69': 75.1, 'old': 55.25, 'new': 75.1}
2023-05-23 00:04:08,875 [trainer.py] => CNN top1 curve: [89.8, 77.95, 72.6, 63.72, 58.54, 54.17, 52.29]
2023-05-23 00:04:08,876 [trainer.py] => CNN top5 curve: [99.5, 96.25, 93.77, 90.4, 87.38, 83.02, 80.8]
2023-05-23 00:04:08,876 [trainer.py] => NME top1 curve: [89.8, 79.1, 74.27, 68.75, 64.44, 60.77, 58.09]
2023-05-23 00:04:08,876 [trainer.py] => NME top5 curve: [99.4, 96.0, 94.37, 91.35, 89.32, 85.45, 83.87]
2023-05-23 00:04:08,876 [trainer.py] => All params: 468704
2023-05-23 00:04:08,876 [trainer.py] => Trainable params: 468704
2023-05-23 00:04:08,877 [icarl.py] => Learning on 70-80
2023-05-23 00:10:58,431 [icarl.py] => Task 7, Epoch 170/170 => Loss 1.712, Train_accy 99.93
2023-05-23 00:10:58,432 [base.py] => Reducing exemplars...(25 per classes)
2023-05-23 00:11:16,920 [base.py] => Constructing exemplars...(25 per classes)
2023-05-23 00:11:25,218 [icarl.py] => Exemplar size: 2000
2023-05-23 00:11:25,218 [trainer.py] => CNN: {'total': 47.32, '00-09': 40.2, '10-19': 30.1, '20-29': 48.8, '30-39': 36.7, '40-49': 43.6, '50-59': 34.8, '60-69': 60.2, '70-79': 84.2, 'old': 42.06, 'new': 84.2}
2023-05-23 00:11:25,219 [trainer.py] => NME: {'total': 54.55, '00-09': 53.9, '10-19': 40.6, '20-29': 57.7, '30-39': 45.8, '40-49': 57.1, '50-59': 45.1, '60-69': 63.8, '70-79': 72.4, 'old': 52.0, 'new': 72.4}
2023-05-23 00:11:25,219 [trainer.py] => CNN top1 curve: [89.8, 77.95, 72.6, 63.72, 58.54, 54.17, 52.29, 47.32]
2023-05-23 00:11:25,219 [trainer.py] => CNN top5 curve: [99.5, 96.25, 93.77, 90.4, 87.38, 83.02, 80.8, 77.85]
2023-05-23 00:11:25,219 [trainer.py] => NME top1 curve: [89.8, 79.1, 74.27, 68.75, 64.44, 60.77, 58.09, 54.55]
2023-05-23 00:11:25,219 [trainer.py] => NME top5 curve: [99.4, 96.0, 94.37, 91.35, 89.32, 85.45, 83.87, 82.19]
2023-05-23 00:11:25,219 [trainer.py] => All params: 469354
2023-05-23 00:11:25,219 [trainer.py] => Trainable params: 469354
2023-05-23 00:11:25,219 [icarl.py] => Learning on 80-90
2023-05-23 00:18:11,022 [icarl.py] => Task 8, Epoch 170/170 => Loss 1.937, Train_accy 99.87
2023-05-23 00:18:11,023 [base.py] => Reducing exemplars...(22 per classes)
2023-05-23 00:18:32,082 [base.py] => Constructing exemplars...(22 per classes)
2023-05-23 00:18:40,388 [icarl.py] => Exemplar size: 1980
2023-05-23 00:18:40,388 [trainer.py] => CNN: {'total': 44.53, '00-09': 39.7, '10-19': 26.6, '20-29': 40.9, '30-39': 33.7, '40-49': 40.1, '50-59': 33.4, '60-69': 44.7, '70-79': 56.2, '80-89': 85.5, 'old': 39.41, 'new': 85.5}
2023-05-23 00:18:40,388 [trainer.py] => NME: {'total': 51.34, '00-09': 51.1, '10-19': 33.7, '20-29': 53.4, '30-39': 43.4, '40-49': 53.7, '50-59': 43.7, '60-69': 56.1, '70-79': 60.8, '80-89': 66.2, 'old': 49.49, 'new': 66.2}
2023-05-23 00:18:40,388 [trainer.py] => CNN top1 curve: [89.8, 77.95, 72.6, 63.72, 58.54, 54.17, 52.29, 47.32, 44.53]
2023-05-23 00:18:40,389 [trainer.py] => CNN top5 curve: [99.5, 96.25, 93.77, 90.4, 87.38, 83.02, 80.8, 77.85, 75.43]
2023-05-23 00:18:40,389 [trainer.py] => NME top1 curve: [89.8, 79.1, 74.27, 68.75, 64.44, 60.77, 58.09, 54.55, 51.34]
2023-05-23 00:18:40,389 [trainer.py] => NME top5 curve: [99.4, 96.0, 94.37, 91.35, 89.32, 85.45, 83.87, 82.19, 79.02]
2023-05-23 00:18:40,389 [trainer.py] => All params: 470004
2023-05-23 00:18:40,389 [trainer.py] => Trainable params: 470004
2023-05-23 00:18:40,390 [icarl.py] => Learning on 90-100
2023-05-23 00:25:32,625 [icarl.py] => Task 9, Epoch 170/170 => Loss 2.071, Train_accy 99.90
2023-05-23 00:25:32,625 [base.py] => Reducing exemplars...(20 per classes)
2023-05-23 00:25:57,312 [base.py] => Constructing exemplars...(20 per classes)
2023-05-23 00:26:06,251 [icarl.py] => Exemplar size: 2000
2023-05-23 00:26:06,251 [trainer.py] => CNN: {'total': 42.06, '00-09': 35.4, '10-19': 23.3, '20-29': 34.9, '30-39': 29.6, '40-49': 38.7, '50-59': 29.2, '60-69': 43.1, '70-79': 44.5, '80-89': 60.9, '90-99': 81.0, 'old': 37.73, 'new': 81.0}
2023-05-23 00:26:06,251 [trainer.py] => NME: {'total': 49.4, '00-09': 47.4, '10-19': 32.0, '20-29': 48.5, '30-39': 40.2, '40-49': 51.3, '50-59': 42.1, '60-69': 55.2, '70-79': 53.2, '80-89': 59.3, '90-99': 64.8, 'old': 47.69, 'new': 64.8}
2023-05-23 00:26:06,251 [trainer.py] => CNN top1 curve: [89.8, 77.95, 72.6, 63.72, 58.54, 54.17, 52.29, 47.32, 44.53, 42.06]
2023-05-23 00:26:06,251 [trainer.py] => CNN top5 curve: [99.5, 96.25, 93.77, 90.4, 87.38, 83.02, 80.8, 77.85, 75.43, 72.11]
2023-05-23 00:26:06,251 [trainer.py] => NME top1 curve: [89.8, 79.1, 74.27, 68.75, 64.44, 60.77, 58.09, 54.55, 51.34, 49.4]
2023-05-23 00:26:06,251 [trainer.py] => NME top5 curve: [99.4, 96.0, 94.37, 91.35, 89.32, 85.45, 83.87, 82.19, 79.02, 77.53]
2023-07-04 17:19:05,818 [trainer.py] => config: ./exps/icarl.json
2023-07-04 17:19:05,818 [trainer.py] => prefix: reproduce
2023-07-04 17:19:05,818 [trainer.py] => dataset: cifar100
2023-07-04 17:19:05,818 [trainer.py] => memory_size: 2000
2023-07-04 17:19:05,818 [trainer.py] => memory_per_class: 20
2023-07-04 17:19:05,818 [trainer.py] => fixed_memory: False
2023-07-04 17:19:05,818 [trainer.py] => shuffle: True
2023-07-04 17:19:05,818 [trainer.py] => init_cls: 10
2023-07-04 17:19:05,818 [trainer.py] => increment: 10
2023-07-04 17:19:05,818 [trainer.py] => model_name: icarl
2023-07-04 17:19:05,818 [trainer.py] => convnet_type: resnet32
2023-07-04 17:19:05,818 [trainer.py] => device: [device(type='cuda', index=1)]
2023-07-04 17:19:05,819 [trainer.py] => seed: 1993
2023-07-04 17:19:07,166 [data_manager.py] => [68, 56, 78, 8, 23, 84, 90, 65, 74, 76, 40, 89, 3, 92, 55, 9, 26, 80, 43, 38, 58, 70, 77, 1, 85, 19, 17, 50, 28, 53, 13, 81, 45, 82, 6, 59, 83, 16, 15, 44, 91, 41, 72, 60, 79, 52, 20, 10, 31, 54, 37, 95, 14, 71, 96, 98, 97, 2, 64, 66, 42, 22, 35, 86, 24, 34, 87, 21, 99, 0, 88, 27, 18, 94, 11, 12, 47, 25, 30, 46, 62, 69, 36, 61, 7, 63, 75, 5, 32, 4, 51, 48, 73, 93, 39, 67, 29, 49, 57, 33]
2023-07-04 17:19:07,273 [trainer.py] => All params: 464154
2023-07-04 17:19:07,273 [trainer.py] => Trainable params: 464154
2023-07-04 17:19:07,274 [icarl.py] => Learning on 0-10```
Not an Issue
Hi,
The reason I open this issue is not that it is an issue it is more of a request. PYCIL is a good resource to use. I am actually using it, especially for regularization-based approaches. I was wondering if you have plans to add more reg-based methods like Synaptic Intelligence: https://arxiv.org/pdf/1703.04200.pdf That would be awesome!
Weird Training Result with train_acc=0 & loss=0
Thanks for your excellent work!
But when I tried my own dataset on it, I found that the training result was really weired, just take EWC as an example:
================ EWC ================
Task 0, Epoch 200/200 => Loss 0.886, Train_accy 70.60, Test_accy 73.00
Task 1, Epoch 180/180 => Loss 0.002, Train_accy 0.00
CNN: {'total': 7.36, '00-09': 7.8, 'old': 7.8, 'new': 0.0}
Task 2, Epoch 180/180 => Loss 0.002, Train_accy 0.00
CNN: {'total': 38.67, '00-09': 46.4, '10-19': 0.0, 'old': 43.77, 'new': 0.0}
Task 3, Epoch 180/180 => Loss 0.003, Train_accy 0.00
CNN: {'total': 12.99, '00-09': 17.4, '10-19': 0.0, 'old': 14.5, 'new': 0.0}
...
Task 9, Epoch 180/180 => Loss 0.004, Train_accy 0.00
CNN: {'total': 28.66, '00-09': 55.6, '10-19': 0.0, 'old': 30.89, 'new': 0.0}
CNN top1 curve: [74.0, 7.36, 38.67, 12.99, 1.22, 23.83, 34.52, 16.55, 8.56, 28.66]
CNN top5 curve: [98.6, 52.26, 78.0, 45.82, 29.05, 53.58, 57.26, 48.85, 31.78, 48.66]
I also tried it on Lwf, the problem remains.
Questions About Loss Function
I am wondering whether issues exist in the loss function. And I found that the loss function is defined as: loss_clf = F.cross_entropy(logits[:,self._known_classes:],targets-self._known_classes), along with loss_ewc.
In my experiments, I set base_session=10, increment=1. So for task1, when calculating the loss, targets=[10, 10, ..., 10] and self._known_classes=10. Therefore, the target(#2para) of the cross_entropy was [0, 0, ..., 0]. Is that mean the loss function pushes the model to reject the new task, making logits of the new task approach 0?
Tuning Loss Function
I tried to change the #2para to [1, 1, ..., 1] by setting targets-self._known_classes+1, but error arises:
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [0,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [1,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [2,0,0] Assertion `t >= 0 && t < n_classes` failed.
...
Traceback (most recent call last):
File "/tmp/pycharm_project_241/main.py", line 33, in <module>
main()
File "/tmp/pycharm_project_241/main.py", line 13, in main
train(args)
File "/tmp/pycharm_project_241/trainer.py", line 18, in train
_train(args)
File "/tmp/pycharm_project_241/trainer.py", line 48, in _train
model.incremental_train(data_manager)
File "/tmp/pycharm_project_241/models/ewc.py", line 62, in incremental_train
self._train(self.train_loader, self.test_loader)
File "/tmp/pycharm_project_241/models/ewc.py", line 84, in _train
self._update_representation(train_loader, test_loader, optimizer, scheduler)
File "/tmp/pycharm_project_241/models/ewc.py", line 138, in _update_representation
loss.backward()
File "/home/linkdata/data/yaoxinjie/anaconda3/envs/FACT18/lib/python3.7/site-packages/torch/tensor.py", line 245, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/linkdata/data/yaoxinjie/anaconda3/envs/FACT18/lib/python3.7/site-packages/torch/autograd/__init__.py", line 147, in backward
allow_unreachable=True, accumulate_grad=True) # allow_unreachable flag
RuntimeError: Unable to find a valid cuDNN algorithm to run convolution
And I don't why it happens and how to handle this. Could you please help me to clarify it? Thanks a lot!
slow IO speed when using ImageFolder on ImageNet-100
Thanks for your excellent work!
I'm doing experiments on ImageNet-100, but I find that the IO speed is very slow when using ImageFolder on ImageNet-100. GPU utilization is very low and the experiments take a long time.
Do I need to read all the data into memory at the beginning? Could you please offer some instructions about this problem？
Whether the code can be applied to object detection such as yolo?
Hello, is ur code only applicable to classification networks? Can it be used in object detection networks? I want to implement EWC on YOLO, and I wonder if u have relevant suggestions. Thank u for ur work and acvice.
brightness参数设置问题
请问brightness在代码中为什么设置为63/255，使用其它数据集时如何修改
DataManager' object has no attribute 'get_total_classnum
Hello, you did an excellent job and thanks for your contribution.
But I meet AttributeError: 'DataManager' object has no attribute 'get_total_classnum' when run python main.py --config=./exps/coil.json. Here are the details:
Traceback (most recent call last):
File "main.py", line 31, in <module>
main()
File "main.py", line 12, in main
train(args)
File "/root/weiwei/PyCIL/trainer.py", line 18, in train
_train(args)
File "/root/weiwei/PyCIL/trainer.py", line 50, in _train
model.after_task()
File "/root/weiwei/PyCIL/models/coil.py", line 38, in after_task
self.nextperiod_initialization=self.solving_ot()
File "/root/weiwei/PyCIL/models/coil.py", line 45, in solving_ot
if self._total_classes==self.data_manager.get_total_classnum():
AttributeError: 'DataManager' object has no attribute 'get_total_classnum'
The following error occurs when training SSRE
Traceback (most recent call last):
File "main.py", line 31, in
main()
File "main.py", line 12, in main
train(args)
File "/home/isalab203/code/FH_IL/PyCIL-master/trainer.py", line 28, in train
_train(args)
File "/home/isalab203/code/FH_IL/PyCIL-master/trainer.py", line 75, in _train
model.incremental_train(data_manager)
File "/home/isalab203/code/FH_IL/PyCIL-master/models/ssre.py", line 62, in incremental_train
self._network_expansion()
File "/home/isalab203/code/FH_IL/PyCIL-master/models/ssre.py", line 238, in _network_expansion
self._network.convnet.switch("parallel_adapters")
File "/home/isalab203/.conda/envs/CIL/lib/python3.8/site-packages/torch/nn/modules/module.py", line 947, in getattr
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'CifarResNet' object has no attribute 'switch'
#####################################
It seems that inc_net.py lacks the switch method？
The implementation of DER
Hi,
I note the bias-correct method used in your implementation of DER is WA but not the balanced fine-tune, which is not identical with the original paper. I wonder why do you decide to change it to WA? :)
RMM pretraining
Hi,
Thank you for the excellent work.
I'm trying to pre-train the RMM agent and reproduce results on RMM+LUCIR. Can I know how long it took for the training?
Packages import problem
Hi~
I have a problem, about import ot package.
Could you help me slove it? or Did you mind tell me where can i download?
Thank you~
FileNotFoundError
when I run the experiment using :
python main.py --config=./exps/finetune.json
it occurs :
FileNotFoundError: [Errno 2] No such file or directory: '/data7/PyCIL-master/logs/reproduce_1993_finetune_resnet32_cifar100_10_10.log'
so I want to ask if the author forgets to upload the repository "/logs" or I need to train by myself.
how can we get the accuracy of the saved model
Thank you very much for providing the code. I want to know the approach of loading the saved model and get the accuracy of different valid data set.
Missing parameters leading to incorrect code behavior
Hi,
Thanks for your interesting work. I tried to run the cil benchmark on different setting (i.e. different number of classes being learnt for each task). In the evaluation code models/base.py, it seems like you forgot to pass the important parameter "increment" to be processed by the accuracy function inside utils/toolkit.py.
This leads to incorrect behavior of the codebase which always compute the evaluation on only one setting (i.e. 10 classes at a time). Could you please check it again? Thanks!
An error in \utils\autoaugment.py
In line 176 "posterize": np.round(np.linspace(8, 4, 10), 0).astype(np.int) " AttributeError raised for "module 'numpy' has no attribute 'int'", what should i do?
大佬您好，我在运行fetril的时候报错了，报错信息显示在\utils\autoaugment.py的176行中有np.round(np.linspace(8, 4, 10), 0).astype(np.int) ，而np没有int只有int/unint 8/16/32/64，请问应该是哪个呀？？呜呜😭😭😭
NME
Excuse me, I would like to ask the NME classifier on iCaRL, will it use the concept of knowledge distillation to calculate the loss? Or is it only available in the Softmax classifier (CNN)?
Grad-cam
您好，想請教一下在inc_net.py中的grad-cam該如何可視化?
Genral idea for the code framework
Hello, I have some questions for you.Is your code modified from the original DER? Could you give me a general idea of your thinking.Thank you very much. I look forward to your recovery.
DER。mask
Hello, I would like to ask where the mask is located in the DER code?
thanks
having error about code in icarl
Thank you for your work on class incremental code integration. It's wonderful. I found a problem in the icarl code.
Loss is composed of the classification of new classes and the distillation of knowledge of old classes. But your code wrongly changed the classification of new classes to the classification of all classes。
fetril cifar10 got error
config:
{
"prefix": "train",
"dataset": "cifar10",
"memory_size": 0,
"shuffle": true,
"init_cls": 2,
"increment": 2,
"model_name": "fetril",
"convnet_type": "resnet18",
"device": ["0"],
"seed": [1993],
"init_epochs": 200,
"init_lr" : 0.1,
"init_weight_decay" : 0,
"epochs" : 50,
"lr" : 0.1,
"batch_size" : 128,
"weight_decay" : 0,
"num_workers" : 8,
"T" : 2
}
error message:
Traceback (most recent call last):
File "fetril.py", line 31, in <module>
main()
File "fetril.py", line 12, in main
train(args)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/trainer.py", line 18, in train
_train(args)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/trainer.py", line 67, in _train
model.incremental_train(data_manager)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/models/fetril.py", line 88, in incremental_train
self._train(self.train_loader, self.test_loader)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/models/fetril.py", line 105, in _train
self._train_function(train_loader, test_loader, optimizer, scheduler)
File "/home/20031211496/Documents/PYCIL/PyCIL-master/models/fetril.py", line 178, in _train_function
for i, (_, inputs, targets) in enumerate(train_loader):
File "/home/20031211496/.conda/envs/drive/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
data = self._next_data()
File "/home/20031211496/.conda/envs/drive/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/home/20031211496/.conda/envs/drive/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/home/20031211496/.conda/envs/drive/lib/python3.8/site-packages/torch/_utils.py", line 434, in reraise
raise exception
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/20031211496/.conda/envs/drive/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/home/20031211496/.conda/envs/drive/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/20031211496/.conda/envs/drive/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/20031211496/Documents/PYCIL/PyCIL-master/utils/data_manager.py", line 207, in __getitem__
image = self.trsf(Image.fromarray(self.images[idx]))
File "/home/20031211496/.conda/envs/drive/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 61, in __call__
img = t(img)
File "/home/20031211496/.conda/envs/drive/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 98, in __call__
return F.to_tensor(pic)
File "/home/20031211496/.conda/envs/drive/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 114, in to_tensor
raise TypeError('pic should be PIL Image or ndarray. Got {}'.format(type(pic)))
TypeError: pic should be PIL Image or ndarray. Got <class 'torch.Tensor'>
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.