Comments (2)
Hi, @Moqixis
Sorry for the late reply. Since we got a very, very limited video dataset for this task before submission, we adopted two-stage training to achieve promising performance. But, in general, the two-stage models indeed have some tricky training treatments, incurring them hard to train. Thus, please fine-tune your model after ensuring the best pre-training performance you can ever get, and then try to fine-tune your model on video datasets by using a small learning rate, fixing the static branch, increasing the fine-tuning epochs, or something else. You can feel free to discuss me with your experience for more inspiration.
I also note that there has a similar issue #13. Today, we retrain our model on the pre-trained weight and get a similar performance reported in our conference paper. Next is the comparison; please check it out.
Original Performance:
- (Dataset:CVC-ClinicDB-612-Test) Sm:0.903;meanEm:0.903;MAE:0.038;maxEm:0.923;maxDice:0.860;maxIoU:0.795;maxSpe:0.992.
- (Dataset:CVC-ClinicDB-612-Valid) Sm:0.923;meanEm:0.944;MAE:0.012;maxEm:0.962;maxDice:0.873;maxIoU:0.800;maxSpe:0.991.
- (Dataset:CVC-ColonDB-300) Sm:0.909;meanEm:0.921;MAE:0.013;maxEm:0.942;maxDice:0.840;maxIoU:0.745;maxSpe:0.996.
New Performance (that we retrained again):
- (Dataset:CVC-ClinicDB-612-Test) Sm:0.891;meanEm:0.876;MAE:0.043;maxEm:0.921;maxDice:0.854;maxIoU:0.785;maxSpe:0.995.
- (Dataset:CVC-ClinicDB-612-Valid) Sm:0.917;meanEm:0.923;MAE:0.013;maxEm:0.952;maxDice:0.861;maxIoU:0.787;maxSpe:0.991.
- (Dataset:CVC-ColonDB-300) Sm:0.897;meanEm:0.892;MAE:0.012;maxEm:0.936;maxDice:0.833;maxIoU:0.734;maxSpe:0.997.
These numerical results show some performance fluctuations due to different random seed or PyTorch versions or something. But I think it is acceptable for us. We don't take further action to improve the performance as we just fine-tune our model with one epoch. Overall It does not show a 'huge' drop after the fine-tuning process.
Thank you all guys to pay attention to our work. Further problems and discussions are welcome here.
ONE MORE THING We also release a large-scale VPS dataset, named SUN-SEG to promote this field. By using this dataset, you can feel free to train your large model without worries about over-fitting problems. You can also discard the two-stage training strategy using our enhanced method, PNS+, which only needs training once. We strongly suggest you use them.
Best,
Ge-Peng.
from pns-net.
Hi, @Moqixis
Sorry for the late reply. Since we got a very, very limited video dataset for this task before submission, we adopted two-stage training to achieve promising performance. But, in general, the two-stage models indeed have some tricky training treatments, incurring them hard to train. Thus, please fine-tune your model after ensuring the best pre-training performance you can ever get, and then try to fine-tune your model on video datasets by using a small learning rate, fixing the static branch, increasing the fine-tuning epochs, or something else. You can feel free to discuss me with your experience for more inspiration.
I also note that there has a similar issue #13. Today, we retrain our model on the pre-trained weight and get a similar performance reported in our conference paper. Next is the comparison; please check it out.
Original Performance:
- (Dataset:CVC-ClinicDB-612-Test) Sm:0.903;meanEm:0.903;MAE:0.038;maxEm:0.923;maxDice:0.860;maxIoU:0.795;maxSpe:0.992.
- (Dataset:CVC-ClinicDB-612-Valid) Sm:0.923;meanEm:0.944;MAE:0.012;maxEm:0.962;maxDice:0.873;maxIoU:0.800;maxSpe:0.991.
- (Dataset:CVC-ColonDB-300) Sm:0.909;meanEm:0.921;MAE:0.013;maxEm:0.942;maxDice:0.840;maxIoU:0.745;maxSpe:0.996.
New Performance (that we retrained again):
- (Dataset:CVC-ClinicDB-612-Test) Sm:0.891;meanEm:0.876;MAE:0.043;maxEm:0.921;maxDice:0.854;maxIoU:0.785;maxSpe:0.995.
- (Dataset:CVC-ClinicDB-612-Valid) Sm:0.917;meanEm:0.923;MAE:0.013;maxEm:0.952;maxDice:0.861;maxIoU:0.787;maxSpe:0.991.
- (Dataset:CVC-ColonDB-300) Sm:0.897;meanEm:0.892;MAE:0.012;maxEm:0.936;maxDice:0.833;maxIoU:0.734;maxSpe:0.997.
These numerical results show some performance fluctuations due to different random seed or PyTorch versions or something. But I think it is acceptable for us. We don't take further action to improve the performance as we just fine-tune our model with one epoch. Overall It does not show a 'huge' drop after the fine-tuning process.
Thank you all guys to pay attention to our work. Further problems and discussions are welcome here.
ONE MORE THING We also release a large-scale VPS dataset, named SUN-SEG to promote this field. By using this dataset, you can feel free to train your large model without worries about over-fitting problems. You can also discard the two-stage training strategy using our enhanced method, PNS+, which only needs training once. We strongly suggest you use them.
Best, Ge-Peng.
谢谢您的回复,我再尝试尝试。
from pns-net.
Related Issues (19)
- 作者您好,问您一个问题 HOT 3
- Training Code? HOT 1
- 文件缺失 HOT 1
- Thanks for the excellent job! HOT 3
- Could you provide the training file? HOT 5
- 训练技巧和数据集的分配 HOT 1
- 缺少文件 HOT 2
- 想问一下怎么提高代码cuda使用版本 HOT 6
- 从头训练没有达到论文中的精度 HOT 12
- ModuleNotFoundError: No module named 'self_cuda_backend' HOT 4
- ModuleNotFoundError: No module named 'self_cuda_backend' HOT 1
- Some questions about the NL module HOT 3
- 无法复现论文精度 HOT 5
- 提示缺少cuda.h如何解决 HOT 3
- Training problem HOT 1
- 关于训练模型的问题 HOT 11
- 数据集已经失效了 HOT 1
- 执行setup.py报错问题 HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pns-net.