UIMNET
This is UIMNET an Imagenet scale uncertanity quantification benchmark suite.
Usage
Setup and installation
Install environment
conda env create -f conda_env.yamlActivate environement
. ./script/setupInstall package in development mode
pip install -e .Run Unit tests
pytests ./testsTraining
Single model
./scripts/run_trainer.py -a ALGORITHM -m MODEL_DIR -a ARCHITECTURE -c CLUSTERING_FILEpython3 -m torch.distributed.launch --nproc_per_node=8 ./scripts/run_trainer.py -a ALGORITHM -m MODEL_DIR -a ARCHITECTURE -c CLUSTERING_FILE -dSweep over all models
./run_training.py -s SWEEP_DIRCalibration
Single model
./scripts/run_calibrator.py -m MODEL_DIR -c CLUSTERING_FILEpython3 -m torch.distributed.launch --nproc_per_node=8 ./scripts/run_calibrator.py -m MODEL_DIR -c CLUSTERING_FILE -dSweep over all models
./run_calibration.py -s SWEEP_DIRIn-domain prediction
Single model
./scripts/run_predictor.py -m MODEL_DIR -c CLUSTERING_FILEpython3 -m torch.distributed.launch --nproc_per_node=8 ./scripts/run_predictor.py -m MODEL_DIR -c CLUSTERING_FILE -dSweep over all models
./run_prediction.py -s SWEEP_DIREnsembling
./scripts/run_ensembles.py -s SWEEP_DIROut-of-domain evaluation
Single model
./scripts/run_evaluator.py -m MODEL_DIR -c CLUSTERING_FILE --measure MEASUREpython3 -m torch.distributed.launch --nproc_per_node=8 ./scripts/run_evaluator.py -m MODEL_DIR-c CLUSTERING_FILE -d --measure MEASURESweep over all models
./scripts/run_evaluation.py -s SWEEP_DIR --measures MEASURE1 MEASURE2 ...License
This source code is released under the MIT license, included Here.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent