Hi authors! How can I evaluate my own trained model?

Hi! You can evaluate the model using this command: <div class="Box Box--condensed

Evaluate my own model about passt HOT 2 CLOSED

kkoutini commented on May 27, 2024

Evaluate my own model

from passt.

Comments (2)

kkoutini commented on May 27, 2024

Hi! You can evaluate the model using this command:

PaSST/ex_audioset.py

Line 352 in 7a0d0a4

def evaluate_only(_run, _config, _log, _rnd, _seed):

which can be called like this:

python ex_audioset.py evaluate_only with trainer.precision=16 models.net.arch=passt_deit_bd_p16_384  trainer.use_tensorboard_logger=True -p

the loading model part is not implemented yet, but you can try for example:

@ex.command
def evaluate_only(_run, _config, _log, _rnd, _seed, preload_modul_path="path.pth"):
    # force overriding the config, not logged = not recommended
    trainer = ex.get_trainer()
    train_loader = ex.get_train_dataloaders()
    val_loader = ex.get_val_dataloaders()
    modul = M(ex)
    modul.val_dataloader = None
    trainer.val_dataloaders = None
    print(f"\n\nValidation len={len(val_loader)}\n")
    #here is the loading part
    ckpt = torch.load(preload_modul_path)
    modul.load_state_dict(ckpt)
    ###
    res = trainer.validate(modul, val_dataloaders=val_loader)
    print("\n\n Validtaion:")
    print(res)

now you can use in the config preload_modul_path to specify the path for your pretrained model:

python ex_audioset.py evaluate_only with trainer.precision=16 models.net.arch=passt_deit_bd_p16_384  trainer.use_tensorboard_logger=True preload_modul_path=/path/to/model.pth -p

from passt.