Pretraining from scratch - Error while creating the h5 dataset file? about protein_bert HOT 5 CLOSED

Hi @fridalon,

It indeed seems to have failed for some reason.
Did you try creating the dataset for UniRef90 (the dataset we used originally to train ProteinBERT). I can't think of any reason why it would work for UniRef90 and not UniRef50, but I think that it's worth starting with the exact same conditions we used (for the sake of debugging). Also, if it still doesn't work, could you attach the full logs you get from the script?

from protein_bert.

Hi @nadavbra ,

I found the problem. in "uniref_dataset.py" file, in the function load_seqs_and_annotations -

I will run and let you know if it worked.
Thanks

from protein_bert.

@nadavbra
can close the issue. thanks

from protein_bert.

@nadavbra @fridalon
Can you say how much compute power is needed to run from scratch?
Say A100 GPU running for 1 month or something else?

If, instead of using the existing model weights, you would like to train from scratch, then follow the steps below. We warn that this is a long process (we pretrained the current model for a whole month), and it also requires a lot of storage (>1TB).

from protein_bert.

It looks like a stronger GPU than the one we used, so yes, 1 month over a single GPU should get you to the same performance.

from protein_bert.