Comments (4)
Hi @mah51,
Just to clarify - you want to fine-tune ProteinBERT to predict the GO annotations which you provide as dataset[1]
? How do you encode these GO annotations? What is your output_spec
?
from protein_bert.
Thank you for the response @nadavbra.
Yes I provide the GO annotations as dataset[1], they aren't encoded. The OutputSpec is generated as follows:
output_spec = OutputSpec(OutputType(True, 'categorical'), unique_labels)
.
I was unaware I had to encode the GO annotations as the model seems to fine-tune with them as a raw input and I assumed they were encoded as part of the encode_dataset
method within evaluate_by_len
. Thanks.
from protein_bert.
The model was indeed pretrained to predict GO annotations, so it already outputs GO annotation predictions (however, as stated in another issue, there is a small problem with figuring out which output corresponds to which GO annotation). In your case, you may want to fine-tune the model anyway (or pretrain it from scratch), so it can predict the specific GO annotations that you care about (which might be different from the GO annotations we used during pretraining).
You can also pretrain a similar model from scratch with your specific sequences and GO annotations. We have instructions for how to pretrain a new ProteinBERT model.
Anyway, what you are trying to do won't work. You specify a sequential-categorical output, which is not the case for your data (there isn't a different output for each protein residue). You can think about your GO annotations as a set of independent binary predictions (one for each GO annotation).
There isn't a ready-to-use output type for that, meaning you will have to write some code if you want to fine-tune the existing ProteinBERT model. Basically, you will likely want to add a Dense(k, activation = 'sigmoid')
layer on top of ProteinBERT's global output, where k
is the number of GO annotations you want to predict.
Hope that helps.
from protein_bert.
Thank you very much for your assistance, I will give that a shot.
from protein_bert.
Related Issues (20)
- Failing to get the weights from the dedicated github repo HOT 5
- Use ProteinBERT with Own Dataset HOT 3
- Original h5 file HOT 5
- loss plot during pretraining HOT 1
- signal peptide detection HOT 1
- KeyError: "Unable to open object (object 'test_set_mask' doesn't exist)" HOT 6
- How to extract the embedding of an amino acid? HOT 10
- Graph execution error HOT 6
- Extract local and global representation using finetune model HOT 1
- Running Benchmarks HOT 4
- Evaluation on larger data set HOT 6
- Using vector representations in the "weights" parameter in the "embedding" section of an LSTM model after fine-tuning my own data HOT 1
- Failing to extract global embedding (1,15599) -> (1,512) HOT 1
- What do the settings mean? HOT 3
- Error when trying to run the finetuning code given in the jupyter notebook HOT 2
- ValueError, set_weights error
- model_generation.py list is not callable error HOT 2
- GO annotations during fine tuning HOT 1
- Missing MajorPTMs train CSV file HOT 1
- Can't get proteinBERT to run on GPU HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from protein_bert.