TensorFlow2 NLP transformer.
No Keras used to create the model
Jupyter Notebook
TensorFlow==2.12.0
Sentencepiece==0.1.99
TensorFlow-text==2.12.1
Numpy==1.23.5
- Download your dataset, save it to datasets/dataset.txt.
- Run train_bpe.py script on it, to create model vocabulary.
- Split sentences in dataset with some special token.
- Launch Google Colab or Jupyter Notebook on local machine and upload spawn.ipynb to it with your vocabulary and dataset.
- Define hyperparameters in this notebook to open dataset, create model and train it.
- The model will be saved after training, download it if you using Google Colab, and run sample.ipynb with your model to perform top-k sampling and infer it.