Implementing wide variety of transformers (improvised version),finetuning as well as trying architectural variants from various research papers and blogs.

1. Roberta Pretraining : Basic Implementation , More detail about the model can be found here.
2. Implementing Fast Transformers: Failed Attempt, FastTransformer Algorithm can be found here.
3. Adaptive Transformers for multimodal Representations: Research paper can be found here.
4. Learning Cross-Modality Encoder Representation from transformers: Paper can be found here.

My text generation model trained on Bhagvad_Gita (Holy Hindu Scripture English version) is now available on huggingface and can be found here.

The easiest way to use them is as below:

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("epsil/bhagvad_gita")

model = AutoModelForCausalLM.from_pretrained("epsil/bhagvad_gita")

Weak Supervision model build wirh Snorkel and with Finance dataset taken from here. Majority Vote Modelling with some heuristics(rules) and currently work on 3 labelling function. Overall accuracy reached is about 61% and can be enhanced further by dealing with mixed sentence heuristics.

1. Finetuning of Sentence Transformer for customized dataset.
2. Improvements in Comparison Methods and Advanced Comparison Techniques.

1. Performance evaluation of llama-7B Model
2. Performance evaluation of Redpajama-3B Model
3. Performance evaluation of Bloke Syntara-7B Model