Comments (1)
Thank you for the appreciation.
Language Models have been trained on their respective Wikipedia corpus - which is available for download in Language specific repositories linked in README. The embeddings that iNLTK provides are essentially the weights of first layer in encoder. Since for classification, the LM has been fine tuned over classification dataset, It didn't make sense to me use those embeddings because they would be biased towards the classification dataset, whereas Wikipedia Dataset, represents, kind of, the whole universe. This was the thinking behind choosing LM for embeddings.
As I explained, because embeddings are essentially weights of the first layer, I expect BERT LM to do slightly better than ULMFiT, and hence embeddings might be slightly better, But I'm not aware of any techniques by which we can quantitatively compare embeddings. Currently we only do this qualitatively by visualizing in 3 dimensions.
Hope this answers your question. I'll close this issue, but feel free to reopen if there's anything!
from inltk.
Related Issues (20)
- How to identify whether the text is Hinglish? HOT 1
- get_similar_vectors, get_similar_sentences fails with AttributeError: 'LSTM' object has no attribute '_flat_weights_names' HOT 3
- Why the Get Embedding Vectors method gives us 400 dimensions? HOT 4
- Dimensionality Reduction HOT 2
- Unable to detect the language HOT 6
- Wordnet for inltk
- Lemmatization for Tamil language HOT 1
- [Documentation Typo] The order of installation in official documentation is wrong HOT 5
- Contribution for Gujarati-English
- How can i contribute? HOT 1
- Adding support for Bhojpuri, Magahi and Maithili HOT 4
- Getting error while calling setup HOT 1
- Not able to use identify language for Indian languages
- LSTM' object has no attribute '_flat_weights_names' HOT 6
- Getting Runtime error while calling the tokenizer
- Import Error while importing tokenize function from inltk.inltk HOT 1
- AttributeError: 'LSTM' object has no attribute '_flat_weights'
- Request for: Code mixed Language support for Kannada (kn)
- Getting error when trying to run devanagari tokenizer
- RuntimeError: Internal: could not parse ModelProto from /home/nlp/miniconda3/lib/python3.9/site-packages/inltk/models/hi/tokenizer.model HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from inltk.