Comments (3)
mattdangerw
commented on May 18, 2024
Note that we will do this only for the learned positional embedding (not the sinusoidal encoding). We believe that the learned embedding will be the most common usage for adding position info, so it's the only one we will provide the extra "glue" layer for.
from keras-nlp.
adhadse
commented on May 18, 2024
Can I try to work on this issue? Please assign me.
from keras-nlp.
mattdangerw
commented on May 18, 2024
Please do. Thanks!
from keras-nlp.
Related Issues (20)
- cannot import name 'CachedMultiHeadAttention' from partially initialized module 'keras_nlp.src.layers.modeling.cached_multi_head_attention' (most likely due to a circular import)
- Keras_NLP and Kaggle Hub: Are models allowed without weights in Kaggle Hub?
- Issue with `BytePairTokenizer`
- Broken link to "Understanding masking and padding" guide HOT 2
- Add CLIP tokenizer to Keras NLP HOT 1
- Mistral kills the process by taking too many RAM HOT 2
- Any guide how to use tools/gemma/run_gemma_xla.py? HOT 4
- Model weights contributions? HOT 5
- Question about Gemma tensor parallel sharding policy HOT 5
- Add `oov_token` Argument to `BytePairTokenizer` HOT 1
- Update ByteTokenizer to remove TensorFlow dependency HOT 1
- How to add a serialized model and weights of a keras model to keras-nlp? HOT 4
- ContrastiveSampler lacks a seed param, while the docstring states it has one HOT 1
- Gemma Model Storing and Loading after Fine tuning HOT 2
- Gemma discrepancies HOT 1
- Dropout is not called in the training regime in TransformerEncoder and others HOT 2
- Data-Parallel Training with KerasNLP and tf.distribute example dataset problem HOT 4
- Feature Request: Transformer Debugger - Debugging and controlling the behavior of transformer based LLM models. HOT 3
- Add Mistral 0.2 models as possible presets HOT 3
- keras-nlp insists I use the (buggy) Tensorflow 2.16.1 which does not work with my GPU HOT 12
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from keras-nlp.