Comments (11)
some function like tf.keras.layers.Embedding
from transform.
You could call either tf.nn.embedding_lookup
directly or tf.keras.layers.Embedding
in the preprocessing_fn.
Have you ran into issues doing that?
from transform.
Apologies for the delay and as of now we don't have such function for embedding transform and you can check all available functions with Tensorflow Transform
here and I have attached one good resource for you with tensorflow transform examples, please have a look once here and if you're looking to explore how Transform component works with TFX pipeline then please check this article I hope it will help you to resolve your issue, If your issue still persists or you need any further assistance please let us know ?
Thank you!
from transform.
You could call either
tf.nn.embedding_lookup
directly ortf.keras.layers.Embedding
in the preprocessing_fn. Have you ran into issues doing that?
because tf.keras.layers.Embedding is return a layer object not a tensor. All of the tfx function in the preprocessing_fn must return a tensor.
if i use tf.keras.layers.Embedding, i must build a tf model, and then model.compile() 、 model.fit() and the predict the input data(It's a whole train flow). but i don't want a whole train flow in the preprocessing_fn code. i want use the embedding function like tft.scale_to_0_1(). is this possible?
from transform.
thanks for the reply. if i can't use the embedding function in the preprocessing_fn function. i will put the embedding layer in the train stage. not the data preprocess stage
from transform.
Could you not call it and apply it directly to your data?
tf.keras.layers.Embedding(dim_a, dim_b)(inputs['x'])
from transform.
Could you not call it and apply it directly to your data?
tf.keras.layers.Embedding(dim_a, dim_b)(inputs['x'])
I had try. but return an error.
return self._stateless_fn._get_concrete_function_garbage_collected( # pylint: disable=protected-access
File "/opt/conda/envs/feast-python-sdk/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2576, in _get_concrete_function_garbage_collected
graph_function, _ = self._maybe_define_function(args, kwargs)
File "/opt/conda/envs/feast-python-sdk/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2760, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/opt/conda/envs/feast-python-sdk/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2670, in _create_graph_function
func_graph_module.func_graph_from_py_func(
File "/opt/conda/envs/feast-python-sdk/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 1247, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/opt/conda/envs/feast-python-sdk/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 677, in wrapped_fn
out = weak_wrapped_fn().__wrapped__(*args, **kwds)
File "/opt/conda/envs/feast-python-sdk/lib/python3.8/site-packages/tensorflow_transform/impl_helper.py", line 683, in transform_fn
transformed_features = preprocessing_fn(inputs_copy)
File "tfx_transform.py", line 133, in preprocessing_fn
outputs['embedding_age'] = tf.keras.layers.Embedding(input_dim=100, output_dim=10)(inputs["age"])
File "/opt/conda/envs/feast-python-sdk/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/opt/conda/envs/feast-python-sdk/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 790, in invalid_creator_scope
raise ValueError(
ValueError: tf.function only supports singleton tf.Variables created on the first call. Make sure the tf.Variable is only created once or created outside tf.function. See https://www.tensorflow.org/guide/function#creating_tfvariables for more information.
from transform.
What about using tf.nn.embedding_lookup
directly? (since keras is creating trainable variables on that layer's build implementation)
from transform.
What about using
tf.nn.embedding_lookup
directly? (since keras is creating trainable variables on that layer's build implementation)
tf.nn.embedding_lookup
The first param of this function is params(A single tensor representing the complete embedding tensor,or a list of tensors all of same shape except for the first dimension, representing sharded embedding tensors following "div" partition strategy.)
I did't have thisembedding tensor
. Thisembedding tensor
should train by embedding function.
from transform.
In that case, the embedding training shouldn't happen in transform.
If there are pre-trained embeddings those can be applied during transform.
from transform.
In that case, the embedding training shouldn't happen in transform. If there are pre-trained embeddings those can be applied during transform.
Get it, i should split embedding transform from pre-trained stage into training stage.
But in other words, embedding transform is one type of feature data transform.
from transform.
Related Issues (20)
- Expose type information HOT 1
- Segmentation Fault on tft.compute_and_apply_vocabulary HOT 8
- Release new version that's compatible with `tfx[kfp]==1.10.0` HOT 7
- Can't install due to dependency on numpy HOT 5
- Python 3.10 support HOT 19
- TypeError: an integer is required [while running 'AnalyzeAndTransformDataset/TransformDataset/InstrumentInputBytes[Transform] HOT 6
- Allow Identity "Transformation" HOT 4
- Beam AnalyzeAndTransformDataset runs expensive transformation _InstanceDictInputToTFXIOInput Twice HOT 3
- Install fails on Apple M2 Ventura HOT 3
- Apple Silicon support for tensorflow-transform not available as tfx-bsl does not support it? HOT 3
- Raise pyarrow upper bound? HOT 12
- apart from the batch dimension, all dimensions must have known size [while running 'AnalyzeAndTransformDataset/AnalyzeDataset/CreateSavedModel[tf_v2_only]/CreateSavedModel'] HOT 2
- Transform component and tf.function HOT 4
- Transform graph returns an empty dictionary at serve_tf_examples_fn function HOT 4
- Graph error when using TFHub Universal-sentence-encoder model HOT 2
- Wheel missing py.typed HOT 2
- universal sentence encoder batch pipeline failing HOT 3
- tfx.components.Transform returns invalid results HOT 1
- TFX Transform layer returns dict with missing keys HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from transform.