Comments (7)
The model is stored in HDF format, which should be easy to read from any language.
Of course, you can also use Returnn to just do forwarding. Or use Returnn as a server process. Or embed Returnn into some other application. We do that for speech recognition, for inference/decoding, where Returnn is embed into our speech recognizer.
from returnn.
Hi,
it depends on what parts of the environment you want to get rid off. You'll still need theano.
A theano function can be pickled and unpickled (at least this was supported at some point), so you could pickle a theano function, which forwards a minibatch and then unpickle it in your target application, so you would get rid of the RETURNN dependency, but you still need theano.
from returnn.
Thank you for the answers. So, I chose to go with serialization in order to get rid of the RETURNN dependency. I tried to serialize the Engine object and then call engine.init_network_from_config() and engine.forward_to_hdf() but it seems that some methods are no picklable. Therefore, maybe I should consider the remaining options.
from returnn.
Ah, I thought you want to implement it purely in C++ or so.
Why exactly do you want to get rid of the Returnn dependency then? You can just put it as a submodule to your project.
from returnn.
Indeed, implement purely in C or C++ it was my first thought. My goal concerns decreasing the recognition time and increase portability, maybe run some tests in a robot as an example. I mean, if I understood correctly I could implement the MDLSTM architecture in C++ and then load the weights, parameters etc. from the HDF model to perform forwarding, but it seems a time-consuming task. So, it didn't occur to me that I could try an embedding approach. Please, could you briefly comment the strategy used to carry out the embedding?
from returnn.
Well, yes, that will be some work, but on the other side, it's mostly straight forward. You can mostly copy the existing CUDA code and with some simple modifications run it like pure C++ code. Do some automatic testing that you get exactly the same outputs. It would also be nice to have that for the TensorFlow backend. Actually, that will be the best option anyway, that you convert everything to TF, and you need to port over the MDLSTM kernel to TF, and then you can dump the computation graph for forwarding and import that from pure C++ and it should be fast to execute. It depends how important the performance is for you. Also, if you want to run on CPU or GPU.
from returnn.
I think from the Returnn side, there is not much to be added, so I'm closing this now.
from returnn.
Related Issues (20)
- CUDA error: initialization error HOT 3
- MultiProcDataset inside PyTorch DataLoader with num_workers>0, multiple issues HOT 4
- RuntimeError: CUDA error: unspecified launch failure HOT 2
- NonDaemonicSpawnProcess hangs at exit HOT 2
- High memory usage with datasets (specifically when multi procs are used)
- Hang at exit in TDL worker in multiprocessing `_run_finalizers`, deadlock in `_wait_for_tstate_lock`? HOT 6
- Hang HOT 2
- Returnn Native after using different apptainer uses old compilation HOT 6
- MetaDataset with sequence list filter file
- HDFDataset (or generic dataset) post processing HOT 15
- Dataset batching like ESPnet support
- torch.nn.functional.conv2d: RuntimeError: GET was unable to find an engine to execute this computation HOT 1
- TensorFlow 2.14 degradation in WER HOT 2
- Updates for recent TensorFlow version
- Hang in dataset iterator HOT 5
- Log GPU device for torch backend HOT 2
- torch.onnx.export requires input_names and output_names to be in order HOT 12
- RF weight dropout HOT 6
- Support for larger scale datasets HOT 33
- RuntimeError: CUDA error: unknown error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from returnn.