Comments (2)
feiga
commented on July 28, 2024
- Yes, the DataBlock defines how to read/write data from/to disk. You may need to change the disk io to your MongoDB io API. DiskDataStream is just a wrapper for out-of-core computing.
- The Write() function write the data to disk file with binary format. The data contains not only the tokens in the corpus, but also the assigned topics to each token, which is part of model parameter and is changed during training. We periodically write data to persistent storage. Then the data file is also the checkpoint file. This is necessary when we do model slice that our paper describes.
- No, we don't have plan to use the new API. Currently LightLDA works with the previous version.
- The latest version of Multiverso only supports BSP/ASP. LightLDA don't use the latest API, the opensourced implementation indeed is SSP mode with s = 1.
- It's just different implementation.
from lightlda.
NicolasWinckler
commented on July 28, 2024
Thanks a lot for your fast reply!
from lightlda.
Related Issues (20)
- a word occurs too many times in all docs that int cannot handle.
- how to install it on multi nodes for distributed training? HOT 10
- How can i use the result to train a topic model HOT 3
- Fatal error in PMPI_Test: A process has failed, error stack: HOT 2
- error occur in Nemesis Network Module HOT 3
- How can i get TOP WORDS for each topic
- when I run the infer it will cause a segmentation fault
- sampling throughput: -nan (tokens/thread/sec)
- Is there any python wrappers for LightLDA?
- Distributed running nytimes through mpi HOT 3
- distributed lightLDA HOT 1
- Size Error while running inference
- lightLDA is killed when traning! HOT 1
- Very Big dataset, Bad Alloc caught: failed memory allocation for documents_buffer in DataBlock
- run example success,but no result
- corpus_size_ > memory_block_size when reading file /data/block.0 HOT 1
- terminate called after throwing an instance of 'zmq::error_t'
- data prepare
- undefined reference to `multiverso:: HOT 2
- The topics don't match when every infer.@feiga
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lightlda.