Comments (3)
Matcher
and RoadMap
are thread-safe in such a way that both are accessed read-only during map matching. However, Matcher
is already running shortest path searches in parallel (data parallelism), i.e. for n candidates (sources) in t-1 and m candidates (targets) in t, it runs n SSMT shortest path searches in parallel (SSMT - single source multiple target); see line 246 and 295 in the Matcher
class. The only non-thread-safe objects relevant during map matching are KState
and MatcherKState
, respectively.
To the best of my knowledge, 'more' parallelization of map matching of a single track requires non-trivial task parallelism, e.g., a parallel implementation of Dijkstra algorithm. However, processing multiple tracks in parallel is just data parallelism which is already implemented if you use the stand-alone server, see lines 252-261 in the Server
class.
All in all, I don't see that much potential for trivial parallelization. You could check the speed-up of your application, if you set matcherNumThreads
to one and to the number of virtual cores, respectively. If you use Barefoot as a library, run StaticScheduler.reset(1)
and StaticScheduler.reset(p)
where p is again the number of virtual cores before the map matching.
Finally, I would recommand to not use threads for further parallelization because it may conflict with Barefoot's internal work-stealing scheduler (planned to be extracted in a separate library in future), that provides (user-level) task parallelism with spawn
and sync
operations as presented in Leiserson et al., Introduction to Algorithms, MIT Press, 3rd edition (see chapter 27).
from barefoot.
Thanks for such a thorough answer. The parallelism in my case is indeed needed to load separate traces in parallel and then match them, with a code something like so in the implementation of Runnable#run()
method:
while (loader.next()){
List<List<MatcherSample>> samples = loader.loadCurrentBatch();
for (List<MatcherSample> trace : samples) {
MatcherKState candidate_state = mapTrace(trace);
saveRoadSpeeds(candidate_state.samples(), candidate_state.sequence());
}
}
thanks for the reference to the Server
class, will check that code and see if it can improve the design I currently have.
Cheers!
from barefoot.
In that case (batch processing), I would just start as many threads as you have cores available and run that code.
If you're having a Spark cluster available, you could check out this to run it on the cluster: scalable map matching.
from barefoot.
Related Issues (20)
- Proximity of road segments ids after map-matching HOT 2
- Docker build failing with osmosis installation HOT 1
- reach of map HOT 3
- Unable to save the output while using the barefoot for offline map matching in scalable manner HOT 10
- Road switch to early. HOT 5
- email validation bug: preventing contact-us submission and access to bmwfs
- Docker build fails at gradlew assemble HOT 3
- Can Matcher be cached?
- Issue downloading gradle during installation. HOT 2
- Spark: matcher response time too long as barefoot servers(stand-alone)
- Can we use any road network with geometry instead of OSM? HOT 2
- Question: Add administrative boundaries to matcher response
- Unable to build maven project
- Unable to build matcher server
- question about online map match
- help!Have you realized it? Please help me deploy the source code for payment
- HMM Break no emssion state
- How to match multi-trajection and save the matching result?
- Possible mistake in path.cost()
- Matching Behaviour is incorrect
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from barefoot.