Need some urgent help about fmm HOT 12 CLOSED

@rbhatia46

• The edges.shp is the edges of the road network.
• The nodes.shp is not used here (it is used for plotting the illustration) as the edges.shp already contains source and target node information.
• The trips.shp is a shapefile where each feature is a trajectory linestring representing a sequence of GPS points. Equivalently, you can also use a CSV file to represent the trajectory as input as show in this file.
  https://github.com/cyang-kth/fmm/blob/master/example/data/trips.csv
• Time information of GPS point is currently not used here.

from fmm.

@rbhatia46
You may refer to this repo for example to run on OSM data.

https://github.com/cyang-kth/osm_mapmatching

In your case, you do not need to run the web app. You need to prepare for the road network in shapefile and the trajectory in shapefile or CSV format. Then run the command line.

from fmm.

@cyang-kth thanks for such a quick and detailed reply. However, on replacing the appropriate files, my matched percentage output on the console is 0.

• My GPS data was originally in points(latitude,longitude) for many users. I grouped the users and converted those to Linestrings for each user. So I have a total of 450 trajectories(450 users) denoted as LineString's, which I am passing in place of edges.shp.

• Also, I am retrieving the network data using ox.graph_from_place('Tokyo, Japan') as denoted in the https://github.com/cyang-kth/osm_mapmatching example and since my gps trajectories are in Tokyo, I should get a match percentage more than 0, right? Am I doing something wrong here.

Please look at the details below :
This is my ubodt_config.xml file

<?xml version="1.0" encoding="utf-8"?>
<ubodt_config>
    <input>
        <network>
            <file>./../../data/tokyo-shape/edges/edges.shp</file>
            <id>osmid</id>
            <source>from</source>
            <target>to</target>
        </network>
    </input>
    <parameters>
        <delta>3</delta>
    </parameters>
    <output>
        <file>ubodt.txt</file>        
    </output>
</ubodt_config>

This is my fmm_config.xml file -

<?xml version="1.0" encoding="utf-8"?>
<fmm_config>
  <input>
    <ubodt>
      <file>ubodt.txt</file>
    </ubodt>
    <network>
      <file>./../../data/tokyo-shape/edges/edges.shp</file>
      <id>osmid</id>
      <source>from</source>
      <target>to</target>
    </network>
    <gps>
      <file>data/user_moving_trajectories.shp</file>
      <id>user_id</id>
    </gps>
  </input>
  <parameters>
    <k>4</k>
    <r>0.4</r>
    <pf>0</pf>
    <gps_error>0.5</gps_error>
  </parameters>
  <output>
    <fields>
      <all/>
      <!-- <ogeom/> <error/> <opath/> <cpath/> -->
    </fields>
    <file>mr.txt</file>
  </output>
  <other>
    <log_level>2</log_level>
  </other>
</fmm_config>

from fmm.

@rbhatia46

I need to check the log file to figure out the reason.

• Your configuration seems to be problematic as the spatial unit is wrong. You are using WGS84 so that the search radius of 0.4 is about 40km, which is too big. You need to specify smaller values. The delta value is also problematic as 3 degree means 300 km, which is also too big. Refer to here as an example
  https://github.com/cyang-kth/osm_mapmatching/blob/master/ubodt_config.xml and https://github.com/cyang-kth/osm_mapmatching/blob/master/fmm_web_config.xml

• But the spatial unit should not be the reason for 0 match percentage here. I guess the reason is that the trajectories are too long and if there are any two consecutive GPS points that cannot be matched (their distance is longer than delta). The whole trajectory will not be matched.

I think you need to provide an example trajectory to figure out the reason.

from fmm.

Yes, my trajectories are too long, I will try to take each day's trajectory of a single user_id and then try.
Thanks

from fmm.

@cyang-kth I tried reducing the trajectory length but that didn't help too. Here is one sample trajectory for your reference(after reducing the length) -

'LINESTRING (130.9234771393204 33.83078184388262, 130.9433757967076 33.81207463216788, 130.9456922564248 33.81112154166963, 130.9635083478821 33.80871120081061, 130.9723575097389 33.80713734036463, 130.973331249883 33.80741815187245, 130.9732351104639 33.80743225692042, 130.9732292700023 33.80742271702733, 130.9732323870297 33.80742032109699, 130.973227738122 33.80740581853526, 130.973230144763 33.80740828773059, 130.9732304743659 33.80740216951512, 130.9732316128148 33.80740980972676, 130.9732334374206 33.80741053665412, 130.9732363575054 33.80741264245972, 130.9732400303775 33.80741956821647, 130.9732366948395 33.80741709873536, 130.9732391829387 33.80742205779672, 130.9732368026937 33.80741788981382, 130.9732497075736 33.80741561283762, 130.9731529630347 33.80702868243831, 130.9589431538129 33.80870179099296, 130.9559500114428 33.80989210123404, 130.9350582063267 33.81921176764175, 130.9182367075363 33.83300829172943, 130.9146003697393 33.83454388599421, 130.8978831638124 33.8397285376067, 130.8948727849097 33.83943039981419, 130.8936573146716 33.83948957980644, 130.8842118668455 33.83018135228659, 130.8762889783002 33.82012775564873, 130.866968439203 33.82334631719893, 130.8654709266048 33.82320452350643, 130.8646923554397 33.82328876870553, 130.8633676750712 33.82333084908883, 130.8585648865529 33.81349126338311, 130.8559253838175 33.81261659611992, 130.8559967550021 33.81275388865756, 130.856945428905 33.81245634808857, 130.8577670023083 33.82407621746614, 130.8573396027422 33.8270159922727, 130.8638582679642 33.84157884755071, 130.8673012600938 33.84640493718487, 130.8718354805905 33.85108369781534, 130.875307534295 33.86036345184726, 130.8726784120396 33.86493790389378, 130.8729592265191 33.86544334122195, 130.8727444120479 33.86578857190862, 130.8727148155766 33.86579816586276, 130.8673494540757 33.87231853251915, 130.8631848865614 33.8723906385047, 130.8633209306194 33.87234869330558, 130.8631100616052 33.87223489584301, 130.8631077267433 33.8722987442311, 130.8632018961073 33.87223882994792, 130.8624921847841 33.87310342433986, 130.8622964295866 33.87391562237063, 130.8616949313946 33.8736572724145, 130.8616910387974 33.87368041438055, 130.8616043133643 33.87434088561928, 130.8615190503871 33.87451482137797, 130.8616565341146 33.87383821900703, 130.8617073395241 33.87376338604203, 130.8617303544702 33.87367672471851, 130.8617082782233 33.8736849087038, 130.8617902993583 33.87371357970708, 130.8617210159495 33.87364891950935, 130.8616775272314 33.87387820376888, 130.8616975944291 33.87451649532778, 130.8616892993408 33.87390484214828, 130.8616769690896 33.87364910785993, 130.8616367209929 33.87433815838578, 130.8608022348562 33.87324487726043, 130.8611567060474 33.87413620832988, 130.8615772425467 33.87432362261316, 130.8666644998391 33.87614335158736, 130.8676801222648 33.87621857173071, 130.8663632159519 33.87620049764877, 130.8663380433561 33.87623933334041, 130.8664874788103 33.87617887816803, 130.8662226740776 33.87617071978313, 130.8664747811067 33.87608104169628, 130.8665274635736 33.87617064506954, 130.8666104166126 33.87621425677693, 130.8664761736399 33.87635995933379, 130.8667174913226 33.87634998960446, 130.8667090161317 33.87632829457937, 130.8667690555314 33.87669951193907, 130.8667608486635 33.87654454071653, 130.8666354000802 33.87638685841939, 130.866630892264 33.87635497479516, 130.8665263627339 33.87592182206784, 130.8673250023655 33.87501329492029, 130.8665772968595 33.87248438174317, 130.8693425510168 33.87231730258684, 130.8788049926904 33.87240096780234, 130.8878332506594 33.86862179213836, 130.8885022266284 33.86906942056467, 130.8885635375798 33.8692257785336, 130.8878799529085 33.86959189773011)'

Is this still long enough, if not can you please suggest the parameters I can try?

from fmm.

@rbhatia46

I think the problem is not about the length of linestring but your trajectory contains some kinds of complex turning back moving behavior. The clusters below shows the problematic regions.

I think you need to preprocess your trajectories by smoothing it or filter by some distance.

For the recommended configuration, I would suggest

• delta: 5km (0.05 degree)
• search radius (300 meters, 0.003 degree)
• k: 8 to 16

If your data is too sparse, you need to increase the value of delta. However, the precomputed UBODT size will also increase.

from fmm.

@cyang-kth I am using the from and to directly obtained from osmnx instead of source and target in the xml file, is that an issue?

from fmm.

@rbhatia46

Yes, it can be a problem. The reason is actually that each edge is considered as a single direction edge. Bidirectional edges with one-way equals false is handled here by adding a reverse edge. Without that reverse edge, some parts of the network are not be connected. That is also one reason for the 0 match percentage.

Generally if you have a topological connected network, your UBODT is supposed to contain millions of rows. If that table is too small it means that a lot of rows are not connected.

from fmm.

@cyang-kth then I will use the way listed in the osm_mapmatching example to convert them to bidirectional edges for one-way equals false using PostGIS, also downloading the network graph using osmnx for a whole country is very time taking(8 hours and still running), can you suggest an alternative to download the entire road network for Japan, this would be very helpful.

from fmm.

@rbhatia46

osmnx is written in Python so it would be extremely slow in processing large dataset. I would suggest you downloading OSM file directly from their server. Then process the data with cmd tools designed for OSM.

• osmosis: https://github.com/openstreetmap/osmosis
• osmfilter: filter OSM data by tags
• osm2pgrouting: https://github.com/pgRouting/osm2pgrouting, import OSM network into a PostGIS database for routing.

An important thing to notice is that fmm is not suitable for too large road network (containing millions of road edges) as the precomputation can take a lot of space. For a city level road network containing less than 100,000 edges it is still fine. For the whole Japan (maybe with millions of edges) it is infeasible to precompute. If your data only covers the highway, then the highway road network of a country might be much smaller, which can still be processed.

In that case of too large network, some other MM algorithms without employing precomputation such as STMatch should be used here.

from fmm.

@rbhatia46

A new algorithm STMATCH has been implemented and added in the repo, which does not need to have the precomputation. If you still need to use it, you are welcome to test.

from fmm.