caesar0301 / s2g Goto Github PK

View Code? Open in Web Editor NEW

24.0 3.0 8.0 87 KB

(S)hapefile to(2) (G)raph/network converter in Python

Home Page: https://pypi.python.org/pypi/s2g

License: MIT License

Python 100.00%

python graph shapefile conversion networkx network-converter fiona s2g

s2g's Introduction

python-s2g

(S)hapefile (2) Graph/network converter in Python

When we process GIS data, a non-trivial problem is the conversion from shape lines to graph or network data structure. The latter may benefit from these out-of-box graphical libraries such as networkx and igraph. But the conversion is a headache to components open communities. This mostly urges me to finish this tiny but useful library.

Install

Requirements: Python 2.7+ or Python 3.3+

sudo apt-get install python python-pip libgeos-dev

Install s2g,

sudo pip install s2g

Extra utilities to run unittests,

sudo apt-get install python-tk
sudo pip install matplotlib

Usage

You have two alternative ways to construct the graph. One is reading from a raw shapefiles with LineString objects. (Under the hood, I involve fiona to read geometries and shapely to analyze the data.). Currently, this tool only supports conversion to undirected graph.

from s2g import ShapeGraph
import networkx as nx

sg = ShapeGraph(shapefile='path/to/roads.shp', to_graph=True)
assert isinstance(sg.graph, nx.Graph)

The other way is designed for programmable usage or time-consuming process where intermediate data could be sniffed or saved. Here is an example to read lines with [fiona]:

from s2g import ShapeGraph
import fiona
from shapely.geometry import shape, LineString

shp = 'path/to/shapefile.shp'

with fiona.open(shp) as source:
    geoms = []
    for r in source:
        s = shape(r['geometry'])
        if isinstance(s, LineString):
            geoms.append(s)

# create ShapeGraph object from a list of lines
sg = ShapeGraph(geoms, to_graph=False)

# detect major components
mc = sg.gen_major_components()
# major components are mc[2]

# convert the largest component to networkx Graph
graph = sg.to_networkx()  # equivalently sg.graph

Dive into source doc to discover other functionalities.

QA

Why not NetworkX's read_shp function? (Issue)

I endeavored to avoid reinventing the wheel at the beginning. It has several limitations to meet common road network processing:

It is not able to detect the major components when the shapefile has disconneted parts
It has no buffer mechsnisms to determine the connectivities of line-line, line-point or point-point pairs
It does not support parameter controlled sampling of road lines when we convert geometry lines into edges
It has no pesudo edges to fix the disconnectivity of geometry elements

References

shp2graph in R by Binbin Lu, as well as his talk on useR! 2011
A Tutorial on Topology Correction of Shapefiles Using GRASS
Why do I make s2g

s2g's People

Contributors

Stargazers

Watchers

Forkers

oztalha cuulee vhffm vishalbelsare pibeg noman-bashir sanigithub karimdatamaster

s2g's Issues

ImportError: No module named 'shapegraph'

In [1]: from s2g import ShapeGraph
   ...: 
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-a9021cd38b6c> in <module>()
----> 1 from s2g import ShapeGraph

/usr/local/lib/python3.5/site-packages/s2g/__init__.py in <module>()
----> 1 from shapegraph import *
      2 from bonus import *

ImportError: No module named 'shapegraph'

Misc geometry type encountered and omit: Polygon

I am getting the following warning for all the data samples:
WARNING:root:Misc geometry type encountered and omit: Polygon

Due to this, the graph created has 0 components.

How does you work differ from NetworkX's read_shp?

Hi there,

I just discovered your project.
It looks similar to NetworkX's read_shp function.
Could you explain how it differs from that function, maybe in your README?

Add/detect road directions then construct directed graph if directions are available

New to python, do we have to save the graph and if so how?

I've had my computer running for seven days converting a shapefile to a graph, I used this code:

from s2g import ShapeGraph
import fiona
from shapely.geometry import shape, LineString

shp = r"C:\Users\Tom\Documents\NorthAmerica.shp"

with fiona.open(shp) as source:
    geoms = []
    for r in source:
        s = shape(r['geometry'])
        if isinstance(s, LineString):
            geoms.append(s)

# create ShapeGraph object from a list of lines
sg = ShapeGraph(geoms, to_graph=False)

# detect major components
mc = sg.gen_major_components()
# major components are mc[2]

# convert the largest component to networkx Graph
graph = sg.to_networkx()  # equivalently sg.graph
plt.savefig(r"C:\Users\Tom\Documents\graph.png")

was I supposed to save it to somewhere because it's finished but I don't see any kind of output in the command line.

How to store node information

After I have converted shapefile to a nx graph, the information in the shapefile (e.g road name, coordinates) seems to be omitted. So how can I obtain the information? I need them to map the graph back to the shapefile.

Error when generating components

Hello, I have a problem during conversion, specifically running the following code on this dataset.

import fiona
from s2g import ShapeGraph
from shapely.geometry import LineString, shape

path = 'extracted_roads.shp'

with fiona.open(path) as source:
    geoms = []
    for r in source:
        s = shape(r['geometry'])
        if isinstance(s, LineString):
            geoms.append(s)

# create ShapeGraph object from a list of lines
sg = ShapeGraph(geoms, to_graph=False)

# detect major components
mc = sg.gen_major_components()

# convert graph to json
G = json_graph.node_link_data(sg.to_networkx())

for node in G['nodes']:
    node['lat'], node['long'] = sg.node_xy[node['id']]

with open('connected_roads.json', 'w') as output:
    output.write(json.dumps(G))

More or less after 400000 entries are processed the conversion stops with this error.

INFO:root:Validating pair-wise line connections of raw shapefiles (total 9756 lines)
100% (47584890 of 47584890) |#################################################################################| Elapsed Time: 0:03:57 Time:  0:03:57
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-0290bc8fe669> in <module>()
----> 1 mc = sg.gen_major_components()

/home/ubuntu/.local/lib/python3.6/site-packages/s2g/shapegraph.py in gen_major_components(self)
    264                 bar.update(k + 1)
    265                 i, j = neighbors[k]
--> 266                 if self.validate_pairwise_connectivity(i, j):
    267                     graph.add_edge(i, j)
    268

/home/ubuntu/.local/lib/python3.6/site-packages/s2g/shapegraph.py in validate_pairwise_connectivity(self, ainx, binx)
    223             valid = True
    224
--> 225         touched = self.validate_intersection(binx, a1)
    226         if touched is not None:
    227             self._pseudo_edges.append([(binx, touched), (ainx, a1)])

/home/ubuntu/.local/lib/python3.6/site-packages/s2g/shapegraph.py in validate_intersection(self, line_index, point)
    195         if line.intersects(buffered_point):
    196             cut = point_projects_to_line(point, line)
--> 197             touched = coords[cut]
    198             self._update_cut(line_index, cut)
    199         return touched

TypeError: list indices must be integers or slices, not NoneType

How can I debug this issue?
Thank you very much.

point_projects_to_edges returns duplicated edges

MemoryError

neighbors = [(i, j) for i, j in product(range(0, L), range(0, L)) if j > i]
MemoryError