adonmo / meos Goto Github PK

This is to make sure that MobilityDB and MEOS have similar naming schemes.

For more discussion on this, check MobilityDB/MobilityDB#18

Pending:

• MEOS: Rename Geometry to GeomPoint
• PyMEOS: Move base type before temporal duration in the naming convention throughout. i.e, for example, remove TSequenceSetGeom and add TGeomPointSeqSet.
• MEOS: Add typedefs, ex. typedef TSequenceSet<int> TIntSeqSet; so that both are available, and with the latter, C++ API can stay in line with PyMEOS API as well.

Aim to have consistent versioning across the project, always.

Currently, our python packaging is based on pyxmolpp2 project's setup.py file, (based on this comment). However, I have skipped the version detection part to keep things simple for now.

• &&: Do the time values overlap (have points in common)?
• @>: Does the first time value contain the second one?
• <@: Is the first time value contained by the second one?
• -|-: Is the first time value adjacent to the second one?

Invalid operations should throw accordingly

Refer MobilityDB docs for more information: https://docs.mobilitydb.com/MobilityDB/master/ch02.html#time_topo_operators

Todo Checklist

• overlaps
  • {timestampset, period, periodset} && {timestampset, period, periodset}
• contains
  • {timestampset, period, periodset} @> time
• contained_by
  • time <@ {timestampset, period, periodset}
• adjacent_to
  • time -|- time

Right now we are using doxygen out of the box - which is fine, although, honestly I think it could be better - both in terms of usability and looks. I have two different approaches in mind, which I think are worth considering:

Approach 1. Use MCSS theme

• Pros: Best part, IMO, is the quick search feature as soon as you press tab. It increases usability by a lot.
• https://mcss.mosra.cz/documentation/doxygen/
• Example (magnum): https://doc.magnum.graphics/magnum/

Approach 2. Integrate with sphinx/readthedocs

• sphinx/readthedocs seems to have support for C++ code. Although I have yet to look into how viable an option this is.
• Pros: This would bring both Python and C++ documentation to a single place. This means common concepts could be documented together. However, it is worth evaluating this approach further if coupling them together might complicate some other things.
• https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#cpp-domain

Right now I am leaned a bit towards Approach 1.
Any ideas/suggestions/contributions towards this would be greatly appreciated.

• +: Union of the time values
• *: Intersection of the time values
• -: Difference of the time values

We can override the respective operators in both C++ and Python
Invalid operations should throw accordingly

Todo Checklist

• TimestampSet, TimestampSet
  • Union
  • Intersection
  • Difference
• Period, Period
  • Union
  • Intersection
  • Difference
• PeriodSet, PeriodSet
  • Union
  • Intersection
  • Difference
• PeriodSet, Period
  • Union
  • Intersection
  • Difference

Sample MobilityDB queries for reference/test cases

mobilitydb=# select period '[2020-01-01, 2020-01-10]';
                      period                      
--------------------------------------------------
 [2020-01-01 00:00:00+00, 2020-01-10 00:00:00+00]
(1 row)

mobilitydb=# select period '[2020-01-01, 2020-01-10]' + '[2020-01-08, 2020-01-10]';
                      ?column?                      
----------------------------------------------------
 {[2020-01-01 00:00:00+00, 2020-01-10 00:00:00+00]}
(1 row)

mobilitydb=# select period '[2020-01-01, 2020-01-10]' + '[2020-01-08, 2020-01-12]';
                      ?column?                      
----------------------------------------------------
 {[2020-01-01 00:00:00+00, 2020-01-12 00:00:00+00]}
(1 row)

mobilitydb=# select period '[2020-01-01, 2020-01-10]' + '[2020-01-18, 2020-01-20]';
                                               ?column?                                               
------------------------------------------------------------------------------------------------------
 {[2020-01-01 00:00:00+00, 2020-01-10 00:00:00+00], [2020-01-18 00:00:00+00, 2020-01-20 00:00:00+00]}
(1 row)

For each of the box types defined in MobilityDB, a class that implements the data structure and relevant functions needs to be defined, similar to how we already defined for time types (like Period etc), range types (like Range<int> etc) and temporal types (like TSequence<float> etc).

Specifically:

• TBox
  • C++ implementation and tests
  • Python bindings and tests
• STBox
  • C++ implementation and tests
  • Python bindings and tests

Check MobilityDB's documentation on Box Types: https://docs.mobilitydb.com/nightly/ch04.html

For an example implementation, check code for Period class: https://github.com/chaitan94/meos/blob/master/meos/types/time/Period.cpp (along with its header file in include folder)

Existing implementations of TInstant, TInstantSet and TSequence can be used as a reference.

• C++ implementation
• C++ tests
• Python bindings
• Python tests
• Serdes
• Documentation

Hi Krishna

My name is Aniket and I am one of the engineers of MoveTK (https://github.com/heremaps/movetk). I am reaching out to you to see if you think there is a scope for collaboration between the two projects?

While the code compiles and wheel are being built on Mac OS right now, they are really linked properly. As a result, when using the wheel to import pymeos, we see the following:

     from pymeos.box import STBox
E   ImportError: dlopen(/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp96l8_7pd/lib/python3.5/site-packages/pymeos.cpython-35m-darwin.so, 2): Symbol not found: __ZN10SerializerI8GeometryE5writeEPK11TInstantSetIS0_E

A C-API would allow the library to be used by many other languages through C-based bindings.
Especially, it would also allow MobilityDB to depend on MEOS and use it where appropriate.

Some possibly useful references

• to get started on learning how to do this: https://www.teddy.ch/c++_library_in_c/
• auto generation of C interface: http://samanbarghi.com/blog/2016/12/06/generate-c-interface-from-c-source-code-using-clang-libtooling/
• Also we can always refer to GEOS's code as well.

The TInstants should always be ordered according to time, we need to add appropriate checks everywhere.

• +: Union of the bounding boxes
• *: Intersection of the bounding boxes

We can override the respective operators in both C++ and Python
Invalid operations should throw accordingly

Todo Checklist

• TBox, TBox
  • Union
  • Intersection
• STBox, STBox
  • Union
  • Intersection

MobilityDB queries for reference/test cases

mobilitydb=# select intrange '[10, 20)' + '[15, 40)';
 ?column? 
----------
 [10,40)
(1 row)

mobilitydb=# select intrange '[10, 20)' * '[15, 40)';
 ?column? 
----------
 [15,20)
(1 row)

mobilitydb=# select intrange '[10, 20)' - '[15, 40)';
 ?column? 
----------
 [10,15)
(1 row)

mobilitydb=# select intrange '[10, 20)' + '[30, 40)';
ERROR:  result of range union would not be contiguous

This is just a thought as of now, because I am not exactly sure how this can be done and whether it will add any unnecessary complications elsewhere, say while implementing various type of functions.

Basically idea is simple: instead of having multiple classes defined, one for each type like for example TSequenceFloat and TSequenceGeom, can we have a single TSequence class, and infer the type based on the values provided? This could be similar to how numpy does through its dtype logic. One major advantage with this approach could be that we might be much more in line with many tools like numpy, pandas and friends.

Currently Geometry class is very simple - it represents a point. We should also add support to specify SRID.

SRID can also be specified for temporal types of geometry, like TSequence<Geometry> and TSequenceSet<Geometry>.

On a side note, should we rename Geometry to Point?

• Geometry
• TInstant
• TInstantSet
• TSequence
• TSequenceSet

Invalid operations should throw accordingly

Refer MobilityDB docs for more information: https://docs.mobilitydb.com/MobilityDB/master/ch05s17.html

Todo Checklist

• contains({geo, tgeompoint}, {geo, tgeompoint}): boolean
• containsproperly({geo, tgeompoint}, {geo, tgeompoint}): boolean
• covers({geo, tpoint}, {geo, tpoint}): boolean
• coveredby({geo, tpoint}, {geo, tpoint}): boolean
• crosses({geo, tgeompoint}, {geo, tgeompoint}): boolean
• disjoint({geo, tgeompoint}, {geo, tgeompoint}): boolean
• equals({geo, tgeompoint}, {geo, tgeompoint}): boolean
• intersects({geo, tpoint}, {geo, tpoint}): boolean
• overlaps({geo, tgeompoint}, {geo, tgeompoint}): boolean
• touches({geo, tgeompoint}, {geo, tgeompoint}): boolean
• within({geo, tgeompoint}, {geo, tgeompoint}): boolean
• dwithin({geo, tpoint}, {geo, tpoint}, double): boolean
• relate({geo, tgeompoint}, {geo, tgeompoint}): text
• relate({geo, tgeompoint}, {geo, tgeompoint}, text): boolean

We can make use of cibuildwheel and Github Actions for this, similar to how I implemented in my other project - daylight.

• Publish sdist to PyPI
• Build and publish wheels for Linux (64-bit)

Moved these to seperate issues and out of scope for 0.1.0

• Fix build for 32-bit architectures Moved to #14
• Build and publish wheels for Mac OS Moved to #15
• Build and publish wheels for Windows Moved to #16

Invalid operations should throw accordingly

Refer MobilityDB docs for more information: https://docs.mobilitydb.com/MobilityDB/master/ch05s17.html

Todo Checklist

• tcontains({geo, tgeompoint}, {geo, tgeompoint}): tbool
• tcovers({geo, tpoint}, {geo, tpoint}): tbool
• tcoveredby({geo, tpoint}, {geo, tpoint}): tbool
• tdisjoint({geo, tgeompoint}, {geo, tgeompoint}): tbool
• tequals({point, tgeompoint}, {point, tgeompoint}): tbool
• tintersects({geo, tpoint}, {geo, tpoint}): tbool
• ttouches({geo, tgeompoint}, {geo, tgeompoint}): tbool
• twithin({geo, tgeompoint}, {geo, tgeompoint}): tbool
• tdwithin({geo, tpoint}, {geo, tpoint}, double): tbool
• trelate({geo, tgeompoint}, {geo, tgeompoint}, text): tbool
• trelate({geo, tgeompoint}, {geo, tgeompoint}): ttext

• &&: Do the range values overlap (have points in common)?
• @>: Does the first range value contain the second one?
• <@: Is the first range value contained by the second one?
• -|-: Is the first range value adjacent to the second one?

Invalid operations should throw accordingly

Refer MobilityDB docs for more information: https://docs.mobilitydb.com/MobilityDB/master/ch02.html#time_topo_operators

Todo Checklist

• overlaps
• contains
• contained_by
• adjacent_to

Temporal Types of Sequence and SequenceSet durations can specify interpolation.

For discrete base types, only Stepwise interpolation is allowed.

For continuous base types, only interpolation can be either Stepwise or Linear.

When serialized, the text representation of these objects would start like 'Interp=Stepwise;[10@2020-01-01, ...'

• Sequence
• Sequence Set

Right now when we are returning point data, we return a custom Geometry class. But when it comes to Python ecosystem, it would be great if we can return this data as Shapely objects. This would make the library play nicely with lots of other geospatial processing tools. Although I haven't looked into it how this can be achieved yet - but since pybind11 allows access to external python libraries, I think this should be possible.

Right now only WKT is supported.

Right now MEOS does not support 32 bit architectures. I am not sure if there is any demand for this issue right now, so I might not invest more into this right now. However, putting the details here in case anyone wants to take a stab at it.

How to test in a 32 bit environment

We can spin up a temporary container with 32 bit ubuntu with a volume mapping using:

$ docker run --rm -it -v $(pwd):/app i386/ubuntu:rolling bash

Then we can build and test MEOS within the container

# apt update
# apt install python3-pip git
# cd /app
# python3 -m pip install .
# python3 -m pip install -r test/python/requirements.txt
# pytest

Right now most errors are because of loss of information in datetime, example:

E       AssertionError: assert 'STBOX ZT((11...:00:43+0000))' == 'STBOX ZT((11...:00:00+0000))'
E         - STBOX ZT((11, 12, 13, 2011-01-01T00:00:00+0000), (21, 22, 23, 2011-01-02T00:00:00+0000))
E         ?                       ^ --    -   ^    ^^                     ^ --    ^^  ^    ^^
E         + STBOX ZT((11, 12, 13, 1970-01-13T05:00:43+0000), (21, 22, 23, 1970-01-14T05:00:43+0000))
E         ?                       ^^^      +  ^    ^^                     ^^^     ^^  ^    ^^

• <<: Is the first numeric or range value strictly left of the second one?
• >>: Is the first numeric or range value strictly right of the second one?
• &<: Is the first numeric or range value not to the right of the second one?
• &>: Is the first numeric or range value not to the left of the second one?
• -|-: Is the first numeric or range value adjacent to the second one?
• <<#: Is the first time value strictly before the second one?
• #>>: Is the first time value strictly after the second one?
• &<#: Is the first time value not after the second one?
• #&>: Is the first time value not before the second one?

Invalid operations should throw accordingly

Refer MobilityDB docs for more information: https://docs.mobilitydb.com/MobilityDB/master/ch02.html#time_relpos_operators

Todo Checklist

• always_strictly_less_than
• always_strictly_greater_than
• never_greater_than
• never_less_than
• adjacent_to
• always_before
• always_after
• never_after
• never_before

• <<: Is the first numeric or range value scritly left of the second one?
• >>: Is the first numeric or range value strictly right of the second one?
• &<: Is the first numeric or range value not to the right of the second one?
• &>: Is the first numeric or range value not to the left of the second one?
• -|-: Is the first numeric or range value adjacent to the second one?
• <<#: Is the first time value scrictly before the second one?
• #>>: Is the first time value strictly after the second one?
• &<#: Is the first time value not after the second one?
• #&>: Is the first time value not before the second one?

Invalid operations should throw accordingly

Refer MobilityDB docs for more information: https://docs.mobilitydb.com/MobilityDB/master/ch02.html#time_relpos_operators

Todo Checklist

• always_strictly_less_than
• always_strictly_greater_than
• never_greater_than
• never_less_than
• adjacent_to
• always_before
• always_after
• never_after
• never_before

• &&: Do the bounding boxes overlap?
• @>: Does the first bounding box contain the second one?
• <@: Is the first bounding box contained in the second one?
• ~=: Are the bounding boxes equal in their common dimensions?
• -|-: Are the bounding boxes adjacent?

Invalid operations should throw accordingly

Refer MobilityDB docs for more information: https://docs.mobilitydb.com/MobilityDB/master/apas02.html#idp23

Todo Checklist

• overlaps
• contains
• equal_in_common_dimensions
• adjacent_to

Just wanted to share with you a similar project with C++ core data structures and algorithms combined with Python API: https://github.com/sandialabs/tracktable

This issue is regarding just setting up the basic boilerplate code for docs. Actual content could come later and would of course be constantly improving.

But at a high level, the doc structure could contain:

• API reference for python
• Setup sphinx boilerplate code
• Setup doxygen boilerplate code
• Publish PyMEOS docs on readthedocs, with automatic builds
• Publish doxygen docs on Github Pages, with a Github Action
• Link both in README appropriately

Quick link for docs of sphinx

Python + autodoc

https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html

C++

https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#cpp-domain
https://devblogs.microsoft.com/cppblog/clear-functional-c-documentation-with-sphinx-breathe-doxygen-cmake/

• <<: Are the X values of the first bounding box strictly less than those of the second one?
• >>: Are the X values of the first bounding box strictly greater than those of the second one?
• &<: Are the X values of the first bounding box not greater than those of the second one?
• &>: Are the X values of the first bounding box not less than those of the second one?
• <<: Are the X values of the first bounding box strictly to the left of those of the second one?
• >>: Are the X values of the first bounding box strictly to the right of those of the second one?
• &<: Are the X values of the first bounding box not to the right of those of the second one?
• &>: Are the X values of the first bounding box not to the left of those of the second one?
• <<|: Are the Y values of the first bounding box strictly below of those of the second one?
• |>>: Are the Y values of the first bounding box strictly above of those of the second one?
• &<|: Are the Y values of the first bounding box not above of those of the second one?
• |&>: Are the Y values of the first bounding box not below of those of the second one?
• <</: Are the Z values of the first bounding box strictly in front of those of the second one?
• />>: Are the Z values of the first bounding box strictly back of those of the second one?
• &</: Are the Z values of the first bounding box not back of those of the second one?
• /&>: Are the Z values of the first bounding box not in front of those of the second one?
• <<#: Are the T values of the first bounding box strictly before those of the second one?
• #>>: Are the T values of the first bounding box strictly after those of the second one?
• &<#: Are the T values of the first bounding box not after those of the second one?
• #&>: Are the T values of the first bounding box not before those of the second one?

Invalid operations should throw accordingly

Refer MobilityDB docs for more information: https://docs.mobilitydb.com/MobilityDB/master/ch04s10.html

Todo Checklist

• always_strictly_less_than ("<<")
• always_strictly_greater_than (">>")
• never_greater_than ("&<")
• never_less_than ("&>")
• strictly_to_left ("<<")
• strictly_to_right (">>")
• not_to_the_right_of ("&<")
• not_to_the_left_of ("&>")
• strictly_below ("<<|")
• strictly_above ("|>>")
• does_not_extend_to_left ("&>")
• does_not_extend_to_right ("&<")
• does_not_extend_below ("&<|")
• does_not_extend_above ("|&>")
• strictly_in_front ("<</")
• strictly_in_back ("/>>")
• does_not_extend_in_front ("&</")
• does_not_extend_in_back ("/&>")
• always_before ("<<#")
• always_after ("#>>")
• never_after ("&<#")
• never_before ("#&>")

• +: Union of the range values
• *: Intersection of the range values
• -: Difference of the range values

We can override the respective operators in both C++ and Python
Invalid operations should throw accordingly

Todo Checklist

• TimestampSet, TimestampSet
  • Union
  • Intersection
  • Difference
• Period, Period
  • Union
  • Intersection
  • Difference
• PeriodSet, PeriodSet
  • Union
  • Intersection
  • Difference
• PeriodSet, Period
  • Union
  • Intersection
  • Difference

Sample MobilityDB queries for reference/test cases

mobilitydb=# select intrange '[10, 20)' + '[15, 40)';
 ?column? 
----------
 [10,40)
(1 row)

mobilitydb=# select intrange '[10, 20)' * '[15, 40)';
 ?column? 
----------
 [15,20)
(1 row)

mobilitydb=# select intrange '[10, 20)' - '[15, 40)';
 ?column? 
----------
 [10,15)
(1 row)

mobilitydb=# select intrange '[10, 20)' + '[30, 40)';
ERROR:  result of range union would not be contiguous

The classes TInstantSet and TSequence do have some significant overlap. We can have a common class call TemporalSet to abstract the common logic between them.

Need to fix all the errors and warnings when compiling on the MSVC platform.

Some sample errors and warnings:

  ./source\io\Deserializer.cpp(235): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data
  ./source\io\Deserializer.cpp(254): error C2908: explicit specialization; 'T Deserializer<T>::nextValue(void)' has already been instantiated