istSOS3 is currently not supported and its advancement not foreseen. see istSOS2 is for the current supported version

In this repository you can find the istSOS3 Python3 module. By following the instruction you will be able to run and use the core of istSOS3 on your machine. For full packaged versions please go to (todo: link of the dowload pages).

To install istSOS3, you ca use pip:

pip3 install istsos

Make a fork of the official repository if you think to contribute to the development, then clone your fork to your workstation.

If you want just to make some tests you can clone directly the official repository.

git clone https://github.com/istSOS/istsos3.git

Install all the dependencies

sudo apt-get install postgresql postgis python3-pip

And some Python3 dependencies:

sudo pip3 install lxml \
    jsonschema \
    python-dateutil \
    tornado \
    psycopg2 \
    aiopg \
    pytest

Installa and create a PostgreSQL database named istsos3 and execute the sql file (postgresql_9.4_istsos_3.0.0.sql) that you can find in the istsos3/assets folder.

sudo -u postgres createdb -E UTF8 istsos3
sudo -u postgres psql \
  -d istsos3 -a \
  -f istsos3/istsos/assets/postgresql_9.4_istsos_3.0.0.sql

Set your postgres user password

sudo -u postgres psql
alter user postgres password 'postgres';

Ctrl-D to exit from psql console

Create the configuration file adapting the loader attribute.

cd istsos3
cp config-example.json config.json

Edit your config file:

{
    "type": "aiopg",
    "host": "localhost",
    "port": "5432",
    "user": "postgres",
    "password": "postgres",
    "database": "istsos3"
}

There is a basic server implementation using Townado Web

python3 examples/server_tornado.py

Listening at http://localhost:8887/sos

Lib used: https://docs.pytest.org/en/latest/

[#todo emprove this] For testing in the istsos/test/ folder there same tests that has been implemented.

in the folder istsos/test/actions/servers/sos_2_0_0/ you can find some examples. To run a single test, execute this from the terminal:

pytest -s istsos/test/actions/servers/sos_2_0_0/test_getCapabilitiesOp.py

or

pytest -s istsos/test/actions/servers/sos_2_0_0/test_describeSensorOp.py

[#todo emprove this] You can also do some basic benchmarking with the files in the examples/speed folder.

Maybe we can take a look at this tools: https://github.com/wg/wrk

python curl.py http://localhost/sos

or directly using curl:

curl -s -w '\ntime_namelookup=%{time_namelookup}\ntime_pretransfer=%{time_pretransfer}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\n\n' -o /dev/null "http://the.request?to=test"

In the docs folder there the sphings file that can be used to generate the docs html page.

To build the docs:

    cd docs
    make html

https://about.gitlab.com/features/gitlab-ci-cd/

The tests are executed automatically at each commit on the remote repository. The test are performed inside a docker environment.

It's possible to execute the test locally, to do that follow the steps below:

Add the GPG key for the official Docker repository:

    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Add Docker repo to APT sources:

    sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

Install Docker

    sudo apt-get update
    sudo apt-get install docker-ce

Run docker without sudo

    sudo usermod -aG docker ${USER}
    su - ${USER}

Download the correct deb (gitlab-ci-multi-runner_amd64.deb) from the following uri:

    https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/v9.5.0/deb/gitlab-ci-multi-runner_amd64.deb

Install gitlab-ci-runner:

    sudo dkpg -i gitlab-ci-multi-runner_amd64.deb

Locally you can only run one test at time, the test run only on commited changes.

    cd git-root
    gitlab-runner exec docker [test_name]

The .gitlab-ci.yml file defines sets of jobs with constraints of how and when they should be run. The jobs are defined as top-level elements with a name (in our case rest) and always have to contain the script keyword.

Each job run independently from each other. If a test fails the job will be stopped.

Example:

rest:
    stage: test

    script:
        - pytest -s istsos/test/actions/servers/rest/test_uom.py
        - pytest -s istsos/test/actions/servers/rest/test_observedProperties.py
        - pytest -s istsos/test/actions/servers/rest/test_material.py
        - pytest -s istsos/test/actions/servers/rest/test_method.py
        - pytest -s istsos/test/actions/servers/rest/test_offering.py
        - pytest -s istsos/test/actions/servers/rest/test_specimen.py

[#todo to be improved]

import asyncio

from istsos.application import Server
from istsos.entity.httpRequest import HttpRequest

@asyncio.coroutine
def execute():
    server = yield from Server.create()
    request = HttpRequest(
        "GET",
        "sos",
        parameters={
            "service": "SOS",
            "version": "2.0.0",
            "request": "GetObservation",
            "offering": "T_LUGANO",
            "temporalFilter": (
                "om:phenomenonTime,"
                "2009-01-01T00:00:00+0100/"
                "2009-01-02T00:00:00+0100"
            )
        }
    )
    response = yield from server.execute_http_request(request, stats=True)
    print("\nLoaded %s observations" % len(response['observations']))

loop = asyncio.get_event_loop()
loop.run_until_complete(
   asyncio.gather(execute())
)
loop.close()

Initializing the istSOS Server with a config dictionary. A new config file will be created in the given path (default to config.pickle).

from istsos.application import State

state = State(
    config={
        "proxy": "http://localhost/istsos3/",
        "cache": False,
        "loader": {
            "type": "aiopg",
            "host": "localhost",
            "port": "5432",
            "user": "postgres",
            "password": "postgres",
            "database": "istsos3"
        }
    }
)