Giter VIP home page Giter VIP logo

open-pryv.io's Introduction

Open Pryv.io

Pryv-Logo

Personal Data & Privacy Management Software

A ready-to-use solution for personal data and consent management.

Pryv.io is a solid foundation on which you build your own digital health solution, so you can collect, store, share and rightfully use personal data.

Maintained and developed by Pryv.

Solution

Features

  • Provides latest Pryv.io core system ready for production
  • User registration and authentication
  • Granular consent-based access control rights
  • Data model made for privacy, aggregation and sharing Data in Pryv
  • Full data life-cycle: collect - store - change - delete
  • REST & Socket.io API
  • Ease of software integration and configuration
  • Seamless connectivity and interoperability

Documentation

Summary

  1. Choose your setup
  2. Download the required files / Run the installation scripts
  3. Edit the configuration files
  4. Start the services
  5. Try the API
  6. Customize your platform

Setup

Pryv.io is designed to be exposed by a third party SSL termination such as NGINX.

Choose your Set-up

  • Discover Open Pryv.io on your local environment, this will only allow localhost apps to connect to your platform.
    • Download docker images without SSL (quick start)
    • Download docker images with SSL
    • Native installation
  • Launch Pryv.io on a server exposed to the Internet with built-in SSL, this requires to have a hostname pointing to the public IP of your server.
    • Download docker images (quick start)
    • Native installation
  • Launch Pryv.io on a server with an external SSL termination. You know what you are doing.
    • Download docker images
    • Native installation

Docker

The dockerized versions and their instructions are available at this link: Download link.

If you wish to build the images yourself, refer to the following README: docker/README-build.md.

Once it is running, you can continue with the tutorials.

Native

Prerequisites:

The installation script has been tested on Linux Ubuntu 18.04 LTS and MacOSX.

  1. npm run setup-dev-env to setup local file structure
  2. npm install to install node modules

Native setup with no SSL

setup the environment

Each service independently - logs will be displayed on the console

  • npm run database start mongodb or start ferretDB
  • npm run api start the API server on port 3000 (default)
  • npm run mail start the mail service

Local native setup with backloop.dev loopback SSL

backloop.dev certificates facilitate local developpment by enabling https on localhost.

setup the environment

  • npm run database to start mongodb or start ferretDB
  • (optional) npm run mail start the mail service
  • npm run apibackloop to start api server using configs/api-backloop.yml

You can now access you API from you own computer with SSL on

  • https://my-computer.backloop.dev:4443

You can check by opening https://my-computer.backloop.dev:4443/reg/service/info

And create new users or access token from the Pryv Access Token Generation Page

Native setup with custom SSL

setup the environment

  1. Edit http:ssl part in ./configs/api.yml file to point to your certificates an key files.
  2. Update dnsLess:publicUrl in ./configs/api.yml to match
  3. Run npm run pryv to start the API

Config

For the native installation, edit ./configs/api.yml

dnsLess:
  publicUrl: http://localhost:3000
http:
  port: 3000
  ip: 127.0.0.1
auth:
  adminAccessKey: REPLACE_ME 
  trustedApps: "*@https://pryv.github.io*, *@https://*.rec.la*, *@https://*.backloop.dev*"
eventFiles:
  attachmentsDirPath: var-pryv/attachment-files
service:
  name: Open-Pryv.io
  support: https://pryv.com/open-pryv-non-configured-page/
  terms: https://pryv.com/open-pryv-non-configured-page/
  home: https://pryv.com/open-pryv-non-configured-page/
  eventTypes: https://pryv.github.io/event-types/flat.json
services:
  email:
    enabled:
      welcome: true
      resetPassword: true
  • publicUrl Is the "Public" URL to reach the service, usually exposed in https by a third party SSL service such as NGNIX.
  • http
    • port The local port to listen
    • ip The IP adress to use. Keep it 127.0.0.1 unless you explicitely want to expose the service in http to another network.
  • auth
    • adminAccesskey key to use for system calls such as /reg/admin/users. A random key should be generated on setup.
    • trustedApps list of web apps that can be trusted-app functionalities API for trusted apps: API reference see: SETUP Guide - customize authentication
  • service API documentation on Service Information
  • services:email see Options & Customization below

Start

At this moment you should have your application running on the public URL you defined.

If you are using another public URL, replace https://my-computer.backloop.dev:4443 by it in the link above.

Design your Data Model

Data in Pryv is stored in streams and events. We provide you with all necessary information to design your own data model in our Data Modelling Guide through a broad range of use cases and scenarios you might encounter.

Try the API

After this process, you should have an account on your Open Pryv.io platform with a valid authorization token in the form of an API endpoint, you can try various API requests using Postman following this guide https://pryv.github.io/open-api/.

You can also try our example apps with guides and tutorials.

Options & Customization

Authentication & Registration web app.

Open Pryv.io comes packaged with app-web-auth3, the default web pages for app authentication, user registration and password reset.

During the set-up process it has been built and published in public_html/access/. To customize it, refer to its README in app-web-auth3/.

To use a new build, simply copy the contents of the generated files from app-web-auth3/dist/ to public_html/access/

Event types

Open Pryv.io comes with default event types. The default ones are fetched at boot from the URL defined in service:eventTypes in the .yml config file, set to https://pryv.github.io/event-types/flat.json.

To customize your own, clone the Data Types repository and follow the guide there.

MongoDB data folder

By default the MongoDB data are stored in var-pryv/mongodb-data. If you want to modify the folder where the MongoDB data files are stored, read ferretDB README.

FerretDB data folder

By default the FerretDB data are stored in var-pryv/ferretdb-data. If you want to modify the folder where the FerretDB data files are stored, you can modify in scripts/setup-mongodb the variable MONGO_DATA_FOLDER.

Visual assets and icons

Your platforms visuals can be customized in public_html/assets/, please refer to the README inside. These assets are a clone of the assets-open-pryv.io.

E-Mails

Pryv.io can send e-mails at registration and password reset request.

The emails can be sent either by local sendmail (default) or SMTP.

This service, its documentation and mail templates can be found in service-mail/.

Backup

Prerequisites:

  • rsync

To make a backup of your data:

Backup: native

Run ./scripts/backup-database-native ${BACKUP_FOLDER} to generate a dump of the current database contents Run ./scripts/backup-usersfiles-native ${BACKUP_FOLDER} to copy the current usersfiles files.

To restore the database, run ./scripts/restore-database-native ${BACKUP_FOLDER} to restore data from the provided backup folder. To restore attachments, run ./scripts/restore-usersfiles-native ${BACKUP_FOLDER} to restore data from the provided backup folder. Depending on your setup, you may need additional access rights.

Backup: dockerized

Follow the guide in the docker package.

Contributing

Contributions are welcome. Get in touch with the Pryv team or submit a PR with your changes or adaptation.

License

Copyright (c) 2019-2023 Pryv S.A. https://pryv.com

This file is part of Open-Pryv.io and released under BSD-Clause-3 License

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

  3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

SPDX-License-Identifier: BSD-3-Clause

License

BSD-3-Clause

open-pryv.io's People

Contributors

ievanavikiene avatar ievavaisnoraitenavikiene avatar kebetsi avatar perki avatar pryv-tech avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

open-pryv.io's Issues

Ability to use Pryv in tests

I'd like to use Pryv in kind of integration tests and therefore I need somehow to run each test in isolated scope, e.g under a transaction that I can start before and rollback after each test, or just an API endpoint that I can use to erase all data safely. Is it possible right now? Right now I see the following way only:

Connect to mongo DB directly and remove all collections via:

mongo [database] --eval "db.getCollectionNames().forEach(function(n){db[n].remove()});"

What would you recommend?

Partially missing error explanation in user registration response

When both username and email are registered, the email member is missing from the error explanation, only the username problem described.

Scenarios:

if only the username reserved, the response is:

{
   "error":{
      "id":"item-already-exists",
      "message":"An user with username 'gabortest001' already exists",
      "data":{
         "username":"gabortest001"
      }
   },
   "meta":{
      "apiVersion":"1.6.2-2-open",
      "serverTime":1607330501.756,
      "serial":"t1607325000"
   }
}

if the email only:

{
   "error":{
      "id":"item-already-exists",
      "message":"An user with email \"[email protected]\" already exists",
      "data":{
         "email":"[email protected]"
      }
   },
   "meta":{
      "apiVersion":"1.6.2-2-open",
      "serverTime":1607330390.87,
      "serial":"t1607325000"
   }
}

if both are registered

{
   "error":{
      "id":"item-already-exists",
      "message":"An user with username \"gabortest001\" already exists",
      "data":{
         "username":"gabortest001"
      }
   },
   "meta":{
      "apiVersion":"1.6.2-2-open",
      "serverTime":1607331212.625,
      "serial":"t1607325000"
   }
}

Probably it designed this way but because the data property holds an object it may contain the email too, to indicate to all problems for the frontend.

What I expected in the data member (regardless of the message):

{
   "error":{
      "id":"item-already-exists",
      "message":"An user with username 'gabortest001' already exists",
      "data":{
         "username":"gabortest001",
         "email": "[email protected]"
      }
   },
   "meta":{
      "apiVersion":"1.6.2-2-open",
      "serverTime":1607330501.756,
      "serial":"t1607325000"
   }
}

open-pryv.io / Docker / Windows

Hello all,

I am trying to dockerize an open-pryv.io to be able to deploy it on Windows machines (and Openshift later) but I am currently facing difficulties with the access to the server API.

Actually, when I run the image on my local machine :
docker run -p 3000:3000 open-pryv.io:v0.1, the container runs successfully.

I am able to connect to the container send requests and get responses:

~ $ docker exec -it be4dd3879d42 bash
root@be4dd3879d42:/open-pryv.io# wget 127.0.0.1:3000
--2020-06-25 14:30:50-- http://127.0.0.1:3000/
Connecting to 127.0.0.1:3000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 150 [application/json]
Saving to: 'index.html'
[...]

But outside the container, the same call fails (directly in Windows):

~ $ wget 127.0.0.1:3000
--2020-06-25 15:33:22-- http://127.0.0.1:3000/
Connecting to 127.0.0.1:3000... connected.
HTTP request sent, awaiting response... No data received.
Retrying.

I tried to access it in many ways (changing port, investing the exposing IP, changing client) but the result is always the same.

Would you have best practices to share about it ?


For information, I used the following Docker file content to build my image:

FROM node:12.13.1

RUN apt-get update
RUN apt-get -y install rsync

RUN npm install -g yarn

COPY . /open-pryv.io
WORKDIR /open-pryv.io

RUN npm install -g yarn

RUN yarn setup
RUN yarn release

RUN apt-get install nano

EXPOSE 3000

CMD ["yarn", "local"]
#CMD ["yarn", "pryv"]


Thanks in advance for your support and kind regards,
Sébastien

Socket.io connection path does not follow Documentation

We noticed that to initialize a socket.io connection, Open pryv.io accept the following format:

https://{hostname}/{username}?auth={token}&resource={username}

Instead of

https://{hostname}/{username}/{username}?auth={token}&resource={username}

as per the documentation

Pryv's team is working on a fix..

Incorrect status code for existing items

Hi there

In the case of POSTing an existing item (e.g. register a user) the server response 400. It seems to be incorrect since by MDN:

...to be a client error (e.g., malformed request syntax, 
invalid request message framing, or deceptive request routing).

In this scenario, the request is formed properly so definitely not a bad request. 409 conflict maybe?

happens here:

Thanks,

Migrating to 1.9

Hello there,

We are trying to upgrade from 1.7.1 to 1.9.0.

The result of the migration seems ok:

db.versions.find()
{
  _id: '1.7.1',
  initialInstall: 1708090032.24
}
{
  _id: '1.7.5',
  migrationStarted: 1708090032.247,
  migrationCompleted: 1708090032.251
}
{
  _id: '1.8.0',
  migrationStarted: 1708090032.252,
  migrationCompleted: 1708090033.201
}
{
  _id: '1.9.0',
  migrationStarted: 1708090033.202,
  migrationCompleted: 1708090033.22
}

Now it looks like users are not available anymore.

For example:

  • the admin/getUsers (GET {register}/admin/users) returns an empty array
  • the previously working login information now fails (Unknown user)

Do you see what the problem could be? (the migrations to 1.9 clearly deletes the system/username events and seems to move them to an sqlite DB but those DBs seem empty in our case).

Is there an additional migration step for this version?

Thanks,
Stéphane.

Pryv docker cannot start

I downloaded latest docker archive to run pryv in a local environment, edited the dockerized-config.yml file as stated in the readme, but the open-pryv.io container fails to start and enters an infinite restart loop. The following error is displayed:

open-pryv.io     |   value: {
open-pryv.io     |     adminAccessKey: 'REPLACE_ME',
open-pryv.io     |     trustedApps: '*@https://pryv.github.io, *@https://*.rec.la*'
open-pryv.io     |   },
open-pryv.io     |   scope: 'base',
open-pryv.io     |   info: 'From file: ./config.json'
open-pryv.io     | }

But the file was correctly modified and I set another value for adminAccessKey.

Error when starting version 1.7.1-6-open on a mongo 4.4 version

The error (warning?) message is:

2022-09-21 13:24:44 [error]: [api-server:Application:error-middleware]: Unhandled API error (MongoError): Invalid command argument. Expected '4.4' or '4.2', found 4.0 in: { setFeatureCompatibilityVersion: "4.0", lsid: { id: UUID("209d35ce-6032-4158-99b9-0620426af71f") }, $db: "admin" }. See https://docs.mongodb.com/master/release-notes/4.4-compatibility/#feature-compatibility.
MongoError: Invalid command argument. Expected '4.4' or '4.2', found 4.0 in: { setFeatureCompatibilityVersion: "4.0", lsid: { id: UUID("209d35ce-6032-4158-99b9-0620426af71f") }, $db: "admin" }. See https://docs.mongodb.com/master/release-notes/4.4-compatibility/#feature-compatibility.
    at MessageStream.messageHandler (/app/dist/node_modules/mongodb/lib/cmap/connection.js:299:20)
    at MessageStream.emit (node:events:390:28)
    at processIncomingData (/app/dist/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/app/dist/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at writeOrBuffer (node:internal/streams/writable:389:12)
    at _write (node:internal/streams/writable:330:10)
    at MessageStream.Writable.write (node:internal/streams/writable:334:10)
    at Socket.ondata (node:internal/streams/readable:754:22)
    at Socket.emit (node:events:390:28)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Socket.Readable.push (node:internal/streams/readable:228:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:199:23) 
}

The consequences are unclear though...

The error is there (this should probably not be hardcoded and go to the config, no?):
components/storage/src/Database.js:152: client.db('admin').command({ setFeatureCompatibilityVersion: "4.0" }, {}, callback);

Cheers,
Stéphane.

Ability to specify configuration via environment variables using docker

I'm using docker-compose based Pryv installation, and it would be great to have the ability to specify some configuration via environment variables instead of mounting the config file. E.g:

services:
  open-pryv.io:
    image:  pryvio/open-pryv.io:latest
    command: "yarn api"
    container_name: "open-pryv.io"
    environment:
        DNSLESS_PUBLIC_URL:  http://localhost:3000
        AUTH_ADMIN_ACCESS_KEY:  CHANGE_ME
        AUTH_TRUSTED_APPS: "*@https://pryv.github.io, *@http://*localhost*"
        DATABASE_HOST: open-pryv-mongo
        DATABASE_PORT: 27017
        DATABASE_NAME: pryv-node
    ports:
        - 3000:3000
    depends_on:
        - open-pryv-mongo
    networks:
        - app-network

Is it possible?

Error on connection page

Hello everybody!
Thanks for the open source initiative!

We tried to install Open Pryv on docker to test the plateform. Unfortunately we have a problem when we try to access the login page at http://pryv-address/www/assets/access
The form is displayed and it is possible to create an account. But when we click on "Login" we have an error.
The console says:

TypeError: "t.accessState is undefined"
    e AccessStates.js:7
    w runtime.js:62
    _invoke runtime.js:296
    e runtime.js:114
    i asyncToGenerator.js:17
    default asyncToGenerator.js:35
    e _export.js:36
    default asyncToGenerator.js:14
    H login.js:9
    s controller.js:21
    w runtime.js:62
    _invoke runtime.js:296
    e runtime.js:114
    i asyncToGenerator.js:17
    default asyncToGenerator.js:35
    e _export.js:36
    default asyncToGenerator.js:14
    e Authorization.vue:129
    w runtime.js:62
    _invoke runtime.js:296
    e runtime.js:114
    i asyncToGenerator.js:17
    default asyncToGenerator.js:35
    e _export.js:36
    default asyncToGenerator.js:14
    submit Authorization.vue:124
    VueJS 4
    click vuetify.js:1391
    VueJS 3
AppError.js:37:12
    value AppError.js:37
    e AppError.js:33
    s controller.js:26
    w runtime.js:62
    _invoke runtime.js:296
    e runtime.js:114
    i asyncToGenerator.js:17
    i asyncToGenerator.js:30

Thank you in advance for any tip

Issue with creating access with 'create-only' level.

Hello everyone.

I'm writing here, because there is issue with creation access with 'create-only' level. For others levels it works fine, but for 'create-only' I get next error:

{
  'error': {
    'id': 'invalid-parameters-format',
    'message': "The parameters' format is invalid.",
    'data': [
      {
        'code': 'ANY_OF_MISSING',
        'params': [
          
        ],
        'message': "Data does not match any schemas from 'anyOf'",
        'path': '#/',
        'schemaId': 'pryv.core.access-create',
        'inner': [
          {
            'code': 'OBJECT_ADDITIONAL_PROPERTIES',
            'params': [
              [
                'expireAfter'
              ]
            ],
            'message': 'Additional properties not allowed: expireAfter',
            'path': '#/'
          },
          {
            'code': 'ONE_OF_MISSING',
            'params': [
              
            ],
            'message': "Data does not match any schemas from 'oneOf'",
            'path': '#/permissions/0',
            'inner': [
              {
                'code': 'ENUM_MISMATCH',
                'params': [
                  'create-only'
                ],
                'message': 'No enum match for: create-only',
                'path': '#/permissions/0/level',
                'schemaId': 'streamPermission'
              },
              {
                'code': 'OBJECT_MISSING_REQUIRED_PROPERTY',
                'params': [
                  'tag'
                ],
                'message': 'Missing required property: tag',
                'path': '#/permissions/0',
                'schemaId': 'tagPermission'
              }
            ]
          },
          {
            'code': 'ENUM_MISMATCH',
            'params': [
              'app'
            ],
            'message': 'No enum match for: app',
            'path': '#/type'
          }
        ]
      }
    ]
  },
  'meta': {
    'apiVersion': '1.4.23',
    'serverTime': 1597152581.771,
    'serial': '2019120401'
  }
}

Please let me know how it can be fixed. Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.