Hosting core api with web app on localhost about keeper-web-app HOT 27 CLOSED

Hello,
I'am verry sorry, but I don't really understand what you're trying to do.
You have installed the API and Webapp on your server using localhost as URL (on different ports or with a reverse proxy in front of them I guess?). But you still want to use login.nunux.org as authentication portal? Is that correct?

Did you give a try to this sample installation: https://github.com/nunux-keeper/keeper-docker ?

Regards

from keeper-web-app.

Hello,

Thanks for your response.

Yes you guess the right. I have installed API and Webapp on local server with different ports and i can't figure out how to authenticate whether login.nunux.org or withe localhost to access api on local server.

Yes I have tried keeper-docker it's working fine on localhost but using this I can't customize code for webapp and API for some changes.

I will appreciate your help.

Thanks.

from keeper-web-app.

OK.
Using login.nunux.org will not work because the client configuration only allows https://app.nunux.org/*` as redirect URL.
I recommend that you deploy your own Keycloak instance.
Configuring a Keycloak tenant is not that hard (if you already know Keycloak ;)).
You can do it using the admin console or with the API.
This script configures a Keycloak instance: https://github.com/nunux-keeper/keeper-docker/blob/master/dockerfiles/keycloak/setup.sh

The goal at this point is to provides:

• The Web App the client configuration (keycloak.json file)
• The API server the public key used to validate the access token (Env var: APP_TOKEN_PUB_KEY)

So to get this you need to create a tenant and a client inside your Keycloak instance:

You have to create a client for the web app:

• public client: true
• base URL: http://localhost:xxx
• redirect URIs: http://localhost:xxx/*
• web origins: +

With xxx your web app port. Instead using port I recommend you to use a reverse proxy and configure a context path: http://localhost/app for the app, http://localhost/api for the API, etc.
Note that redirect URI is given to your browser so if you are using a FQDN you should set your FQDN here.

Once your tenant created (with roles and users) you have to provide the configuration to the web app.
The web app needs a keycloak.json file. You can get it from the installation tab of the client configuration or by using the API.
Replace the following file into the repo if you build the web site using the build chain. Or simply replace the file at the root dir of the final web site.

For the API server side you have to provide the public key used to sign the access token.
You can get this key form the keycloak.json file or using the web console of keycloak or using the API.
You also have to set the URL of the Real to the API: APP_AUTH_REALM

Once this tricky part done you should be able to authenticate yourself using your own Keycloak server.

Hopefully it will help :/

from keeper-web-app.

Hello,

Thanks for your reply,

I have configured Keycloak server and able to authenticate and redirect to the app in localhost but when i call any API it gives unauthorized error.
I also tried from postman.

from keeper-web-app.

On API side you should have this kind of configuration(environment variables) :

• APP_AUTH_REALM=https://keycloak-url/auth/realms/nunux-keeper
  URL of the keycloak Realm (only used as information for API clients)
• APP_TOKEN_PUB_KEY=./var/cert/staging-pub.pem
  Path of the Keycloak client public key (PEM format) (extracted form keycloak.json file)
• APP_ALLOW_AUTO_CREATE_USERS=true
  To allow users to be created as soon as JWT is validated

Can you check that ?

from keeper-web-app.

Yes I have done these settings. here are the console outputs:

You can also notice that the API gets called twice first time with status 200 OK and second time 401 (Unauthorized)

from keeper-web-app.

Hello,
Actually this is the part of my research here in university lab but I am unable to resolve these configuration problems. I will appreciate if you help me to setup this project.
Thanks...

from keeper-web-app.

The first API call is an OPTION HTTP request to check that CORS is allowed (because API is not on the same URL than web app). So it's normal to have a 200 response. The true request is the next one. Can you provides HTTP Headers of an API request? The request should contains a cookie access_token=xxx or a HTTP header Authorization: Bearer xxx. The xxx part is the access token. You can decode this access token by using this website: https://jwt.io/. You should be able to see if the token is valid (for instance exp is the expiration date of the token, iss the issuer, aud the client name, ). You can also validate the signature of the token by putting the public key (the exact same key you should provide to the API server). Can you share the content of this token (without personal data!)?

BTW can you tell me which version of Keycloak you are using?
I ask that because the Web App use the keycloak.js of my Keycloak instance: https://github.com/nunux-keeper/keeper-web-app/blob/master/public/index.html#L20
This can lead to some trouble if your instance version is not the same that mine: 3.4.0.Final

from keeper-web-app.

Hello,

I am using keycloak-4.5.0.Final version.

The request is sent in a similar way, I also tried this from Postman but the same error.

{
"error": {
"name": "JsonWebTokenError",
"message": "invalid signature"
}
}

The header for (http://localhost:9000/v2/documents?from=0&size=20&order=desc) is something like:

General:
Request URL: http://localhost:9000/v2/documents?from=0&size=20&order=desc
Request Method: GET
Status Code: 401 Unauthorized
Remote Address: [::1]:9000
Referrer Policy: no-referrer-when-downgrade

Response Header
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Content-Type, Authorization, Content-Length, X-Requested-With
Access-Control-Allow-Methods: GET,PUT,POST,DELETE,OPTIONS
Access-Control-Allow-Origin: http://localhost:3000
Connection: keep-alive
Content-Length: 68
Content-Type: application/json; charset=utf-8
Date: Tue, 02 Oct 2018 03:07:39 GMT
ETag: W/"44-XE9I4Eov2V0u0jPwEUOEavDctfc"
Vary: Accept-Encoding

Request Headers
Accept: application/json
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJPcVpNcWJjU0ZIRkFXWDctUGdES01oMFpwZGxYTW1uV0E5RmNwMDNCX2ZRIn0.eyJqdGkiOiIyN2JiOWVkNi02ODM3LTRmYmQtOTU4Ni1iMTU4Mzk3NjEyY2UiLCJleHAiOjE1Mzg0NDk5NTksIm5iZiI6MCwiaWF0IjoxNTM4NDQ5NjU5LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvbnVudXgta2VlcGVyIiwiYXVkIjoia2VlcGVyLWFwcCIsInN1YiI6IjVhOGM2ZDlkLTAxNGItNGNlOS05ZGI0LTQ0YzdlMDhjZTFjZiIsInR5cCI6IkJlYXJlciIsImF6cCI6ImtlZXBlci1hcHAiLCJub25jZSI6IjdhZWRhMDIwLTlkYWQtNDlkMy1iMDE0LTkxN2ZjNjNiY2U4NyIsImF1dGhfdGltZSI6MTUzODQ0OTYyMSwic2Vzc2lvbl9zdGF0ZSI6IjkyZDZmYjc2LTBmMDktNGE2NC04YzUyLTgwMDIwM2UxNTY5NSIsImFjciI6IjAiLCJhbGxvd2VkLW9yaWdpbnMiOlsiaHR0cDovL2xvY2FsaG9zdDozMDAwIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsImFkbWluIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6Im9wZW5pZCBlbWFpbCBwcm9maWxlIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJtdXphbWlsIn0.USKr2X4bSeUC-DZhCIpkjBW5YCrEZOBR7WLCggln9VR9N-EAuyBjOaQ3OUq3gNf52QE-zy1ZhPCAppXbcyMTpQDKxLVmT7cjsyrtZcbOVpWxzerJ3Xc8jbmgpIHBjZBoA_mNJMbiCtUwCh0pvPZNVZnNRehfZgQioqyfFtU6sSQWvq72Difcbw-j9BG3Ft1xTgpTTKBeX9XQDyMNUNNCg9JCR_yelSTQ_VVUqNxJQuh4QxG0D80_1OPBDuRtT0I_dp7kPgM0zA0AERHsbKXK4WDQwIoZMbtQMqi4Sq4Z8GwkLbNi4BvDeoZaUnotXMDlJx0e_yG1I_Ez8assADOLGg
Connection: keep-alive
Cookie: KEYCLOAK_STATE_CHECKER=ms5XHwXUoLYw1raP8UKjUJAvuDxxcQwJJW5XIsqRQoc
Host: localhost:9000
Origin: http://localhost:3000
Referer: http://localhost:3000/documents
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36

Query String parameter
from: 0
size: 20
order: desc

I decode my access token using JWT Tool tool, the exp, iss, and aud look OK but it also says "Invalid signature".

from keeper-web-app.

OK. That's a good news because that means that the pb is only related to the public key.
Can you post your http://localhost:3000/keycloak.json file AND the file targeted by the APP_TOKEN_PUB_KEY env var of the API?

from keeper-web-app.

APP_TOKEN_PUB_KEY=./var/cert/staging-pub.pem

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEApp9HXQdHXNfyhpK4xJldcNO3+KitOxxv6p35B07d4XqrLGvvrtDDHhuMMW4c50vkZT8vBSatvlD4koBqNavb3KF5ZodGM+JB4/M5Ci29IJYRD+CtFmrZQyK8eDzxcNCbhfOaoipYgahrLU7L7EImxpdo2YqIbRSdOwm0ssXvYAx3MMGs4/J6pJUBptDY22A6QvxPKYORWjImQDk+s/G+lfrONxHMIqczgNayErzjNjDyEIayzweGqu+YC7hasrqjF547sIMM+XsBIaxAIQ/9g0K96WAdqIRc3wF0HCFRJoWlS9/iilnrdGRIL6zCCrIgPDkwtwzPnJUvc13yi8TN1QIDAQAB
-----END PUBLIC KEY-----

keycloak.json
{
"realm": "nunux-keeper",
"realm-public-key": "-----BEGIN PUBLIC KEY-----\neyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEApp9HXQdHXNfyhpK4xJldcNO3+KitOxxv6p35B07d4XqrLGvvrtDDHhuMMW4c50vkZT8vBSatvlD4koBqNavb3KF5ZodGM+JB4/M5Ci29IJYRD+CtFmrZQyK8eDzxcNCbhfOaoipYgahrLU7L7EImxpdo2YqIbRSdOwm0ssXvYAx3MMGs4/J6pJUBptDY22A6QvxPKYORWjImQDk+s/G+lfrONxHMIqczgNayErzjNjDyEIayzweGqu+YC7hasrqjF547sIMM+XsBIaxAIQ/9g0K96WAdqIRc3wF0HCFRJoWlS9/iilnrdGRIL6zCCrIgPDkwtwzPnJUvc13yi8TN1QIDAQAB\n-----END PUBLIC KEY-----\n",
"auth-server-url": "http://localhost:8080/auth",
"ssl-required": "external",
"resource": "keeper-app",
"public-client": true,

"confidential-port": 0
}

from keeper-web-app.

This is strange! The public key of the API side is OK and validate the JWT correctly!
BUT the realm-public-key of the keycloak.json is not correct! There is \n chars and the key begin with eyMII instead of MII. It seems to be a pb on the Keycoak side. However, JWT signature is correct and validated by the API public key. You should be able to pass. I think that the API server don't load correctly the APP_TOKEN_PUB_KEY. Are you sure that this variable env is transmitted to the process?
If you can update the code I suggest you to add a console.log(key) here: https://github.com/nunux-keeper/keeper-core-api/blob/master/src/middleware/auth_jwt.middleware.js#L18 to see which key is used.

from keeper-web-app.

The public key in API and keycloak.json are the same and eyMII just got when copying to the text file.

Well, this file https://github.com/nunux-keeper/keeper-core-api/blob/master/src/middleware/auth_jwt.middleware.js doesn't get called, Even I tried to comment the whole file it's still the same.

from keeper-web-app.

Well that means you are not running the code (and the configuration) you expect to run. How do you start your API server?

from keeper-web-app.

Hello,

I am using this command to start the API server:
sudo make - with-app deploy logs.

I guess, running with above command pull core-api and start in a docker container. Any changes in our source code are not reflecting.

Can you tell me any workaround to start it without docker?

from keeper-web-app.

Ok this is the explanation!
You can build your own Docker image using make build into the keeper-core-api project. Then the make deploy will use the local Docker image. And you should see your modifications.

OR, you can even run the API without Docker by using directly Node.js:

$ # Load configuration
$ source ./etc/default/staging.env
$ # Intall dependencies
$ npm install
$ # Start the API server
$ npm start

If you don't use Docker Compose you will have to start Redis, MongoDB and ElasticSearch separately.

from keeper-web-app.

Getting this error now

from keeper-web-app.

Your APP_SEARCH_ENGINE_URI is not correct. The scheme should be elasticsearch not http.
Ex: APP_SEARCH_ENGINE_URI=elasticsearch://localhost:9200/keeper

from keeper-web-app.

Hello,

I tried to run using Docker build and deploy
but the API server doesn't start and API container is also not created. Here is the screenshot:

Then I tried with node:
I started Mongo, Elastic and Redis services but when we do npm start it gives me following error:

Here is my staging.env file:

###
# Development configuration
###

# Env
# Defaults: development
# NODE_ENV=development

# Server port
# Defaults: 3000
APP_PORT=9000

# Server base URL
# Defaults: http://localhost:3000
#APP_BASE_URL=https://api.nunux.org/keeper
APP_BASE_URL=http://localhost:8080

# Auth server base URL
# Default: none
APP_AUTH_REALM=http://localhost:8080/auth/realms/nunux-keeper

# Application logger level
# Defaults: debug
APP_LOG_LEVEL=debug

# Database URI
# Defaults: mongodb://localhost/keeper
APP_DATABASE_URI=mongodb://localhost/keeper
#APP_DATABASE_URI=elasticsearch://elasticsearch:9200/keeper

# Search engine URI
# Default: elasticsearch://localhost:9200/keeper
APP_SEARCH_ENGINE_URI=elasticsearch://localhost:9200/keeper

# Redis server
# Defaults: redis://localhost:6379/0
APP_REDIS_URI=redis://localhost:6379/0

# Stats server URI
# Defaults: none
APP_STATS_SERVER_URI=statsd://telegraf:8125

# Allow login to auto create users
# Default: false
APP_ALLOW_AUTO_CREATE_USERS=true

# Allow admin to remove users
# Default: false
APP_ALLOW_REMOVE_USERS=true

# Token secret
APP_TOKEN_SECRET=xyKFaqKHLG
#APP_TOKEN_PUB_KEY=./var/cert/test-pub.pem
#APP_TOKEN_PRIV_KEY=./var/cert/test.pem
APP_TOKEN_PUB_KEY=./var/cert/staging-pub.pem
#APP_TOKEN_PRIV_KEY=./var/cert/staging.pem

# Jobs to handle (comma separated list)
# Defaults: all
#APP_JOBS=export-user,import-user,download,ghostbuster,rebuild-index

# Embedded job worker
# Should not be used in production.
# Defaults: false
APP_EMBEDDED_WORKER=true

# Download document's resources
# Default: default
# Values:
# - disabled: no resource downloaded
# - async: async download using queuing system
# - default: direct download
APP_DOWNLOADER=default

# Storage backend
# Default: local
# Values:
# - local: local file system
# - s3: S3 object storage (UNSTABLE)
APP_STORAGE=local

# Local storage directory
# Default: ./storage
APP_STORAGE_LOCAL_DIR=/var/opt/app/storage


#APP_STORAGE_S3_HOST=fr-1.storage.online.net
#APP_STORAGE_S3_BUCKET=keeper
#APP_STORAGE_S3_ACCESS_KEY=###
#APP_STORAGE_S3_SECRET_KEY=###

# Global event broker for some app events
# Events: user.create, user.remove
# Defaults: disabled
#APP_EVENT_BROKER_URI=https://test:[email protected]/keeper-events

# Initial cleint registration asccess token
# Default: none
#APP_CLIENT_INITIAL_ACCESS_TOKEN=###

And here is my dev.env file:

###
# Development configuration
###

# Env
# Defaults: development
NODE_ENV=development

# Server port
# Defaults: 3000
APP_PORT=9000

# Server base URL
# Defaults: http://localhost:3000
#APP_BASE_URL=https://api.nunux.org/keeper
APP_BASE_URL=http://localhost:8080

# Auth server base URL
# Default: none
APP_AUTH_REALM=http://localhost:8080/auth/realms/nunux-keeper

# Application logger level
# Defaults: debug
APP_LOG_LEVEL=debug

# Database URI
# Defaults: mongodb://localhost/keeper
APP_DATABASE_URI=mongodb://localhost:27017/keeper
#APP_DATABASE_URI=elasticsearch://elasticsearch:9200/keeper

# Search engine URI
# Default: elasticsearch://localhost:9200/keeper
APP_SEARCH_ENGINE_URI=elasticsearch://localhost:9200/

# Redis server
# Defaults: redis://localhost:6379/0
APP_REDIS_URI=redis://localhost:6379/0

# Stats server URI
# Defaults: none
APP_STATS_SERVER_URI=statsd://telegraf:8125

# Allow login to auto create users
# Default: false
APP_ALLOW_AUTO_CREATE_USERS=true

# Allow admin to remove users
# Default: false
APP_ALLOW_REMOVE_USERS=true

# Token secret
APP_TOKEN_SECRET=xyKFaqKHLG
#APP_TOKEN_PUB_KEY=./var/cert/test-pub.pem
#APP_TOKEN_PRIV_KEY=./var/cert/test.pem
APP_TOKEN_PUB_KEY=./var/cert/staging-pub.pem
#APP_TOKEN_PRIV_KEY=./var/cert/staging.pem

# Jobs to handle (comma separated list)
# Defaults: all
#APP_JOBS=export-user,import-user,download,ghostbuster,rebuild-index

# Embedded job worker
# Should not be used in production.
# Defaults: false
APP_EMBEDDED_WORKER=true

# Download document's resources
# Default: default
# Values:
# - disabled: no resource downloaded
# - async: async download using queuing system
# - default: direct download
APP_DOWNLOADER=default

# Storage backend
# Default: local
# Values:
# - local: local file system
# - s3: S3 object storage (UNSTABLE)
APP_STORAGE=local

# Local storage directory
# Default: ./storage
APP_STORAGE_LOCAL_DIR=/var/opt/app/storage


#APP_STORAGE_S3_HOST=fr-1.storage.online.net
#APP_STORAGE_S3_BUCKET=keeper
#APP_STORAGE_S3_ACCESS_KEY=###
#APP_STORAGE_S3_SECRET_KEY=###

# Global event broker for some app events
# Events: user.create, user.remove
# Defaults: disabled
#APP_EVENT_BROKER_URI=https://test:[email protected]/keeper-events

# Initial cleint registration asccess token
# Default: none
#APP_CLIENT_INITIAL_ACCESS_TOKEN=###

from keeper-web-app.

Hello,

I have created a new docker image and deploy the app with that image but now this error is coming.
can you please identify this error:

from keeper-web-app.

Hello,
the module when is not installed.
This means that you have not installed dependencies into your Docker image.
Run npm install into your Dockerfile or use the 'on-build' official node image.

from keeper-web-app.

I tried this but still no luck.

Can you please make a repository of independent project or provide proper steps and documentation for up and running this Curation tool on localhost.

from keeper-web-app.

That is the very purpose of this project: https://github.com/nunux-keeper/keeper-docker

from keeper-web-app.

Yes that project works fine but I could not customize it’s code.

from keeper-web-app.

You should be able to:

$ # Go to the API project
$ cd keeper-core-api
$ # ... do your code update and build a new image
$ make build
$ # Go back to docker project
$ cd ../keeper-docker
$ # And deploy the stack
$ make deploy

The stack will use your local updated image.

In order to be sure to use your custom image you can even change the image version (latest ) by something like custom. And build the API using this command: docker build --rm -t ncarlier/keeper-core-api:custom .
Then change the compose file (https://github.com/nunux-keeper/keeper-docker/blob/master/docker-compose.app.yml#L7) to use this custom label:

...
  keeper-core-api:
    image: "ncarlier/keeper-core-api:custom"
    restart: always
...

from keeper-web-app.

Hello,

I followed these steps:

created a new image of keeper-core-api muzamil47/keeper-core-api:latest and also node image

and then make build to build muzamil47/keeper-core-api:latest

Then i chenged to this file (https://github.com/nunux-keeper/keeper-docker/blob/master/docker-compose.app.yml#L7)

This is my docker-compose.app.yml file

keeper-core-api:
    image: "muzamil47/keeper-core-api:latest"
    restart: always
    env_file: etc/keeper-core-api.env
    volumes:
      - conf-data:/var/opt/app/config
      - storage-data:/var/opt/app/storage
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000"]
    depends_on:
      - redis
      - mongo
      - elasticsearch
    labels:
      - "traefik.port=8080"
      - "traefik.frontend.rule=PathPrefixStrip:/api"

  #######################################
  # Job Worker
  #######################################
  keeper-job-worker:
    image: "muzamil47/keeper-core-api:latest"
    restart: always
    env_file: etc/keeper-core-api.env
    command: run job-worker
    volumes:
      - conf-data:/var/opt/app/config
      - storage-data:/var/opt/app/storage
    depends_on:
      - redis
      - mongo
      - elasticsearch
      - keeper-core-api
    labels:
      - "traefik.enable=false"

  #######################################
  # Web App
  #######################################
  keeper-web-app:
    build: ./dockerfiles/nginx
    image: "ncarlier/keeper-web-app-www:latest"
    restart: always
    volumes:
      - www-data:/var/www/html:ro
    labels:
      - "traefik.port=80"
      - "traefik.frontend.rule=PathPrefixStrip:/keeper"

and when i run make deploy the container is listed in docker but it's status is always restarting.

and when i access http://localhost:900 the response is 404 page not found

from keeper-web-app.

Your kinematics is correct (however, step 1 and step 2 are basically the same).
Beware of the Docker cache. (because in your example the image is built essentially from the Docker cache).
Another thing is that steps 3 and 4 of Docker's construction are strange. Do you use the genuine Dokerfile ?
If the container restarts, it means that the start has failed. You should print the container log to see what's going on:

$ # Display ALL containers (also those that have been stopped)
$ docker ps -a
$ # Show logs of the failed container
$ docker logs <id of the stopped container>

from keeper-web-app.