Giter VIP home page Giter VIP logo

localega-tsd-proxy's Introduction

LocalEGA-TSD-proxy

TSD File API documentation

https://test.api.tsd.usit.no/v1/docs/tsd-api-integration.html

NB: access is restricted to UiO network. Please, contact TSD support for the access, if needed.

Configuration

Environment variables used:

Variable name Default value Description
SSL_ENABLED true Enables/disables TLS for DOA REST endpoints
SERVER_KEYSTORE_PATH /etc/ega/ssl/server.cert Path to server keystore file
SERVER_CERT_PASSWORD Password for the keystore
BROKER_HOST public-mq Public RabbitMQ broker hostname
BROKER_PORT 5671 Public RabbitMQ broker port
BROKER_VHOST / Public RabbitMQ broker virtual host
BROKER_VALIDATE true Validate server MQ certificate or not
ROOT_CERT_PATH /etc/ega/ssl/CA.cert Path to the CA file for RabbitMQ connectivity
ROOT_CERT_PASSWORD Passphrase to unlock RabbitMQ CA file
TSD_ROOT_CERT_PATH /etc/ega/ssl/CA.cert Path to the CA file for TSD connectivity
TSD_ROOT_CERT_PASSWORD Passphrase to unlock TSD CA file
CLIENT_CERT_PATH /etc/ega/ssl/client.cert Path to the client cert for RabbitMQ connectivity
CLIENT_CERT_PASSWORD Passphrase to unlock client cert file
BROKER_USERNAME admin Public RabbitMQ broker username
BROKER_PASSWORD guest Public RabbitMQ broker password
EXCHANGE cega RabbitMQ exchange to publish messages to
ROUTING_KEY files.inbox RabbitMQ routing key
CLIENT_ID Elixir AAI client ID
CLIENT_SECRET Elixir AAI client secret
CEGA_AUTH_URL https://egatest.crg.eu/lega/v1/legas/users/%s?idType=username Central EGA authentication REST endpoint URL
CEGA_USERNAME Central EGA auth endpoint username
CEGA_PASSWORD Central EGA auth endpoint password
TSD_HOST api.tsd.usit.no TSD File API URL
TSD_PROJECT p11 TSD project code
TSD_APP_ID ega TSD application ID
TSD_ACCESS_KEY TSD File API access key
PASSPORT_PUBLIC_KEY_PATH /etc/ega/jwt/passport.pem Path to the public key for passport JWT validation
OPENID_CONFIGURATION_URL https://login.elixir-czech.org/oidc/.well-known/openid-configuration URL of the OpenID configuration endpoint
VISA_PUBLIC_KEY_PATH /etc/ega/jwt/visa.pem Path to the public key for visas JWT validation

Sample Docker Swarm entry

...
  proxy:
    image: uiobmi/localega-tsd-proxy:latest
    ports:
      - 443:8080
    deploy:
      restart_policy:
        condition: on-failure
        delay: 5s
        window: 120s
    environment:
      - ROOT_CERT_PASSWORD
      - SERVER_CERT_PASSWORD
      - CLIENT_CERT_PASSWORD
      - CLIENT_ID=test
      - CLIENT_SECRET=test
      - CEGA_USERNAME
      - CEGA_PASSWORD
      - TSD_HOST
      - TSD_ACCESS_KEY
    secrets:
      - source: rootCA.p12
        target: /etc/ega/ssl/CA.cert
      - source: server.p12
        target: /etc/ega/ssl/server.cert
      - source: client.p12
        target: /etc/ega/ssl/client.cert
      - source: jwt.pub.pem
        target: /etc/ega/jwt/passport.pem
      - source: jwt.pub.pem
        target: /etc/ega/jwt/visa.pem
...

localega-tsd-proxy's People

Contributors

dtitov avatar joshbaskaran avatar parisa68 avatar a-ghanem avatar dependabot[bot] avatar kjellp avatar dependabot-preview[bot] avatar

Watchers

James Cloos avatar  avatar  avatar Kjetil Klepper avatar Milena Pavlovic avatar Milen Kouylekov avatar  avatar

localega-tsd-proxy's Issues

No tests

Is your feature request related to a problem? Please describe.
Seems to be no tests /checks present for this service ( is tested by overall integration tests only). I. e.there is no support to see if a dependabot update or other changes breaks this particular service before approving a PR .

Describe the solution you'd like
A mminimal set of tests for the proxy service that do not require a full blown stack to interact with. Build, ++

Release localega-tsd-proxy v2.0

The PR with relevant changes is in place:

Todo:

  • Push the v2.0.0 tag
  • Make sure the release notes document the group ID change

Start https server with defined timeout values

Describe the bug
HTTP timeouts are necessary to expire inactive connections and failing to do so might make the application vulnerable to attacks like slowloris which work by sending data very slow, which in case of no timeout will keep the connection active eventually leading to a denial-of-service (DoS) attack. (CWE-400)

Lifted from:
neicnordic/sda-s3proxy#293

May be relevant for other https servers in setup? Web?

Missing delete message to CEGA upon deletion of inbox file by user

Describe the bug
If a user is deleting an uploaded file from the inbox, it is correctly removed by lega-commander/TSD in co-operation, but the proper "remove" message to CEGA is not having the intended effect. I.e. CEGA still lists the file as available for user to pick for ingestion (when defining a Run or DataSet in the submission portal).

A "remove" message is being sent (seen in the logs), unknown if:

  • if the format conforms to the json schema for this message type
  • if a proper correlationID is set for the message (CEGA known to fail/ignore messages without a mandatory correlationID)
  • if all fields of the message has the proper content. Both fileSize and lastModified fields are set to value "0" which probably should be corrected.

Expected behavior
When the delete end-point is executed, a properly formatted message with correct content is sent to CEGA from the proxy service (similar pattern to the upload message when the upload end-point is utilized)

Update README.md

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
The current version of README.md does not provide up-to-date information on the LocalEGA-TSD-proxy or shortly proxy service.

Invalid mime type errors

Describe the bug
A clear and concise description of what the bug is.
The java server spits out errors about invalid mime type specifically for the { token.

Additional context
Add any other context about the problem here.
Specifying MediaType to JSON (for proxyController)/ HTML (for authcontroller) might help with this.

Circular dependency

Describe the bug
The service has a circular dependency.

To Reproduce
running the integration tests in LocalEGA-deploy-swarm reproduces the problem.

Expected behavior
In #42 spring-boot-starter-parent was upgraded to 2.6.2 which does not allow circular referencing.

Screenshots
| The dependencies of some of the beans in the application context form a cycle:
|
| ┌─────┐
| | localEGATSDProxyApplication
| ↑ ↓
| | org.springframework.boot.autoconfigure.web.servlet.WebMvcAutoConfiguration$EnableWebMvcConfiguration
| ↑ ↓
| | org.springframework.security.config.annotation.web.configuration.OAuth2ClientConfiguration$OAuth2ClientWebMvcSecurityConfiguration
| └─────┘
|
|
| Action:
|
| Relying upon circular references is discouraged and they are prohibited by default. Update your application to remove the dependency cycle between beans. As a last resort, it may be possible to break the cycle automatically by setting spring.main.allow-circular-references to ***.

Refactor for new users response API

Is your feature request related to a problem? Please describe.
API refactor needed based on API changes in Central EGA NSS

Describe the solution you'd like
refactor for:

  1. New API response from Central EGA NSS
  2. New endpoint URL from Central EGA NSS

Describe alternatives you've considered

Additional context
neicnordic/sda-inbox-sftp#287

Delete inbox file operation leads to incomplete remove message sent to CEGA

Describe the bug
When issuing a "delete file in inbox" operation in the REST interface (i.e. using lega-commander), the message being posted by the MQ Aspect to CEGA is incomplete. It has "filesize":0 and "file_last_modified":0.

To Reproduce
Steps to reproduce the behavior:

  1. List your files in your inbox using "lega-commander inbox -l"
  2. Select a file to delete: "lega-commander inbox -d delete-this-file.c4gh"
  3. Log into ega.elixir.no and inspect proxy server log to see MQ message posted to CEGA: "docker logs proxy-container-id | less"
  4. Observe wrong message

Expected behavior
filesize should be set correctly.
file_last_modified: check docs for what info this should contain

Screenshot

Screenshot from 2022-03-01 11-46-40

Review status of localega-proxy and update for production deployment

Review current dependabot updates
NB: clearing-house and TSD-File-API-client latest releases not detected yet
Decide which features too merge for a new release for the planned Sept production deployment

  • NB certificate renewal (read cert from file and not docker secret)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.