Giter VIP home page Giter VIP logo

sda-metadata-reviewer's Introduction

Querying collections in the metadata store

Dependency

brew install mongodb-community

Start the mongo and s3 servers

cd dev_tools/
docker-compose up -d
  • The minio is used only for the cross reference, you can skip it using
cd dev_tools/
docker-compose up -d database

Exporting a mongo dump

mongodump -u admin -p admin

Restoring a mongo dump

mongorestore -u admin -p admin metadata.bson

Reviewing the submission metadata

  • Find the user that you want to review the submission for:
./main --action list-users

Minimally the user id needs to be provided in the filter.json file to fetch the folder identifiers that belong to a user's submission folder. To obtain them, fill in the user id in the aforementioned file:

{
    "userId": "myuserid"
}
./main --action list-folders

Now find the folder id of the submission and specify it in the filter.json file:

{
    "folderId": "d28e77a17a6a4c19ac53891a678054a5"
}
  • In order to fetch all user specific metadata objects from a given folder run:
./main --action list-objects
  • If you only want to see the metadata from a given metadata object, it is possible to specify its accessionId as a filter.
{
    "folderId": "d28e77a17a6a4c19ac53891a678054a5",
    "accessionId": ""
}

Cross reference metadata files with S3

The cross reference part is comparing the files in the metadata with the ones uploaded in the S3 backend. It can be run with the folderId for the specific submission using the following filter:

{
    "folderId": "d28e77a17a6a4c19ac53891a678054a5",
    "accessionId": ""
}

or with the accessionId of the analysis using the following filter:

{
    "folderId": "",
    "accessionId": "9fd29e35a82e49d999528a5f3c6d49aa"
}

to cross reference based on the file names run the following command:

./main --action cross-ref-inbox

to cross reference based on the checksums of the files run the following command:

./main --action cross-ref-ingestion
  • Fix docker-compose for postgres and s3

sda-metadata-reviewer's People

Contributors

dbampalikis avatar dependabot[bot] avatar jonandernovella avatar norling avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.