Giter VIP home page Giter VIP logo

chanjo2's People

Contributors

dependabot[bot] avatar northwestwitch avatar ramprasadn avatar vince-janv avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar

chanjo2's Issues

Track code coverage using codecov

To do:

  • Add codecov report part in GitHub tests action
  • Register the report in codecov
  • Send the first coverage report from main branch to codecov

Enable d4 on hasta

  • Set up d4 on hasta using singularity.
  • Document installation and maintenance instructions

Validate region files on post

Description

When region definition files gets added to the database they should be validated.
It is not decided what format these files should be on yet, bed is probably the way to go. csv could be an alternative.

This can be closed when

  • The app validates that the file exists
  • The app validates that the file is on the correct format

Updating genes can't be done unless transcripts are removed first

This happens in a database pre-populated with data, If database already contains genes AND transcripts, you get an error, since transcripts table contains reference to the genes one, so the genes can't be removed.

What to do:

When the command to update genes is launched, then also eventual old transcripts and exons data should be dropped, which makes sense!

Endpoint to load gene intervals

Create an endpoint that loads the intervals for each single gene (the entire gene). There will be a similar thing for transcripts and exons as well.

it should accept genome build.

Should download and parse the genes file and create one entry for it in the intervals table with the following cols:

  • id
  • chromosome
  • start
  • stop

Additionally it should create and link tags for the gene above so it's searchable using the following parameters:

  • build (required)
  • one of the following items: [Ensembl id, HGNC symbol, HGNC ID] (optional)

Deprecated code in dbutil.py

It will raise an error when switching to SQLAlchemy 2

/Users/chiararasi/Documents/work/GITs/chanjo2/src/chanjo2/dbutil.py:32: MovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
Base = declarative_base()

Establish MVP for chanjo-report

  • Determine minimum viable product which replaces current functionality of chanjo-report

Discussion might be needed with Scout team and potentially some Scout customers.
Are all important metrics present in current report?
Do we need an additional microservice for chanjo-report or should chanjo2 have a report functionality?

Create a skeleton

Description

Create a skeleton of the app that describes how the structure should be.

This can be closed when

  • There is a skeleton with a couple of relevant endpoints

Add option to display only MANE transcripts in coverage module

(I don't know if this issue should be reported in the chanjo repository instead, feel free to redirect)

In the Chanjo Report in Scout, it would be nice to have an option (perhaps a tick box) to narrow down the information about fully/incompletely covered transcripts so only information on MANE Select (and perhaps also Plus Clinical) transcripts is shown when requested.

Create database models

  • Create a sketch of database models and relationships
  • Create mock objects of database entries according to the model for development and testing

Some warnings from the docker-image-push GitHub action

Warning: The save-state command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
Docker info
Buildx version
Warning: The set-output command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Generate d4 test data

Test files in *.d4 format generated

  • MIP-DNA TGS
  • MIP-DNA WES
  • MIP-DNA WGS
  • BALSAMIC TGS
  • BALSAMIC WES
  • BALSAMIC WGS

VARCHAR requires a length on dialect mysql

Testing the app with a real MySQL database. I get the following error:

sqlalchemy.exc.CompileError: (in table 'samples', column 'coverage_file_path'): VARCHAR requires a length on dialect mysql

The demo is working because it's using another SQL dialect (sqlite)

Logs printed twice

I have the feeling that metadata object is called/imported from the wrong place. Bug is present in current main.

Demo app is launched twice because of gunicorn settings

After introducing the logs, I've realised that the demo app is launched twice:

Basically it's creating the tables twice and printing the Running a demo instance of Chanjo2 twice. This is not unexpected, but annoying. It's caused by gunicorn running with 2 workers

We could solve this in 2 ways:

  • Changing the gunicorn config to use 1 processor
  • Launching the demo app with the uvicorn command instead: uvicorn src.chanjo2.main:app, which is also easier..

I would opt for the second option. We just need to change a line in the README file. I'll fix!

Add instructions on how to use a dockerized chanjo2 on the fly to get coverage data from files

For instance having a d4 file and a bed file with the intervals.

Chanjo2 can be used also in projects not related at all with Scout, but as a container that can be runned to gather quick coverage data when running a pipeline, for instance.

Another useful application for non-scout users: run it as a container and provide a d4 file and a list of genomic coordinates (doesn't require a populated database at all).

Can't launch the app via docker-compose or direct gunicorn command

Hi @Vince-janv and @ramprasadn. I'd like to revive this project and I'm trying to launch the application.

Using docker-compose (docker-compose up) doesn't work. I get the following error:

(chanjo2) chiararasi@ChiaraRMBP:~/Documents/work/GITs/chanjo2$ docker-compose up
WARNING: The D4DB_NAME variable is not set. Defaulting to a blank string.
WARNING: The D4DB_USER_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The MYSQL_HOST_PORT variable is not set. Defaulting to a blank string.
WARNING: The MYSQL_CONTAINER_PORT variable is not set. Defaulting to a blank string.
WARNING: The DATA_VOLUME variable is not set. Defaulting to a blank string.
WARNING: The D4DB_USER_NAME variable is not set. Defaulting to a blank string.
WARNING: The CHANJO_HOST_PORT variable is not set. Defaulting to a blank string.
WARNING: The CHANJO_CONTAINER_PORT variable is not set. Defaulting to a blank string.
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.chanjo2-demo.ports contains an invalid type, it should be a number, or an object
services.d4database.ports contains an invalid type, it should be a number, or an object
(chanjo2) chiararasi@ChiaraRMBP:~/Documents/work/GITs/chanjo2$

Same when I invoke the gunicorn command: gunicorn --config gunicorn.conf.py src.chanjo2.main:app (I guess?):

  File "/Users/chiararasi/Documents/work/GITs/chanjo2/src/chanjo2/main.py", line 9, in <module>
    from chanjo2.dependencies import engine, get_session
  File "/Users/chiararasi/Documents/work/GITs/chanjo2/src/chanjo2/dependencies.py", line 11, in <module>
    engine = create_engine(mysql_url, echo=True)
  File "/Users/chiararasi/miniconda3/envs/chanjo2/lib/python3.8/site-packages/sqlmodel/engine/create.py", line 139, in create_engine
    return _create_engine(url, **current_kwargs)
  File "<string>", line 2, in create_engine
  File "/Users/chiararasi/miniconda3/envs/chanjo2/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned
    return fn(*args, **kwargs)
  File "/Users/chiararasi/miniconda3/envs/chanjo2/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 530, in create_engine
    u = _url.make_url(url)
  File "/Users/chiararasi/miniconda3/envs/chanjo2/lib/python3.8/site-packages/sqlalchemy/engine/url.py", line 731, in make_url
    return _parse_rfc1738_args(name_or_url)
  File "/Users/chiararasi/miniconda3/envs/chanjo2/lib/python3.8/site-packages/sqlalchemy/engine/url.py", line 787, in _parse_rfc1738_args
    components["port"] = int(components["port"])
ValueError: invalid literal for int() with base 10: 'None'
[2023-01-11 10:34:06 +0100] [49258] [INFO] Worker exiting (pid: 49258)
[2023-01-11 10:34:06 +0100] [49255] [WARNING] Worker with pid 49258 was terminated due to signal 15
[2023-01-11 10:34:06 +0100] [49255] [INFO] Shutting down: Master
[2023-01-11 10:34:06 +0100] [49255] [INFO] Reason: Worker failed to boot.

I see where the error is in both cases. If you don't mind I'd like to write a basic fix and some instructions to run the app in README?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.