Giter VIP home page Giter VIP logo

ustc-software-2017's Introduction

Biohub 2.0 - iGem Project by USTC Software 2017

Build Status

our wiki

What is Biohub?

Biohub is a web-based software authored by USTC Software 2016, focusing on more convenient data handling for synthetic biologists. Technically Biohub is a extensible plugin system, where each functional module is packed and integrated as a plugin. Several plugins (Pano, PathFinder, etc.) are initially included, but users are also allowed to develop and deploy their own plugins, with the help of documentation and demonstration. In this way, new algorithms can be acquainted and discussed by other scholars as early as possible.

What is Biohub 2.0?

Biohub 2.0 is a synthetic biology community, focusing on more efficient data retrieving and ideas sharing. It provides a user-friendly interface to interact with standard biological parts, or BioBricks in short. With the help of a high-performance search backend and well-designed filtering and ordering algorithms, users can obtain information of the parts they want more precisely. For those interest them, users may leave a mark (called a 'Star') for later using. If one has already use a part in practice, he or she may grade the part, or post an article (called an 'Experience') to share ideas. Experiences can also be voted or discussed. All the data collected (rating scores, number of stars, etc.) will be used for more accurating filtering and ordering.

More than a community, Biohub 2.0 is also a flexible plugin system. A plugin is an optional and pluggable module that can be integrated into the software and take advantage of all the internal data. Plugin developers can either use BioBricks data for further analysis, or just implement a synthetic biological algorithm for the community. There is no specific restriction about the code of plugins, but they will be preapproved before installed into the system as a consideration of security. By default, two plugins, ABACUS and BioCircuit, are available as two handy tools and, in a sense, some kinds of demonstration.

Despite of the similarity in their names, Biohub 2.0 has few common points compared with Biohub. The two projects differ largely in their emphasis and implementation. The only thing Biohub 2.0 borrows from Biohub is the idea of the plugin system, which is further improved and implemented in a more stable way. The name 'Biohub', inspired by another active developer community Github, is very suitable for a biology community but was previously used. For disambiguation, the project is named Biohub 2.0.

Motivation

The numerous parts provided by iGEM Parts Registry are precious to the community, but unfortunately displayed in a terrible way. For synthetic biologists with little or no knowledge about programming, the only approach to access the parts is by submitting the search box on the official pages, which is however too crude to use. The default matching algorithm defies multi-dimensional searching, and the default ordering is just wasting users' time. It's hopeless to get anything useful via this tool. Thus, a software to mine high-quality parts accurately is urgently needed.

With the data dumped from iGEM Parts Registry, which contains information like sample status or used times, the parts can be roughly ranked. But further ranking can only be accomplished after experiments are done, which requires the assistance from the whole community. Such feedback mechanism does exist in iGEM Parts Registry, which is called Experiences, but is seldom used probably because of its hard-to-use interface. We think a user-friendly forum centred on Biobricks is necessary, where users can grade the parts they've used, or share experiences with other scholars. The forum on the one hand complements the data used for more accurate ranking, and on the other hand provides a platform for idea exchanging.

The parts data can be used for various purposes, but apparently we are not able to achieve all of them. In consideration of this, we decide to make our software extensible. Developers with ideas to better make use of the data may develop their own plugins, and embed them into the software. With the help of forum, such plugins will be acquainted quickly, and accelerate the development of synthetic biology.

How to use Biohub 2.0?

For biologists, just register an account on biohub.technology and sign into it, and you can enjoy all the services we provide.

How to develop a Biohub 2.0 plugin?

NOTE Since Biohub 2.0 has been developed, tested and deployed on Ubuntu 16.04, we recommend you to use similar working environment to avoid unexpected problems.

Firstly, you should make sure the following requirements are satisfied:

  • python >= 3.5. Related packages: python3, python3.5-dev. (libbz2-dev, zlib1g-dev are required if you compile python from source code).
  • mysql >= 5.7. Related packages: mysql-server, mysql-client, libmysqlclient.
  • redis. Related packages: redis-server.
  • jre. Related packages: default-jre.
  • elasticsearch < 3. Can be downloaded via: https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/deb/elasticsearch/2.4.6/elasticsearch-2.4.6.deb
  • nodejs >= 6. Related packages: nodejs.

Then you can clone down the repository by:

$ git clone https://github.com/igemsoftware2017/USTC-Software-2017
$ cd USTC-Software-2017
$ git submodule update --init --recursive

We recommend to use virtual environment to manage python dependencies:

$ python3 -m venv .env
$ source .env/bin/activate
(.env) $ pip install -r requirements/dev.txt

Setting up the server

The main configuration file should lie at ~/config.json. You can finish it with the help of ~/config.json.example:

{
    "DATABASE": {
        "NAME": "",     // database name
        "USER": "",     // database username
        "PASSWORD": "", // database password
        "HOST": "",     // database host, here should be "localhost"
        "PORT": 3306,   // database port
        "TEST": {       // test database
            "NAME": "",
            "CHARSET": "utf8",
            "COLLATION": null,
            "MIRROR": null
        }
    },
    "PLUGINS": [        // list of installed plugins
        "biohub.abacus",
        "biohub.biocircuit",
        "biohub.biomap"
    ],
    "TIMEZONE": "UTC",
    "UPLOAD_DIR": "",   // path to directory where uploaded files are placed
    "REDIS_URI": "",    // redis connection URL. format "redis://<username>:<password>@<host>:6379"
    "SECRET_KEY": "",   // secret key of the site
    "MAX_TASKS": 20,    // the maximum number of tasks can be executed at a time
    "TASK_MAX_TIMEOUT": 180,  // the maximum seconds a task can be executed
    "ES_URL": "http://127.0.0.1:9200/", // the connection URL of ElasticSearch
    "EMAIL": {          // email configuration
        "HOST_PASSWORD": "",
        "HOST_USER": "",
        "HOST": "",
        "PORT": 465
    },
    "CORS": [],         // authenticated domains. fill the field if cross-domain issues occur during debugging
    "THROTTLE": {       // throttle configuration for each modules
        "rate": 15,
        "experience": 86400,
        "post": 15,
        "vote": 15,
        "register": 3600
    }
}

Then run ./biohub-cli.py init to initialize the database. This step may take several minutes as it will download and preprocess the initial data. After that you can run ./biohub-cli.py runserver to test if the server can be normally started.

Setting up the frontend

The frontend of Biohub 2.0 lies in frontend/main. Firstly you should install the dependencies:

$ cd frontend/main
$ npm install

During developing, Biohub 2.0 uses webpack-dev-server for hot-reloading. You should configure the proxies in frontend/main/domain.config.json to make it work properly. As above, an example file is available:

{
    "dev": {
        // during developing, plugins and the main frontend are compiled and served separately
        // the following field maps each plugin to a port where its frontend files served
        // to serve a plugin frontend files, run `npm run dev <port>` in its frontend directory
        "plugins": {
            "biohub.abacus": 10000,
            "biohub.biocircuit": 10001,
            "biohub.biomap": 10002
        },
        // the address to your local developing server
        "proxy_domain": "localhost:8000"
    },
    // configuration for production environment
    // you can simply ignore it
    "prod": {
        "domain": "biohub.technology",
        "static": "https://ustc-software2017-frontend.github.io/Biohub-frontend/dist/assets/"
    }
}

Then run npm run dev, and you can access the site at localhost:8080.

Create your own plugin

To create and activate a new plugin, simply run:

(.env) $ ./biohub-cli.py plugin new <plugin_name> --with-frontend
(.env) $ ./biohub-cli.py plugin install <plugin_name>

where <plugin_name> is the dot-separated module name of the plugin (e.g. biohub.abacus). You may type ./biohub-cli.py plugin new --help to find more details.

The second step is to install dependencies for the frontend of the new plugin. Simply run:

(.env) $ cd path/to/your/plugin/frontend
(.env) $ npm install
(.env) $ npm run dev <port>  # randomly choose a number for <port> from 10000~65535

Then compiled files will be served at localhost:<port>. Don't forget to add a new item in frontend/main/domain.config.json:

{
    "dev": {
        // ...
        plugins: {
            // ...
            "<your_plugin_name>": "<port>"
        }
    }
}

Now you can start to write your own plugin. The templates generated by biohub-cli contains some useful comments, which may help you during developing. Also, you can take our integrated plugins (e.g. biohub.biomap) as references.

ustc-software-2017's People

Contributors

hsfzxjy avatar lanyj avatar lsj1292 avatar mlzeng avatar songchaow avatar testpersonal avatar victorsu000 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ustc-software-2017's Issues

Drop plugin framework out of the current architecture.

According to a series of research, there is no perfect solution to support a plugin framework. Stability contrasts with hot-reloadability, and varied utilities contrasts with security. On the other hand, our server with limited resources cannot support plugins uploaded from even tens of users. The plugin framework prevents the core part of the project from scaling up.

But this doesn't mean that Biohub will change to be conservative. There are multiple ways to help the community, and integrating plugins is just one of them. In the new version, we may open up some interfaces for public data accessing.

new ways for scraping data.

Due to the weird design of WikiMedia, Igem Parts Registry may use non-semantic tags for documentation rendering (e.g. use <table> for typography). This may lead to incorrect parsing by html2markdown, and mess up the final result. Comparatively, directly accessing the Edit page (take BBa_K2042000 as an example) can fetch the raw page code in WikiText format, which can make the parsing simpler and preciser.

Besides, the History page in wikitools will list out recent changes on the page, so there is no need to re-fetch the whole page entirely during upgrading.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.