Giter VIP home page Giter VIP logo

bridge-pipeline's Introduction

The Bridge pipeline.

####The Bridge pipeline uses Apache Object Oriented Data Technology (OODT) for data processing. Apache OODT integrates and archives your processes, your data, and its metadata (https://oodt.apache.org/).

OODT code structure:
  • bin -- Holds env.sh (where all environment variables are located) and oodt script (used to start and stop oodt)
  • data
    -- archive -- Holds all the datasets (format for datasets is name_of_dataset i.e. 'nanograv5' and include all .tim and .par files and nothing else)
    -- jobs -- Holds all the processed information from workflow (format - web app sets a session_id for each run. there will be a directory with that session_id and all the output data will be stored within this directory. All plots generated by residual plotting is in 'plots'. All optimal stat data is stored in 'optimal_stat'. All f stat data is stored in 'f_stat'. and average epoch output is stored in 'avg_epoch'. The dataset used for the session id is stored in 'session_data'. This directory can be clean nightly or weekly to regain some space.
    -- workflow -- Stores a catalog of all workflows run. If you ever want to remove all the workflow data which appears in the OODT opsui, delete the complete directory (not just the contents).
  • logs โƒ oodt.out - Holds all the logs for the entire OODT system. When running into issues, this is the first place you should check for possible reasons.
  • pge --
    -- bridge -- All the functional bridge code from the git repository should be cloned here. (This directory does not exist. Deployer should create it.)
    -- bin -- Holds helpful Python scripts to run workflows
    -- policy -- Holds PGE configuration files. Each workflow has it's own configuration file used to set env vars and commands to run.
  • workflow
    -- etc -- Set logging properties or workflow properties -- logs -- Holds workflow logs -- policy -- Define workflows here. For an addition of a workflow, you will need to edit the events.xml, tasks.xml and create a new workflow file to define where the pge configuration file is located (so create a pge config (pge/policy) for you workflow as well)

####The User Interface (UI) code uses AngularJS , an open-source (javascript) web application framework. (https://angularjs.org/)

UI code structure:
  • app - Angular web application โƒ css -- This directory holds all CSS stylesheets.
    -- img -- This directory holds all images needed for the UI. -- js -- This directory holds all Javascript files. -- lib -- This directory holds all libraries used. i.e. Bootstrap, Angular -- partials -- This directory holds different views (pages)

Partials directory: In this directory you can find 2 main HTML files. The about.html file is the home page of the web app. This is where the introduction and instructions for the application are given. In the analysis.html file, you will find all the HTML and angular data-binding for the main application.

For example, you can see that the data selection "accordion", advanced options, the engage buttons, and the result views are all defined within this page.

Note: Comments in HTML are enclosed in <!-- comment --> (and it appears in blue in vi)

js directory: In this directory you will find all the JS code that controls the HTML. In this application we use an angular route library (found in lib/angualr dir) to route the appropriate pages when navigation buttons are selected. All the control is defined in the app.js file.

All the control for the about.html and analysis.html page is defined in the controller.js file. The controller specifically for the analysis.html page is called 'AnalysisController'. On line 3, you should set the proper 'api_url' that point to your flask API. within the 'AnalysisController', you will see that on line 30 (at the time of documentation) all the defaults for the analysis view is set.

bridge-pipeline's People

Contributors

shakeh avatar yliang1 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.