Giter VIP home page Giter VIP logo

360composer's Introduction

360 Composer

360-Composer is a software package containing all the components needed to deploy a 360 video story line editing suite.

Changes in this branch (f.k.a. robin-variable-quality)

  • Improved docker-compose development stack:
    • Updated base images
    • Development stack no longer uses production build of the frontend, but instead a development server with live reloading (much faster!)
    • Proxying is now done by nginx, instead of http-proxy-middleware.
  • HLS transcoding in backend (video path now returns m3u8 playlist instead of mp4 file)
  • Modified a-frame video player for HLS support
  • (WIP) Modified annotation editor (babylon.js video player) for HLS support

Installation and usage

When installing you can choose to run a production or development build.

Development

First, run npm install once (do this again when package.json is changed):

docker compose run frontend install

Create the assets directory: mkdir ./assets

Then, start the entire stack:

docker compose up --build

The web interface will be available at http://localhost:8080.

Production

Save the docker-compose.prod.yaml file to an empty directory:

curl -o docker-compose.yaml https://raw.githubusercontent.com/robert-belleman/360Composer/main/docker-compose.prod.yaml

Create a .env file with the required settings (this is an example, you should generate your own passwords!):

DATABASE_PASSWORD=j4jOVqaIp60cnZfUeWNpvsukSJT2JZ
JWT_SECRET_KEY=d0V83DrVapK9pK8j85nPqtSUCE3HJq

Start the stack:

docker compose up

Tips:

  • The web interface is available at http://localhost:8080
  • Exit using Ctrl+C
  • Start in background using docker compose up -d. Containers will automatically start after a system reboot with the restart: unless-stopped restart policy.
  • Stop and remove containers using docker compose down
  • Members of the visualisation lab can push updated docker images to GitHub Container Registry using the ./docker-push.sh script

360composer's People

Contributors

derkades avatar baleineus avatar dynamometer67 avatar cvisl avatar markwijz avatar ricoround avatar robert-belleman avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

360composer's Issues

Add ability to "brand" the environment

There are currently hard-coded logos (Levvel) and texts in the frontend and player that should be configurable by the user of 360Composer. This would allow everyone to "brand" 360Composer as they wish.

Use flask extensions instead of self-created code

Currently multiple parts are made by the students themselves with potential security risks while this is already solved by numerous extensions for Flask. In this issue we'll keep a list of potential Flask extensions that can replace code from the students.

  • Hash password creation could be replaced by bcrypt. This provides better hash-algorithms which we don't have to maintain ourselves.
  • Authentication is now created from scratch while that could be solved by Flask-login

Consistency in UI design

Currently it is very clear that it has been made by multiple people with different ideas on how to implement the UI design. For example the LoginForm component has its own custom made style while other components make use of MUI. This makes the user experience highly irregular and should be made consistent. I would suggest to use MUI consistently throughout the application.

Ability to add (interactive) 3D polygonal objects to a Scene

There are many situations in which it would be useful to add 3D objects to a 360 camera scene so that they appear to be part of the scene. For example; adding a 3D model of a machine to a filmed factory floor.
This is a potential Pandora's box: once this would be added, chances are high that plenty of other features are going to be asked for next: add animation, physics, interaction types, etc. (basically: add all functionality of Unity/Unreal)!

Analytics needs rethinking

The current "analytics" functionality records events as they occur during a session; as described in section 4.2.2 of Emil van Veen's thesis): "1) the start and end of a timeline, 2) the start and end of a scenario, 3) the start and end of a scene, and 4) the occurrence of a key interaction moment". The problem is that the records that are produced are not "ACID" structured, which makes robust analysis hard. For example: "start" and "end" events of a timeline, scenario or scene are not always balanced (i.e. there is no matching "end" with every "start", which can happen when a client is interrupted). It is also difficult to tell apart sessions and interaction, which is wanted in the dashboard by Sharon Monk.

Database password doesn't except special characters

Currently the database password does not except special characters, my local postgresql has a random 32 character random master password which is not accepted as password and gives me this error:

pg_isready: error: invalid percent-encoded token:

I tried to URI encode the password but then docker-compose trips over the percentage sign which you can then escape with double percentage signs but then the database trips on it again. This needs a proper solution because special characters are commonly used in passwords. For now I simplified my local db password because there is no sensitive data in it anyways.

Ability to control the position of an annotation

There is currently no way to control where a text annotation appears in a Scene. Add an ability by which a scene creator can control this, either at a fixed location in a Scene or (relative to) the current viewpoint.

Ability to record where the end-user is looking

For analytics purposes it would be nice to be able to record where a end-user is looking while playing a Scenario. This can be used to monitor if the end-user is acquiring the correct information from the Scenes before answering the questions.

Unify video playback in scene editor and XR player

The video players in the scene editor is based on Babylon while the XR player is built on A-frame. This results in discrepancies, such as different locations of annotations. It would be good to use the same code for both video players.

Load videos on demand rather than all at once

In the current implementation for each video fragment a separate <video> element exists in the A-Frame assets list. This causes all videos to be loaded at the same time, which may cause an excessive memory usage especially for larger number of videos or longer/higher quality (i.e. 4K) videos. This may cause problems especially on weaker devices.

Scenario editor does not make full use of available screen space

The scenario editing environment does not scale with the available screen space. This is inconvenient if you're creating a complex scenario that runs off the screen. You can scroll through a scenario by dragging the background of the scenario, but is not completely clear for someone new to the editor. In that case it would be helpful to show scrollbars.

Add interaction/annotation based on where the end-user is looking

Add an annotation/interaction possibility that controls the progression through a Scenario based on where a user is looking. This would result in a possibility for a scenario creator to connect Scenes based on the region in which the end-user is looking at a Scene.

Improve connectivity layout in Scene editor

In the Scenario editor; the lines that connect Scenes are often invisible if they pass through other Scenes. This quickly results in difficult to understand "spaghetti", but sometimes causes important side effects. For example: a Scene output that goes into its own input is possible, but is completely invisible and can therefore not be deleted!

In Timeline; add the ability to select which Scenarios are randomized

Currently, all Scenarios in a Timeline are randomized. In some cases it is necessary to exclude Scenarios from randomization, for example: the "introduction" scenario should always come first, or the "exit" scenario should always come last, or a scenario that introduces the next scenarios should always be at a specific moment.

Add a possibility to select the scenarios that are randomized.

Deleting a scenario doesn't work

In the scenario panel: if you click on the trashcan in a scenario, a pop-up appears "Are you sure you want to remove this scenario?", but if you then press "Remove" the scenario is not deleted and a red message appears "Unable to delete scenario.".

Use flask migrations to automate most of the database changes

Currently there is a Alembic workflow by changing everything by hand. This created a severe difference between the Flask-sqlalchemy models and the database migrations. One automatic migration already showed about a 100 differences between the models and the database (from the migrations directory).

Add ability to play Scenarios off-line

There are situations where the end-user requires an optimal Quality of Service or has limited connectivity and is unable to reach the server backend. For these cases it would be nice to have an "export" facility that contains a completely self-contained package that can be installed on the client device and that has the same functionality as the on-line version.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.