Giter VIP home page Giter VIP logo

opens3d's Introduction

Build Status Codacy Badge codecov

Open-Source Real-Time Assistance Framework for Stereoscopic Content Production

The goal of this library is to provide users with tools (C++ library and software) to create stereoscopic 3D content. These tools should be sufficient to create stereoscopic 3D for cinematic purposes.

S3DAnalyzer

Features

Stereoscopic 3D content analysis is available through a modern C++ library.

Most algorithms are implemented using OpenCV but the library uses interfaces so that it could be implemented with any other computer vision frameworks.

The usage of the library is demonstrated with a stereoscopic 3D analysis software (S3DAnalyzer):

  • 3D Video player (anaglyph, side-by-side)
    • Side-by-side
    • Above-below
    • Separate files (left, right)
    • Supports many video formats and codecs (with FFmpeg)
    • Variable horizontal image translation to adjust the image plane (useful for raw videos for which all points are projected in front of the screen).
  • Camera alignment
    • Detect misalignments (roll angle, vertical offset, zoom ratio, tilt, pan, etc.)
  • Camera rectification
    • Correct misalignments detected to minimize vertical disparities
  • Disparity analysis
    • Disparity range
    • Colored feature points from disparity (red = too close in front of the screen, purple = too far behind screen plane)
  • Viewer-centric depth analysis
    • Depth from screen plane of detected features (from viewer distance, screen size...)

Build Instructions

This project uses C++14 and CMake.

This project is mainly developped on Linux (Ubuntu, Arch) but has recently been ported to Mac OS and Windows (see Windows Setup). FFmpeg can be installed from source (or with brew on Mac OS) and OpenCV compiled from source.

Dependencies are:

  • FFmpeg >= 3.0
  • OpenCV >= 3.0
  • Qt >= 5.7

If all dependencies are correctly configured, the project can be built with cmake:

mkdir build
cd build
make

Compilers:

  • Continuous development: linux 64 bits, GCC (Build + Tests)
  • Internal development: linux 64 bits, Clang (Build + Tests)

Coding Standards

This project uses clang-format Chromium coding standard.

License

This software is licensed under a BSD 3-clause license. See LICENSE file.

Citations

Cite as:

@misc{bedard2018opens3d,
  author = "Hugo Bédard",
  title = "OpenS3D, an Open-Source Real-Time Assistance Framework for Stereoscopic Content Production",
  howpublished = "\url{https://github.com/hugbed/OpenS3D}",
}

opens3d's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

opens3d's Issues

Image Mode Reuses Rectified Images

If you press compute again, instead of rectifying the original image, it rectifies the previous rectified image since it retrieves the image from the textureManager.

Draw matches from points and images

Implement a function to draw features from a list of 3d points on cv image (same behaviour as cv::drawMatches)

Would prevent having to carry around cv::DMatch everywhere.

Prevent Rectification on Small Number of Inliers

Dealing with videos, a frame may have a large number of detected features (lets say more than 100) while the next frame may have not enough features to give a correct answer (not enough inliers for ransac --> not large enough probability for a good answer).

The alignment and rectification parameters should not be updated if there are not enough features found.

ImageOperations to abstract computations in mainWindow

A list of ImageOperation that can be toggled (on/off).
ImageOperations will have an input image, and an output image that will be the output of all the operation chain (each activated operation in order transform the image).

This will help with #31 since it will be possible the operations will always be done on the input image instead of the output image as of right now.

Question about dependencies

This seems promising. Can I use it to just produce the stereoscopic image without any sort of putting in the file or rendering, so to get rid of ffmpeg or qt?

Thanks a lot.

noexcept and const

Check for missing noexcept or const (for member functions not modifying member variables)

Epipolar lines display tool

Implement a class to display epipolar lines from fundamental matrix, 2 images and 2 sets of points (1 in each image).

Prepare demo examples (alignment, rectification)

In order to compare the current algorithms to the articles and to visualize results. There should be simple programs using S3D features:

  • Feature matching
  • Epipolar line visualization
  • Rectification (with epipolar lines)

Create Rectifier class

To rectify images from rectification matrix.

  • Add function to compute rectification matrix from STAN model.
  • Include debugging tools to see images after rectification.

Better variable names for StanAlignment

Give better variable names for StanAlignment. Currently the variable names are based on the article (a, c, f). They should have readable names such as verticalOffset, rollAngle, Zoom, tiltOffset.

Note: where to put the units for this so it's not confusing and variable names are not too long?

Support multiple resolutions/fps for Decklink video device

It is hardcoded to 720p 60fps. It was 1080p 30fps before since the camera used previously only supported these specs. It would be nice to be able to set the resolution fps to different resolutions supported by the Decklink interface.

STAN solver numerical stability issue

As stated in the article cited below, solving STAN alignment with tiltKeystone and zParallaxDeformation may lead to numerical stability issues since they depend on multiple parameters. These parameters should be removed from the estimation for use with RANSAC since a small error in the inputs may lead to inconsistent large output errors.

Note: it would be nice to control the number of parameters to solve without having to change the StanFundamentalMatrixSolver class.

Zilly, Frederik, et al. "Joint estimation of epipolar geometry and rectification parameters using point correspondences for stereoscopic TV sequences." Proceedings of 3DPVT. 2010.

Implement base class for rule of five

To prevent having to define (and repeat in each class) all the operations (move, copy) when defining a virtual destructor for an abstract class.

So:

  1. Define a virtual destructor (or the destructors of the implementations classes will not be called when deleting them through the base interface)
  2. Delete copy operations (construction, assignment) for base class since we don't know if derived class will be copiable.
  3. Default move operations (construction, assignment) for base class to prevent the compiler from deleting them (move operations can be good optimizations).
  4. Define default constructor (that would be otherwise deleted by deleting construction operations).
  5. Define virtual clone operation to replace copy (owner pointer to use covariant return types that can't be used with smart pointers. States clearly that the returned value will be owned by the caller.

Interfaces should inherit privately this base class to use the features of this base class without any conceptual relationship between the 2 classes.

Quote from Scott Meyers' book :

  • "Private inheritance is most likely to be a legitimate design strategy when you're dealing with two classes not related by is-a where one either needs access to the protected members of another or needs to redefine one or more its virtual functions."
  • "Private inheritance means is-implemented-in-terms-of. It's usually inferior to composition"
    "If you make a class D privately inherit from a class B, you do so because you are interested in taking advantage of some of the features available in class B, not because there is any conceptual relationship between objects of types B and D."
  • "Private inheritance means nothing during software design, only during software implementation."

Example:

#include <iostream>
#include <memory>
using namespace std;

class Empty {};

// defines virtual destructor for interface (pure abstract) class
// deletes copy (don't know if derived will contain incopiable types)
// allows move (everything is movable)

template <class B = Empty>
class rule_of_five_interface : B {
public:
  virtual ~rule_of_five_interface() = default;
  rule_of_five_interface(const rule_of_five_interface&) = delete;
  rule_of_five_interface& operator=(const rule_of_five_interface&) = delete;
  rule_of_five_interface(rule_of_five_interface&&) = default;
  rule_of_five_interface& operator=(rule_of_five_interface&&) = default;
  virtual gsl::owner<rule_of_five_interface*> clone() = 0;
protected:
  constexpr rule_of_five_interface() = default;
};

class Interface : rule_of_five_interface<> {
public:
   virtual int next() = 0;
};

class Impl : public Interface {
public:
  Impl() {}
  ~Impl() override = default;
  
  int next() override { return 1; }    
};

int main() {
  std::unique_ptr<Interface> p = std::make_unique<Impl>();
  cout << p->next() << endl;

  return 0;
}

References:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.