Comments (7)
There's an open source project that does this: https://github.com/Niek/superview
from gyroflow.
Superview is implemented as of ae8ecdd
It's not 100% correct, because I still see some minor warping, but I think it's pretty usable.
SuperView is essentially full sensor 4:3 video, non-linearly stretched to 16:9
I took 9 angles of a chessboard with a GoPro in both SuperView and Wide mode (using app so the camera stays in exactly the same spot), then I detected chessboard corners on all 18 images and I tried to figure out an algorithm to make points from SuperView match the ones from Wide shot.
I set up a canvas in html, because we can have live preview in VS Code so the prototyping becomes really quick, I tried to figure out an algorithm and then using Wolfram Alpha I converted the formula to the inverse, so we have the calculation both ways.
I attach the development files, maybe someone wants to pick this up and improve or find out why there's still some warping.
The points are not matching perfectly there, but when I made them aligned better - it was actually producing worse RMS when calibrating the lens, I'm not sure why that is
2022-04-28-18-03-37_Trim.mp4
from gyroflow.
+1, would be great to have SuperView checkbox in export settings.
Obviously there are ways to achieve this without gyroflow, but that would be inconvenient and lower quality
from gyroflow.
Optically it's not anamorphic, it's probably just squeezed in the rescaler.
I would GUESS that performing the cal results in a significantly different Fx and Fy in the camera matrix, do you happen to have an example matrix?
If it does, based on the review of the distortion model and how it works, I suspect that handling this should not be difficult at all, with the possible exception of the FOV math being impacted and/or while I have a pretty good idea of how the stabilization is handled intra-frame in frame_transform.rs - I don't know if there's some additional interframe rotation handling I ahven't noticed yet. (I've got an idea for how to handle it, but I want to confirm my understanding of what's going on is correct, will elaborate more either tonight or tomorrow.)
Specifically, instead of deriving Fx and Fy for new_k in https://github.com/gyroflow/gyroflow/blob/master/src/core/undistortion/frame_transform.rs#L58 (which appears to be effectively creating a "virtual camera" that is panned/rotated to compensate for the real camera's rotation) from the original camera, allowing them to continue to be different, set Fx = Fk = ???? (derived from either original Fx or Fk plus your FOV scaling) - setting Fx = Fk for the "virtual camera" when the recording camera has Fx != Fk should handle the nonsquare pixels of the input.
Also likely would handle the Runcam 5 Orange's Superview mode more easily than the workflow outlined in https://www.youtube.com/watch?v=4JL5R-W9ZzU
I don't have a Gopro or Runcam 5 so can't do the cal experiments myself, although I am seriously considering the R5 Orange.
I wonder if we should possibly create a rollup distortion/lens model issue and/or project since we have quite a few different issues in the tracker that are probably best solved together.
I suspect the math provided in the references of https://github.com/Niek/superview may not be correct, but happens to be an "ok-ish" approximation for semi-defishing something.
from gyroflow.
The runcam 5 XV mode is a linear stretching of the pixels and can be corrected by changing the Fx/Fy values (see #71 ). GoPro superview is a bit more complex and is more akin to a horizontal quadratic curve, but that still has to be confirmed and modelled by comparing 4:3 and superview frames of the same subject to each other.
from gyroflow.
Hm, i guess i misunderstood the original issue. I would simpy like to be able to import 4:3 video, stabilize it and export as superview stretched. Can that be implemented?
from gyroflow.
There's no big practical reason for doing that. The purpose of superview is to allow for a 16:9 image to contain the full 4:3 image captured by the sensor for action shots etc. Due to the way undistortion is applied, the result is also nonlinearly stretched in the horizontal axis, meaning rendering in 16:9 works well for 4:3 input. Applying additional superview leads to bad distortion.
One use case would be to re-add the lens distortion (#30) and then apply superview-like stretching, but the only reason would be to emulate the superview-like effect as opposed to making cinematic footage where any distortion is typically unwanted.
from gyroflow.
Related Issues (20)
- Building on Gentoo linux HOT 21
- (Android build) Preview on S22 Ultra Android 14 does not play with videos 5min+ HOT 9
- Improve "Browse" feature HOT 1
- Inquiry Regarding Implementation Details of lock_horizon_angle Function in horizon.rs HOT 11
- Switching "Device for video processing" from OpenCL to wgpu, then back to OpenCL results in broken render HOT 1
- mkv file format not supported HOT 8
- Full Sony metadata support HOT 7
- ffmpeg error for the export HOT 9
- 2d Optical-flow stabalization HOT 3
- Export-Folderaccess for Render queue under MacOS HOT 2
- BRAW crashing when loading new file after successfully loaded another one HOT 1
- Trimm settings is not working correctly from created settings preset HOT 8
- Implement AI-based optical flow
- Automatic limit of lens profile radial distortion
- Motion blur magic
- Broken on Nobara Linux HOT 2
- smoothed gyro data or final applied rotation matrics for image stabilization HOT 3
- Collect stats which lens profiles from the database are used for rendering
- Loading second BRAW file on Mac hangs
- Wrong "dynamic zooming" russian translation HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gyroflow.