darbyjohnston / tlrender Goto Github PK

With similar features to DJV:

• Customizable radius
• Convert to different pixel types
• Sample both "scene" and "display" colors

We currently use OpenGL "GL_LINEAR" mode for image scaling which suffers from artifacts when making thumbnail images. It would be nice to have a high quality image scaling mode for thumbnails, maybe similar to the one in DJV 1.3.

https://www.newtek.com/ndi/applications/

A feature of mrViewer is that it allows changing versions of a clip (movie or sequence). This would be an addition to the FilesModel to allow replacing a file with another one (and the likely recreation of the underlying timeline).

The versioning (using a regex or other convention to match versions) can be done by the viewing application.

I was trying to erase the drawn shapes in an FBO and found out I could not as drawMesh() would reset my setting of:

    glBlendFunc( GL_ZERO, GL_ONE_MINUS_SRC_ALPHA );

to:

  glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

Similar to DJV, with the ability to use the grid to identify individual pixels.

To do so, FFmpeg needs to be compiled with the --enable-libvpx option. This optonal BSD codec/format is very important as it is specifically designed to work at 8K. Most 8K movies use it.
Currently, albeit I added code to turn on the codec (trivial code change), tlRender reads the movies partially and then shows the frames black. Just like what happened with the TopGun clip I showed you once.

It would be nice to have a histogram tool similar to DJV. Are there other tools that would be helpful for inspection and QA?

Also related:
#29
#31
#34

When libdeflate.a (or libdeflate.so) is found in one of the library dirs, compilation of the example programs fails with errors like:

tif_zip.c:(.text+0xfe): undefined reference to `libdeflate_free_decompressor'

The quick fix is to add:

-Dlibdeflate=OFF to the CMAKE_ARGS of BuildTIFF.cmake.

A better fix would be to include the deflate library in the linking phase, albeit I was unable to find where:

find_package(Deflate)

is found to see what library variable it sets.

There's an expat dependency in OCIO. I think the super-build should be responsible for fetching that bit... Building on macOS yields this (after I hack in TLR_PLATFORM_MACOS):

ld: library not found for -lOCIO_expat-NOTFOUND

I think the issue is perhaps with OCIO? expat in theory is packaged as a system library...

https://opentimelineio.readthedocs.io/en/stable/api/python/opentimelineio.adapters.otioz.html

The Mac build puts the dylibs in the install folder, and references them there. The Otio libs expect to be copied as siblings to the executable.

Most likely, all the install/lib/ dylibs including otio's should be referenced as @loader_path/.../lib/libfoo.dylib

This likely involves install_name_tool shenanigans at the end of the build. Currently, I'm working around it by manually copying the otio dylibs into the bin dir (the example then runs properly)

otool -L ./tlrplay-glfw | grep dylib
	/usr/lib/libexpat.1.dylib (compatibility version 7.0.0, current version 8.0.0)
	@loader_path/libopentimelineio.dylib (compatibility version 0.0.0, current version 0.0.0)
	@loader_path/libopentime.dylib (compatibility version 0.0.0, current version 0.0.0)
	/Users/nporcino/dev/tlRender/build/install/lib/libavdevice.58.dylib (compatibility version 58.0.0, current version 58.13.100)
	/Users/nporcino/dev/tlRender/build/install/lib/libavfilter.7.dylib (compatibility version 7.0.0, current version 7.110.100)
	/Users/nporcino/dev/tlRender/build/install/lib/libavformat.58.dylib (compatibility version 58.0.0, current version 58.76.100)
	/Users/nporcino/dev/tlRender/build/install/lib/libavcodec.58.dylib (compatibility version 58.0.0, current version 58.134.100)
	/Users/nporcino/dev/tlRender/build/install/lib/libswresample.3.dylib (compatibility version 3.0.0, current version 3.9.100)
	/Users/nporcino/dev/tlRender/build/install/lib/libswscale.5.dylib (compatibility version 5.0.0, current version 5.9.100)
	/Users/nporcino/dev/tlRender/build/install/lib/libavutil.56.dylib (compatibility version 56.0.0, current version 56.70.100)

Currently we rely on using FFmpeg for converting the audio in files to a format supported by the audio device. This seems to work fairly well, but since FFmpeg is a large dependency it might be nice to investigate alternative approaches like libsamplerate.

ociobake allows offline creation of look up tables. https://opencolorio.readthedocs.io/en/latest/tutorials/baking_luts.html If tlRender allowed as an option that a precomputed LUT were passed in, the runtime dependency on OCIO could be removed, or made optional while still supporting OCIO color profiles. Another option would be to take tlRender's existing texture generating function and wrap it in a simple command line tool. The issue at hand is that the OCIO routines used by tlRender use glsl shaders from OCIO to do then work, which is a can of worms when the run time is not an OpenGL runtime.

The latest "main" branch has a regression in that .otio files don't show movie files properly, when the movie file has a timecode in it and the .otio file has a source_range that does not begin at the beginning of the movie file.

The latest "main" branch will show the movie file from the beginning of the movie, not from the source_range.

For reference, mrv2 v0.6.4 would display these type of files properly. mrv2 v0.7.1 no longer does.

Find a .zip file with a sample .otio and .mov file here:

https://mega.nz/file/zD4UkAgB#jauJY7hfP9bAqwR4IFIKlquutyMyTWs4WPbQsdj3wfE

When checking renders with alpha, it is often useful to switch from seeing them without compositing (with a black background) and with compositing (some other background like a gray one as I have in mrv2).

I thought that:
timeline::AlphaBlend::None

would do it, but it doesn't.

Currently the renderer only supports a single OCIO configuration.

While the Read Behind / Read Ahead cache settings are easy to understand and work fine for movies. When dealing with image sequences, you want to maximize the memory (ie. set a specific value in GB and forget about it).

Both background color and checkers pattern.

I am seeing an issue on Linux (Windows on the same machine is fine) where the cache has data, yet there's a stutter during playbacks. Looking into the code, the FLTK timeouts are being called properly, but the currentVideoObserver in TimelinePlayer.cpp is not getting called sometimes, skipping several frames even. The cache line shows the buffer is full, so that's not it. Maybe the thread reading code is blocking?
This is with a merged scene branch and with the SolLevante movie (other movies are fine).

Using play-glfw on a 4K mp4 movie takes tlRender about 4 seconds to start. With tlplay creating the thumbnails it is much worse and playback is not smooth at least until the thumbnails are created.

In comparison, mrViewer opens the 4K movie in 1 second or less.

A commit made the audio start late. When the audio was added with the -a flag, all was still good.
I didn't do a bisect, but audio was fine at least up to:

b476f32

BART_2021-02-07.m4v, PSR63_2012-06-02.mov and Dinky_2015-06-11.m4v display black on all players (linux tested only), at least on this commit:

d4f302f

       _timelinePlayer->tick();
        const auto& frame = _timelinePlayer->observeFrame()->get();
        if (frame != _frame)
        {
            _frame = frame;
            _renderDirty = true;
        }

Efficiency question... It's sufficient just to compare frame.time, isn't it? the frame comparison also compares layers. Even if layers changed mid frame, we've already rendered for the time, in other words if time didn't change and layers did, I don't think it's necessary to trap it.

It looks like there is a slight color shift with the YUV to RGB color conversion, for example with this file:
https://richardssam.github.io/ffmpeg-tests/tests/greyramp-fulltv/greyscale-full.mp4

The pixels in the center of the image have RGB values of about 127, 129, 126.

Is there a better equation to use for the conversion that won't introduce a color shift?

Add support for playing every frame, whether the playback is realtime or not.

Where these intended to be set by Cmake? If no, it might be sufficient to replace them with a test against __APPLE__.

Not essential, but useful. Reverse playback with audio.

For example when viewing a rendered sequence that is still in progress, it would be nice to reload the rendered images as they are updated.

Either free-form rotation or presets like 0, 90, 180, and 270 degrees.

I created an .otio file from mrv2's Playlist and albeit it loads and reports no errors, the first clip plays without audio. Jumping to the second clip makes audio play. This happens on mrv2 (all platforms) and tlplay (tested Linux only), and only with some movie files (one's with timecode different than 0?).

You can download the two movies and the .otio file from here:

https://mega.nz/file/SSIg3KBI#vQwCgpggxriqbqx7UvXWMHRJygEoUB_bV2FjnWW_FGg

It seems either something is broken on the new otio scheme or in tlRender's handling of it.

• Aspect ratio
• Safe area
• Rule of thirds
• Center cross

Add a new application, maybe "tlrbake", that can read an .otio file and turn it into a movie.

Feature request:

Add scrubbing of timeline with audio.