katajakasa / sdl_kitchensink Goto Github PK

I've packaged SDL_kitchensink for Gentoo in https://gitlab.com/src_prepare/src_prepare-overlay and I had to overcome CMake's strange behavior of generating a static library no matter what. This is not needed if a user does not set USE=static, so there's an exception in the ebuild that uses find with -delete to remove the static library if it is undesired.

Currently SDL_kitchensink uses older (deprecated) decoder API from ffmpeg to make sure things work with earlier ffmpeg versions. However, sooner or later it's going to be deprecated, and the library needs to move to the new API.

Implement the new api beside the old one and support both for now.

Positions are currently somewhat wrong

Current threading is a mess. We should ideally have two threads:

• An IO thread which reads input and does demuxing
• A decoder thread which reads packets and actually decodes stuff
• Some sort of command buffer for instructions instead of a global lock
• Use only new ffmpeg api -- take advantage of ffmpegs internal buffering
• Proper SDL_cond waits

This assumption is just wrong. There are Unix development environments for Windows like Cygwin and msys2, and there are ways to detect if we are running in them. Cygwin and msys2 should for example be able to call uname -a on Windows and return something like MINGW64_NT-10.0 hostname 2.11.1(0.329/5/3) 2018-09-10 14:19 x86_64 Msys, while NMake would have no clue what did we just ask for.

Hi,

I started playing with your library today to see if it could help in fast prototyping, it does. That's said, i spotted a bug where you won't be able to seek backward or even stop the player when you are in the last seconds of a video stream (say around 10 second before the end of the stream).
I didn't find the time yet to go deeply into your sources, but i think the "dec_lock" is never unlocked, and so the program hang.

I have fatal issu when i try to compile SDL_kitchensink:

In function ‘_HandleBitmapSubtitle’:

/SDL_kitchensink-master/src/kitplayer.c:498:17: 
error: ‘AVSubtitleRect {aka struct AVSubtitleRect}’ has no member named ‘data’
 rect->data[0],

/src/kitplayer.c:500:17: 
error: ‘AVSubtitleRect {aka struct AVSubtitleRect}’ has no member named ‘linesize’
 rect->linesize[0],

SDL_kitchensink-master/src/kitplayer.c:503:66:
 error: ‘AVSubtitleRect {aka struct AVSubtitleRect}’ has no member named ‘data’
SDL_SetPaletteColors(s->format->palette, (SDL_Color*)rect->data[1], 0, 256);

Cordially

Text subtitles. SDL_ttf ?

Doxygen ?

Release version 1.0.0.

• Run a valgrind and asan runs
• Test all example videos
• Add docs to public headers #6
• Publish docs somewhere
• Set correct version number in cmakelists.txt
• Make sure that SONAME etc. are correct in result library

Hello, I can not run make -j command on my computer. Here is the log:

[ 90%] Building C object CMakeFiles/SDL_kitchensink.dir/src/internal/utils/kitringbuffer.c.o
[ 92%] Building C object CMakeFiles/SDL_kitchensink_static.dir/src/internal/subtitle/renderers/kitsubrenderer.c.o
[ 95%] Building C object CMakeFiles/SDL_kitchensink.dir/src/internal/subtitle/renderers/kitsubass.c.o
[ 97%] Linking C shared library libSDL_kitchensink.so
[100%] Linking C static library libSDL_kitchensink_static.a
[100%] Built target SDL_kitchensink_static
/usr/bin/ld: /usr/local/lib/libavcodec.a(vc1dsp_mmx.o): relocation R_X86_64_PC32 against symbol `ff_pw_9' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
CMakeFiles/SDL_kitchensink.dir/build.make:568: recipe for target 'libSDL_kitchensink.so.1.0.7' failed
make[2]: *** [libSDL_kitchensink.so.1.0.7] Error 1
CMakeFiles/Makefile2:136: recipe for target 'CMakeFiles/SDL_kitchensink.dir/all' failed
make[1]: *** [CMakeFiles/SDL_kitchensink.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2
Whate shall I do to make it work?

While trying to make videos loop, I ran into something. Basically, the video simply quits decoding. The way I do looping is when the video stops without an error, I assume it's at the end, so I seek to 0 and start it playing again.

Kit_PlayerSeek(cin->player, 0.0);
Kit_PlayerPlay(cin->player);

In looking at the code, it looks like a mutex is left locked. In the thread main loop, there's this:

        if(SDL_LockMutex(player->dec_lock) == 0) {
            while((ret = _DemuxStream(player)) == -1);
            if(ret == 1) {
                player->state = KIT_STOPPED;
                continue;
            }

There's no corresponding unlock before the continue.

Also, all my videos end about 7 to 8 seconds early, no matter the length. Not sure what the deal with that is. :)

We should support fetching extended information about streams (+ other playback information) from the player.

Stream selection should be done in player, not in source.

• Source should be a pure representation of source data
• Player should decide on what to play and how to play it.
• Player should have stream selection in create functions and during-playback selector functions.

libass dep required ?

Subtitle parsing should be implemented.

Old ffmpeg api support should be dropped, and library internals rewritten to only work with the new ffmpeg decoder api. This should make code cleaner and behave better.

There are still cases where errors may happen; those should be found and fixed.

On the decoder input side, we don't need the locks anymore since we no longer use multiple threads.

=> Remove extra mutexes.

Buffer sizes should be decided runtime from incoming stream bitrates. Currently they are hardcoded, and take either too much memory for low bitrate streams or have too little space for the high bitrate streams.

SDL_kitchensink builds /usr/lib/libSDL_kitchensink.so which gets installed in a public library directory, but has an SONAME that does not contain any versioning information, either after the .so or before it and set off by a hyphen. It cannot therefore be represented in the shlibs system, and if linked by binaries its interface cannot safely change. There is no backward-compatible way to migrate programs linked against it to a new ABI.

(See https://lintian.debian.org/tags/shlib-without-versioned-soname.html for more documentation )

... Seems wrong, currently.

This was originally part of v1 plans, but moved to v2 now.

SDL_Texture is basically a wrapper for OpenGL/Direct3D textures. It might be a good idea to let the library decode directly on those (In addition to / instead of) SDL_Textures.

Consider:

• SDL_Surface or custom bitmap surface
• OpenGL Texture
• Direct3D Texture
• Vulkan ?

Hi,

I trusted the last changes, and maybe I was too fast. Testing all examples lead to instant crash.

CreatePlayer seems to be the root. data, pts seem to be null. Just in case it could help, I've put the stack.

Waiting, I'll reverse the changes on my side.

HTH

Thread 1 "simple" received signal SIGSEGV, Segmentation fault.
0x0000000000405331 in dec_decode_audio_cb (dec=0x2392640, in_packet=0x24c63c0)
    at /home/eric/Devel/minidart/miniDart_GitLab/SDL_Kitchensink/DEBUG/SDL_kitchensink/src/internal/audio/kitaudio.c:175
175	            out_packet = _CreateAudioPacket(
(gdb) p in_packet 
$1 = (AVPacket *) 0x24c63c0
(gdb) p *in_packet 
$2 = {buf = 0x23c37c0, pts = 0, dts = 0, 
  data = 0x2428510 "\vw:\001\032@\353\370@>\225^Ye\226PE/\awF@t\346\023؎:\237\352\345\025\272o\322BuNj\250j]o|\346\264e\356l\270\200\345\065\067\266\022\270~\375\372Y\317_\276T\247\202UO\323&S=\322X\202\353[\204\372\070\235\223Nw\210b\270i`ҧF\352\vp\211V\205Z\222\245\066ߪy\276\025uI\235=\256\366\033\362\060\225\221R\272\324\067\317\333)~\251\363\330ob%\234\372\223\335\357\237V\205\rK\347Ы\216\371.\272\230\206@\"\231\352d\273\367®\251\025\272O\324oK]\330\327O\237WL\351\355Z\357\351Q\261\071\315wϳ8\247\r\364%O\237>U\006\275", <incomplete sequence \352>..., size = 1280, stream_index = 1, 
  flags = 1, side_data = 0x0, side_data_elems = 0, duration = 1536, pos = 48, convergence_duration = 0}
(gdb) p *dev
No symbol "dev" in current context.
(gdb) p *dec
$3 = {stream_index = 1, clock_sync = 0, clock_pos = 0, output = {format = 32784, is_signed = 1, bytes = 2, 
    samplerate = 48000, channels = 2, width = 0, height = 0}, codec_ctx = 0x23a01c0, format_ctx = 0x238ed40, 
  output_lock = 0x2428a60, buffer = {0x2396990, 0x2391f10}, userdata = 0x2396520, 
  dec_decode = 0x405126 <dec_decode_audio_cb>, dec_close = 0x4053ce <dec_close_audio_cb>}
(gdb) q
A debugging session is active.

This could affect future releases. Compiling against FFmpeg 3.2.

/home/rmoog/git/SDL_kitchensink/src/kitplayer.c: In function ‘_InitCodecs’:
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:90:9: warning: ‘codec’ is deprecated [-Wdeprecated-declarations]
         acodec = avcodec_find_decoder(format_ctx->streams[src->astream_idx]->codec->codec_id);
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:9:0:
/usr/include/libavformat/avformat.h:893:21: note: declared here
     AVCodecContext *codec;
                     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:98:9: warning: ‘avcodec_copy_context’ is deprecated [-Wdeprecated-declarations]
         if(avcodec_copy_context(acodec_ctx, format_ctx->streams[src->astream_idx]->codec) != 0) {
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:8:0:
/usr/include/libavcodec/avcodec.h:4240:5: note: declared here
 int avcodec_copy_context(AVCodecContext *dest, const AVCodecContext *src);
     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:98:9: warning: ‘codec’ is deprecated [-Wdeprecated-declarations]
         if(avcodec_copy_context(acodec_ctx, format_ctx->streams[src->astream_idx]->codec) != 0) {
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:9:0:
/usr/include/libavformat/avformat.h:893:21: note: declared here
     AVCodecContext *codec;
                     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:116:9: warning: ‘codec’ is deprecated [-Wdeprecated-declarations]
         vcodec = avcodec_find_decoder(format_ctx->streams[src->vstream_idx]->codec->codec_id);
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:9:0:
/usr/include/libavformat/avformat.h:893:21: note: declared here
     AVCodecContext *codec;
                     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:124:9: warning: ‘avcodec_copy_context’ is deprecated [-Wdeprecated-declarations]
         if(avcodec_copy_context(vcodec_ctx, format_ctx->streams[src->vstream_idx]->codec) != 0) {
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:8:0:
/usr/include/libavcodec/avcodec.h:4240:5: note: declared here
 int avcodec_copy_context(AVCodecContext *dest, const AVCodecContext *src);
     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:124:9: warning: ‘codec’ is deprecated [-Wdeprecated-declarations]
         if(avcodec_copy_context(vcodec_ctx, format_ctx->streams[src->vstream_idx]->codec) != 0) {
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:9:0:
/usr/include/libavformat/avformat.h:893:21: note: declared here
     AVCodecContext *codec;
                     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:141:9: warning: ‘codec’ is deprecated [-Wdeprecated-declarations]
         scodec = avcodec_find_decoder(format_ctx->streams[src->sstream_idx]->codec->codec_id);
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:9:0:
/usr/include/libavformat/avformat.h:893:21: note: declared here
     AVCodecContext *codec;
                     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:149:9: warning: ‘avcodec_copy_context’ is deprecated [-Wdeprecated-declarations]
         if(avcodec_copy_context(scodec_ctx, format_ctx->streams[src->sstream_idx]->codec) != 0) {
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:8:0:
/usr/include/libavcodec/avcodec.h:4240:5: note: declared here
 int avcodec_copy_context(AVCodecContext *dest, const AVCodecContext *src);
     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:149:9: warning: ‘codec’ is deprecated [-Wdeprecated-declarations]
         if(avcodec_copy_context(scodec_ctx, format_ctx->streams[src->sstream_idx]->codec) != 0) {
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:9:0:
/usr/include/libavformat/avformat.h:893:21: note: declared here
     AVCodecContext *codec;
                     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c: In function ‘_HandleVideoPacket’:
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:348:9: warning: ‘avcodec_decode_video2’ is deprecated [-Wdeprecated-declarations]
         int len = avcodec_decode_video2(vcodec_ctx, player->tmp_vframe, &frame_finished, packet);
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:8:0:
/usr/include/libavcodec/avcodec.h:4811:5: note: declared here
 int avcodec_decode_video2(AVCodecContext *avctx, AVFrame *picture,
     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c: In function ‘_HandleAudioPacket’:
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:423:9: warning: ‘avcodec_decode_audio4’ is deprecated [-Wdeprecated-declarations]
         len = avcodec_decode_audio4(acodec_ctx, aframe, &frame_finished, packet);
         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:8:0:
/usr/include/libavcodec/avcodec.h:4762:5: note: declared here
 int avcodec_decode_audio4(AVCodecContext *avctx, AVFrame *frame,
     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c: In function ‘Kit_CreatePlayer’:
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:999:13: warning: ‘codec’ is deprecated [-Wdeprecated-declarations]
             if(st->codec->codec_type == AVMEDIA_TYPE_ATTACHMENT && attachment_is_font(st)) {
             ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:9:0:
/usr/include/libavformat/avformat.h:893:21: note: declared here
     AVCodecContext *codec;
                     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:1009:25: warning: ‘codec’ is deprecated [-Wdeprecated-declarations]
                         (char*)st->codec->extradata,
                         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:9:0:
/usr/include/libavformat/avformat.h:893:21: note: declared here
     AVCodecContext *codec;
                     ^
/home/rmoog/git/SDL_kitchensink/src/kitplayer.c:1010:25: warning: ‘codec’ is deprecated [-Wdeprecated-declarations]
                         st->codec->extradata_size);
                         ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitplayer.c:9:0:
/usr/include/libavformat/avformat.h:893:21: note: declared here
     AVCodecContext *codec;
                     ^
[ 85%] Building C object CMakeFiles/SDL_kitchensink_static.dir/src/kitringbuffer.c.o
[ 90%] Building C object CMakeFiles/SDL_kitchensink_static.dir/src/kitsource.c.o
/home/rmoog/git/SDL_kitchensink/src/kitsource.c: In function ‘Kit_GetSourceStreamInfo’:
/home/rmoog/git/SDL_kitchensink/src/kitsource.c:66:5: warning: ‘codec’ is deprecated [-Wdeprecated-declarations]
     switch(stream->codec->codec_type) {
     ^
In file included from /home/rmoog/git/SDL_kitchensink/src/kitsource.c:5:0:
/usr/include/libavformat/avformat.h:893:21: note: declared here
     AVCodecContext *codec;
                     ^

Currently we blend the subtitles on the fly on every frame. This is slow and should be replaced with hardware accelerated solution.

Instead, let's make a texture atlas containing the subtitle surfaces, and render from there. Texture size should be as small as possible, but still large enough to be able to contain enough images. There should also be some sort of max limit + culling of old surfaces.

• Find a suitable texture size. Anything up to 4096x4096 seems to be well supported. We should probably scale this up or down according to video size.
• Implement some sort of atlas manager that can squash images to the texture and then upload the texture
• Reorganize, cull and reupload the atlas texture when necessary.
• Implement a public API call for reading the atlas contents
• Update the video playback example

Like SDL_RWFromFile, it should be possible to open a video/audio file from the Android assets directory.

This happens on FreeBSD 11.0 on Raspberry Pi B+

This happens when trying to run example video player on FreeBSD 11.0 on amd64 in a VM.

Currently we don't support HW decoding. This one is something for rainy days and boredom :P

It has been reported in Debian that sdl-kitchensink fails to build from source with gcc-10, see https://bugs.debian.org/957784

/usr/bin/ld: CMakeFiles/SDL_kitchensink.dir/src/internal/subtitle/renderers/kitsubass.c.o:./obj-x86_64-linux-gnu/./include/kitchensink/internal/libass.h:53: multiple definition of `ass_process_chunk'; CMakeFiles/SDL_kitchensink.dir/src/internal/audio/kitaudio.c.o:./obj-x86_64-linux-gnu/./include/kitchensink/internal/libass.h:53: first defined here
…

As documented here https://gcc.gnu.org/gcc-10/porting_to.html;

A common mistake in C is omitting extern when declaring a global variable in a header file. If the header is included by several files it results in multiple definitions of the same variable. In previous GCC versions this error is ignored. GCC 10 defaults to -fno-common, which means a linker error will now be reported.

Indeed, I can confirm that passing -fcommon to gcc will let the build pass, but this is just a temporary workaround I would say.

Platform

• OS: Debian unstable/sid as of 2020-07-23, with GCC-10
• SDL version 2.0.12+dfsg1-1
• FFMPEG version: 7:4.3-3+b1
• SDL_Kitchensink version : 1.0.8

Hi,

In src/internal/video/kitvideo.c, we have:

    17	enum AVPixelFormat supported_list[] = {
    18	    AV_PIX_FMT_YUV420P,
    19	    AV_PIX_FMT_YUYV422,
    20	    AV_PIX_FMT_UYVY422,
    21	    AV_PIX_FMT_NV12,
    22	    AV_PIX_FMT_NV12,
    23	    AV_PIX_FMT_RGB24,
    24	    AV_PIX_FMT_BGR24,
    25	    AV_PIX_FMT_RGB555,

Looks like the AV_PIX_FMT_NV12 is declared twice.

Suggested change: https://github.com/ebachard/SDL_kitchensink/commit/66c320153ce3ec0b6f102efce2e3972332883bcb

There are more information about source that can be fetched from ffmpeg, we should support fetching it.

Currently, when audio needs to be synced, we either offer silence or just skip samples. We should probably try to stretch or shrink the audio buffer instead, and correct the sync problem gradually.

I've got a use case for this player in an opensource game engine, but it's required that the media player be able to play from char *, given the size of the cached media file. I've forked and done a commit of my work so far, but it segfaults. Could you please take a look?
https://github.com/rmoog/SDL_kitchensink/commit/04c315e88692573418fc84fc37804df2379f8410

Currently there is only a single thread which is used to decode everything. Audio, video and subtitle decoding should all be moved to their own threads.

The current library will create a source from a URL, but what would be handy is another function that created a source using custom read and seek routines. Looking at the source for Kit_CreateSourceFromUrl and at examples of creating a custom ffmpeg io context (like this one: https://www.codeproject.com/Tips/489450/Creating-Custom-FFmpeg-IO-Context ), it seems like that should be fairly easy. This would be a major boon to people doing things like reading files through PHYSFS or the like.

Maybe, maybe not. Afaik encoding is relatively easy to do with just native ffmpeg, so this is probably not required anyway.

The current libass renderer can only handle small amounts of images at once. Larger amounts will completely choke the renderer.

Right now the decoder thread just uses SDL_Delay to make sure that the thread does not use all resources. Instead we should use SDL_cond to signal the thread to decode more when there is room in output buffers.

msys2 holds includes and libs in respective toolchain directories, and it has start scripts that let you fire up the console fully configured for the desired compile target (amd64 or x86). In the case of amd64, SDL.h lives in /mingw64/include/SDL2/SDL.h and the include dir in amd64 msys2 gcc is /mingw64/include.

kitaudio.c searches for SDL.h in the configured usr catalog, so to support that, maybe we could use something that asks sdl2-config for locations, or, add preprocessor defines that try to guess if we're on msys2 and then ask for SDL.h from <SDL2/SDL.h>? Both solutions would work fine:

$ sdl2-config --cflags
-I/mingw64/include/SDL2 -Dmain=SDL_main

When starting playback and heavy amounts of subtitles hit the screen, playback may halt due to the time it takes for libass to handle rendering glyphs to cache etc.

It seems that it might be helpful to have looping functionality builtin to the player.

• Looping selector in create functions.
• -1 for infinite, 0 for no looping, >0 for loop count selection ?

Currently we seek to keyframes; we should probably seek to any frames. Just jumping to a random non-keyframe will cause garbage though, maybe first seek to keyframe and then forward or rewind one frame at a time until wanted pts is reached (?).

For some videos, there is either no audio or audio stutters at video start. Find out why and fix.

Currently only 1 or 2 channels supported. SDL can do more, we should too.