dawelter / h264decoder Goto Github PK

h264 decoding module for python based on libav

CMake 7.14% C++ 49.64% Python 43.23%

ffmpeg h264 pybind11 python video-decoding

h264decoder's Introduction

H264 Decoder Python Module

The aim of this project is to provide a simple decoder for video captured by a Raspberry Pi camera. At the time of this writing I only need H264 decoding, since a H264 stream is what the RPi software delivers. Furthermore flexibility to incorporate the decoder in larger python programs in various ways is desirable.

The code might also serve as example for libav and pybind11 usage. /blob/CI/CD-improvements/.github/workflows/python-package.yml

Examples

You can do something like this

import h264decoder
import numpy as np

f = open(thefile, 'rb')
decoder = h264decoder.H264Decoder()
while 1:
  data_in = f.read(1024)
  if not data_in:
    break
  framedatas = decoder.decode(data_in)
  for framedata in framedatas:
    (frame, w, h, ls) = framedata
    if frame is not None:
        #print('frame size %i bytes, w %i, h %i, linesize %i' % (len(frame), w, h, ls))
        frame = np.frombuffer(frame, dtype=np.ubyte, count=len(frame))
        frame = frame.reshape((h, ls//3, 3))
        frame = frame[:,:w,:]
        # At this point `frame` references your usual height x width x rgb channels numpy array of unsigned bytes.

There are simple demo programs in the examples folder. display_frames.py is probably the one you want to take a look at.

Requirements

Python 3
cmake for building
libav / ffmpeg (swscale, avutil and avcodec)
pybind11 (will be automatically downloaded from github if not found)

For the example scripts

matplotlib
numpy

I tested it on

Ubuntu 18, gcc 9, Anaconda environment with Python 3.7, Libav from Ubuntu repo.
Windows 10, Visual Studio Community 2017, Anaconda environment with Python 3.7, FFMPEG from vcpkg.

Building and Installing

Windows

The suggested way to obtain ffmpeg is through vcpkg. Assuming all the setup including VC integration has been done, we can install the x64 libraries with

vcpkg.exe install ffmpeg:x64-windows

We can build the extension module with setuptools almost normally. However cmake is used internally and we have to let it know the search paths to our libs. Hence the additional --cmake-args argument with the toolchain file as per vcpkg instructions.

python setup.py build_ext --cmake-args="-DCMAKE_TOOLCHAIN_FILE=[path to vcpkg]/scripts/buildsystems/vcpkg.cmake"
pip install -e .

The -e option installs symlinks to the build directory. Useful for development. Leave it out otherwise.

Alternatively one can build the extension module manually with cmake. From the project directory:

mkdir [build dir name]
cd [build dir name]
cmake -DCMAKE_TOOLCHAIN_FILE=[path to vcpkg]/scripts/buildsystems/vcpkg.cmake -A x64 ..
cmake --build .

Linux

Should be a matter of installing the libav or ffmpeg libraries. On Debian or Ubuntu:

sudo apt install libswscale-dev libavcodec-dev libavutil-dev

And then running

pip install .

in the project directory.

History

v2

For Python 3. Switch to PyBind11. Module renamed from libh264decoder to h264decoder! Support installation via setuptools.

v1

For Python 2.7. Depends on Boost Python. Project/Build file generation with CMake.

Credits

Michael Welter. Original author.
Martin Valgur. Switch to pybind11, nice build configuration and more.

License

The code is published under the Mozilla Public License v. 2.0.

h264decoder's People

Contributors

Stargazers

Watchers

h264decoder's Issues

Error LNK2001: unresolved external symbol

after I type

python setup.py build_ext --cmake-args="-DCMAKE_TOOLCHAIN_FILE=[path to vcpkg]/scripts/buildsystems/vcpkg.cmake"

I found error in powershell
error LNK2001: unresolved external symbol av_parser_parse2 [D:\h264decoder\h264decoder-master\bu
ild\temp.win-amd64-3.7\Release\h264decoder.vcxproj]

How to fix it?

libavcodec.so.57: Cannot open shared object file

Hi all,

I was trying out both Tello_Video & Tello_Video_With_Pose_Recognition packages in my Linux - ubuntu 16.04 LTS

At first, I installed Tello_Video with the one click install and it has successfully run, however, when i tries to install and run Tello_VIdeo_With_Pose_Recognition, it shows the following errors, and now it no longer runs again.

I have checked and I moved the 'libh264decoder.so' file into my python's site-packages file & /usr/lib/ , the above move was suggested by the internet. But still getting the error:

spark@spark:~/Downloads/Tello-Python-master/Tello_Video_With_Pose_Recognition$ python main.py

Traceback (most recent call last):
File "main.py", line 1, in
import tello
File "/home/spark/Downloads/Tello-Python-master/Tello_Video_With_Pose_Recognition/tello.py", line 5, in
import libh264decoder
ImportError: libavcodec.so.57: cannot open shared object file: No such file or directory
spark@spark:~/Downloads/Tello-Python-master/Tello_Video_With_Pose_Recognition$ cd ..
spark@spark:~/Downloads/Tello-Python-master$ cd Tello_Video

spark@spark:~/Downloads/Tello-Python-master/Tello_Video$ python main.py
Traceback (most recent call last):
File "main.py", line 1, in
import tello
File "/home/spark/Downloads/Tello-Python-master/Tello_Video/tello.py", line 5, in
import libh264decoder
ImportError: libavcodec.so.57: cannot open shared object file: No such file or directory
spark@spark:~/Downloads/Tello-Python-master/Tello_Video$ `

Any help will be greatly appreciated.

support h264decoder for python3 and macOS

hi together,

I love your decoder since this one really works with python2 and devices like the Tello drone.
Unluckily, python2 is not supported anymore :(
your decoder is really awesome, I would love to get a current version for python3

Install failed on ubuntu22.04 with python 3.8 on ffmpeg 3.4.9

I'm running
pip install .
and it went wrong, here is the output

yuanzl@yuanzl-pi5:~/h264decoder-master$ pip install .
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Processing /home/yuanzl/h264decoder-master
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: h264decoder
  Building wheel for h264decoder (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [69 lines of output]
      running bdist_wheel
      running build
      running build_ext
      -- The C compiler identification is GNU 9.5.0
      -- The CXX compiler identification is GNU 9.5.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found PythonInterp: /usr/bin/python3 (found suitable version "3.8.18", minimum required is "3.6")
      -- Found PythonLibs: /usr/lib/aarch64-linux-gnu/libpython3.8.so
      -- Performing Test HAS_FLTO
      -- Performing Test HAS_FLTO - Success
      -- Found pybind11: /home/yuanzl/.local/include (found version "2.11.1")
      Using existing pybind11 v2.11.1
      -- Configuring done
      -- Generating done
      -- Build files have been written to: /home/yuanzl/h264decoder-master/build/temp.linux-aarch64-3.8
      [ 25%] Building CXX object CMakeFiles/h264decoderlib.dir/src/h264decoder.cpp.o
      /home/yuanzl/h264decoder-master/src/h264decoder.cpp: In constructor ‘H264Decoder::H264Decoder()’:
      /home/yuanzl/h264decoder-master/src/h264decoder.cpp:40:28: error: ‘AV_CODEC_CAP_TRUNCATED’ was not declared in this scope; did you mean ‘AV_CODEC_CAP_HARDWARE’?
         40 |   if(codec->capabilities & AV_CODEC_CAP_TRUNCATED) {
            |                            ^~~~~~~~~~~~~~~~~~~~~~
            |                            AV_CODEC_CAP_HARDWARE
      /home/yuanzl/h264decoder-master/src/h264decoder.cpp:41:23: error: ‘AV_CODEC_FLAG_TRUNCATED’ was not declared in this scope; did you mean ‘AV_CODEC_FLAG_DROPCHANGED’?
         41 |     context->flags |= AV_CODEC_FLAG_TRUNCATED;
            |                       ^~~~~~~~~~~~~~~~~~~~~~~
            |                       AV_CODEC_FLAG_DROPCHANGED
      gmake[2]: *** [CMakeFiles/h264decoderlib.dir/build.make:76: CMakeFiles/h264decoderlib.dir/src/h264decoder.cpp.o] Error 1
      gmake[1]: *** [CMakeFiles/Makefile2:85: CMakeFiles/h264decoderlib.dir/all] Error 2
      gmake: *** [Makefile:91: all] Error 2
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/home/yuanzl/h264decoder-master/setup.py", line 79, in <module>
          setup(
        File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/home/yuanzl/h264decoder-master/setup.py", line 46, in run
          self.build_extension(ext)
        File "/home/yuanzl/h264decoder-master/setup.py", line 77, in build_extension
          subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp)
        File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j2']' returned non-zero exit status 2.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for h264decoder
  Running setup.py clean for h264decoder
Failed to build h264decoder
ERROR: Could not build wheels for h264decoder, which is required to install pyproject.toml-based projects

I'm quite confused because It seems to be a incompatible question on ffmpeg, which AV_CODEC_CAP_TRUNCATED has been removed on ffmpeg 6.0, but I'm using ffmpeg 3.4.9 and I don't know how to fix that.

Is there anyone have the example file for testing the setup?

Is there anyone who has the example file mentioned in the example code?

I just want this file to run the demo code to make sure that my setup is correct.

...
f = open(thefile, 'rb')
...

Thank you all,

memory leak on the lastest version.

Hi, i am using your project to decoder video stream from drones. I found that there is a memory leak when decoding the data. Here is the code:

import socket
import h264decoder
from scanner import Scanner
import queue
from memory_profiler import profile

@profile
def _h264_decode(packet_data, decoder, queue):
    frames = decoder.decode(packet_data)
    for frame_data in frames:
        (frame, w, h, ls) = frame_data
        if frame is not None:
            frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
            frame = (frame.reshape((h, ls // 3, 3)))
            frame = frame[:, :w, :]
            while queue.qsize() > 1:
                queue.get()
            queue.put(frame)

def _receive_video_thread(video_socket, decoder, queue):
    pack_data = ''
    print("receive video thread start....")
    while True:
        try:
            res_string, ip = video_socket.recvfrom(2048)
            pack_data += res_string.hex()
            if len(res_string) != 1460:
                tmp = bytes.fromhex(pack_data)
                _h264_decode(tmp, decoder , queue)
                pack_data = ''
        except socket.error as exc:
            print("Caught exception socket.error(video_thread): %s" % exc)

scanner = Scanner('192.168.1.')
scanner.find_available_tello(1)
tello_list = scanner.get_tello_info()
print(tello_list)
video_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
video_socket.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 10 * 1024)
video_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
video_socket.bind(('', tello_list[0][2]))
soc = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
soc.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
soc.bind(('', tello_list[0][1]))
soc.sendto('streamon'.encode('utf-8'), (tello_list[0][0], 8889))
queue = queue.Queue()
decoder = h264decoder.H264Decoder()
_receive_video_thread(video_socket, decoder, queue)

The memory monitoring result is:

get image.
Filename: /home/jakeluo/Documents/tello_project/video_test.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     8   3080.6 MiB   3080.6 MiB           1   @profile
     9                                         def _h264_decode(packet_data, decoder, queue):
    10   3082.5 MiB      1.8 MiB           1       frames = decoder.decode(packet_data)
    11   3082.5 MiB      0.0 MiB           2       for frame_data in frames:
    12   3082.5 MiB      0.0 MiB           1           (frame, w, h, ls) = frame_data
    13   3082.5 MiB      0.0 MiB           1           if frame is not None:
    14   3082.5 MiB      0.0 MiB           1               print("get image.")
    15   3082.5 MiB      0.0 MiB           1               frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
    16   3082.5 MiB      0.0 MiB           1               frame = (frame.reshape((h, ls // 3, 3)))
    17   3082.5 MiB      0.0 MiB           1               frame = frame[:, :w, :]
    18   3082.5 MiB      0.0 MiB           2               while queue.qsize() > 1:
    19   3082.5 MiB      0.0 MiB           1                   queue.get()
    20   3082.5 MiB      0.0 MiB           1               queue.put(frame)


get image.
Filename: /home/jakeluo/Documents/tello_project/video_test.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     8   3082.5 MiB   3082.5 MiB           1   @profile
     9                                         def _h264_decode(packet_data, decoder, queue):
    10   3084.5 MiB      2.1 MiB           1       frames = decoder.decode(packet_data)
    11   3084.5 MiB      0.0 MiB           2       for frame_data in frames:
    12   3084.5 MiB      0.0 MiB           1           (frame, w, h, ls) = frame_data
    13   3084.5 MiB      0.0 MiB           1           if frame is not None:
    14   3084.5 MiB      0.0 MiB           1               print("get image.")
    15   3084.5 MiB      0.0 MiB           1               frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
    16   3084.5 MiB      0.0 MiB           1               frame = (frame.reshape((h, ls // 3, 3)))
    17   3084.5 MiB      0.0 MiB           1               frame = frame[:, :w, :]
    18   3084.5 MiB      0.0 MiB           2               while queue.qsize() > 1:
    19   3084.5 MiB      0.0 MiB           1                   queue.get()
    20   3084.5 MiB      0.0 MiB           1               queue.put(frame)


get image.
Filename: /home/jakeluo/Documents/tello_project/video_test.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     8   3084.5 MiB   3084.5 MiB           1   @profile
     9                                         def _h264_decode(packet_data, decoder, queue):
    10   3086.6 MiB      2.1 MiB           1       frames = decoder.decode(packet_data)
    11   3086.6 MiB      0.0 MiB           2       for frame_data in frames:
    12   3086.6 MiB      0.0 MiB           1           (frame, w, h, ls) = frame_data
    13   3086.6 MiB      0.0 MiB           1           if frame is not None:
    14   3086.6 MiB      0.0 MiB           1               print("get image.")
    15   3086.6 MiB      0.0 MiB           1               frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
    16   3086.6 MiB      0.0 MiB           1               frame = (frame.reshape((h, ls // 3, 3)))
    17   3086.6 MiB      0.0 MiB           1               frame = frame[:, :w, :]
    18   3086.6 MiB      0.0 MiB           2               while queue.qsize() > 1:
    19   3086.6 MiB      0.0 MiB           1                   queue.get()
    20   3086.6 MiB      0.0 MiB           1               queue.put(frame)


get image.
Filename: /home/jakeluo/Documents/tello_project/video_test.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     8   3086.6 MiB   3086.6 MiB           1   @profile
     9                                         def _h264_decode(packet_data, decoder, queue):
    10   3088.4 MiB      1.8 MiB           1       frames = decoder.decode(packet_data)
    11   3088.4 MiB      0.0 MiB           2       for frame_data in frames:
    12   3088.4 MiB      0.0 MiB           1           (frame, w, h, ls) = frame_data
    13   3088.4 MiB      0.0 MiB           1           if frame is not None:
    14   3088.4 MiB      0.0 MiB           1               print("get image.")
    15   3088.4 MiB      0.0 MiB           1               frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
    16   3088.4 MiB      0.0 MiB           1               frame = (frame.reshape((h, ls // 3, 3)))
    17   3088.4 MiB      0.0 MiB           1               frame = frame[:, :w, :]
    18   3088.4 MiB      0.0 MiB           2               while queue.qsize() > 1:
    19   3088.4 MiB      0.0 MiB           1                   queue.get()
    20   3088.4 MiB      0.0 MiB           1               queue.put(frame)


get image.
Filename: /home/jakeluo/Documents/tello_project/video_test.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     8   3088.4 MiB   3088.4 MiB           1   @profile
     9                                         def _h264_decode(packet_data, decoder, queue):
    10   3090.4 MiB      2.1 MiB           1       frames = decoder.decode(packet_data)
    11   3090.4 MiB      0.0 MiB           2       for frame_data in frames:
    12   3090.4 MiB      0.0 MiB           1           (frame, w, h, ls) = frame_data
    13   3090.4 MiB      0.0 MiB           1           if frame is not None:
    14   3090.4 MiB      0.0 MiB           1               print("get image.")
    15   3090.4 MiB      0.0 MiB           1               frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
    16   3090.4 MiB      0.0 MiB           1               frame = (frame.reshape((h, ls // 3, 3)))
    17   3090.4 MiB      0.0 MiB           1               frame = frame[:, :w, :]
    18   3090.4 MiB      0.0 MiB           2               while queue.qsize() > 1:
    19   3090.4 MiB      0.0 MiB           1                   queue.get()
    20   3090.4 MiB      0.0 MiB           1               queue.put(frame)


get image.
Filename: /home/jakeluo/Documents/tello_project/video_test.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     8   3090.4 MiB   3090.4 MiB           1   @profile
     9                                         def _h264_decode(packet_data, decoder, queue):
    10   3092.5 MiB      2.1 MiB           1       frames = decoder.decode(packet_data)
    11   3092.5 MiB      0.0 MiB           2       for frame_data in frames:
    12   3092.5 MiB      0.0 MiB           1           (frame, w, h, ls) = frame_data
    13   3092.5 MiB      0.0 MiB           1           if frame is not None:
    14   3092.5 MiB      0.0 MiB           1               print("get image.")
    15   3092.5 MiB      0.0 MiB           1               frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
    16   3092.5 MiB      0.0 MiB           1               frame = (frame.reshape((h, ls // 3, 3)))
    17   3092.5 MiB      0.0 MiB           1               frame = frame[:, :w, :]
    18   3092.5 MiB      0.0 MiB           2               while queue.qsize() > 1:
    19   3092.5 MiB      0.0 MiB           1                   queue.get()
    20   3092.5 MiB      0.0 MiB           1               queue.put(frame)


Filename: /home/jakeluo/Documents/tello_project/video_test.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     8   3092.5 MiB   3092.5 MiB           1   @profile
     9                                         def _h264_decode(packet_data, decoder, queue):
    10   3092.5 MiB      0.0 MiB           1       frames = decoder.decode(packet_data)
    11   3092.5 MiB      0.0 MiB           1       for frame_data in frames:
    12                                                 (frame, w, h, ls) = frame_data
    13                                                 if frame is not None:
    14                                                     print("get image.")
    15                                                     frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
    16                                                     frame = (frame.reshape((h, ls // 3, 3)))
    17                                                     frame = frame[:, :w, :]
    18                                                     while queue.qsize() > 1:
    19                                                         queue.get()
    20                                                     queue.put(frame)


Filename: /home/jakeluo/Documents/tello_project/video_test.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     8   3092.5 MiB   3092.5 MiB           1   @profile
     9                                         def _h264_decode(packet_data, decoder, queue):
    10   3092.5 MiB      0.0 MiB           1       frames = decoder.decode(packet_data)
    11   3092.5 MiB      0.0 MiB           1       for frame_data in frames:
    12                                                 (frame, w, h, ls) = frame_data
    13                                                 if frame is not None:
    14                                                     print("get image.")
    15                                                     frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
    16                                                     frame = (frame.reshape((h, ls // 3, 3)))
    17                                                     frame = frame[:, :w, :]
    18                                                     while queue.qsize() > 1:
    19                                                         queue.get()
    20                                                     queue.put(frame)


get image.
Filename: /home/jakeluo/Documents/tello_project/video_test.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     8   3092.5 MiB   3092.5 MiB           1   @profile
     9                                         def _h264_decode(packet_data, decoder, queue):
    10   3094.3 MiB      1.8 MiB           1       frames = decoder.decode(packet_data)
    11   3094.3 MiB      0.0 MiB           2       for frame_data in frames:
    12   3094.3 MiB      0.0 MiB           1           (frame, w, h, ls) = frame_data
    13   3094.3 MiB      0.0 MiB           1           if frame is not None:
    14   3094.3 MiB      0.0 MiB           1               print("get image.")
    15   3094.3 MiB      0.0 MiB           1               frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
    16   3094.3 MiB      0.0 MiB           1               frame = (frame.reshape((h, ls // 3, 3)))
    17   3094.3 MiB      0.0 MiB           1               frame = frame[:, :w, :]
    18   3094.3 MiB      0.0 MiB           2               while queue.qsize() > 1:
    19   3094.3 MiB      0.0 MiB           1                   queue.get()
    20   3094.3 MiB      0.0 MiB           1               queue.put(frame)


get image.
Filename: /home/jakeluo/Documents/tello_project/video_test.py

Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     8   3094.3 MiB   3094.3 MiB           1   @profile
     9                                         def _h264_decode(packet_data, decoder, queue):
    10   3096.4 MiB      2.1 MiB           1       frames = decoder.decode(packet_data)
    11   3096.4 MiB      0.0 MiB           2       for frame_data in frames:
    12   3096.4 MiB      0.0 MiB           1           (frame, w, h, ls) = frame_data
    13   3096.4 MiB      0.0 MiB           1           if frame is not None:
    14   3096.4 MiB      0.0 MiB           1               print("get image.")
    15   3096.4 MiB      0.0 MiB           1               frame = np.fromstring(frame, dtype=np.ubyte, count=len(frame), sep='')
    16   3096.4 MiB      0.0 MiB           1               frame = (frame.reshape((h, ls // 3, 3)))
    17   3096.4 MiB      0.0 MiB           1               frame = frame[:, :w, :]
    18   3096.4 MiB      0.0 MiB           2               while queue.qsize() > 1:
    19   3096.4 MiB      0.0 MiB           1                   queue.get()
    20   3096.4 MiB      0.0 MiB           1               queue.put(frame)

the function decode() is the cause of the memory leak. I have tried to fix it by myself, but i am not familiar with C++ and libav. Could you please tell me how to solve this problem?

how to build this lib on python3 on ubuntu?

I have follow the guide, use setup.py to build this library, and get errors" by not providing "Findpybind11.cmake" , but I have installed pybind11 in the machine..
Could you please provide the details steps of setuptools to install this library?

install error

  Fetching pybind11
  -- Found PythonInterp: /usr/bin/python3.7 (found version "3.7.14")
  -- Found PythonLibs: python3.7m
  -- Performing Test HAS_CPP14_FLAG
  -- Performing Test HAS_CPP14_FLAG - Success
  -- pybind11 v2.5.0
  -- Performing Test HAS_FLTO
  -- Performing Test HAS_FLTO - Success
  -- LTO enabled
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /home/tello/h264decoder/build/temp.linux-aarch64-cpython-37
  Scanning dependencies of target h264decoderlib
  [ 25%] Building CXX object CMakeFiles/h264decoderlib.dir/src/h264decoder.cpp.o
  [ 50%] Linking CXX static library libh264decoderlib.a
  [ 50%] Built target h264decoderlib
  Scanning dependencies of target h264decoder
  [ 75%] Building CXX object CMakeFiles/h264decoder.dir/src/h264decoder_python.cpp.o
  In file included from /home/tello/h264decoder/build/temp.linux-aarch64-cpython-37/_deps/pybind11-src/include/pybind11/pytypes.h:12:0,
                   from /home/tello/h264decoder/build/temp.linux-aarch64-cpython-37/_deps/pybind11-src/include/pybind11/cast.h:13,
                   from /home/tello/h264decoder/build/temp.linux-aarch64-cpython-37/_deps/pybind11-src/include/pybind11/attr.h:13,
                   from /home/tello/h264decoder/build/temp.linux-aarch64-cpython-37/_deps/pybind11-src/include/pybind11/pybind11.h:44,
                   from /home/tello/h264decoder/src/h264decoder_python.cpp:6:
  /home/tello/h264decoder/build/temp.linux-aarch64-cpython-37/_deps/pybind11-src/include/pybind11/detail/common.h:112:10: fatal error: Python.h: No such file or directory
   #include <Python.h>
            ^~~~~~~~~~
  compilation terminated.
  CMakeFiles/h264decoder.dir/build.make:62: recipe for target 'CMakeFiles/h264decoder.dir/src/h264decoder_python.cpp.o' failed
  make[2]: *** [CMakeFiles/h264decoder.dir/src/h264decoder_python.cpp.o] Error 1
  CMakeFiles/Makefile2:109: recipe for target 'CMakeFiles/h264decoder.dir/all' failed
  make[1]: *** [CMakeFiles/h264decoder.dir/all] Error 2
  Makefile:83: recipe for target 'all' failed
  make: *** [all] Error 2
  Traceback (most recent call last):
    File "<string>", line 36, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/tello/h264decoder/setup.py", line 84, in <module>
      zip_safe=False,
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
      self.run_command(cmd)
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/command/install.py", line 68, in run
      return orig.install.run(self)
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/_distutils/command/install.py", line 698, in run
      self.run_command('build')
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/_distutils/command/build.py", line 132, in run
      self.run_command(cmd_name)
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
      self.distribution.run_command(command)
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
      super().run_command(command)
    File "/home/tello/.local/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/home/tello/h264decoder/setup.py", line 46, in run
      self.build_extension(ext)
    File "/home/tello/h264decoder/setup.py", line 77, in build_extension
      subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp)
    File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j2']' returned non-zero exit status 2.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for h264decoder
Running setup.py clean for h264decoder
Failed to build h264decoder

subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j2']' returned non-zero exit status 2.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> h264decoder

error: legacy-install-failure

× Encountered error while trying to install package.
╰─> h264decoder

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Install in windows

env: visual studio 2017+win10
I have successfully installed ffmpeg with vspkg
But there are some problems when installing 264decoder，it seems like vcpkg can't find pybind11.

(base) D:\h264decoder-master\h264decoder-master>python setup.py build_ext --cmake-args="-DCMAKE_TOOLCHAIN_FILE=C:/dev/vcpkg/scripts/buildsystems/vcpkg.cmake"
running build_ext
-- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.19044.
CMake Warning at C:/dev/vcpkg/scripts/buildsystems/vcpkg.cmake:816 (_find_package):
By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "pybind11",
but CMake did not find one.

Could not find a package configuration file provided by "pybind11" with any
of the following names:

pybind11Config.cmake
pybind11-config.cmake

Add the installation prefix of "pybind11" to CMAKE_PREFIX_PATH or set
"pybind11_DIR" to a directory containing one of the above files. If
"pybind11" provides a separate development package or SDK, be sure it has
been installed.
Call Stack (most recent call first):
CMakeLists.txt:29 (find_package)

Fetching pybind11
-- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.19044.
CMake Error at C:/Program Files/CMake/share/cmake-3.23/Modules/ExternalProject.cmake:2540 (message):
error: could not find git for clone of pybind11-populate
Call Stack (most recent call first):
C:/Program Files/CMake/share/cmake-3.23/Modules/ExternalProject.cmake:3640 (_ep_add_download_command)
CMakeLists.txt:23 (ExternalProject_Add)

-- Configuring incomplete, errors occurred!
See also "D:/h264decoder-master/h264decoder-master/build/temp.win-amd64-3.9/Release/_deps/pybind11-subbuild/CMakeFiles/CMakeOutput.log".

CMake Error at C:/Program Files/CMake/share/cmake-3.23/Modules/FetchContent.cmake:1076 (message):
CMake step for pybind11 failed: 1
Call Stack (most recent call first):
C:/Program Files/CMake/share/cmake-3.23/Modules/FetchContent.cmake:1217:EVAL:2 (__FetchContent_directPopulate)
C:/Program Files/CMake/share/cmake-3.23/Modules/FetchContent.cmake:1217 (cmake_language)
C:/Program Files/CMake/share/cmake-3.23/Modules/FetchContent.cmake:1260 (FetchContent_Populate)
CMakeLists.txt:39 (FetchContent_MakeAvailable)

-- Configuring incomplete, errors occurred!
See also "D:/h264decoder-master/h264decoder-master/build/temp.win-amd64-3.9/Release/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
File "D:\h264decoder-master\h264decoder-master\setup.py", line 79, in
setup(
File "E:\conda\lib\site-packages\setuptools_init_.py", line 153, in setup
return distutils.core.setup(**attrs)
File "E:\conda\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "E:\conda\lib\distutils\dist.py", line 966, in run_commands
self.run_command(cmd)
File "E:\conda\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "D:\h264decoder-master\h264decoder-master\setup.py", line 46, in run
self.build_extension(ext)
File "D:\h264decoder-master\h264decoder-master\setup.py", line 76, in build_extension
subprocess.check_call(['cmake', ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env)
File "E:\conda\lib\subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', 'D:\h264decoder-master\h264decoder-master', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=D:\h264decoder-master\h264decoder-master\build\lib.win-amd64-3.9\', '-DPYTHON_EXECUTABLE=E:\conda\python.exe', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE=D:\h264decoder-master\h264decoder-master\build\lib.win-amd64-3.9\', '-A', 'x64', '-DCMAKE_TOOLCHAIN_FILE=C:/dev/vcpkg/scripts/buildsystems/vcpkg.cmake']' returned non-zero exit status 1.

Support for hardware accelerated video decoding

Hi, does h264decoder support hardware acceleration (opencl, cuda, vaapi). If not, is it planned to implement this feature in the future?

Could some one help me. How to solve Oak-D pro camera data decoding and display

#every time different frame data size
#May be it is commpress data

#this is my code

import cv2
import config
import depthai as dai
import subprocess as sp
from os import name as osName
import numpy as np
import h264decoder
import matplotlib.pyplot as pyplot
import sys

''' Create pipeline'''
pipeline = dai.Pipeline()

''' Define sources and output '''
camRgb = pipeline.create(dai.node.ColorCamera)
videoEnc = pipeline.create(dai.node.VideoEncoder)
xout = pipeline.create(dai.node.XLinkOut)

xout.setStreamName("h264")

''' Properties '''
camRgb.setBoardSocket(dai.CameraBoardSocket.RGB)
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
videoEnc.setDefaultProfilePreset(camRgb.getVideoSize(), camRgb.getFps(), dai.VideoEncoderProperties.Profile.H264_MAIN)

''' Linking '''
camRgb.video.link(videoEnc.input)
videoEnc.bitstream.link(xout.input)

''' video_in = cv2.VideoCapture("udpsrc port=12345 ! application/x-rtp-stream,encoding-name=JPEG ! rtpstreamdepay ! ''' rtpjpegdepay ! jpegdec ! videoconvert ! appsink", cv2.CAP_GSTREAMER)
video_out = cv2.VideoWriter("appsrc ! videoconvert ! jpegenc ! rtpjpegpay ! rtpstreampay ! udpsink host=[destination-ip] port=12344", cv2.CAP_GSTREAMER, 0, 24, (1920, 1080), True)

img = None
fig, ax = pyplot.subplots(1,1)

''' video_in = cv2.VideoCapture("udpsrc port=12345 ! application/x-rtp-stream,encoding-name=JPEG ! rtpstreamdepay !''' rtpjpegdepay ! jpegdec ! videoconvert ! appsink", cv2.CAP_GSTREAMER)
video_out = cv2.VideoWriter("appsrc ! videoconvert ! jpegenc ! rtpjpegpay ! rtpstreampay ! udpsink host=[destination-ip] port=12344", cv2.CAP_GSTREAMER, 0, 24, (1920, 1080), True)

img = None
fig, ax = pyplot.subplots(1,1)

def display(framedata):
global img, fig, ax
(frame, w, h, ls) = framedata
print(w,' ',h,' ',ls)
if frame is not None:
print('frame size %i bytes, w %i, h %i, linesize %i' % (len(frame), w, h, ls))
frame = np.frombuffer(frame, dtype=np.ubyte, count=len(frame))
frame = frame.reshape((h, ls//3, 3))
frame = frame[:,:w,:]

if not img:
    img = ax.imshow(frame)
    pyplot.show(block = False)
else:
    img.set_data(frame)
    pyplot.draw()
pyplot.pause(0.001)

''' Two APIs to decode frames'''
def run_decode_frame(decoder, data_in):
while len(data_in):
''' One frame at a time, indicating how much of the input has been consumed.'''
framedata, nread = decoder.decode_frame(data_in)
data_in = data_in[nread:]
display(framedata)
def run_decode(decoder, data_in):
''' Consume the input completely. May result in multiple frames read.'''
''' This runs a bit faster than the other way.'''
framedatas = decoder.decode(data_in)
for framedata in framedatas:
display(framedata)

def send():
''' Connect to device and start pipeline'''
with dai.Device(pipeline) as device:
print("Oak-D")
''' Output queue will be used to get the encoded data from the output defined above'''
q = device.getOutputQueue(name="h264", maxSize=30, blocking=True)
decoder = h264decoder.H264Decoder()
try:
while True:
frame = q.get().getData() # Blocking call, will wait until new data has arrived
run_decode(decoder, frame)
'''cv2.imshow('Original', data)'''
key = cv2.waitKey(1) & 0xff
if key == 27:
break
except:
print("Exception")
pass
'''cv2.destroyAllWindows()'''
video_out.release()

if name == 'main':
send()

ImportError: No module named libh264decoder

Hi I am trying to get this code to work in conjuction with a tello drone. But when i install all your packages it says theres a problem importing the libh264decoder. Is there a way you could help me get this working?