Blackmagic HDMI capture devices don't show up in capture list about processing-video HOT 14 CLOSED

Hi, the video library relies on gstreamer to handle capture devices. The version of gstreamer bundled with Processing is rather old (from the 0.10 branch that is no longer maintained) and probably lacks the plugins that properly support the Blackmagic hardware.

We are trying to gather some momentum to update the underlying java bindings so we can move to gstreamer 1.x, which might have better hardware support for capture devices. While we work on that, perhaps you could check (outside Processing) if gstreamer 1.x does in fact support Blackmagic capture devices.

from processing-video.

An interim measure may be at: #32

from processing-video.

I'm still testing, but so far so good... another approach uses BlackSyphon and @codeanticode's Syphon Library to pipe the video from a BlackMagic device into Processing as a PImage.

from processing-video.

I happened to have access to a windows 10 machine with a blackmagick intensity pro 4k pcie card.

I used processing 3.0.6 and the latest version of the processing-video lib.

When I run the getting started with capture, I got a list of about 70 options, but I couldn't get an image from the camera for now. I'll keep investigating though.

from processing-video.

@b2renger what version of the video library are you using? We've uploaded the first v2 beta release, which uses gstreamer 1.x, perhaps it has better hardware support, did you try it out?

from processing-video.

@codeanticode great ! I'll check it out on monday then and get back to you. Thanks

from processing-video.

@codeanticode
The beta doesn't work, I have less options to choose from. Here is what I get in the console with the beta, when launching 'GettingStartedCapture' example :

Available cameras:
[0] "Blackmagic DeckLink WDM Streaming"

And I got this message when closing the proccessing windows :

WARNING: no real random source present!

The cam.available() command evaluates to false.

FIY1 : when using the v1.0.1 from contribution manager with the 'GettingStartedCapture' example I have this list :
Available cameras:
[0] "name=Blackmagic DeckLink WDM Streaming,size=720x576,fps=25"
[1] "name=Blackmagic DeckLink WDM Streaming,size=720x576,fps=25"
[2] "name=Blackmagic DeckLink WDM Streaming,size=1280x720,fps=50"
[3] "name=Blackmagic DeckLink WDM Streaming,size=1280x720,fps=50"
[4] "name=Blackmagic DeckLink WDM Streaming,size=1280x720,fps=60"
[5] "name=Blackmagic DeckLink WDM Streaming,size=1280x720,fps=60"
[6] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=25"
[7] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=25"
[8] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=30"
[9] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=30"
[10] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=24"
[11] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=24"
[12] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=50"
[13] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=50"
[14] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=60"
[15] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=60"
[16] "name=Blackmagic DeckLink WDM Streaming,size=720x576,fps=25"
[17] "name=Blackmagic DeckLink WDM Streaming,size=720x576,fps=25"
[18] "name=Blackmagic DeckLink WDM Streaming,size=1280x720,fps=50"
[19] "name=Blackmagic DeckLink WDM Streaming,size=1280x720,fps=50"
[20] "name=Blackmagic DeckLink WDM Streaming,size=1280x720,fps=60"
[21] "name=Blackmagic DeckLink WDM Streaming,size=1280x720,fps=60"
[22] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=25"
[23] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=25"
[24] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=30"
[25] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=30"
[26] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=24"
[27] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=24"
[28] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=50"
[29] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=50"
[30] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=60"
[31] "name=Blackmagic DeckLink WDM Streaming,size=1920x1080,fps=60"

I tried to set the camera with this :

cam = new Capture(this, cameras[31]);

or

cam = new Capture(this, 1920,1080, "Blackmagic DeckLink WDM Streaming", 25);

in both case cam.available() evaluates to false.

FIY2 : BlackSpout also returns a black image whatever configuration I choose.

Let me know if I can help further, I would love to get this to work :)

from processing-video.

Another feedback :

OBS (https://github.com/jp9000/OBS) does get an image from the card. When running OBS and launching the GettingStarted example I get those errors :

BaseSrc: [ksvideosrc0] : failed to start capture (0x00000020)
BaseSrc: [ksvideosrc0] : Internal data stream error.

from processing-video.

@b2renger Thanks for looking into this! My recommendation would be to start by making sure that gstreamer 1.x does indeed support this hardware, and figuring out how to construct a command line pipeline that works. Then we can see how to translate that into the video library.

from processing-video.

@codeanticode well I've looked around but I'm not really an expert in video drivers and such.

So I worked on a live usb stick running ubuntu 16.04. First of all I made sure I could get an image from my camera with their dedicated software. Then I tried to construct a pipeline with gst-launch, but I didn't manage to find a device that would represent the camera with gst-inspect.

Navigating the web I found that some people manage to get Intensity pro card to work, but not really any info on the one I use "intensity pro 4k". Would you have suggestions or ressources to be able to dig out the interface name to be used with gst-launch ?

from processing-video.

HI, i just tried to open the blackmagic decklink mini device directly in gst-launch-1.0 with:

gst-launch-1.0 -v decklinkvideosrc mode=17 connection=0 ! d3dvideosink sync = false

It worked. This is for 1280x720 in 50fps. Is there a easy way how i could update the capture class from this repository to get the blackmagic decklink work with a new build? Or is the Problem of the Decklink not showing content in processing more complex?

Thank you for help!

from processing-video.

Hi,

now I tried Gstreamer Java Core Implementation directly:
https://github.com/gstreamer-java/gst1-java-core
to open the Decklink Mini.

The capture worked with the capture-example from here:
https://github.com/gstreamer-java/gst1-java-examples
I changed the bin-initialization from the captureTest Example to:

Bin bin = Bin.launch("decklinkvideosrc mode=17 connection=0 ! capsfilter caps=\"video/x-raw, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, framerate=(fraction)50/1, format=(string)UYVY, colorimetry=(string)bt709, chroma-site=(string)mpeg2\" ! videoconvert ", true);

How can we now change this method (initGStreamer) from Capture.java ?
Maybe we can create a different initGStreamerMethod for Blackmagic-Decklink Devices?

    pipe = new Pipeline();
    Element videoscale = ElementFactory.make("videoscale", null);
    Element videoconvert = ElementFactory.make("videoconvert", null);
    Element capsfilter = ElementFactory.make("capsfilter", null);

    String frameRateString;
    if (frameRate != 0.0) {
      frameRateString = ", framerate=" + fpsToFramerate(frameRate);
    } else {
      frameRateString = "";
    }
    capsfilter.set("caps", Caps.fromString("video/x-raw, width=" + width + ", height=" + height + frameRateString));

    rgbSink = new AppSink("sink");
    rgbSink.set("emit-signals", true);
    newSampleListener = new NewSampleListener();
    rgbSink.connect(newSampleListener);
    // XXX: unsure about BGRx
    rgbSink.setCaps(Caps.fromString("video/x-raw, format=BGRx"));

    pipe.addMany(srcElement, videoscale, videoconvert, capsfilter, rgbSink);
    pipe.linkMany(srcElement, videoscale, videoconvert, capsfilter, rgbSink);

    makeBusConnections(pipe.getBus());

from processing-video.

@jitterhorse not sure if this is still relevant, but the latest beta of the video library is incorporating support for custom pipelines (see this example to get a sense of how it would work). It is not yet finalized so your full pipeline to get the video from the Blackmagic device will not work, it should with the v2 stable release. Planning to upload it in the next weeks.

from processing-video.

The pipeline parsing functionality is available in the latest version of the video library (see capture/CustomPipeline), which should help capturing video from specialized capture hardware.

from processing-video.