Giter VIP home page Giter VIP logo

escapi's Introduction

ESCAPI

Extremely Simple Capture API

Copyright (c)2015 Jari Komppa http://iki.fi/sol

Binaries

Binaries available at http://iki.fi/sol/zip/escapi3.zip

Usage

Add escapi.cpp to your project. This file contains code to load the escapi.dll. See the samples ('simplest' recommended) for API usage.

Rust Bindings Usage

Just add the crate in your project and start using it. NOTE: make sure that Windows SDK and Media Foundation headers are accessible for your Rust toolchain, otherwise the library build will fail (i.e. it should work fine with MSVC toolchain and it might require some changes if you're using mingw toolchain).

License

ESCAPI is released under the unlicense. In short, use for any purpose as long as you don't hold me responsible for anything. It would be nice if you'd toss me a mail if you play with this thing.

Some examples use external libraries with different licenses.

What is ESCAPI?

A fairly easy to use webcam (or other video input device) capture API.

Version 2.0 adds support for multiple capture devices and requesting the capture device names, as well as new examples.

Version 2.1 updates the examples, including purebasic and blizmax examples, as well as an OpenGL based 'funny mirrors'-example, and fills out the top 8 "alpha" bits in the captured data with 0xff.

Version 3.0 is complete rewrite using windows media foundation instead of directshow. The version adds interface for playing with camera properties, new examples, automatic camera resolution selection, 64 bit builds, and source release. Version 3.0 onwards requires windows vista or later (7, 8, 8.1, 10..).

Motivation

One of the last things that I added to 'textmedia' was webcam support. I got interested in webcams for a while, thinking I might whip up some kind of "eye toy"-ish game using a web cam.

One of the bad sides about webcams is that the programming API is so complicated - the only way to use them back then was through DirectShow. Simply getting data from the camera is a fairly complicated process, while not giving you much control.

So, to get rid of the directmedia SDK requirement, I split the required code into a separate DLL, and now I present to you the ESCAPI:

  • setupESCAPI - Initialize the whole library. (in escapi.cpp)
  • countCaptureDevices - Request number of capture devices available.
  • getCaptureDeviceName - Request the printable name of a capture device.
  • initCapture - Tries to open the video capture device. Returns 0 on failure, 1 on success.
  • doCapture - Requests a video frame to be captured.
  • isCaptureDone - Returns 1 when the requested frame has been captured.
  • deinitCapture - Closes the video capture device.

So basically, you call setup to initialize the library, call init to start the capture device, and call doCapture to start the capture process. When the capture is done, you can ask for another frame. Etc.

Unfortunately, "eye toy"-wise, the webcams on PCs are quite laggy, and this varies from a camera to the next. My logitech messenger camera has a lag or about one second, while my creative instant camera is has a more tolerable lag. Both cameras also perform some automatic contrast trickery and other stuff like that which won't be quite game-friendly.

Thus, I never ended up doing that "eye toy"-like game. Maybe one day =)

Dependencies

Some of the examples use external libraries:

escapi's People

Contributors

daniel-abramov avatar jarikomppa avatar kp0c2 avatar nathanw2 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

escapi's Issues

Is the build script still working?

Hi, I tried to build the project (used it as a Rust dependency), but it failed at the last step (copying) of build.rs, I tried to download the project and build it on my local machine as a separate crate, but it still fails.

I tried to uncomment the last line (where the copy is done) and now it works. Seems like the build.rs do not always build the DLL, even though the cmd.output().expect(...); does not fail.

By the way, I think that it makes sense to build a static library when building a library for Rust, in this case the users of the crate do not need to ship a DLL with their executable/library.

Memory Leak

Memory leak while initializing/deinitializing cameras. I placed sleeps here to highlight the problem.

std::this_thread::sleep_for(std::chrono::milliseconds(100));
SimpleCapParams params;
params.mTargetBuf = new int[640 * 480];
params.mWidth = 640;
params.mHeight = 480;
std::this_thread::sleep_for(std::chrono::milliseconds(100));

for (int i = 0; i < 4; ++i)
{
	//activate
	std::this_thread::sleep_for(std::chrono::milliseconds(150));
	initCapture(0, &params);

	//deactivate
	deinitCapture(0);
}

photo5260354087784917031

Cannot change settings while camera is in use in OBS

I'm trying to utilize escapi for a livestream situation so I can change settings on the fly. Escapi doesn't seem to like when the camera is in use. I'm using propgui and have trimed out all the capture stuff as far as I can see but once I try to change settings, the setting just snaps back to where it was. I also noticed just using enum and enumprop take a long time to return settings when OBS is running. Any help would be great.

Static link for escapi.dll?

I really like your library, but only one problem. If i want to make program one file executable then it cannot be single file, because library requires dll. Can you compile dll as .lib or something that i can link staticly.

Version 2.1

Not so much of an issue, more a request - form the readme, it looks like version 2 was targeting DirectShow rather than MediaFoundation - is the v2 code available anywhere, since I have a DirectShow device that I need to capture from...

Initialize function pointer values with NULL in escapi.cpp

Description

Not initializing function pointer values with NULL in escapi.cpp function declarations is quite dangerous. All these pointers get random addresses and could point to something completely different. Also even if GetProcAddress does not find a function in the DLL it still is not set to NULL for some C++ compilers (i.e. Embarcadero RAD Studio C++) and so can end up loading a completely different library.

I've seen this myself when we first used ESCAPI 3.0, but then decided to go back to 2.1 because of the DirectShow support. Replacing the 3.0 DLL with 2.1 DLL did not cause any errors and the protection where it checks for function pointers != NULL did nothing and allowed to load the 2.1 DLL into process. Which is scary, cause you don't know what these random pointers can end-up modifying in the running process memory.

Reproduction steps

  • Add escapi.cpp and escapi.h from 3.0 release into your project.
  • Compile
  • Copy escapi.dll version 2.1 to the binary output directory

Expected result

Application should not allow to load the older version of escapi.dll

Actual result

Application loads the older version of escapi.dll even if the Property functions are not found.

Getting list of supported formats / resolutions / framerates

Would it be possible for escapi to allow querying for the supported formats of a camera, and then query for the supported resolutions of a given format?
Similar to how it's possible with the rscam Rust crate:

extern crate rscam;

use std::io::prelude::*;
use std::fs;

fn main() -> rscam::Result<()> {
	let mut cam = rscam::new("/dev/video2").unwrap();
	for fmt in cam.formats() {
		let fmt = fmt?;
		println!("{:?}", fmt);
		if let rscam::ResolutionInfo::Discretes(v) = cam.resolutions(&fmt.format)? {
			for res in v {
				println!("{:?} {:?}", res, cam.intervals(&fmt.format, res)?);
			}
		}
	}

	cam.start(&rscam::Config {
		interval: (1, 30),      // 30 fps.
		resolution: (1920, 1080),
		format: b"MJPG",
		..Default::default()
	}).unwrap();

	for i in 0..30 {
		let frame = cam.capture().unwrap();
		let mut file = fs::File::create(&format!("frame-{}.jpg", i)).unwrap();
		file.write_all(&frame[..]).unwrap();
	}
	Ok(())
}

which prints:

YUYV (YUYV 4:2:2)
(1920, 1080) Discretes: 5fps
(160, 120) Discretes: 30fps, 25fps, 20fps, 15fps
(176, 144) Discretes: 30fps, 25fps, 20fps, 15fps
(320, 240) Discretes: 30fps, 25fps, 20fps, 15fps
(352, 288) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 360) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 480) Discretes: 30fps, 25fps, 20fps, 15fps
(800, 600) Discretes: 10fps, 5fps
(848, 480) Discretes: 10fps, 5fps
(1024, 768) Discretes: 10fps, 5fps
(1280, 800) Discretes: 10fps, 5fps
(1280, 720) Discretes: 10fps, 5fps
MJPG (Motion-JPEG, compressed)
(1920, 1080) Discretes: 30fps, 25fps, 20fps, 5fps
(160, 120) Discretes: 30fps, 25fps, 20fps, 15fps
(176, 144) Discretes: 30fps, 25fps, 20fps, 15fps
(320, 240) Discretes: 30fps, 25fps, 20fps, 15fps
(352, 288) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 360) Discretes: 30fps, 25fps, 20fps, 5fps
(640, 480) Discretes: 30fps, 25fps, 20fps, 15fps
(800, 600) Discretes: 30fps, 25fps, 20fps, 5fps
(848, 480) Discretes: 30fps, 25fps, 20fps, 5fps
(1024, 768) Discretes: 30fps, 25fps, 20fps, 5fps
(1280, 800) Discretes: 30fps, 25fps, 20fps, 5fps
(1280, 720) Discretes: 30fps, 25fps, 20fps, 5fps
RGB3 (RGB3, emulated)
(1920, 1080) Discretes: 30fps, 25fps, 20fps, 5fps
(160, 120) Discretes: 30fps, 25fps, 20fps, 15fps
(176, 144) Discretes: 30fps, 25fps, 20fps, 15fps
(320, 240) Discretes: 30fps, 25fps, 20fps, 15fps
(352, 288) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 360) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 480) Discretes: 30fps, 25fps, 20fps, 15fps
(800, 600) Discretes: 30fps, 25fps, 20fps, 5fps
(848, 480) Discretes: 10fps, 5fps
(1024, 768) Discretes: 30fps, 25fps, 20fps, 5fps
(1280, 800) Discretes: 30fps, 25fps, 20fps, 5fps
(1280, 720) Discretes: 30fps, 25fps, 20fps, 5fps
BGR3 (BGR3, emulated)
(1920, 1080) Discretes: 30fps, 25fps, 20fps, 5fps
(160, 120) Discretes: 30fps, 25fps, 20fps, 15fps
(176, 144) Discretes: 30fps, 25fps, 20fps, 15fps
(320, 240) Discretes: 30fps, 25fps, 20fps, 15fps
(352, 288) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 360) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 480) Discretes: 30fps, 25fps, 20fps, 15fps
(800, 600) Discretes: 30fps, 25fps, 20fps, 5fps
(848, 480) Discretes: 10fps, 5fps
(1024, 768) Discretes: 30fps, 25fps, 20fps, 5fps
(1280, 800) Discretes: 30fps, 25fps, 20fps, 5fps
(1280, 720) Discretes: 30fps, 25fps, 20fps, 5fps
YU12 (YU12, emulated)
(1920, 1080) Discretes: 30fps, 25fps, 20fps, 5fps
(160, 120) Discretes: 30fps, 25fps, 20fps, 15fps
(176, 144) Discretes: 30fps, 25fps, 20fps, 15fps
(320, 240) Discretes: 30fps, 25fps, 20fps, 15fps
(352, 288) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 360) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 480) Discretes: 30fps, 25fps, 20fps, 15fps
(800, 600) Discretes: 30fps, 25fps, 20fps, 5fps
(848, 480) Discretes: 10fps, 5fps
(1024, 768) Discretes: 30fps, 25fps, 20fps, 5fps
(1280, 800) Discretes: 30fps, 25fps, 20fps, 5fps
(1280, 720) Discretes: 30fps, 25fps, 20fps, 5fps
YV12 (YV12, emulated)
(1920, 1080) Discretes: 30fps, 25fps, 20fps, 5fps
(160, 120) Discretes: 30fps, 25fps, 20fps, 15fps
(176, 144) Discretes: 30fps, 25fps, 20fps, 15fps
(320, 240) Discretes: 30fps, 25fps, 20fps, 15fps
(352, 288) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 360) Discretes: 30fps, 25fps, 20fps, 15fps
(640, 480) Discretes: 30fps, 25fps, 20fps, 15fps
(800, 600) Discretes: 30fps, 25fps, 20fps, 5fps
(848, 480) Discretes: 10fps, 5fps
(1024, 768) Discretes: 30fps, 25fps, 20fps, 5fps
(1280, 800) Discretes: 30fps, 25fps, 20fps, 5fps
(1280, 720) Discretes: 30fps, 25fps, 20fps, 5fps

http://loyd.github.io/rscam/rscam/struct.Camera.html

I'm asking because I often encounter the case where a 30 FPS FullHD capture is only possible with MJPEG, like in this case. (The emulated modes are done by v4l2 in software afaik.)
When I open this camera with escapi, it chooses the slow 5 FPS YUYV mode :(

In my modified version of escapi I added MJPEG as a supported format and then pass through the undecoded MJPEG buffer to my Rust application where I use the mozjpeg-sys crate to decode the frames.
But it's very blind (not querying what's supported) & hardcoded to my webcam, I want it to work for any webcam.
So what I would ideally want to do with escapi is: find the highest resolution with at least 20 fps and use that format (and when it's MJPEG I can decode it myself).

Is there any plan to allow querying for supported formats / resolutions / framerates like this? :)

Setting a custom capture framerate

I bought a webcam that can do 30ps Full Hd. I tested it in OBS-Studio, at first I got very low capture framerate, about 5 fps. But then I read this post and it worked: By setting the "Resolution/FPS Type" to Custom and setting Resolution to 1920x1080 I got the 30 FPS capture framerate.

Then I tried to use the webcam in my opengl application, capturing it with Escapi and rendering it to the screen.
But in my application I ALSO only get that low capture framerate with Escapi :(
(Btw, I get high FPS with a friend's webcam so it's not my application's fault.)
I talked on IRC with the developer of OBS about what OBS does when you set a custom capture FPS, it uses DirectShow functions to change the pin type. He pointed me to this:
https://github.com/jp9000/libdshowcapture/blob/master/source/device.cpp#L304

Could the same be done with Windows Media Foundations?
It seems that some cameras (like mine) use a lower capture framerate by default than what they could, so it would be useful to be able to set a custom capture framerate with Escapi, too :)

Escapi not working in Visual Studio

Can someone tell the the whole process of using escapi in visual studio? I am following Javidx9's 8 bits of Image Processing video on Youtube and he uses escapi to display his webcam on his console in C++ but he didn't show the steps of setting up escapi so I am a bit lost

Memory leak in ver3.0

Memory leak in ver3.0 - in function deinitCapture(unsigned int deviceno) dont free memory..

in V2.1 - work ok, but use more RAM ..

Can you fix V3.0 ?

Rust API suggestions, finding device id from its name

In the Rust API it would be useful to be able to enumerate the capture devices so that one can get a device's name and id (and possibly max resolution) before initializing the capture on it with init().
Because the installed devices on a system change over time and don't always have the same ID, so I store the camera that I want to use by its name in my application's config file and then I want to iterate over the available cameras and find its ID by its name before calling init() with that id.
This could be done by having an Iterator that yields DeviceInfo which has .id() -> usize, .name() -> String, .max_resolution() -> (u32, u32).

What do you think?

(Btw, I love ESCAPI, it makes dealing with a webcam so much easier.)

propgui not showing all webcams

I compiled the project using Visual Studio 2019 (v142) with SDL2 imported through vcpkg. All projects compiled and worked well except propgui.

progui uses RGBA for each camera texture, so it seems that some of my cameras were not filling in the A component (thereby causing their textures to become transparent).

The fix is rather simple in main.cpp - lines 48-51:

		for (i = 0; i < 256 * 256; i++)
			capture[device].mTargetBuf[i] = (capture[device].mTargetBuf[i] & 0xff00ff00) |
			((capture[device].mTargetBuf[i] & 0xff) << 16) |
			((capture[device].mTargetBuf[i] & 0xff0000) >> 16) | (0xff000000); // last bit adds alpha = 100%

I will submit a pull-request with my changes in case the @jarikomppa wants to merge it, but I am also leaving this note here in case others experience a similar problem ;).

Escapi library not showing any output

Hi, I am using the escapi binaries for my project....... I have created a Windows Desktop wizard empty application using visual studio 16 and added the escapi.h, escapi.cpp, and escapi.dll to my project but when I run my project it does not show anything and it does not even detect my camera....where should I add the escapi.dll or do I need to do anything else to make it work .

Detecting when webcam was unplugged

How can I detect when the webcam was unplugged?

In my loop I could check if countCaptureDevices() changed from last frame, and if it did, I can check if the currently opened cam is still in the list of connected ones, but is there a better way?

thread 'main' panicked at 'Could not capture an image: CaptureTimeout'

Hi,I got an error: CaptureTimeout.
I'm using a notebook. When the app starts, I can see the LED beside the camera is turn on for about 200ms.
My PC ENV:

$ rustup show
Default host: x86_64-pc-windows-msvc
rustup home:  E:\rustup

installed toolchains
--------------------

nightly-i686-pc-windows-msvc (default)
nightly-x86_64-pc-windows-msvc

active toolchain
----------------

nightly-i686-pc-windows-msvc (default)
rustc 1.41.0-nightly (27d6f55f4 2019-12-11)

$ /e/rustdir/target/debug/test_escapi.exe devices: 1 capture initialized, device name: USB HD Webcam Frame #0, captured and saved as image.png thread 'main' panicked at 'Could not capture an image: CaptureTimeout', src\libcore\result.rs:1189:5 stack backtrace: 0: core::fmt::write at /rustc/27d6f55f47e8875e71083a28ed84ea5a88e1b596\/src\libcore\fmt\mod.rs:1025 1: std::io::Write::write_fmt<std::sys::windows::stdio::Stderr> at /rustc/27d6f55f47e8875e71083a28ed84ea5a88e1b596\/src\libstd\io\mod.rs:1426 2: std::panicking::default_hook::{{closure}} at /rustc/27d6f55f47e8875e71083a28ed84ea5a88e1b596\/src\libstd\panicking.rs:193 3: std::panicking::default_hook at /rustc/27d6f55f47e8875e71083a28ed84ea5a88e1b596\/src\libstd\panicking.rs:210 4: std::panicking::rust_panic_with_hook at /rustc/27d6f55f47e8875e71083a28ed84ea5a88e1b596\/src\libstd\panicking.rs:471 5: std::panicking::begin_panic_handler at /rustc/27d6f55f47e8875e71083a28ed84ea5a88e1b596\/src\libstd\panicking.rs:375 6: core::panicking::panic_fmt at /rustc/27d6f55f47e8875e71083a28ed84ea5a88e1b596\/src\libcore\panicking.rs:84 7: core::result::unwrap_failed at /rustc/27d6f55f47e8875e71083a28ed84ea5a88e1b596\/src\libcore\result.rs:1189 8: core::result::Result<slice<u8>*, escapi::Error>::expect<slice<u8>*,escapi::Error> at /rustc/27d6f55f47e8875e71083a28ed84ea5a88e1b596\src\libcore\result.rs:984 9: test_escapi::main at H:\work\rust_projects\test_escapi\src\main.rs:23 10: std::rt::lang_start::{{closure}}<()> at /rustc/27d6f55f47e8875e71083a28ed84ea5a88e1b596\src\libstd\rt.rs:67 note: Some details are omitted, run withRUST_BACKTRACE=fullfor a verbose backtrace.

The captured frame format

I think it would be nice to update the documentation a bit, currently it does not mention the format of the returned frame. Is it the same format regardless of the camera? If so, does it mean that the conversion is applied to the captured frame before the frame is returned to the user? Would it make sense to allow user to specify the format of the captured frame (so that we can skip the conversion to improve the capturing performance)?

Escapi can't see virtual cameras like OBS Virtual Camera

skype_webcame_input
(pic of skype showing all cams. Wow, that pic came out big)

Escapi (using the propgui.exe example) can see my Game Capture HD60 Pro, but it does not see virtual cameras like "Camera (NVIDIA Broadcast" and "OBS Virtual Camera".

I suspect this is an issue with Media Foundation (MFEnumDeviceSources just won't see them) and the only "fix" is to use DirectShow (which we're not supposed to use anymore?!) , but figured I'd better ask in case I'm totally wrong and there actually is a way to access these.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.