This code repo is now deprecated and the project is unsupported
AI Skills for Windows
Summary
Implementing and integrating efficient AI and Computer Vision (CV) solutions is a hard task for developers. The industry is moving at a fast pace and the amount of custom-tailored solutions coming out makes it almost impossible for app developers to keep up easily. We preview a base framework to expose AI solutions called AI Skills for Windows as well as pre-built common solutions (i.e. detection, classification, segmentation, etc.) developed by Microsoft and partners.
The AI Skills for Windows framework is meant to standardize the way AI and CV is put to use within a Windows application (i.e.: UWP, Desktop Win32, .Net Core 3.0+) running on the edge. It aims to abstract away the complexity of AI techniques by simply defining the concept of skills which are modular pieces of code that process input(s) and produce output(s). The implementation that contains the complex details (i.e. pre and post processing of data, model inference, algorithm, transcoding, applying the right heuristics, etc.) is encapsulated by WinRT APIs that inherits the base class present in the Microsoft.AI.Skills.SkillInterface namespace, which leverages built-in Windows primitives which in-turn eases interop with built-in hardware acceleration leveraged by frameworks such as Windows ML. All AI Skills for Windows derivatives follow the same programatic paradigm and flow from a developer consumer standpoint: if you understand how to use one AI Skill for Windows, you understand how to use them all. (See key AI Skills API concepts)
While this release focuses on vision-oriented scenarios and primitives, this API is meant to accommodate any kind of input and output variable and a wide range of scenarios (Vision, Audio, Text, etc.). Any developer can extend this API set and expose their own AI skills. See skills released by Intel
If you are looking for the earlier preview release samples and documentation, we archived them in a branch here: Preview branch
Searches an image for quadrilateral shapes and returns the coordinates of their corners if found. The LiveQuadDetector is a stateful version of the QuadDetector that attempts to detect only 1 quadrangle and keeps track of the previous quad detected to be used as guide which optimizes tracking performance as new frames are bound over time. This is well suited for most scenarios operating over a stream of frames over time. QuadDetector can be set to detect more than 1 quadrangle and will search the whole frame everytime unless a previous quadrangle is provided.
Request for ObjectDetectorPreview :
Is it possible to output the confidence level with each match - like what is returned for Windows ML / Custom Vision ONNX files? The Evaluations are returning some poor results in some cases (ie. mixing up leaves on a tree branch as a 'Person' from a 4k security camera is one ongoing issue) - I'm pretty sure ability to filter out low confidence matches would make a big difference here.
I'm writing a win32 dll and using Windows Vision Skill to process input image, my dll will be loaded automatically by system process, however I don't have permission to edit application manifest.
I got exception "ClassFactory cannot supply requested class".
Does anyway to load Windows Vision Skill inside a win32 dll without using application manifest?
It's built successfully but when run it, exception happens. Do I need additional configuration to get it run? Thanks.
Exception thrown at 0x00007FFF88F9A599 (KernelBase.dll) in SkeletalDetectorSample_Desktop.exe: WinRT originate error - 0x80040154 : 'Class not registered'.
'SkeletalDetectorSample_Desktop.exe' (Win32): Loaded 'C:\Windows\System32\bcrypt.dll'.
'SkeletalDetectorSample_Desktop.exe' (Win32): Loaded 'C:\Windows\System32\sechost.dll'.
Exception thrown at 0x00007FFF88F9A599 in SkeletalDetectorSample_Desktop.exe: Microsoft C++ exception: winrt::hresult_error at memory location 0x000000CDA515E970.
It would be nice if you can provide the NuGet packages that are compatible with .NET Standard 2.0, iOS and Android along with UWP. That way we could AI Vision apps for the mobile platforms. After all, mobile is nothing without cloud support, and cloud needs mobile frontend in addition to desktop, web, etc.
I searched several nuget sources, I did find three Windows.AI.Vision,xxx packages, but not the FaceSentimentAnalyzer. Is this sample still in development?
I see there is a similar sample Emoji8, I'll give that a try. This is for the Hackathon, so its not super critical โฆ but it would be cool :)
Hi!
I'm kind of new to C++/WinRT and trying to test desktop samples with VS2017.
I started from VisionSkillsSamples.sln, build and run Desktop/ObjectDetectorSample_Desktop.
While dealing with this issue, I found that my executable doesn't have dependencies on skill-related dlls(e.g. Microsoft.AI.Skills.SkillInterfacePreview.dll ...). I'm not sure this is related to my issue, but I expected to have those dependencies.
Please let me know if I missed any configuration required.
Thank you!
I had to run unblock-file on prebuild and postbuild scripts to make them work.
FYI:
There's a problem with the 1.0.0.2 'Intel' Vision Skills packages that were published by Intel. (not yet tried to see if the 1.0.0.3 updates suffer same problem).
All the embedded attibution 'text' files appear to have been included with the 'Compile' build action - which prevents any samples from being built once you download the packages from NuGet. (VS2019 throws a bunch of compile errors as it tries to treach all these text files as code).
To fix this I had to go into the NuGet package cache folder and manually delete all the .txt files from those subfolders before I could compile anything.
As per above - just saw 1.0.03 has just been published (with new skills) - will check to see if it's still a problem there (can't find any release notes for your stuff so I'm unable to see what was fixed).
There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.
I am trying to develop an Object Detection Application for Hololens 2 via Unity. I just want to know whether the Namespaces and the Code Snippets used in this sample for Object Detection can also be used inside unity editor to be deployed to Hololens 2. Is there a possiblity? if not is there any other sample references which could be used for Object detection via Hololens 2. Please do hlep me with this.
Hello,
I want to study the use of AI in the context of pattern recognition. Not that of human faces that has been done for a long time, but in that of the recognition of mushroom spores. Visually we can compare these spores to seeds of various plants.
I start in this area but I think it is already building models of these objects.
Can Windows Vision Skills help with this project?
On my local machine (Win 10, Win 18362.1082) the skeletaldetector example runs very well with ~200ms eval time on the CPU (i7-6700k) and less than 100ms on the GPU (1080Ti).
When trying it out on the Hololens 2 (10.0.19041.1382), the framerate drops drastically with eval time of ~2000ms on the CPU (Armv8 64-Bit Family 8 Model 803 Revision 70C) and ~10000ms on the GPU (Qualcomm Adreno 630GPU).
Any way to make this faster on the HL2?
I use AISkill - Concepttagger in C++/WinRT project to analyze Images
It work great most of time
however, sometimes, I could not get any result from same picture I get result success before
and I didn't get any error message
I just know that Microsoft.AI.Skills.SkillInterface.dll doesn't work