ritchielozada / unitywithwebrtc Goto Github PK

View Code? Open in Web Editor NEW

57.0 57.0 24.0 7.07 MB

Unity using WebRTC UWP Libraries

License: MIT License

C# 43.10% C++ 53.62% Objective-C 0.44% C 2.84%

unitywithwebrtc's People

Stargazers

Watchers

unitywithwebrtc's Issues

Unable to compile

Hi, I have the same environment as you and I can't compile the project.

If I compile in il2cpp it works but I can't compile UWP solution in visual studio and I need to compile in .NET.

When I compile in .NET, it doesn't works, I have this error : UnityException: Failed to run serialization weaver with command ...

Full error :

UnityException: Failed to run serialization weaver with command "Temp\StagingArea\Data\Managed\WebRtcWrapper.dll" "-pdb" "-verbose" "-unity-engine=Temp\StagingArea\Data\Managed\UnityEngine.dll" "Temp\StagingArea\TempSerializationWeaver" "-lock=UWP\project.lock.json" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\UnionMetadata\10.0.15063.0\Facade" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Calls.CallsVoipContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.SocialInfo.SocialInfoContract\2.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Devices.DevicesLowLevelContract\3.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Devices.Printers.PrintersContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Foundation.FoundationContract\3.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Foundation.UniversalApiContract\4.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Graphics.Printing3D.Printing3DContract\3.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Networking.Connectivity.WwanContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Services.Store.StoreContract\2.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Services.TargetedContent.TargetedContentContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.System.Profile.ProfileHardwareTokenContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.System.Profile.ProfileSharedModeContract\2.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.UI.ViewManagement.ViewManagementViewScalingContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Activation.ActivatedEventsContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Activation.ActivationCameraSettingsContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Activation.ContactActivatedEventsContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Activation.WebUISearchActivatedEventsContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Background.BackgroundAlarmApplicationContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Calls.Background.CallsBackgroundContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Calls.LockScreenCallContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.FullTrustAppContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Preview.InkWorkspace.PreviewInkWorkspaceContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Preview.Notes.PreviewNotesContract\2.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Resources.Management.ResourceIndexerContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Search.Core.SearchCoreContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Search.SearchContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.StartupTaskContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Wallet.WalletContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Devices.Custom.CustomDeviceContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Devices.Portable.PortableDeviceContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Devices.Printers.Extensions.ExtensionsContract\2.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Devices.Scanners.ScannerDeviceContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Devices.Sms.LegacySmsApiContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Gaming.Input.GamingInputPreviewContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Gaming.Preview.GamesEnumerationContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Gaming.UI.GameChatOverlayContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Globalization.GlobalizationJapanesePhoneticAnalyzerContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Management.Deployment.Preview.DeploymentPreviewContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Management.Workplace.WorkplaceSettingsContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Media.Capture.AppBroadcastContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Media.Capture.AppCaptureContract\4.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Media.Capture.CameraCaptureUIContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Media.Capture.GameBarContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Media.Devices.CallControlContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Media.MediaControlContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Media.Playlists.PlaylistsContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Media.Protection.ProtectionRenewalContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Networking.NetworkOperators.LegacyNetworkOperatorsContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Networking.Sockets.ControlChannelTriggerContract\2.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Perception.Automation.Core.PerceptionAutomationCoreContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Security.EnterpriseData.EnterpriseDataContract\4.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Security.ExchangeActiveSyncProvisioning.EasContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Services.Maps.GuidanceContract\2.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Services.Maps.LocalSearchContract\3.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.System.Profile.SystemManufacturers.SystemManufacturersContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.System.Profile.ProfileRetailInfoContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.System.UserProfile.UserProfileContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.System.UserProfile.UserProfileLockScreenContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.UI.ApplicationSettings.ApplicationsSettingsContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.UI.Core.AnimationMetrics.AnimationMetricsContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.UI.Core.CoreWindowDialogsContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.UI.Xaml.Hosting.HostingContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Web.Http.Diagnostics.HttpDiagnosticsContract\2.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.System.SystemManagementContract\3.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.Calls.CallsPhoneContract\3.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.ApplicationModel.CommunicationBlocking.CommunicationBlockingContract\2.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Devices.SmartCards.SmartCardBackgroundTriggerContract\3.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Devices.SmartCards.SmartCardEmulatorContract\4.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Embedded.DeviceLockdown.DeviceLockdownContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Networking.NetworkOperators.NetworkOperatorsFdnContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Phone.PhoneContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.Phone.StartScreen.DualSimTileContract\1.0.0.0" "-additionalAssemblyPath=C:\Program Files (x86)\Windows Kits\10\References\10.0.15063.0\Windows.UI.WebUI.Core.WebUICommandBarContract\1.0.0.0" "-additionalAssemblyPath=Temp\StagingArea\Data\Managed".
Symbols will be read from Temp\StagingArea\Data\Managed\UnityEngine.pdb
Symbols will be read from Temp\StagingArea\Data\Managed\WebRtcWrapper.dll.mdb
Weaving assembly C:\Users\ancardin\Documents\workspace\r3as\UnityWithWebRTC-master\Temp\StagingArea\Data\Managed\WebRtcWrapper.dll
Mono.CompilerServices.SymbolWriter.MonoSymbolFileException: Symbol file `C:\Users\ancardin\Documents\workspace\r3as\UnityWithWebRTC-master\Temp\StagingArea\Data\Managed\WebRtcWrapper.dll.mdb' does not match assembly
   � Mono.CompilerServices.SymbolWriter.MonoSymbolFile.ReadSymbolFile(String mdbFilename, Guid assemblyGuid)
   � Mono.Cecil.Mdb.MdbReaderProvider.GetSymbolReader(ModuleDefinition module, String fileName)
   � Mono.Cecil.ModuleReader.ReadSymbols(ModuleDefinition module, ReaderParameters parameters)
   � Mono.Cecil.ModuleReader.CreateModuleFrom(Image image, ReaderParameters parameters)
   � Mono.Cecil.ModuleDefinition.ReadModule(String fileName, ReaderParameters parameters)
   � usw.Weaver.WeaveAssembly(String assemblyPath, AssemblyDefinition unityEngineAssemblyDefinition, ReaderParameters readerParameters)
   � usw.Weaver.Weave()
   � usw.Program.RunProgram(ConversionOptions options)
   � usw.Program.Main(String[] args)

PostProcessWinRT.RunSerializationWeaver () (at C:/buildslave/unity/build/PlatformDependent/WinRT/SharedSources/CSharp/PostProcessWinRT.cs:492)
PostProcessWinRT.Process () (at C:/buildslave/unity/build/PlatformDependent/WinRT/SharedSources/CSharp/PostProcessWinRT.cs:149)
UnityEditor.WSA.BuildPostprocessor.DoPostProcess (BuildPostProcessArgs args) (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/ExtensionModule.cs:155)
UnityEditor.WSA.BuildPostprocessor.PostProcess (BuildPostProcessArgs args) (at C:/buildslave/unity/build/PlatformDependent/MetroPlayer/Extensions/Managed/ExtensionModule.cs:162)

Render Performance of DirectX Plugin in HoloLens

Render performance of DirectX plugin in Unity to process and display graphics need significant improvement compared to a pure DirectX project. A native DirectX implementation is able to achieve 30+ frames per second update while a plugin with Unity is less than 5 frames per second.

Gray panel not show video streaming of the camera

Hi Ritchie,
congratulations for your great job!
I'm trying to show a video on the gray screen but it doesn't works.
The system is connected because I see that it sand frames but can't see the video.
Any suggests??

Thanks,
BR
Simone

Video not sent from Unity

I'm testing the code, in two differente scenario:

Machine A and Machine B running XamlTestApp application with two MediaElement for preview: all works as expexted and i can see both video streaming
Machine A running Unity project (build for UWP) and Machine B running XamlTestApp: the video stream from the Unity client is not sent (i can see the webcam turned on).

The wrapper is the same, so i suppose it is something related to unity.

I'm using VP8 codec.

Release Build Exception--DllNotFoundException: Unable to load DLL "TexturesUWP"

Release Build Exception--DllNotFoundException: Unable to load DLL "TexturesUWP".

Does it work well in hololens?

I have made it work with hololens. But the frame rate is very low, and the usage of CPU is very high. Is there any resolution to deal with this problem. Btw, it seems some "TODO" work in choose the video encoder. Does it use the Raw Data to communicate?

Having stair step problem on image bounderies and texture filtering(bilinear, trilinear or anisotropic) is having no effect on image quality.

Hi,
I have imported your TextureUWP project for plugin build. I am streaming H264 encoded frames(webcam via Gstreamer) and rendering it on the cube(Hololens). I am having stair step issue on the edges of object in image frame. I have tried different texture sampling techniques but these have no(zero) effect on image quality. I have added below code snippet in TextureUWP.cpp file:

void CreateSampler()
{ //Called at intialization
ZeroMemory(&samplerDesc, sizeof(D3D11_SAMPLER_DESC));
D3D11_TEXTURE_ADDRESS_MODE addressMode = D3D11_TEXTURE_ADDRESS_CLAMP
samplerDesc.AddressU = addressMode;
samplerDesc.AddressV = addressMode;
samplerDesc.AddressW = addressMode;
samplerDesc.ComparisonFunc = D3D11_COMPARISON_FUNC::D3D11_COMPARISON_NEVER;
samplerDesc.Filter = D3D11_FILTER::D3D11_FILTER_MIN_MAG_LINEAR_MIP_POINT;//D3D11_FILTER_ANISOTROPIC;
samplerDesc.MaxAnisotropy = 16;
samplerDesc.MaxLOD = FLT_MAX;
samplerDesc.MinLOD = FLT_MIN;
samplerDesc.MipLODBias = 0;
}

Here I'm creating the sampler.
void UpdateUnityTexture(void* textureHandle, int rowPitch, void* dataPtr)
{
ID3D11Texture2D* d3dtex = (ID3D11Texture2D*)textureHandle;
assert(d3dtex);

ID3D11DeviceContext* ctx = NULL;
m_Device->GetImmediateContext(&ctx);
//New inserted lines
ID3D11SamplerState* sampler = NULL;
m_Device->CreateSamplerState(&samplerDesc, &sampler);
ctx->PSSetSamplers(0, 1, &sampler);
//
ctx->UpdateSubresource(d3dtex, 0, NULL, dataPtr, rowPitch, 0);
ctx->Release();
}

I have not used directX before. Am I doing it right? Please help.

Thanks
Azam

Crash on both apps (client and Unity) when peer connection is established

I've spent a few hours now trying to figure out the reason of this issue but no dice.

I can build and deploy the UWP Peer application on a my laptop as well as build and deploy the Unity UWP client application on my desktop which are both on the same network. I start both apps as well as the server application executable and can connect both peers to the server.

Whenever I click connect peer (on either application) both apps throw an unhandled win32 exception and crash.

Has anyone else experienced this problem?

Where can I get the code of peerconnction_server.exe

I want to write an Android client.Now I can connect to the server and receive offer message in Android client,but can't send anser message.I want to debug through the server code,so can you provide me the server code?

When buidling cant find UnityWithWebRTC-master\UWP\project.lock.json

When I try to build the Unity Project I get the following error: Exception: ExtractAssemblyTypeInfo: Failed to process Assets/Plugins/WebRtcWrapper/WebRtcWrapper.dll, System.IO.DirectoryNotFoundException: Could not find a part of the path "C:\Users\Rouven\Desktop\UnityWithWebRTC-master\UWP\project.lock.json".

I followed your instructions in the ReadMe file, but still not lucky. Any information on this?

Cheers
Rouven

import Org.WebRtc.dll problem

Hi,
I have problem with importing Org.WebRtc.dll into my project.
Is there any other action needed, other then copy the file into project Asset folder?
Thank you.

Is there an interface for the audio play in Unity?

I made it work out in unity, but there is a problem that it has no audio play in Unity.

Repeated calls do not send Video/Audio

Hey, I was testing repeated calls within one application session and figured out that the camera/microphone do not get dropped safely. From this point on it is a little "random" if the next call will work with video and audio.

You have "GC.Collect()" in several parts of your code. Enabling the one I linked at the bottom does let the camera/microphone drop immediately and every next call does work fine. However this only works in Debug mode. As soon as this is built as "Master" for UWP, this call does cause a crash.

Could you provide some information about the "TO-DO" you placed there? I'm not really sure which references are meant to be cleared by the GC.Collect() call to enable repeated calls in one session. I would like to replace this by the safe calls.

GC.Collect() 1

Question Server

Hi @ritchielozada ,

one question, where is the server from? Is it opensource?

Regards

Nothing In Hololens

When I deploy it on Hololens Device and start the application ,but I can not see anything in Hololens Except the TextureQuad. Anyone can tell me why?

Sync data with frames

I need to send some kind of data together with audio/video streaming.
I've succesfully used the data channel, but i can't have control of timing.
Data sent on the data channel are not synched to frames and i strongly need it (a video with text subtitles for example)

A solution could be packing this data (few bytes) in the raw frame, but i'm afraid there is not entry point where to update the local frame before it has been sent to the peer.

Any suggestion?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.