versatica / mediasoup-client Goto Github PK
View Code? Open in Web Editor NEWmediasoup client side JavaScript library
Home Page: https://mediasoup.org
License: ISC License
mediasoup client side JavaScript library
Home Page: https://mediasoup.org
License: ISC License
Given that mediasoup v2 uses a mediasoup protocol the native SDK for Android and iOS should implement that protocol.
Help, ideas and proposal are welcome.
nvr mnd
Safari is receiving the streams but can't play them. I don't get any errors in the console and I don't really know if it is the server, the client or the browser itself that has a problem.
I attached the debug output from mediasoup for server and both clients (publisher and subsriber).
Publishing with Safari works only in direction to Chrome as subscriber and Firefox only receives 1fps.
But this is a different problem and I will create a new issue if you think it's a bug.
What I tried to do:
MacOS: 10.14
Safari: 12
Firefox: 63
const MEDIA_CODECS = [
{
kind: "audio",
name: "opus",
clockRate: 48000,
channels: 2,
parameters: {
useinbandfec: 1
}
},
{
kind: "video",
name: "H264",
clockRate: 90000,
parameters: {
"packetization-mode": 1,
"profile-level-id": "42e01f",
"level-asymmetry-allowed": 1
}
}
]
Logs
mediasoup-client-publisher.log (Firefox)
mediasoup-client-subscriber.log (Safari)
mediasoup-server.log
Hello everyone,
having kind of a weird problem that I just can't seem to resolve. I am using version 2.1.0 (also tested 2.0.16) of the mediasoup-client and have a local demo server set up for testing. Using a mediasoup-demo client with gulp live works just fine, I am having an Android implementation that also works fine with the server I am running.
However, I can't seem to get it working when including it into an existing application. I used the simple example from the readme, but whenever I try to add the second producer, I get a "InvalidAccessError: Track already exists".
It does not matter in which order I do it (video first or audio first), the second call always gives me this error. Debugged a bit and it happens specifically when doing "addStream" to the peerconnection in the addProducer call. The streams I am getting from the getUserMedia calls do have different IDs.
Current code snippet (added a 5 sec delay for the second since I suspected racing conditions): https://pastebin.com/uSkFb4xR
I am getting into the "video send error" case. Started off with an exact copy of the example given (just one getUserMedia call etc.) and it was just the same.
Maybe I am just blind and oversaw something critical, but at the moment I am completely at a loss after 12 hours of trying to get this running, and am wondering if there is some kind of bordercase bug that leads to adding the same stream twice.
Thanks in advance!
After updating chrome to version 73, using H264 codec, can't see the video of broadcaster (black screen), it seems the problem in getting requestConsumerKeyFrame, because after effective profile change, the video is appeared. It regards v2 of mediasoup, can see on mediasoup demo as well
mediasoup-client already handles them (094601f). However is better to track the Edge issue reported here:
Would be great if we have a way to generate mediasoup requestObjects on demand(without using the event), something like Room.createJoinRequest(peerName, [appdata]).
It would be great if we could get client side stats exposed (https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/getStats). I know browser specs still different pretty wildly on delivering those, but just retrieving what we can would be great.
Let me know if you'd like me to add it as a PR.
As explained here (thanks @alvestrand!), we can reuse a SDP m= section for sending media by calling stop()
on the corresponding transceiver
first. Then, when creating a new transceiver
it may reuse the existing and "stopped" SDP m= section.
NOTE: transceiver.stop()
implemented in Firefox and Safari (but not in Chrome yet). So we can improve our Firefox and Safari handlers.
Chrome55 and Chrome67 handlers, as well as two Firefox handlers have a lot of common code.
Shared code can be moved out to some "abstract" base class with overrides for methods that are actually different.
Are there any plans to cleanup handlers?
I can do a pull request if you are fine with these changes.
When joining a room with Chrome72 (Windows10), which was started with Firefox65 (Linux) I'm unable to see the webcam of the Firefox participant. Chrome is throwing the following error. I'm using the same config as used in the mediasoup demo app. (config.example.js).
I'm not quite sure what is causing this, I haven't had this issue before Chrome < 72.
Snippet:
streamWebcam(stream){
const videoTrack = stream.getVideoTracks()[0];
this._webcamProducer = this._room.createProducer(videoTrack, null, {type: 'webcam'});
this._webcamProducer.send(this._sendTransport)
.then(() => console.log('sending our webcam'));
}
streamMic(stream){
const audioTrack = stream.getAudioTracks()[0];
this._micProducer = this._room.createProducer(audioTrack, null, {type: 'webcam'});
audioTrack.stop();
this._micProducer.send(this._sendTransport)
.then(() => console.log('sending our mic'));
}
BTW Firefox -> Firefox, goes without any issue.
When placing mediasoup-client
within a package.json "dependencies"
using the GIT URL (to test a branch or whatever) it does not work:
"dependencies": {
"mediasoup-client": "github:versatica/mediasoup-client#master"
}
It happens that the retrieved code does not include the lib-es5
folder (babelized via gulp babel
). Must work on this.
Can mediasoup client sdk be used by a Node.js Client Application? So audio from a browser can be streamed in real-time to this node.js client? Signalling in this case handled by a central server (another Node.js application).
In my case the Node.js client application will be running on a Raspberry Pi.
There's a transpiler issue that failed to transpile codes into lib-es5
.
For instance, in line 21 of lib/handlers/sdp/commonUtils.js
:
for (const m of sdpObj.media) { ...
transpiled to:
for (var _iterator = sdpObj.media[Symbol.iterator](), _step; !(_iteratorNormalCompletion = (_step = _iterator.next()).done); _iteratorNormalCompletion = true) { ...
it will throw Uncaught TypeError: sdp.media[Symbol.iterator] is not a function
as Iteration_protocols, pure js object is not iterable.
The best way to solve it is to refactor the codebases by replacing
for (const m of sdpObj.media) { ...`
with
for (const m of Object.values(sdpObj.media)) { ...`
for reference: babel/babel-loader#84
BTW, I'm working on React-Native with mediasoup-client, the workaround to this issue just changes the nodejs entrypoint from "lib-es5/index.js" to "lib/index.js" as package.json
below:
...
"homepage": "https://mediasoup.org",
"license": "ISC",
"main": "lib/index.js",
"name": "mediasoup-client",
...
any suggestion? mediasoup
is great ๐
4ed7948#diff-baa1c0fc131ac17847da1f8864df2863
Add Chrome67 handler with support for rtpSender.replaceTrack()
Isn't Chrome 65 already supporting it? https://www.chromestatus.com/features/5700232381726720
Unified-Plan issues in all browsers (with the collaboration of @fippo):
And some items not clear in the spec:
InvalidSessionDescriptionError: Invalid description, mid length greater than 16 unsupported until 2-byte rtp header extensions are supported in webrtc.org
Firefox limited mid
length to 16 characters: mozilla/gecko-dev@69d270a
Can mid
be made shorter?
Would be nice to have a possibility to connect clients directly. This is something implemented in jitsi-meet as well and is important for scaling and privacy.
For jitsi description see: https://jitsi.org/news/p2p4121
Can we have the option to enforce the use of H264 codec on all clients (chrome, edge, firefox) when a Safari 11 user joins, so that the safari user can see all videos instead of using a browser plug in.
you can look in this fiddle in order to enforce H264 codec
https://jsfiddle.net/rainzhao/s5tadxgq/
What am I doing wrong? I followed the sample code...
const peers = await this.room.join(this.id);
this.sendTransport = this.room.createTransport('send');
this.recvTransport = this.room.createTransport('recv');
peers.forEach(this.handlePeer);
const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });
this.localVideoComponent.srcObject = stream;
const audioTrack = stream.getAudioTracks()[0];
const videoTrack = stream.getVideoTracks()[0];
const audioProducer = this.room.createProducer(audioTrack);
const videoProducer = this.room.createProducer(videoTrack);
audioProducer.send(this.sendTransport);
videoProducer.send(this.sendTransport);
This code gives me the error Error: no transport remote parameters
.
The case is, I want to change the video resolution after created a producer like this
videoProducer.track.applyConstraints({
width: { ideal:1280 },
height: { ideal: 960 }
}).then(()=> {
console.log("success")
})
After success, I have checked video tag videoHeight and videoWidth properties and it showed 640*480. It will throw overConstraintsException
if I use exact
instead of ideal
. And I have done the same action for videoProducer.originalTrack
, not work either.
But, if I close the video producer using videoProducer.close()
or never create any producer use the track, it becomes 1280 * 960. So my webcam can support this resolution.
I have found video producer will copy the track and hold it, I don't know if it's the case and how to resolve it. Now, I have to close producer first when I want to change any constraints, is there a more graceful solution?
When doing ICE restart in Firefox (so getting new remote iceParameters
from the server-side WebRtcTransport and then calling transport.restartIce({ iceParameters })
with them) the media freezes in both send and recv directions. It's reproducible in the demo app.
It also happens in mediasoup v2.
NOTE: No idea about "ICE restart" support in FF.
No investigated yet. Basically audio/video from Firefox does not properly reach remote consumers.
according to source we send 'direction' to server during transport creation
it's my packages.json
:
"mediasoup-client": "^2.4.0",
"react": "16.3.1",
"react-native": "0.55.1",
When I'm launching my RN app (android) in dev mode, get this error:
error: bundling failed: Error: Couldn't find preset "@babel/env" relative to directory "....\\node_modules\\mediasoup-client"
what babel packages or configs I should do? thanks~
As mentioned here, mediasoup-client is using ES6 syntax. With meteor 1.6+ version, the support to ES6 syntax is inbuilt. But I still get the following errors when I use "import { getDeviceInfo } from 'mediasoup-client'".
Uncaught SyntaxError: Unexpected token import
on Chrome and
SyntaxError: import declarations may only appear at top level of a module
on Firefox
Hello!
I'm trying to set a TURN server, because of firewall restrictions for some users. So, on the v2.4.9, I've set this roomOptions to the mediasoupClient.Room(roomOptions) :
roomOptions: {
turnServers: [
new RTCPeerConnection({ iceServers: [
{ urls: 'stun:turn.myserver.com' },
{ urls: 'turns:turn.myserver.com', username: 'myuser', credential: 'mypassword', credentialType: 'password' },
]}),
],
}
And I get this error on Firefox ( 66 on Linux, 65 on Windows )
error communication TypeError: "Missing required 'urls' member of RTCIceServer"
And this error on Chrome 72 (Windows and Linux):
error communication TypeError: Failed to construct 'RTCPeerConnection': Malformed RTCIceServer
I known the format have changed ( before the property was 'url' and not 'urls', and username wasn't a property). Here is the good format: https://w3c.github.io/webrtc-pc/#rtciceserver-dictionary
I've also tried to replace 'turns:' (port 443) to 'turn:', but doesn't help.
May be the lib mediasoup client doesn't support the new format, or may be there is something wrong in my configuration ?
Thanks for your help!
According to https://github.com/versatica/mediasoup-client/blob/master/MEDIASOUP_PROTOCOL.md#createproducer
createProducer method should return {}
(empty object) but return underfined
Should server return empty object or underfined?
Scenario (assuming the mediasoup room is configured with VP8 and H264 codecs in that order of preference):
transports
of type "send" and apply the preferred codec in them.producers
of type "video" and apply the same video track to them.consumer
plays the VP8 track.consumer
plays the H264 track.Now i use this client v2, but this does not support a lot of Chrome based browsers.
YandexBrowser is popular in Russia, is based on Chrome 71. A lot of various browser the same.
Plese add some code to your Device.js file in _detect() at last:
else if (ua.search('Chrome')!=-1)
{
Device._flag = 'chromium';
Device._handlerClass = Chrome70;
}
In both mediasoup-client and mediasoup v2.
Reported in https://bugzilla.mozilla.org/show_bug.cgi?id=1437980
Relevant snippet from handlers/Firefox50.js
.then(() =>
{
const newRtpReceiver = this._pc.getReceivers()
.find((rtpReceiver) =>
{
const { track } = rtpReceiver;
if (!track)
return false;
return track.id === consumerInfo.trackId;
});
if (!newRtpReceiver)
throw new Error('remote track not found');
return newRtpReceiver.track;
});
newRtpReceiver track id will not match consumerInfo trackId because Firefox no longer uses the msid from the offer as the track id.
No idea how to handle this: https://bugzilla.mozilla.org/show_bug.cgi?id=1393407
Something coming to my mind is:
removeProducer
or removeConsumer
is called, and it corresponds to the first m=
section, abort.when i build a personal client to connect your demo's server, i have a error what is 'Failed to execute 'send' on 'WebSocket': Still in CONNECTING state'. i think i have connected to server, why?(may be my mySignalingChannel has a error, i don't know react , could you give me a simple demo?)
Is there any example of implementation with react-native?
Where can I find it?
I try to install mediasoup in my server from code. I clone the demo repo into my server and run npm install on both app and server. Then I run server.js, and set my nginx to point out server/public folder.
But I get this error which results in no media being transfered:
mediasoup-client:Transport Transport connection state changed to failed
mediasoup-client:Transport Transport connection state changed to failed
Does any body know how to fix this?
Thank you
Hi!
When I disable an audio track that i'm sending to a room (audio levels reach 0, tested with hark), but on the receivnig end there is noise in the audio.
AFAIK it's common practice to add comfort noise on packet loss, but in this case I have no packet loss. The audio track is just disabled.
I'm not sure if this is supposed to add comfort noise, but either way I'd like to disable it somehow.
There are multiple issues with Edge transport for consumers:
_setupTransport
is not called when calling addConsumer
MediaStreamTrack
is not returned from addConsumer
methodTo be tested and so on. In fact, right now Chrome M74 does not respect the user given rid
values.
Must also change the way we are enabling simulcast in FF since we don't do it in the addTransceiver()
but later via `sender.setParameters()' . However the spec says:
The addTransceiver method establishes the simulcast envelope which includes the maximum number of simulcast streams that can be sent, as well as the ordering of the encodings. While characteristics of individual simulcast streams can be modified using the setParameters method, the simulcast envelope cannot be changed.
The problem is that Firefox does not yet implement RTCRtpTransceiverInit
: issue report.
rid
values).
When receiving any Consumer in mediasoup-client v3 in Firefox:
"InvalidSessionDescriptionError: Invalid description, mid length greater than 16 unsupported until 2-byte rtp header extensions are supported in webrtc.org"
This is because we are setting a=mid
with the value of the consumer.id
which in v3 is an UUID (prefixed with "a-" or "v-" if audio or video), such as:
a=mid:v-c5a4e034-9aef-4179-8689-d2f443f181d0
Chrome happily ignores it because it does not send MID RTP header extension, but Firefox does. Anyway those long a=mid
just happens in the PeerConnection for reception but Firefox complains anyway (because it may thing that, eventually, it should also send media on it).
Such as, for example in Chrome:
{
googCpuOveruseDetection: false,
googCpuOveruseEncodeUsage: false
}
For this, we need a new (and complex) setting in RoomOptions:
const room = mediasoupClient.Room(
{
requestTimeout : 8000,
googPeerConnectionConstraints :
{
googCpuOveruseDetection: false,
googCpuOveruseEncodeUsage: false
}
});
The thing is:
This must be defined.
Moving https://github.com/versatica/mediasoup-demo/issues/16 to this repo.
Currently there is no way to flip a camera on a mobile device using mediasoup-client. I tried mediasoup-demo, and it didn't work. The current workaround is to stop the tracks both on mediaStream and videoProducer:
Typescript example:
export const switchCamera = async (
videoDevicesCycle: Iterator<MediaDeviceInfo>,
videoStream: MediaStream,
videoProducer: any
): Promise<MediaStream> => {
const videoDevice = videoDevicesCycle.next().value
videoStream!.getVideoTracks()[0].stop()
videoProducer.track.stop()
videoStream = await getStream(
config.videoMediaStreamConstraints(videoDevice!.deviceId)
)
await videoProducer.replaceTrack(videoStream.getVideoTracks()[0])
return videoStream
}
Issue and some rationale described here: #47
Hi, I'm interested in this sdk. Mediasoup is very nice and it's exactly what I wanted to use.
By the way, I wonder if there is a very simple example using <script src="mediasoup-client.js"/> and its API.
I tried to find it but I've got only mediasoup-client.js.
Could you please give me any advice about this? Thank you.
On mediasoup-client 2.0.17
With react-native 0.55.3
, I get this error when running the app on android:
error: bundling failed: ReferenceError: Unknown plugin "transform-builtin-classes" specified in "xxx/node_modules/mediasoup-client/.babelrc" at 0, attempted to resolve relative to "xxx/node_modules/mediasoup-client"
at xxx/node_modules/babel-core/lib/transformation/file/options/option-manager.js:180:17
at Array.map (<anonymous>)
at Function.normalisePlugins (xxx/node_modules/babel-core/lib/transformation/file/options/option-manager.js:158:20)
at OptionManager.mergeOptions (xxx/node_modules/babel-core/lib/transformation/file/options/option-manager.js:234:36)
at OptionManager.init (xxx/node_modules/babel-core/lib/transformation/file/options/option-manager.js:368:12)
at File.initOptions (xxx/yumii-mobile/node_modules/babel-core/lib/transformation/file/index.js:212:65)
at new File (xxx/yumii-mobile/node_modules/babel-core/lib/transformation/file/index.js:135:24)
at Pipeline.transform (xxx/yumii-mobile/node_modules/babel-core/lib/transformation/pipeline.js:46:16)
at Object.transform (xxx/yumii-mobile/node_modules/metro/src/transformer.js:135:5)
However, I didn't encounter that error when using react-native 0.50.4
I'm quite unsure about what could cause that...
I would like to update the ICE server after the room has been created in mediasoup.client,
but unfortunately I didn't find a room function that I could use to update the turnServers.
Would it be possible to add a new function to Room object that updates RoomOptions->turnServers and this way I could restart ICE with the new turnServer?
I get a time limited long term (turn) credential from an API, so this is why I need this feature.
After receiving the TypeError I don't get any more logs and nothing happens.
Normally I would expect logs for transports/peers/consumers after, but as you can see, it ends after the last entry.
In the code I can see that it is in a try catch. Does the logger also stop further execution?
Sadly I don't have enough time to look deeper into the problem and provide a PR.
Microsoft Edge: 40.15063.674.0
Microsoft EdgeHTML: 15.15063
mediasoup-client:Room constructor() [options:%o] +0ms undefined
mediasoup-client:Device browser supported [flag:msedge, name:"Microsoft Edge", version:15.15063, handler:Edge11] +0ms
mediasoup-client:Room join() [peerName:"ym6czcx"] +3ms
mediasoup-client:Room _sendRequest() [method:queryRoom, request:%o] +561ms [object Object]
mediasoup-client:Room request succeeded [method:queryRoom, response:%o] +7s [object Object]
mediasoup-client:Room join() | got Room settings:%o +4ms [object Object]
mediasoup-client:Edge11 getNativeRtpCapabilities() +0ms
mediasoup-client:Room join() | native RTP capabilities:%o +519ms [object Object]
mediasoup-client:Room join() | extended RTP capabilities:%o +3ms [object Object]
mediasoup-client:Room join() | effective local RTP capabilities for receiving:%o +4ms [object Object]
mediasoup-client:Room _sendRequest() [method:join, request:%o] +4ms [object Object]
mediasoup-client:Room request succeeded [method:join, response:%o] +328ms [object Object]
mediasoup-client:Room join() | joined the Room +4ms
mediasoup-client:Room createTransport() [direction:recv] +2ms
mediasoup-client:Transport constructor() [direction:recv, extendedRtpCapabilities:%o] +0ms [object Object]
mediasoup-client:Edge11 constructor() [direction:recv, extendedRtpCapabilities:%o] +866ms [object Object]
mediasoup-client:Edge11 iceGatherer.gather() failed: TypeError: Object doesn't support property or method 'gather' +5ms
Low stream must have "high" priority and so on:
mediasoup-client/lib/handlers/Firefox59.js
Line 150 in 60909bc
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.