Comments (7)
@hoch I think the only place that would need to be edited would be the definition of allowed to start.
https://webaudio.github.io/web-audio-api/#allowed-to-start
This is the current version:
An AudioContext is said to be allowed to start if the user agent allows the context state to transition from "suspended" to "running". A user agent may disallow this initial transition, and to allow it only when the AudioContext's relevant global object has sticky activation.
Should I go ahead and create a PR to change this paragraph to explicitly disallow the user agent to block the transition if the sinkId
is set to { type: 'none' }
?
@mjwilson-google I might be wrong but I think it should not provide any additional fingerprinting capabilities compared to an OfflineAudioContext
.
@danrossi In theory there is an option to get a MediaStream from a muted audio element using captureStream()
. It should completely ignore the volume of the audio element. But as far as I can tell current implementations are very incosistent.
It's the second to last paragraph in here: https://w3c.github.io/mediacapture-fromelement/#dom-htmlmediaelement-capturestream.
... Muting the audio on a media element does not cause the capture to produce silence, ... Similarly, the audio level or volume of the media element does not affect the volume of captured audio.
from web-audio-api.
This is the corresponding bug for Chromium: https://bugs.chromium.org/p/chromium/issues/detail?id=1467591.
Should the spec explicitly say that the autoplay policy should not affect a muted AudioContext
?
from web-audio-api.
Probably worth it to make explicit yes.
from web-audio-api.
@chrisguttandin Could you take a look at the possible spec change for this? The impl change might be trivial, but we need a written spec to initiate the work.
from web-audio-api.
There are two benefits I see to the autoplay restriction:
- It stops websites from playing annoying sounds.
- It stops websites from using WebAudio for user fingerprinting on sites they haven't interacted with.
Muted AudioContexts don't play annoying sounds, but they could still potentially be used for fingerprinting by running audio graphs and looking at the results. Note that microphone input should be gated behind a permission, so direct audio capture shouldn't be possible without user interaction.
Is this something we should be concerned about? My guess is that the autoplay restriction wasn't supposed to be a privacy or security feature so we should handle anti-fingerprinting another way, but I wanted to bring this up just in case.
from web-audio-api.
I've been putting together some tests. There seems to be an oversight here. For web admin consoles behind logins that need autoplay players muted for live streaming with analyser working to show an audio meter. It requires some kind of modal requesting gesture each load. Which will enable context and unmute the video element to get analyser data. Even with gain attached and the gain muted, the video requires to be unmuted to get data, which breaks autoplay. I'm unsure how to get the analyser working with muted video elements.
The gain node is supposed to take over volume control and the video element stay unmuted with volume set to 1. So to override volume control with a gain node because a muted video element gets no data, a context with no sink just for the analyser isn't helpful unless multiple contexts can be setup one for volume control and another for analyser but then I still run into the problem of requiring a muted video element for autoplay and analyser gets no data,
I found a hack by temporarily requesting microphone device permissions like with webrtc features completely works around autoplay mute and context restrictions and with gain muted also. unmuted video element still fails to play in safari, it has to go back to muted. The autoplay should somehow identify the gain is muted rather than the video element to keep receiving data. For both video playback and audio context.
There should have been a permission feature so there is no requirement to show a modal dialog to keep requesting user gesture and permission saved in the site settings. Like with camera and microphone. I may have to implement such a hack showing modal dialog in my feature to turn on context and unmute the video element and may be annoying for the end user as it's impossible to save the setting. And provide the mediadevice hack as another option.
- mediadevice work around https://electroteque.org/dev/audio/video-mediadevice.html
- click to enable context and unmute video element per page load https://electroteque.org/dev/audio/video.html
- same as above but unmute and play to work around browser restriction with unmuted video element - https://electroteque.org/dev/audio/video-unmuteplay.html
- mediadevice option has no restriction - https://electroteque.org/dev/audio/webcam.html
from web-audio-api.
@chrisguttandin captureStream also requires an unmuted video element to analyse audio. Like with canvas capture it's software encoding so incurs unnecessary cpu. Maybe there should be an option to just capture audio not video. I've implemented features to show the requirement of a permission system like mediadevices so acceptance of audio contexts is stored in site settings. Why I think it's an oversight. You can see here it requires to display a modal dialog to capture gestures per page load as nothing is saved or remembered.
I did find however clicking from another page in chrome, will work around unmuted video and context autoplay restrictions. So the modal will only show up when visiting the page directly and suspended context states. I developed a hack requesting microphone temporarily with a dialog to notify what is going on. Works around all autoplay restructions. And the permission is saved to keep requesting it only if the context is suspended.
I guess with a context with an empty sink can be used for the analyser, and due to the problem requiring unmuted video elements to get audio data. A second context is needed to mute audio through a gain node which I have a feature for so muting keeps audio analysers going by keeping the element unmuted with volume set to 1. Autoplay will still fail due to muted restrictions. Webkit is more touchy with this and will immediately stop playback if the element is unmuted.
Should I try and make a report about captureStream with chrome or provide a way to get audio data with muted video elements ?
https://electroteque.org/plugins/videojs/audio-visualiser/demos/autoplay/
https://electroteque.org/plugins/videojs/audio-visualiser/demos/autoplay-mediadevice/
Chrome remembers clicks from another page so no autoplay restriction. Firefox and Safari does not.
https://electroteque.org/plugins/videojs/audio-visualiser/demos/
from web-audio-api.
Related Issues (20)
- Should copyToChannel accept a shared Float32Array? HOT 2
- Handling unconnected AudioWorkletNode's output HOT 11
- Device-related error reporting via AudioContext.onerror HOT 18
- Event Loop execution in `closed` state HOT 3
- Using `default` device ID for setSinkId() HOT 7
- Mono audio plays on only the left speaker if a ChannelSplitterNode is used HOT 11
- Review mute/unmute/ended and constraints on track in audioContext.createMediaStreamDestination().stream HOT 1
- AudioWorkletProcessor difficult to use i have new Idea
- add AudioNode methods function for manipulation
- Whether process() is called for an AudioWorkletNode should depend on whether the active source flag is true and whether the AudioNodes connected to inputs are actively processing HOT 4
- AudioDestinationNode numberOfOutputs=1 according to the spec, but all browsers return 0 HOT 3
- AudioWorkletProcessCallback should not be a callback type HOT 8
- Audio routed to the earpiece speaker HOT 1
- AudioContext stuck on "interrupted" in Safari HOT 1
- Can ASSN.start() start an AudioContext that would otherwise not be allowed to start? HOT 5
- error event uses ErrorEvent but it doesn't fill in any of its fields HOT 2
- Wrong "Fire an event" syntax in recent updates HOT 1
- Sometimes often no sound/audio but not always HOT 1
- Specify different behaviors for no sink ID and the default sink ID (""). HOT 4
- AudioContextOptions sinkId should only be allowed in a secure context
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from web-audio-api.