Background

If XRSession.inputSources is accessed it calls device.getInputSources() where device is WebVRDevice.

  get inputSources() {
    return this[PRIVATE].device.getInputSources();
  }

https://github.com/immersive-web/webxr-polyfill/blob/master/src/api/XRSession.js#L314

WebVRDevice.getInputSources() returns an array composed from its .gamepadInputSources

  getInputSources() {
    let inputSources = [];
    for (let i in this.gamepadInputSources) {
      inputSources.push(this.gamepadInputSources[i].inputSource);
    }
    return inputSources;
  }

https://github.com/immersive-web/webxr-polyfill/blob/master/src/devices/WebVRDevice.js#L567

.gamepadInputSources is empty first, and they're set up in onFrameStart().

  onFrameStart(sessionId) {

          ...

          this.gamepadInputSources[i] = inputSourceImpl;

          ...

  }

https://github.com/immersive-web/webxr-polyfill/blob/master/src/devices/WebVRDevice.js#L295

I faced a problem in Three.js with this behavior. Three.js keeps session.inputSources and uses later here.

			inputSources = session.inputSources;

https://github.com/mrdoob/three.js/blob/r108/src/renderers/webvr/WebXRManager.js#L144

But the above line is called before the first immersive animation frame is invoked. So inputSources is an empty array even if some controllers exist.

Question

Is this behavior intentional? If so, should the problem be fixed in Three.js side?

The spec mentions

Each XRSession has a list of active XR input sources (a list of XRInputSource) which MUST be initially an empty list.

The inputSources attribute returns the XRSession's list of active XR input sources.

When new XR input sources become available for XRSession session, the user agent MUST run the following steps:
...
Extend session’s list of active XR input sources with added.
...

https://immersive-web.github.io/webxr/#xrsession-interface

I'm not really sure exactly when they should be set up. (Sorry if I'm missing) In immersive animation frame? Or synchronously when XRSession is instanciated if controllers are already available?

The README for the polyfill is presenting this as "an implementation of WebXR".

But WebXR is not finished, and will be continuing to change for a little while.

Please add a banner to the top of the README saying that this will track WebXR as it is being defined, and that it will change without notice.

We don't want people writing code assuming this is the final standard.

hey!
after requesting a session, the AR session starts as expected. unfortunately there is no camera feed on the display. This is my code:

`import React from 'react';
import PropTypes from 'prop-types';
import { withStyles } from '@material-ui/core/styles';
import WebXRPolyfill from 'webxr-polyfill';
import * as THREE from 'three';

class WebXR extends React.Component {
    constructor(props) {
        super(props);
        // this.onXRFrame = this.onXRFrame.bind(this);
        this.onEnterAR = this.onEnterAR.bind(this);
        

        this.init();
    }

init = async () => {

    if (navigator.xr && window.XRSession.prototype.requestHitTest) {
        try {
            this.device = await navigator.xr.requestDevice();
            this.message = 'device supported';


        } catch (e) {
     
            this.message = 'device not supported';

            return;
        }
    } else {
        // If `navigator.xr` or `XRSession.prototype.requestHitTest`
        // does not exist, we must display a message indicating there
        // are no valid devices.
        this.message = 'device not supported';

        return;
    }

    document.getElementById('enter-ar').addEventListener( 'click', this.onEnterAR);
   
    
}
onEnterAR = async () => {

    let outputCanvas = document.createElement('canvas');
    
    let ctx = outputCanvas.getContext('xrpresent');
    try {
        
  
        const session = await this.device.requestSession({
          
            outputContext: ctx,
            environmentIntegration: true
        });            

       

 
        document.body.appendChild(outputCanvas);
        this.onSessionStarted(session)
        
    } catch (e) {

        this.message = 'device not supported';


    }
    
}



onSessionStarted = async (session) => {
     this.message = 'session started';
    this.session = session;

    this.renderer = new THREE.WebGLRenderer({
        alpha: true,
        preserveDrawingBuffer: true,
      });
    this.renderer.autoClear = false;

    this.renderer.shadowMap.enabled = true;
    this.renderer.shadowMap.type = THREE.PCFSoftShadowMap;

    this.gl = this.renderer.getContext();


    await this.gl.setCompatibleXRDevice(this.session.device);


    this.session.baseLayer = new window.XRWebGLLayer(this.session, this.gl);

    const framebuffer = this.session.baseLayer.framebuffer;
    this.renderer.setFramebuffer(framebuffer);

    this.scene = new THREE.Scene() // The scene will be rotated and oriented around the camera using the head pose

    this.camera = new THREE.PerspectiveCamera(70, 1024, 1024, 0.1, 1000) // These values will be overwritten by the projection matrix from ARKit or ARCore
    this.scene.add(this.camera)

    

    this.frameOfRef = await this.session.requestFrameOfReference('eye-level');
    this.session.requestAnimationFrame(this.onXRFrame);

    }
    render() {

        return (
            <div id="wrapper" style={styles.root}>WebXR {this.message}
                <button id="enter-ar" >Enter AR</button>
                <div> Output: {this.output}</div>
            </div>

        );
    }
}




export default withStyles(styles)(WebXR);

I am not sure, whats wrong. I know this is more a stackoverflow question, but I didn't get any response. Thanks a lot!

It'd be amazing if we could hook into some/all of the web platform tests that browsers run to ensure spec compliance:

https://github.com/web-platform-tests/wpt/tree/b6523af4883a9b9db729fbcd6c411ae1441676d9/webxr

Background

The polyfill dispatches an event with an object like

this.dispatchEvent('selectstart', {
  frame: this[PRIVATE].frame,
  inputSource: evt.inputSource
});

https://github.com/immersive-web/webxr-polyfill/blob/master/src/api/XRSession.js#L104-L107

But if I understand correctly the second argument should be XR*Event inheriting Event.

https://www.w3.org/TR/webxr/#events

So the code above should be like this

class XRInputSourceEvent extends Event {
  constructor(type, initDict) {
    super(type, initDict);
    this.frame = initDict.frame;
    this.inputSource = initDict.inputSource;
  }
}

...

this.dispatchEvent('selectstart', new XRInputSourceEvent('selectstart', {
  frame: this[PRIVATE].frame,
  inputSource: evt.inputSource
});

Problem

Because the polyfill currently dispatches an event with an object, the user code relying on Event API doesn't work. For example Three.js has a code using Event.type.

	controllers[ i ].dispatchEvent( { type: event.type } );

https://github.com/mrdoob/three.js/blob/r108/src/renderers/webvr/WebXRManager.js#L80

Question

Is there any special reasons why the polyfill doesn't define XR*Event yet? If it's because just lack of resources we'd help.

There are some Google usages of the polyfill currently, and it requires usage of Closure Compiler and JSDoc comments. I think the current JSDocs do provide sufficient value, and we should have a test to ensure that they are valid for tools that consume this data for any release.

Currently we only check that it's a valid configuration on recognized keys exclusive and outputContext. That means if there's a new session feature foo implemented by BlinkFoxEdge, then the polyfill would think this is a valid, supported session: supportsSession({ foo: true, exclusive: true })

This is a meta bug for discussion around updating the polyfill to the latest version of the spec now that things have settled down a bit.

I've created a handful of categorized issues, detailing what I'm aware of that has changed at least, and areas that probably need to be confirmed. The issues are all tagged in the WebXR 2019 milestone.

Contributing/development

If you're interested in contributing, express interest in an issue! We will be using the webxr-2019 branch for PRs. There are examples (originally from webxr-samples) in the repo that we can use to test progress as we iterate inbetween two APIs, hopefully ending up with something functionally similar to what could be on webxr samples in the future.

The polyfill has two main areas: the API plumbing (src/api/*) and providers (src/devices/*. Not only will the API have to be updated to match the spec, but the input between the API and the underlying provider may also need to change.

We can test against WebVR and non-WebVR mobile browsers using the WebVRDevice provider and CardboardXRDevice provider. If we can incrementally make these changes while updating the local examples, then we should still be able to support the examples in these browsers as a means of testing. In addition to the traditional unit tests here to help with the plumbing.

Happy to answer any questions about the structure of the current code!

The polyfill currently is (was) targeting the latest WebXR API spec. The webxr-examples repo contained a shim that would patch various iterations of the implementation in Chrome (patching changes between Chrome 68, 69, 70, etc.). With the WebXR API stabilizing, it's now appropriate for the polyfill to handle these smaller 'patches' between versions. We should be explicit on what browsers/devices we attempt to support. We should:

• Define support range: e.g. last 2 major versions, plus Chrome M73 (Magic Leap, Samsung)
• Document support range in README plus any extra config/modules needed to support different browsers. <model-viewer> has great matrix+docs for its support, which require devs to include different polyfills, which aids in configuring the tradeoffs of your choosing

Maybe just a z-index issue, not reproducible on iOS11

From the README:

"Download the build at build/webxr-polyfill.js and include it as a script tag, or use a CDN. You can also use the minified file in the same location as webxr-polyfill.min.js."

There is a scenario where inline sessions are supported (all WebXR implementations must support this), but no immersive session (e.g. an AR headset?).

If and only if an immersive session is needed and unsupported, the polyfilled stereoscopic view (cardboard-vr-display) could be included and supported, but it's a bit of a strange scenario where sessions can either be native or polyfilled.

A similar scenario in the webvr-polyfill existed, where Firefox for Android supported WebVR, but provided no VRDisplays. In this case without intervention, the WebVR spec is providing a worse experience than not at all. The solution was to (optionally), if on mobile, query for VRDisplays if the API exists, and inject a polyfilled VRDisplay only if none exist.

This is a bit of a gray area and could introduce some complexity. Practically speaking, I'd like to wait until there's a device/browser combo that could cause this where the polyfill could produce a more desirable experience.

Via immersive-web/webxr#366

Defaults to 'opaque' -- possible that we could detect WebVR 1.1 implementations that use AR and return 'alpha-blend', but probably best to let extensions of PolyfilledXRDevice provide their own values.

There doesn't seem to be too much missing from the current input implementation, but this hasn't been tested in awhile. A lot of TBD here. AFAIK things that are new:

• gripSpace (dependent on #50 XRSpace)
• Pointer events
• Gamepad integration(?)

See #51 for more information on this effort, and development goals.

I'm wondering about the possibility to polyfill AR, akin to how Carboard is polyfilled, but for "magic window" (not stereo) AR.

There are a few reasons to do this:

• when developing AR content, it is often convenient to use desktops for debugging. Yes, you can't get at the platform capabilities (e.g., full motion control, anchors, etc), but I have personally found that being able to do some of the debugging of content, etc., can by much easier on desktop. Having an app "see that there is AR" and then follow that path of the code, even though there are limitations, is useful
• on mobile, with WebRTC available, doing simple magic window AR (especially for geospatial AR or rotation-only situations) is something people ask for a lot
• simple marker-based AR (e.g., ARToolkit, OpenCV Aruco markers, and probably more) is possible using video from WebRTC streams. Again, as a fallback for developers who want to hit more users in the next while, providing the basic setup and display of video, exposing the device orientation via the Cardboard code, and then letting devs do the trivial work of grabbing video frames and tracking with it, may be useful in the short term

There are significant limitations, most obviously that the camera intrinsics aren't known, so the field of view of the video isn't known. But given the ability to build the polyfill with different features turned on / off (right now, WebVR and Cardboard), it seems like a potentially useful addition.

On iOS, users need to make sure their device orientation is not locked in portrait mode before they're able to put their device in a headset. This may not be obvious to new users - could we add something that says "Make sure your device orientation is not locked" or something similar on the screen that pops up and shows them how to insert their phone into the headset?

I'm trying to create an AR test from the latest chrome stable (76.0.3809.111) browser on my pixel 2, but after enabling all the WebXR flags, I get an exception while calling the navigator.xr.requestDevice() with TypeError: navigator.xr.requestDevice is not a function

I get the above error with and without using the webxr-polyfill.js within my loading page. Any idea if or how and get around this? Or is it just implemented for supporting VR for now?

The above works fine on the Chrome Dev (71.0.3574.0).

immersive-web/webxr#357 (not yet merged)

@jsantell: While it's not 100% complete the code in the webxr-2019 branch is relatively functional and can run a majority of the code written against the current WebXR spec, the VR portion of which we no longer expect to break backwards compatibility with.

Given that the master branch now reflects a version of the spec that will never be shipped, it's currently pretty awkward to tell people how to get to the polyfill for testing. As such I think it would be appropriate to merge the webxr-2019 branch into master relatively soon, and continue development there. What, if anything, would you like to see happen prior to that merge?

It seems that session type "immersive" instead of "exclusive" is not supported yet in oculus browser (Oculus Go). Is it possible to support both terms until the transition is done in all browsers?

This works:

navigator.xr.requestDevice().then((device) => {
            device.supportsSession({exclusive: true}).then(() => {
                console.log("Supported");
            });
        });

For Mixed Reality through SteamVR, the primary input button for the hand controller is the same button that triggers the SteamVR UI, so you can't really interact with the webxr scene.

Is there a way to change the primaryButtonIndex for an input source (or any other workaround)?

Hi,

I'm trying to get this polyfill work with my OnePlus 3 (Android 8.0) device. Unfortunately, I get the No DPDB device match warning, and it doesn't work. If I switch the WebXR flag to enabled in chrome, then my app works, so I guess it should work with the flag disabled with this polyfill. So, what should I do?

Thanks

Added session to XRPresentationFrame
immersive-web/webxr@add98af

s/eyeLevel/eye-level
s/headModel/head-model

via #65

Currently, a valid "XRDevice" must be generated (out of the box, either backed via WebVR or Cardboard). There are reasons to only support polyfilling inline without immersive-vr (perf overhead of cardboard polyfilling, 270kb for cardboard (#56)), and according to spec should always be supported, which currently can fail for some configurations of cardboard and allowCardboardOnDesktop.

It might make sense to add a new device in src/devices/*, e.g. InlineWebXRDevice which handles this barebones case. Additionally, it seems that this would eliminate the need for the allowCardboardOnDesktop configuration.

The 4 pages in the examples folder which are used as E2E tests seem to have lots of overlap. Consolidating them will make maintenance easier as we work to bring the polyfill up to the latest WebXR spec.

magic-window.html is basically xr-presentation.html with support for showing the initial scene on in an inline session (on the desktop screen, for example). These pages could be merged.

I'm not sure if the mirroring.html page is relevant anymore because mirroring support has been removed from WebXR (but is still possible with WebGL calls).

I'm also not sure what offscreen-canvas.html is supposed to do, but it looks very similar to the other example pages.

While we reject if an exclusive session already exists, we should reject if a previous exclusive session is still pending immersive-web/webxr#360

@jsantell @Squareys
The recent addition of WebGL2RenderingContext to the required partials on the WebXRPolyfill global value breaks our usage of the polyfill in the WebXR Viewer on iOS, since the WKWebView doesn't offer that class.

One fix would be to remove WebGL2RenderingContext as a requirement and then testing for its existence in the code that uses it but I wanted to check with you two before submitting a PR to make sure that this won't break anything else.

I have Mozilla's WebXR Viewer iOS app using the webxr-polyfill as the basis for WebXR support. It creates a new PolyfilledXRDevice that uses ARKit under the covers to provide tracking. Now I'm looking at adding hit-testing and eventually anchors.

So, I have a couple of questions about how best to work with the polyfill.

Approach 1 would be to to make a second polyfill just for hit-testing and anchors that is loaded after the webxr-polyfill and mucks around with the private data structures and methods of classes like XRSession and PolyfilledXRDevice.

Approach 2 would be to fork the webxr-polyfill and modify the base classes like XRSession to add the proposed APIs and to modify PolyfilledXRDevice to provide hit-testing and anchor methods.

I'm leaning toward approach 2 just because there's less monkey patching and if the proposed APIs are accepted then I can relatively easily submit a PR to the webxr-polyfill with the platform-independent changes.

@jsantell Is this something you're already working on or have a different suggested approach?

Complete error:
WEBVR22602: VRDisplay.requestPresent is rejected - presenting HTMLCanvasElement's context is not compatible with the VRDisplay. Event.preventDefault() must be called during the firing of webglcontextlost event on the presenting HTMLCanvasElement.

Edge 42.17134.1.0 Microsoft EdgeHTML 17.17134
Windows 10 1803 Build 17134.407

polyfill samples don't work (webVR works), we had it only in that pc (others work fine), any hints of what's causing the issue?

immersive-web/webxr#325

• XRPose/XRViewerPose need to be implemented, replacing XRDevicePose.
• frame.getViewerPose(), frame.getPose() need to be implemented on XRFrame
• pose.emulatedPosition
• XRRigidTransform

See #51 for more information on this effort, and development goals.

Should match native implementation, need to look into this more.

Currently, the old XRCoordinateSystem is implemented, as well as supporting bounded areas at least.

The following classes are new in WebXR 2019:

• XRSpace
• XRReferenceSpace
• XRBoundedReferenceSpace

As well as some API renaming (session.requestReferenceSpace).

AFAIK, there are no current coordinate system events, but supporting XRReferenceSpaceEvent is also needed

See #51 for more information on this effort, and development goals.

There's a need when extending the polyfill to customize the request call requestSession() with custom values (e.g. WebXR Viewer). Currently, it requires clobbering the function and manually polyfilling. Is there a nicer way of handling this?

Additionally, extending the actual XRSession object with or without these changes.

Most likely dependent on #55

Currently investigating too, but wanted to share in case any of you has any hints. The polyfill stopped working after (what I think it was) an Oculus Browser update.

User Agent is:
Mozilla/5.0 (Linux; Android 7.1.2; Pacific Build/N2G48H) AppleWebKit/537.36 (KHTML, like Gecko) OculusBrowser/5.0.8.123398067 SamsungBrowser/4.0 Chrome/66.0.3359.203 Mobile VR Safari/537.36

The old WebVR examples work, examples in three.js site work. But none of the webxr-samples work anymore. It gets stuck on the white spinning circle.

requestDevice works, device.supportsSession({immersive:true}) also works. It seems to be related to display.requestAnimationFrame, as I can see this warning after cancelling with the back button in the controller.
[Violation] 'requestAnimationFrame' handler took 2814ms

Seems to be stuck there.

Figure out a CDN solution so developers can target the latest polyfill version, keep up with browser latest changes, and take advantage of #61 when ready. Referencing the npm package would suffice, and should update documentation to guide devs to use the CDN.

This may need to be broken up into smaller issues.

Currently needing implementation AFAIK:

• session.renderState
• session.updateRenderState
• layer.getNativeFramebufferScaleFactor
• layer.ignoreDepthValues
• getViewport() moved from XRLayer to XRWebGLLayer
• gl.makeXRCompatible() rename
• Any changes to XRPresentationContext, in light of the potential inline changes? immersive-web/webxr#636

See #51 for more information on this effort, and development goals.

Hi,

I created a quick mockup about the issue in the title, you can find it here: https://vrtual.house/development/webxr-polyfill-test/

The problem:
I cannot make any interaction on the scene, when the WebXR flag is disabled, and the polyfill is used. In three.js, there are some events for interaction, like selectstart, and selectend, but they're not dispatched when the polyfill is used, only when the flag is set to enabled.

I guess it has something to do with the fact that the inputsourceschange isn't fired either when the polyfill is used. (You can find the lines in question starting from line 21752 in the three.js r96 build):

immersive-web/webxr#342

With the modularization work happening in WebXR, the overhead of some optional dependencies and the overhead for the polyfill in general, it'd be great if a small script could include only the needed/requested modules.

This would save the overhead of shipping polyfills to browsers that do not need them, as well as provide a mechanism for optionally including (#56) other parts.

With the exception of Samsung Internet for GearVR, all browsers supporting WebVR/WebXR support modules; putting a collection of modules on a CDN with a lightweight entry may be a good solution for distributing the polyfill as a collection of files. Note that modules are inherently async, and the polyfill currently requires to be synchronously included, such that navigator.xr exists by the time user script runs. We could require await polyfill.ready() before interacting potentially; I know that this caused issues in the past with THREE.VRControls for example that created globals at loadtime (VRFrameData) which could twist developer expectations a bit.

Potential modules:

• cardboard-vr-display (#56) - provide immersive VR on unsupported platforms
• AR features
• Mock Session - generates mock data for an XRSession for development/testing purposes

We had reports of these 2 antivirus:
AVG (Chrome Windows 10)
Avast (Chrome Mac OSX)

Blocking the polyfill from running, flagging it for being infected with JS:Downloader-EAX Trojan.

Doing some debug, we found out that the warning comes up while running Dpdb.prototype.calcDeviceParams_

For example, setting the near plane does not pass in the session ID to the PolyfilledXRDevice, like setting the baseLayer for example. Probably every method in PolyfilledXRDevice should accept a session ID, even though setting different near planes for two concurrent sessions seems uncommon.

A handful of updates to the initialization routine:

• Remove XRDevice
• Move supportsSession/requestSession to XR
• Update supportsSession/requestSession to handle "mode" APIs
• Support only "inline" and "immersive-vr" for now; plumbing already exists for the WebVR polyfilled device (leveraging "exclusive" and "outputContext"'d sessions)

See #51 for more information on this effort, and development goals.

There should be a way to optionally build with cardboard-vr-display: this dependency is a majority of the polyfill's size (~250kb?) and unnecessary in the AR-only extension cases (WebXR Viewer).

webxr-2019 branch has been merged to master #75. But Readme looks still old, for example navigator.xr doesn't have requestDevice() method now. I'm happy if readme is updated.