immersive-web / privacy-and-security Goto Github PK

Home space that educates end users on security elements of what XR can entail for their privacy and security, things like showing what SLAM data can see, how tracking data can be used, might be useful for a browser feature that could allow for teaching users about security risks, how the tech works, and what to be aware of.

One way to reduce the number of annoying consents and notices would be to deliver a general statement: "Watch out. This technology is new. It may handle privacy and security in ways that you do not expect. You should educate yourself about how this technology works and its strengths and weaknesses."

Malicious Entity Injection, or MEI (writeup at mei.ninja), is an attack I've researched and found exploitable in certain situations for Unity3D apps. Prevention mechanisms should be put in place to prevent malicious loading of entities from third party sources such as ad networks and content integrity/signature checks on entities loaded.

Without MEI defenses, an attacker potentially has the ability to inject entities that can be used to manipulate or harm a user. Currently, HTTPS and TLS are adequate for defending against MEI in Unity3D form, however with mixed content loading (IE ad networks), or broken TLS implementations, it's possible to compromise what's loaded. Something similar to CSP needs to be developed to defend against MEI in the JS realm as well.

Use Case: Users with limited dexterity can use a headset, but not buttons, either on the headset or on controllers. (For example, users with Parkinson's, or who do not currently have use of their hands.)

UI both outside and inside VR can be implemented with fuse buttons (where the user places the pointer on a button for some predetermined length of time). This can be implemented with JavaScript, wherein the code creates a click event at the appropriate time. Unfortunately, this runs afoul of the security requirement that requestPresent() be called in a user-interaction handler, like a click handler.

For the immersive web to be accessible to all, there needs to be an approach to allow users who can only control their head to enter VR mode, and other transitions that require user assent, like starting audio play.

A narrow solution would be a browser flag to disable this security requirement.

A broader solution would a mechanism to extend the browser. Such a mechanism may already exist for the flat web.

From @ddorwin on June 27, 2017 19:27

In general, WebVR presentation (exclusive session) requires a user gesture. However, magic window (non-exclusive) sessions do not currently require a user gesture. Providing pose data without requiring a user gesture or clear indication that such data is being provided presents privacy concerns, especially for mobile clients where the users is always holding the device.

While external desktop HMDs, may appear to pose less concern, there are potential issues for external desktop HMDs as well, including:

• Browsing in VR - VR user agents need to apply similar consideration if they choose to expose magic window in the browser in the HMD.
• There may be privacy concerns about access to high-frequency pose data even if the user is not wearing or touching the headset. See, for example, https://crbug.com/421691.

In addition, some future use cases/capabilities, such as Tango-style 6-DoF or "punchthrough" magic window in a VR browser, may enable the application to derive a lot more information, including data that might enable page-wide gaze tracking.

Since requiring a gesture for magic window would break a number of use cases, we need to consider other mitigations, require some, and allow user agents flexibility to implement others.

Examples include:

• Follow the Generic Sensor API’s security and privacy considerations (#249)
• Only expose WebVR on secure origins (#249)
• Only provide poses to the focused frame. (Likely implied by #249.)
• Only provide poses for frames that are same origin to the top level document.
• Throttle frequency and/or provide reduced precision (e.g., a low-pass filter).
  • This could be applied to:
    • All tracking-only/non-presenting/non-gesture instances
    • OR just instances in unfocused or non-same-origin frames frames, though this could create unexpected behavior for users.
• Use permissions or other mechanisms to allow users to opt-out of tracking for individual sites or entirely.
  • Integration with the Permissions API should enable this option for user agents.
• Use indicators or other UI to inform users that device tracking is in use.
• Attempt to ensure that the use of pose data is legitimate.
  • For example, by ensuring it results in changes that are clearly visible to the user.
  • It may be difficult to ensure this since a malicious app could try to make subtle visual changes, but some ideas include:
    • Requiring that the UA only process frames for visible and non-obscured of a specific size or percentage of the window.
    • Requiring that the frame changes with pose changes.
    • Throttling frequency and/or precision as appropriate.
  • Note: Such mitigations may require an explicit link between VRSession.requestFrame() and an output area, such as is proposed for magic window in #237.

We may also want to consider allowing the application to request specific ranges of accuracy. This would allow applications to ensure consistent resolution/frequency for all frames and for the user agent to make more intelligent decisions about whether to require permission, display indicators, etc. Similarly, it might make sense to require the page to request, though not necessarily be given, capabilities such as 6-DoF and "punchthrough."

Copied from original issue: immersive-web/webxr#250

From @ddorwin on June 27, 2017 19:21

WebVR exposes sensor data to the Open Web Platform. The Generic Sensor API specification “defines a framework for exposing sensor data to the Open Web Platform in a consistent way.” [1] WebVR should follow this framework, especially the security and privacy considerations ([2]). The Security check algorithm ([3]) and other parts may also be useful.

The DeviceOrientation API is the most similar to WebVR, which essentially exposes a higher-resolution/frequency version of the same data. This spec defers entirely to [2]. [4]

One might argue that some items may not apply when a user gesture is required, but a gesture is not required for magic window. Thus, such differentiation would place more requirements on non-presenting WebVR than the more powerful presenting mode.

Note that integration with the Permissions API, which is recommended by [2], is also preferred for features that use Feature Policy (#86).

In particular, note that [2] requires that “all interfaces defined by this specification or extension specifications must only be available within a secure context.” Note also that Chrome intends to deprecate DeviceOrientation on insecure origins [5] (this is an older API that predates the guidance in the current spec).

In the context of these similar specs, it is hard to argue that WebVR, especially when not presenting, should be available on insecure contexts. Furthermore, if non-presenting WebVR is not available on insecure contexts, it seems weird to allow the more powerful presenting mode in such contexts.

[1] https://w3c.github.io/sensors/
[2] https://w3c.github.io/sensors/#security-and-privacy
[3] https://w3c.github.io/sensors/#security-check
[4] https://w3c.github.io/orientation-sensor/#security-and-privacy
[5] https://crbug.com/481604

Copied from original issue: immersive-web/webxr#249

Over in https://github.com/immersive-web/real-world-geometry I posted an issue discussing when and how to ask for permissions for access to real-world-geometry (immersive-web/real-world-geometry#7). Posting this here to track it on this site too.

Issue contained this text

Access to real-world-geometry brings up the questions of permissions and levels of permissions and how they might work in practice.

The implementation I am working toward has 3 levels of permissions, with the UA also supporting a "reduced information" (lite) mode.

An app can request one of these progressive levels (each includes the previous):

• minimal, basic WebXR: spatial tracking, hitTesting, creating anchors from absolute matrices or hitTest results
• world sensing: real-world-geometry, illumination, detection and tracking of well-defined things the platform supports (images, objects, QRCodes, etc), world relocalization maps, etc
• camera / sensor data: originally this was "camera" access, but it includes, I think, access to any of the low level sensors a device might have.

There are questions, obviously. How can we standardize on some of the sensing bits. Where do things like eye trackers and other invasive sensors (like "affect" sensors, or facial tracking of the user in an HMD for creating their avatar elsewhere) fit?

For both the "minimal" and "world sensing" levels, there is a "limited" option, which (for our iOS app) has the user select a plane; that is the only world structure that gets exposed, or hit test against.

Finally, we are displaying the current state in the URL bar (when it's exposed), akin to the HTTPS icon or camera or microphone icon on a browser. Clicking on it brings up a detailed permissions sheet, where specific information can be turned on/off, including the most basic access. The web page is NOT notified of these changes; the data just stops flowing. The user can change this at any time.

This relates to the real-world-geometry in two ways:

• I am adding an option to request session, called "worldSensing". If true, when the user is asked for permission to use XR, it also includes permission to use "worldSensing". The advantage of doing this is that the user only gets one permission request, instead of getting a WebXR permission and then a worldSensing permission. We have not decided yet what to do about failure: does the user have the option to say "yes" to "webxr" but "no" to world sensing? Or does the webpage need to deal with requestSession failing, and then request a session with less access (or display and error?)
• if the app requests anything requiring worldSensing later, and didn't put worldSensing in the original request, we could trigger a new permission. Apps might do this if they want to wait until the user does something that requires world sensing, if they don't require it all the time. It would be nice if there was a way for the app to provide a string or description of why they want this. if not, pages can display info before they request (since they know it's going to trigger one).

Thoughts?

An explainer should outline user privacy and security concerns (particularly threat vectors) when sites have access to real-world geometry. An explainer should additionally explore approaches to mitigating those concerns.

One approach would be to create multiple explainers, each on a specific topic. This might make it easier for individual APIs to reference specific topics that are relevant.

Another approach would be to combine all the topics in this repo into a single explainer. That would make it easier for everyone to link to a comprehensive guide (instead of linking to multiple docs).

Any opinions or preferences?

For now, I'm adding issues for a few topics that I'd like to see covered; my assumption is that we can migrate the contents of those discussion threads into whatever explainer format makes sense.

From @ddorwin on January 10, 2017 19:29

https://w3c.github.io/webvr/#security currently says:

Non-focused tabs are allowed to enumerate Gamepads and VRDisplays but will see last received state or default values.

I think it's worth calling out why this is the case. For example, without this limitation, a background tab on a magic window device could track the user's movement for days without the user's knowledge.

In addition:

• "Tabs" should probably be "documents" or similar.
• This should be stated more directly as a user agent requirement. See also #172.

For discussion, I propose something like:

User agents MUST allow non-focused documents to enumerate Gamepads and VRDisplays but MUST NOT provide position, pose, or other state to non-focused documents.

NOTE: Providing data to non-focused documents could...

This could probably be written pore positively, such as stating that the user agent MUST allow all documents to enumerate and only allow the focused document to get state. However, it is also important to call clarify that non-focused documents must not be able to get such state and why.

Copied from original issue: immersive-web/webxr#173

So I don't know where to put this, but I think there's going to need to be a content loading API for browsers. With how the web currently works, we have content for one site loaded from all over the place. We're gonna need a CSP style loading API with a signing system that allows fine grained controls of what loads from where with content validation. If not, a malicious attacker could pop a content repo and swap content that could cause harm to users.

This is probably going to require a new API, but frankly, it relates to immersive as this is when it's going to come in to the "oh god why is there a car coming at me, I better swerve" crash "wait, there was a car, what the fuck?" territory.

An explainer should outline user privacy and security concerns (particularly threat vectors) when sites have the ability to detect real-world objects, including planar image detection. An explainer should additionally explore approaches to mitigating those concerns.

An explainer should outline the concerns of an AR session soliciting permissions and gaining access to system resources (e.g. microphone, camera) for longer than the duration of the AR session, that the site might use later for other, non-AR purposes.

As per the thread here:
https://twitter.com/NellWaliczek/status/1090344367053664256
and my follow up here:
https://twitter.com/aoighost/status/1090344892172304384

The idea that SLAM would be handled by Javascript is scary as hell. A malicious site could use a modified SLAM library to stream data of a user's surroundings, or worse, use it to attack the user themselves in a means to cause physical harm to the user. This should be handled by making it codified in spec that SLAM is not handled by a server side library, but rather by the browser.

It may be possible in the real world to design visual inputs that appear as certain objects to computer vision systems, but which don't look like that to users.

For example, if someone found a visual pattern that an AR system would identify as a particular object, they could place that pattern in the real world.

This has potential threat vectors in situations where the visual pattern is not identifiable to users, and may send false signals to an AR system. This in turn may provide bad information to users about the presence of such objects.

For example, if an AR system warned a user to STOP when a stop sign was detected, then a maliciously-placed false stop sign could trigger that warning; this would be particularly threatening if the user had their vision limited in some way and couldn't disambiguate for themselves whether the stop sign actually existed.

At the Permissions Workshop, @NellWaliczek, @avadacatavra, and I talked and it was suggested that I file an issue on bystander privacy and WebXR. By "bystander privacy", I mean the notion of "how does a user who is in the same room as someone using an XR device know whether an image of them is being captured by the person using the XR device", and, by extension, if the XR device is also profiling them.

This is somewhat related to the Real World Geometry Threat Vectors User PII threat of "Users may be identified by learning the geometry of their face, or through gait analysis" but to others in a room where an XR experience is taking place.

From @ddorwin on November 9, 2017 2:10

On desktop, WebVR presentation results in two separate and isolated displays for a single app. Since the same app is driving both displays, any API call (other than those on VRDevice and related objects) could be related to activity on either or both displays (or from either form of input). This is fine except for UI that is initiated but not controlled by the app. Examples include permission prompts and UI for the Payment Request API.

In order to enable developers to create experiences that are predictable across implementations, we should probably establish expectations for where such UI is displayed. Although UI is often out of scope for web specs, this seems like a necessary exception.

As for what that expectation should be, it would seem that such UI should at least be displayed to the presenting VR display. Perhaps implementations should be allowed to optionally display to the traditional display too. This may be useful for multi-user or "demo" scenarios.

We may also want to provide guidance for implementations that cannot display such UI to the presenting VR display. For example, to reject with or throw INVALID_STATE_ERR.

On a related note, we may also want to recommend in the security/privacy considerations section that indicators, such as for microphone use and other privacy-sensitive features, be displayed in both places.

Copied from original issue: immersive-web/webxr#299

An explainer should outline user privacy and security concerns (particularly threat vectors) when sites offer immersive web experiences that expose camera data, even if the site does not have access to that camera data. An explainer should additionally explore approaches to mitigating those concerns.

It's recommended that IPD be rounded slightly when configured by the user to avoid unique values that could (on their own) be fingerprinted; however it may be possible to combine an IPD value with another signal (e.g. IP address?) to fingerprint a user. The same may be true for bounds geometry and floor height (when set by the user in an emulated mode to a non-default value). Is it possible to address this?