Rcam2 is my second attempt at a real-time volumetric AR VFX system with Unity (the first attempt is Rcam). This time I used iPad Pro with a LiDAR scanner.

The Rcam2 system consists of two software components: RcamController and RcamVisualizer. RcamController runs on an iPad device and sends a video stream and metadata (camera position, control data, etc.) to RcamVisualizer, which runs on a desktop computer and renders VFX. It uses NDI to communicate between these two components, so it doesn't require any special hardware but only a network connection

I used Rcam2 in a Boiler Room stream on 24th September 2020. You can watch the recorded video on Vimeo.

• Unity 2020.1.6
• iPad Pro with LiDAR scanner (2020 model)
• iOS 14 and Xcode 12
• NDI SDK 4.5

• Unity 2020.1.6
• HDRP-compatible desktop system

1. Build and run the RcamController project on the iPad device.
2. Play the RcamVisualizer project on the desktop system.
3. Select a controller from the dropdown list. You can hide the UI by clicking an empty area of the screen.

• Although Rcam2 works with a wireless connection, it's recommended to use a wired ethernet connection for better latency and framerate. NDI works in a zero-configuration fashion, so you can directly connect a device and a computer without a router/switch.

• Holding an iPad device for a long time could be painful in a physical sense. You can relieve it by using a camera handle.

• Under a normal configuration with a wired connection, RcamController runs up to about two hours with a full charge.

• You can blur boundaries between rendered objects and real objects using the color grading and the depth-of-field effects. It's recommended to enable the "Film" and "DoF" toggles whenever these effects are acceptable.

Try turning off the Windows firewall. It solves the problem in most cases.

If you're using a virtual network device, try turning it off. Note that WSL2 implicitly creates a virtual network device (vEthernet). You have to turn it off to establish a connection correctly.

No. Rcam2 requires a LiDAR scanner. iPad Pro 2020 is the only device that supports LiDAR at the moment.

It's technically possible but requires lots of changes. You might have to remove some HD features (like the DoF effect) and reduce some performance-related numbers like particle capacity counts in VFX. You also have to tackle several device issues like thermal throttling, battery life, etc.

Don't expect that I do anything in this direction. The controller-visualizer design of Rcam2 is the most convenient way to avoid these problems.