visionOS 26.4 Released With VR Foveated Streaming & Improved Spatial Audio

Apple released visionOS 26.4 this week, bringing VR foveated streaming and a key improvement to spatial audio in mixed reality.

Foveated Streaming

The key new feature, foveated streaming, was announced last month when the first visionOS 26.4 beta released for Vision Pro developers.

Before you continue reading, note that foveated streaming is not the same as foveated rendering, though the two techniques can be used alongside each other. As the names suggest, while foveated rendering involves the host device actually rendering the area of each frame you’re currently looking at with higher resolution, foveated streaming refers to sending that area to the headset with higher image quality than the rest of the frame.

It’s a term you may have heard in the context of Valve’s Steam Frame, where it’s a fundamental always-on feature of its PC VR streaming offering, delivered via the USB PC wireless adapter by default.

Given that the video decoders in headsets have a limited maximum resolution and bitrate, foveated streaming helps prioritize resolution and compression quality where you’re currently looking.

Valve’s depiction of foveated streaming.

Unlike the macOS Spatial Rendering introduced in the main visionOS 26 release last year, which is a relatively high-level system that only supports a local Mac as a host, Foveated Streaming is a low-level host-agnostic framework, that any PC VR server could in theory implement. For developers who want a ready-to-go implementation, Apple worked closely with Nvidia to add support for its CloudXR SDK. And despite the name, CloudXR can be used for both local and cloud PCs.

Normally, for native visionOS apps, the operating system does not provide developers with any information about where the user is looking – Apple says this is in order to preserve privacy. Instead, developers only receive events, such as which element the user was looking at as they performed the pinch gesture. But crucial to foveated streaming working, the API tells the developer the “rough” region of the frame the user is looking at. It’s the first time visionOS developers get this kind of information.

This should allow the host to render at higher resolution in this region too, not just stream it in higher resolution. As always, this will require the specific VR game to support foveated rendering, or to support tools that inject foveated rendering.

Over the past few weeks, multiple companies have announced supporting Vision Pro’s foveated streaming via Nvidia’s CloudXR SDK:

X-Plane 12 and iRacing are releasing dedicated visionOS clients that automatically connect to the sim on your PC, automatically align your cockpit to your physical peripherals through computer vision, and segment out those peripherals with passthrough so you can see them in VR.Autodesk VRED has already added an immersive mode that pixel streams high fidelity assets to Vision Pro, and it’s already being used by Kia, BMW, Volvo, and Rivian.A lone developer released Clear XR on TestFlight, which lets you stream any OpenXR VR game from your PC with foveated streaming.

One notable limitation is that Nvidia’s CloudXR SDK only supports Nvidia’s Ada and Blackwell GPU architectures, meaning RTX 40-series and 50-series graphics cards.

Improved Spatial Audio In Mixed Reality

Since launch, Apple Vision Pro has supported not just basic positional spatial audio, but audio that scans the features and materials of your space to precisely match sound to your physical environment. Apple calls this Audio Ray Tracing.

With visionOS 26.4, Apple says Vision Pro now stores and remembers the acoustic properties of rooms you use the headset in, such that the Audio Ray Tracing feature can initialize quicker and have a more holistic model of your environment without you needing to move around first.