Mark Zuckerberg has so many VR headset prototypes to show us

Meta’s Reality Labs division has unveiled new prototypes in its roadmap to lightweight, hyper-realistic virtual reality graphics. The breakthroughs are far from consumer-ready, but codenamed Butterscotch, Starburst, Holocake 2, and Mirror Lake designs could result in a sleek, brightly-lit headset that supports finer detail than the current Quest 2 display.

Mark Zuckerberg, CEO of Meta, and Michael Abrash, Reality Labs chief scientist, along with other Reality Labs members, presented their work at a virtual roundtable last week. The event focused on designs that Meta calls “time machines”: bulky proofs of concept meant to test a specific feature, like a super-bright backlight or a super-high-resolution screen. “I think we’re in the middle of a big step toward realism right now,” Zuckerberg told reporters. “I don’t think it’ll be that long before we can create scenes with basically perfect fidelity.” Display technology isn’t the only unsolved piece of this puzzle, but it’s an area where Meta’s VR hardware is intense -Research gives him a head start.

Zuckerberg reiterated his plans to ship a high-end headset, codenamed Project Cambria, in 2022 after it was first announced last year. Cambria supports both full VR and mixed reality, thanks to high-resolution cameras that can relay a video feed to an internal screen. It also ships with eye-tracking, a key feature for future meta headsets. From there, Meta plans two lines of VR headsets, according to Zuckerberg: one that will remain cheap and consumer-focused, like today’s Quest 2, and one that will feature the company’s latest tech and aim at a “prosumer or professional” market. That comes with reports that the company is already planning updates to the Cambria and the Quest 2, although those prototypes weren’t discussed on the call.

Meta’s VR headsets sit alongside a separate line of augmented reality smart glasses designed to project images onto the real world rather than blocking it with a screen. Meta recently scaled back the launch of its first generation AR glasses, and in general VR screens have reached consumers much faster than AR holograms. But Meta’s prototypes show just how far the company thinks it can go.

Butterscotch is an attempt at a near-Retina quality headset display – something you can find in high-end headsets from companies like Varjo, but not in the current Meta range. The design is “far from deliverable” and required roughly halving the Meta Quest 2’s 110-degree field of view. But it offers about 2.5 times the Quest 2’s resolution at (sort of) 1832 x 1920 pixels per eye, allowing users to read the 20/20 line of sight on an eye chart. According to Zuckerberg, it offers about 55 pixels per field of view degree, slightly below Meta’s 60 pixels per degree Retina standard and slightly below Varjo’s 64 pixels per degree.

Starburst is even less available than Butterscotch, but is testing a similarly impressive upgrade. The bulky design uses a powerful lamp that needs handles to support its weight and produces HDR (high dynamic range) lighting with 20,000 nits of brightness. “This is totally impractical to consider as product direction for the first generation, but we’re using it as a test bed for further research and study,” says Zuckerberg. “The goal of all this work is to help us discover what technical avenues will allow us to make meaningful improvements so we can move closer to visual realism.”

Holocake 2 moves in the opposite direction, exploring Meta’s ability to make VR headsets thinner and lighter. It’s the successor to a 2020 design based on holographic optics, a light diffraction technique where a nearly flat panel stands for a thick refractive lens. The result could be as thin as a pair of sunglasses, but Meta is still working on developing a self-contained light source to power them – almost certainly a laser, not the OLEDs commonly used today. “We have to do a lot of engineering to achieve a consumer-grade laser that meets our specifications: that’s safe, inexpensive, efficient, and that a slim VR headset fits,” says Zuckerberg. “To be honest, the decision about a suitable laser source has not yet been made.”

The presentation also discussed Half Dome, a long-running series of prototypes that can shift planes of focus depending on where users are looking. These varifocal optics started as a clunky mechanical system in 2017 and later switched to a series of liquid crystal lenses. According to internal meta-research, they can create a more convincing (and physically comfortable) illusion of depth in VR.

Meta described Half Dome’s technology in 2020 as “almost ready for prime time,” but today, Zuckerberg was more measured. “This stuff is pretty far away,” he said in response to a question about the Prime Time comment. “We’re working on it, we really want to build it into one of the upcoming headsets, I’m confident we will eventually, but I won’t be announcing anything ahead of time today.”

Reality Labs will be discussing additional research at the SIGGRAPH show in August, including how to more accurately capture real-world footage for mixed reality.

The above designs exist as actual hardware that Zuckerberg briefly demonstrated during the event. But Meta also unveiled a prototype called Mirror Lake, which is essentially ambitious and was never built. The design is more akin to ski goggles than Meta’s bulky Quest hardware, and would include Holocake 2’s thin optics, Starburst’s HDR capabilities, and Butterscotch’s resolution. “It shows what a complete next-generation display system could look like,” said Abrash.

In addition to these features, Mirror Lake would include an outward-facing display that projects an image of the user’s eyes, reducing the sense of physical separation for those outside the headset. Meta showed off this somewhat sinister feature in a prototype last year, and it might not be the only company interested in the concept: Apple has reportedly been considering a similar feature for its rumored headset. The idea is tailored for a mixed reality world where Meta has put much of its future on the line – but today the company emphasizes the incremental steps along the way.