Xbox cloud gaming gets mouse and keyboard support and latency improvements

Microsoft is preparing to add mouse and keyboard support for its Xbox Cloud Gaming (xCloud) service, which streams Xbox games to TVs, PCs, mobile devices, and more. The software giant teased the expansion earlier this year and is now encouraging game developers to prepare for mouse and keyboard support and some big latency improvements for Xbox Cloud Gaming soon.

“Xbox has had keyboard and mouse support for a number of years, and we’re working on adding it to streaming for PC users,” explains Morgan Brown, a software engineer on Microsoft’s Xbox game streaming team. “But you can start adding it to your game right away, and your console keyboard and mouse users will appreciate it.” It lights up in streaming once we add it.”

Microsoft flight simulator Boss Joerg Neumann previously teased that the addition of mouse and keyboard support for Xbox Cloud Gaming could arrive this summer. As Microsoft encourages developers to think more about mouse and keyboard support for Xbox games streaming to PC, it’s likely we’ll see this ad soon.

It allows Xbox Cloud Gaming users to stream Xbox games, not PC games, using mouse and keyboard. We could see games like Sea of ​​Thieves, Minecraft, Halo infinityand even Fourteen days All support mouse and keyboard via Xbox Cloud Gaming. However, the list of Xbox games that support mouse and keyboard is still relatively small. It will be especially useful when Microsoft expands the Xbox Cloud Gaming library later this year.

Image: Microsoft

In addition to mouse and keyboard support, Microsoft is also giving developers more ways to improve streaming latency in their games. Microsoft has been working on a new Display Details API that can save up to 72 ms of latency overall. This is achieved through the use of Direct Capture, which reproduces hardware functions in software to eliminate VSync latency and double or triple buffering and even the scaling required for TVs.

Scaling and artifacts add additional latency to game streaming, and many games already support Direct Capture to improve their performance on Xbox Cloud Gaming. Latency can drop as low as 2-12ms compared to 8-74ms for the traditional display pipeline. However, there are some limitations. Direct Capture only supports a maximum resolution of 1440p and does not yet support dynamic resolution or HDR.

The resolution limitation will not be an issue for most game developers right now, with Xbox Cloud Gaming downscaling games to 720p on mobile and 1080p on PC and web. Microsoft eventually expects to support higher resolutions, but there’s no timeline for 1440p or 4K support for the new Xbox TV app. “This is something we expect to change over time based on different devices, network conditions, and streaming stack improvements,” explains Brown. Tools will soon be available to developers to test their games and find out how they can support direct capture.

Image: Microsoft

The latency improvements are key for game streaming services like Xbox Cloud Gaming, and as Direct Capture shows, it’s not just about reducing network latency. Nvidia introduced its RTX 3080 GeForce Now tier last year with impressive latency improvements. Nvidia has developed its own Adaptive Sync technology that adjusts game rendering to a synchronous monitor and allows GeForce Now to sync streamed games to any 60Hz or 120Hz monitor.

Nvidia’s Adaptive Sync also reduces some buffering between the CPU and GPU on the server side, and the end result is some impressive latency improvements over what’s available from Google Stadia or Xbox Cloud Gaming. Nvidia even claims to beat an Xbox Series X running at 60fps locally thanks to 120fps GeForce Now support.