SteamOS gaming performs significantly worse than Windows, Ars analysis shows

Specs at a glance: The Ars Technica Dual-boot Test Rig
OS	Microsoft Windows 10 Pro (64-bit) and SteamOS 2.0 4.1.0-0 (on separate drives)
CPU	Intel Pentium G3220 (Haswell), dual-core, 3.0GHz
GPU	Zotac Geforce GTX660 (2GB) w/ GeForce Game Ready Driver v. 358.91
RAM	8GB DDR3-1600
Motherboard	MSI H81I (mini-ITX)
Storage	Western Digital WD Blue 7200 rpm 500GB HD x 2
Sound	Onboard
Network	Onboard (wired gigabit Ethernet)
PSU	Antec VP-450, 450W
Case	BitFenix Prodigy, arctic white

Since Valve started publicly talking about its own Linux-powered "Steam Boxes" about three years ago now, we've wondered what kind of effect a new gaming-focused OS would have on overall PC gaming performance. On the one hand, Valve said back in 2012 that it was able to get substantial performance increases on an OpenGL-powered Linux port of Left 4 Dead 2. On the other hand, developers I talked to about SteamOS development earlier this year told me that the state of Linux's drivers, OpenGL tools, and game engines often made it hard to get Windows-level performance on SteamOS, especially if a game was built with DirectX in mind in the first place.

With this week's official launch of Valve's Linux-based Steam Machine line (for non-pre-orders), we decided to see if the new OS could stand up to the established Windows standard when running games on the same hardware. Unfortunately for open source gaming supporters, it looks like SteamOS gaming comes with a significant performance hit on a number of benchmarks.

To start our tests, we dragged out the dual-boot SteamOS/Windows machine we first built nearly two years ago (when making our own dual-boot how-to guide) and got all the operating systems and drivers up to date. The hardware on that bare bones machine is a little out of date now, but since that hardware is remaining static for both sides of the test, it should suffice for giving an idea of the relative performance between the operating systems.

Once everything was set up, we put the machine through Geekbench 3 (which, unlike many other benchmark tools we considered, actually has a Linux version). The results show a definite edge for Windows 10, especially in terms of floating point operations. Still, SteamOS is within the same order of power magnitude in many other performance metrics.

Generalized CPU benchmarks are only somewhat useful in judging actual, GPU-powered gaming performance, though. To see how the OS change affected gaming, we looked for benchmarkable top-of-the-line titles we could put through the comparative wringer on both systems. The only problem is that the most graphically intensive recent releases on Windows 10 aren't available to test on SteamOS. We'd love to see how games like Fallout 4 or Call of Duty: Black Ops III run on SteamOS, but until and unless their developers come out with Linux ports, that's going to be impossible.

We finally settled on a couple of mid-to-late-2014 releases that had SteamOS ports suitable for our tests: Middle-Earth: Shadow of Mordor and Metro: Last Light Redux. Both are relatively graphically intensive 3D games with built-in benchmarking tools and a variety of quality sliders to play with (including six handy presets in Shadow of Mordor's case). For all the gaming benchmarks, we ran each test at least three times and took the median number to ensure the results were reliable.

No matter how you slice it, running these two high-end titles on SteamOS comes with a sizable frame rate hit; we got anywhere from 21- to 58-percent fewer frames per second, depending on the graphical settings. On our hardware running Shadow of Mordor at Ultra settings and HD resolution, the OS change alone was the difference between a playable 34.5 fps average on Windows and a stuttering 14.6 fps mess on SteamOS.

While these are two AAA games ported to Linux by respected publishers, it's possible the developers simply weren't able to extract the best performance from the less familiar OpenGL and Linux environment. We figured that Valve's own games wouldn't have this problem; if anyone could get the maximum performance out of its Linux ports, it should be the company behind SteamOS itself.

Unfortunately, Valve's own Source engine games showed the same performance hit when compared to their Windows versions. Portal, Team Fortress 2, and DOTA 2 all took massive frame rate dips on SteamOS compared to their Windows counterparts; only Left 4 Dead 2 showed comparable performance between the two operating systems (though there's no sign of those SteamOS frame rate improvements Valve cited years ago).

Since the Source engine games we tested were on the older side, the frame rate performance wasn't the difference between "playable" and "unplayable" in any case, even on maxed-out settings. For games like these, which don't push the upper limits of our hardware, most gamers wouldn't even notice the difference between the frame rates listed here. Still, it's not a good sign that Valve's own porting efforts generally couldn't get comparable, Windows-level performance out of a SteamOS version.

Testing six games on a single hardware set up is far from comprehensive, of course. Games built from the ground up with OpenGL and Linux in mind might be able to best their Windows counterparts; similar benchmarking by Phoronix showed Unbuntu 15.04 outclassing Windows 10 when running open source Quake clone OpenArena, for instance. Newer graphics hardware might be better suited to take advantage of high-end OpenGL features, though that new hardware is at least as likely to get more power out of Windows' prevalent DirectX standards as well. Upcoming games that support Microsoft's DirectX 12 and/or OpenGL's Vulkan standard could change the performance equation substantially, too.

All that said, right now, it seems that choosing SteamOS over a Windows box means sacrificing a significant amount of performance on many (if not most) graphically intensive 3D games. That's a pretty big cost to bear, considering that Alienware sells its Windows-powered, console-style Alpha boxes at prices that are only $50 more expensive than identically outfitted SteamOS machines. That's not to mention the fact that Steam on Windows currently has thousands of games that aren't on SteamOS—including most AAA recent releases—while SteamOS has no similar exclusives to recommend it over Windows.

Hopefully, Valve and other Linux developers can continue improving SteamOS performance to the point where high-end games can be expected to at least run comparably between Linux and Windows. Until then, though, it's hard to recommend a SteamOS box to anyone who wants to get the best graphical performance out of their PC hardware.