Google Octane Web Browser Benchmark Retired

Google has retired and is no longer maintaining the Octane benchmarkOctane benchmark that measures a JavaScripts engine’s performance by running a suite of tests representative of certain use cases in JavaScript applications. Google first came out with Octane in 2012 and it became one of the most recommended benchmarks to run by the browser community.

Google noted that by 2015 most JavaScript implementations had done compiler optimizations to get high scores in Octane and that the benchmark scores seen in Octane didn’t really translate that well into the performance one would get on real web pages. In fact, some of the optimizations made for higher Octane scores actually hurt performance on real-world websites. So, Google Octane is being put out to pasture just like SunSpider was years ago because the developers felt the industry over-optimized for them and the results no longer lined up with the performance one would get when surfing the web. Google did not say that Octane was being removed from their site, but if you are worried about that happening you should fork it on GitHubfork it on GitHub and use a local copy to run it.

Legit Reviews uses Octane in our CPU benchmarks, so we will be finding a replacement for this web browser performance test. Michael Hablich over at the V8 JavaScript Engine team has a great post up about real world performance and noted that he likes a test called Speedometer, a WebKit benchmark that includes applications written in React, Angular, Ember, and other frameworks. He found that performance test had more appropriate runtime call stats than Octane.

His testing (shown above) demonstrated that the speedometer test suite had a very similar runtime profile to the 25 sites. Speedometer doesn’t exactly match real web pages, but it clearly does a better job than Octane. It looks like this is the benchmark test that we will be moving over to!