2011-06-06
Comparing Performance: Azure vs Cairo
You haven't posted in a while!
This is true! And I do apologize, I've been very busy and hope to post more again in the near future. I particularly apologize for comments which I might've missed, my blog started getting covered with spam comments and it became impossible to separate the good from the bad. I've mass deleted over 2000 spam comments and updated both the blog software and the antispam blacklist. Hopefully things will be better now.
What's Azure?
Azure is a new graphics component that we've been developing for use with Mozilla. Considering Joe Drew has already done an excellent blog post on the subject I won't be going into it in much detail. You can find his blog post here, if you haven't yet I recommend you read it before you continue reading this post! :)
So what's the current status?
Well, we set a goal for ourselves to implement the Azure API on Direct2D for Q2. So this is what I've been working on for the last couple of weeks, in addition to that we've created an implementation of Canvas2D that sits on top of Azure. The sum of that work resulted in a build of mozilla that can run a canvas implementation based on the new Azure API. This has allowed me to do performance and correctness comparisons between the current Canvas code based on the cairo Direct2D backend, and the new code based on Azure. The latest builds feature an implementation of canvas on the Azure D2D backend that is almost as good in terms of correctness as the cairo version. In some cases even better!
Almost? What's broken?
So there's a couple of issues that still need to be resolved, the main correctness problems are:
- No performance optimizations have been done, there's some pitfalls right now
- The Radial Gradient implementation does not follow the spec when the inner circle lies (partially) outside of the outer circle
- Global composite operations which are unbound by their source do not behave properly in the presence of a clip
- isPointInPath returns false for points which are exactly on the edge of a path
- Behavior of zero size canvases is not exactly as it should be
So, what about performance?
In general, you should see much improved performance for Canvas2D in a Direct2D environment. Almost any test will perform either at the same performance level, or better. Some tests will benefit greatly, while others will basically stay the same, this mainly depends on where the bottlenecks for the tests are. There are however still some performance pitfalls, where Azure may be slightly slower than Cairo. Most of these cases should be resolved in the coming time. At the moment also the initialization of Azure canvases is slightly more expensive than normal, this should easily be offset by improved performance though!
Some numbers
I ran several benchmarks comparing Azure canvas performance to the Cairo Direct2D backend canvas performance. Here's a nice chart:
In addition to these tests there were some tests which weren't easy to include in the chart since they didn't report frames per second. Two notable ones are the IE Testdrive 'Speed Reading' test, which ran in 6 seconds both with Azure and Cairo, however reported an average drawing time of 5 ms for Azure, and 8 ms for Cairo. Possibly the total time ended up still being the same due to the nature of timeouts in Firefox. A more outspoken difference was the IE TestDrive Paintball demo, which ran in 10.91 seconds on Azure versus almost 30 seconds on Cairo!
All in all we're very happy with these results and hopefully future optimizations will improve them further.
I want to see this for myself!
Well, you can! I've made a build with Azure available here. Using this build Azure will be enabled by default -if- you have Direct2D support. If your system does not support Direct2D (you can check in the 'about:support' page), this will function practically as a normal nightly build. You can switch azure on and off through the 'gfx.canvas.azure.enabled' preference on the 'about:config' page.
Some info for people who want to do their own testing:
- My testing was done on a dual Xeon E4650 with 24 GB of RAM and a Radeon 6970 GPU
- Performance of this build cannot be compared to a normal firefox build, as Profile Guided Optimization was not performed on this build
What's next?
Well, first we have to iron out the last quirks and get AzureD2D canvases shipped. Once that's done it will mark a major milestone in the Azure project. Soon to follow should be an Azure implementation in Quartz, bringing performance improvements to our Mac audience. The next, possibly most interesting step for Azure will be a larger project: creating a backend that can use the GPU for rendering vector graphics through OpenGL and different Direct3D versions. This will hopefully allow us to bring fast, consistent rendering performance to our users on all platforms!
Eventually all browser rendering, and not just Canvas, should run through Azure, further improving your browsing experience in Firefox!
35 comments
But why was cairo slow? Clearly something pathological is going on in the FishBowl app. What is it?
@Colin:
Indeed. FishBowl is using custom composite operators in a certain manner. These operators behave different in cairo than the canvas spec. Which I believe means some additional back and forth is done that isn’t needed. This is part of the reason Azure was conceived, to create an API that maps more cleanly to our needs :).
Colin, one of the things was the handling of global alpha. This is easier to express directly with Direct2D than going through cairo because cairo doesn’t have the concept of pattern alpha.
@Jeff, Colin: Yeah, global alpha definitely also makes a significant difference. Speed Reading also sees benefits from this.
Hi Bas, this is a great post.
I’m almost completely noob about GFX, but I’d really like to see a blog post about why don’t you pick Skia which has incremental GPU support on the way?
Thanks.
@Felix: There’s a bunch of reasons, first of all Skia’s GPU support hasn’t proven itself yet, whereas Direct2D’s has. There’s also licensing issues with Skia which would need to be resolved, but I don’t know much about that. Finally I personally am not sure if I believe Skia is taking the best approach, but that’s something that would require more research. This was the fastest road to major improvements on the short term, and the possibility to creating our own consistent GPU rendering on the longer term. Of course we could create a Skia Azure backend for testing! :-)
Nvidia users should update to the latest drivers.
Desktop
http://www.nvidia.com/object/win7-winvista-32bit-275.33-whql-driver.html
http://www.nvidia.com/object/win7-winvista-64bit-275.33-whql-driver.html
Notebook
http://www.nvidia.com/object/notebook-win7-winvista-275.33-whql-driver.html
http://www.nvidia.com/object/notebook-win7-winvista-64bit-275.33-whql-driver.html
I’m still thinking that you should have tried to add the features you needed into Cairo rather than writing a new layer :-(
@Bas: Thanks for your reply.
Can’t wait to see a experimental Skia backend working there. :)
Pretty impressive! On my system in the fishbowl test the Azure Firefox manages 34 FPS with 2000 fish, whereas IE 9 and IE 10 preview don’t get more than 17. Beating Microsoft at its own game is quite cool :-)
Nice work Bas. Paintball went from ~16.0 to 10.9. Fishbowl only hit about 25fps at 250 fish but now I get a solid 60 fps at 2000 fish.
Have you got any idea on why the Mister Potato Gun demo runs at widely varying speeds for no apparent reason? I’ll sometimes get ~24,000, 42,000, and 85,000. Can’t figure out why this is happening. It doesn’t seem to be build specific as it can change with the same build in the course of a day.
“The next, possibly most interesting step for Azure will be a larger project: creating a backend that can use the GPU for rendering vector graphics through OpenGL and different Direct3D versions. This will hopefully allow us to bring fast, consistent rendering performance to our users on all platforms!”
Indeed interesting. But isn’t this basically what OpenVG does? As far as I know all OpenVG implementations are built on top of OpenGL. So will you be implementing something custom upon OpenGL or will you be fixing up an existing/creating a new OpenVG implementation?
Once AzureD2D canvases ships does that mean that FF on Windows will see some benefits or will it be until backend for GPU lands that Firefox will be able to take advantage of the tech?
Good to hear people are getting positive results!
@Noel: I can understand that, but I believe, as Joe outlined in his blog post, there’s too many fundamental design problems in the cairo API that would be hard to surmount. Not to mention we wouldn’t be able to upstream due to the large scale changes that would be needed and the lack of resources to adapt all cairo backends. This is not just about features, a mostly stateless vs. a stateful API is a huge difference.
@Gary: Great to hear that! I’ve wondered the same thing about Mister Potato Gun and I have no idea. It seems a higher running time results in a higher score, and the running time seems to vary as well. It’s almost as if some randomness affects it. I didn’t include it in here for exactly that reason.
@Zsolt: No, it will be unrelated to OpenVG. I also don’t know of any very good completely open OpenVG implementation, but maybe I missed something. The idea is it’s general enough to output triangle strips and generate texture coordinates in a way that works across different graphics frameworks. And use additional features like geometry shading and such things to speed things up further on capable hardware. On windows this would still work on D3D9/10/11 because there’s simply performance benefits to that.
@Caspy7: Yes, Firefox users on Windows 7 that have Direct2D for normal browsing will see advantages from Azure. Possibly if Gecko moves to rendering on Azure before the GPU backend lands they will see advantages in page rendering as well. The same will go for Mac users when a Quartz backend lands.
This sounds exciting from a Firefox-is-improving standpoint.
Currently an update to the Cairo port (ports.haiku-files.org/) is needed in Haiku OS (haiku-os.org) for recent Firefox versions to work, will this mean you can run Firefox without the Cairo library completely and it should run?
Also Firefox and Haiku OS are both Coverity users, like buddies! :)
Will the license be MPL, tri-license like Fx or other?
Thank you, I await your reply.
@Anonymous: Not really, we plan on using Cairo as a fallback software renderer for the moment for systems where hardware assisted rendering does not work adequately.
The license will be as open as it can be as far as I’m concerned! Presumably this will be the MPL (v2), but I know fairly little about these things. At the moment the component (without cairo support) can be used and built completely stand-alone, but is obviously useless outside of systems where Direct2D exists.
I tried the build. It worked well! I just wonder why the Flickr Explorer test is still so slow?
Also a tip: if the try build misses some DLLs, install the VC++ 2010 redistributable package.
Bas, the first version of my post contained a tinyurl of the testpage http://ie.microsoft.com/testdrive/Performance/FlickrExplorer/Default.html But i couldn’t send it, because ur spam filter thought it is a spam. :)
@WonderCsabo: I’ll have a look, possibly FlickrExplorer doesn’t use Canvas, that would certainly explain! I’ve had to set my spam filter quite agressively considering how many spam arrives on blogs. I have to moderate all comments manually :(.
Are there any plans to create an XRender backend for Linux, or will those users be stuck with Cairo until you come up with a general OpenGL version?
Also, is the plan to move everything to the GL/D3D versions eventually, or would you first try D2D/Quartz and only fall back to the 3D APIs if those weren’t found?
@Matt: There was discussion, but the small market and the large difference in quality between different X-render implementations make the return on investment too low to justify. So I’m afraid they’ll be stuck with cairo until we come up with our own accelerated backend.
The second part of your question depends largely on how good our versions will turn out to be. If at some point they perform better than D2D, logically we’ll start using them, as long as they don’t, we probably won’t :). Consistency is important but so is a great user experience!
Hi! Can you make a build every week with latest changes from this great API?
@Christian: You’ll be happy to know Azure landed on mozilla-inbound yesterday, and mozilla-central last night. Tomorrow’s nightly will have Azure included for canvas on Direct2D enabled builds, and upcoming nightlies will include improvements and fixes we make!
Sounds exciting. I only think you should have chosen a different name, what with Azure being MS’ cloud computing name.
Steven is right.
“Electolysis", “Telemetry” these Greek words are much better. If someone knows what they mean, is able to understand easily why have been chosen.
So, a similar word for HWA-Project, could be great. :)
Here’s a quick question: How tightly bound is Azure to the Mozilla codebase? It seems like a promising project, and I can think of various uses for it outside of Mozilla. How easy or hard would it be to extract and use as a stand-alone library?
@Anonymous: Not at all, at the moment you’d have to import mozilla’s RefPtr.h file into the project to get it to build stand-alone, that’s the only dependency. There’s a visual studio project in the azure folder to build it.
That sounds great. When I have some time from everything else, I might have a look at it. By then maybe you’ve got some more ports running too.
Hopefully it will stay fairly independent, too.
@Bas, thanks for the info. Keep up the good work! ;)
Yes, Nightly seems to be faster, however, Test3 seems to be broken considerably… do you know about it?
http://www.kevs3d.co.uk/dev/asteroidsbench/
Actually, on fully HW accelerated system on W7/WhiteListed NVidia is SW rendering with turned OFF faster than HW rendering, esp. test 2 and 5 of asteroids…
And DX9 (non D2D) and slower computer is also much faster, is it known and temporary symptom?
@Marek: The correctness bug is known and will be fixed in a future release.
The performance on Test 2 is a known bug that’s being worked on. Test 5 is more complex, it’s a more longer-term symptom due to the global composite operator it uses. (It’s not Azure specific)
Can I ask something that concerns me?
Only Fx’s HWA keeps generally my Graphics Card running with middle-clocks, always…
On IE9 and Chrome, the clocks staying in lower levels while both using normally HWA on normal browsing…
The GC-fan works always faster with Fx… It’s like Fx uses the Card a lot more than the other browsers…
Is there an explanation? To be honest, I don’t like this behavior and thus I turn off HWA.
Interesting work. Will Azure be released as a library and if so what license will the code have?
@Eugene: It -can- currently be built stand-alone, however this isn’t as easy yet as it should be. It will be under the MPL like all other Mozilla code, so it should be quite easy to use in other projects.