Web 2.0 Browser shootout

When I hear "Web 2.0" I think about the next generation of interactive, networked interfaces. Some of these new interfaces will not be attached to a browser at all but many will remain browser-centric, or will at least will be rendered using W3C standards. There are a lot of technologies like Flash and Silverlight that are helping to drive Web 2.0 forward but ultimately the oldest form of client side automation, javascript, is and will continue to be the preferred solution because of its entrenched usage and frameworks like AJAX and JSON. Adding to this is the recent javascript improvements from the Webkit team and the Gecko team. Soon we will be processing javascript at near native code speed.

As you can see from my previous javascript benchmarks the current leader is Gecko when it comes to performance of a stable release rendering engine. Webkit does very well in the SVN version and will be an excellent javascript performer when the current Squirrelfish implementation is stabilized. From the benchmarks released for tracemonkey we will see even greater improvement in javascript when xulrunner 1.9.1 is released. That is estimated to be about 3 months out at this point. Currently 64 bit is not supported as I figured out in an attempt to test tracemonkey performance. Opera seems a long way from Gecko and Webkit when it comes to javascript performance but the browser itself is featureful and Opera produces an excellent mobile browser which will only become increasingly important in years to come.

The next important piece to the Web 2.0 puzzle is standards compliance. This has become increasinly important as more browsers have shown up on the average users' radar. Mac users have Safari (Webkit), Opera and Firefox (Gecko), Linux users have Gecko browsers, KHTML from which Webkit is derived, and Opera, Windows users have Opera, Safari, IE, and Firefox. As you can see only IE is not cross-platfrom. The Unix version of IE died a long time ago and the Mac version more recently but there is no recent version of IE that is supported on another OS other than a Microsoft one. On mobile devices a variety a browsers are offered. Opera is offered on many different platforms. Gecko and Webkit, being open source can be ported to many different platforms and have already been ported to some. IE stands alone again as a MS only product. For this reason I think that IE has a limited future in Web 2.0. Not everything runs a MS OS and this isn't going to change in the future in fact I think this fact is only going to become more pronounced as mobile devices become more popular.

The infamous Acid tests test W3C CSS compliance. The current test is Acid3. It tests CSS3 compliance. Browsers are on their way to achieving Acid3 conformance and in fact a previous SVN build of Webkit passed the Acid3 test but current versions do not. Here is how the current Webkit scores:



Unfortunately at this point Midori (using Webkit) crashes in addition to failing Acid3. The current unstable Xulrunner (Gecko) doesn't crash and make it farther in the test but the visual results aren't quite as good:



The unstable version of Xulrunner produces better results while still not passing:



Opera does okay but still isn't there yet either:



Testing IE with Acid3 shows similary half baked solutions but I don't have the results to show because I don't have Windows on this computer. It makes it to 71 with all gray boxes before it quits, much like the current Gecko engine. Regardless I think IE is going to have very little to do with the Web 2.0 revolution compared to other browsers simply because it is not cross platform.

Taking into consideration both javascript performance and acid3 compliance only Webkit and Gecko seems ready to take on Web 2.0 full force. They both have excellent javascript speed and are getting to acid3 compliance quicker than other browsers. Opera is a good browser but I think they might be better served working on Webkit and integrating it with their web browser than continuing their own engine. This would save resources to work on other aspects of the browser while retaining an excellent browsing experience. Gecko has made huge strides and continues to do so. In the future I think we will be talking about Webkit vs Gecko as opposed to IE vs Netscape like we did over ten years ago. Ultimately both engines are going to drive Web 2.0 development much more so than the alternatives.

Opera Javascript Comparison

Since I've been posting about javascript performance lately I thougt I would compare Opera v.9.52 JS performance to the current release of Xulrunner running under Epiphany 2.22. The results are mixed. Celtic Kane's javascript speed test shows results that best Gecko but not Webkit.



In comparison I was only able to achieve a score of 255 (lower is better) for the current Xulrunner. This looks good for Opera. Let's take a look at the sunspider benchmark now. Xulrunner is show in the FROM column while Opera is shown in the TO column.



As you can see Opera's javascript perfomance is significantly slower than Xulrunner's performance. I am a bit confused by the results considering Opera did pretty well in Celtic Kane's test and I have heard that Opera's JS performance is very good. My sunspider results were so out of whack from what I was expecting that I ran them again but the results were the same. I wonder if anyone else can confirm my findings.

Tracemonkey javascript performance

I thought that my comparison of the latest Webkit vs stable Gecko wasn't fair so I decided today to create an ebuild to fetch the latest xulrunner so I could test Tracemonkey. I had to tweak the ebuild a little (other than fetching with mercurial) because xulrunner would not compile with lcms enabled. I created another USE flag that toggles lcms. With this and some modification to the patch tarball I was able to build the latest xulrunner.

Here is my ebuild:

/usr/local/portage/net-libs/xulrunner/xulrunner-9999.ebuild

This ebuild requires the xulrunner.conf from the original ebuild files.

Next I launched Epiphany and typed "about:config". I searched for "jit" and then toggled "javascript.options.jit.content" to true and restarted my browser. Then I proceeded to run Celtic Kane's javascript speed test and the Sunspider Benchmark. Celtic Kane's test didn't show much of an improvement over the old javascript engine and didn't come close to Squirrelfish's performance. The result was a paltry 23ms decrease in time.



Maybe the Sunspider Benchmark would be different. Here is the comparison of the old Gecko JS engine and Tracemonkey. The old engine is under the FROM title and Tracemonkey is under TO.



We see some significant improvements over the old engine but we see some regression too. The overall picture shows us a slight improvement. Next let's compare Webkit's results to Tracemonkey. Here we have Webkit in the TO column and Tracemonky in the FROM column. Get ready for an all out browser war!



Yikes! That's disappointing. For all the noise we've heard about Tracemonkey these were not the results I was expecting. Squirrelfish is still eating it for lunch. I guess it's a good sign that Mozilla's numbers are improving but I have yet to see the performance they claim. There could be many reasons for this and I'm sure things will only improve but as it stands now I cannot achieve the kind of numbers I've seen put up on the web about Tracemonkey.

UPDATE: Apparently JIT cannot yet be enabled in 64-bit browsers so that is why perfromance is so close between the new Tracemonkey JS engine and the old one. When that changes I'll post a real performance comparison.

Webkit vs Gecko Javascript performance

If you haven't noticed I have a fascination with bleeding edge software. One of the pieces of software I have been following since its inception is the GTK port of Webkit. It is an excellent browser engine that is just as capable if not more so than other major competing engines. One of Webkit's strong points is javascript performance. The Squirrelfish javascript engine that was introduced a few months ago offers incredible performance.

To prove my point I tested the latest Webkit from subversion againt Xulrunner-1.9 using two different javascript tests. The first test is I used was Celtic Kane's Javascript speed test. I ran the test a couple dozen times on each browser. The Webkit based Midori browser scored an 89ms (lower is better) for its best case while the best I was able to acheive with Epiphany on Xulrunner was 255ms which makes it over 2.5 times slower. In fact Midori's worst time wasn't even close to Epiphany's best time. I took screenshots of the two best times I achieved. On the left are the results for Webkit/Midori while on the right are results for Gecko/Epiphany.

The second test I ran was Webkit's own Sunspider Javascript test. It is a much more complete javascript test and takes several minutes to complete. Again Webkit outshines Gecko in the Sunspider tests but the results are much closer. Webkit's javascript engine, Squirrelfish averages about 1.5 times faster than Gecko's engine. Webkit results are shown in the FROM column while Gecko results are in the TO column.



It seems this is all about to change. Firefox 3.1 is supposed to include a new and improved javascript engine called Tracemonkey that outshines Squirrelfish. It is not yet complete so it is hard to tell how much better it is actually going to be but preliminary tests show some amazing results. When I get a chance to test it I'll post another comparison.

With all the improvements going into both the Gecko engine and the Webkit engine it should make web browsing on either platfrom a much better experience. I do have to give the Mozilla foundation credit for Gecko. I was starting to prefer a Webkit based GNOME environment over embedded Gecko but Xulrunner-1.9/Firefox-3 was a very good release and the next release is only going to be better. Things are really heating up now in the browser wars again.

Web 2.0 Browser shootout

When I hear "Web 2.0" I think about the next generation of interactive, networked interfaces. Some of these new interfaces will not be attached to a browser at all but many will remain browser-centric, or will at least will be rendered using W3C standards. There are a lot of technologies like Flash and Silverlight that are helping to drive Web 2.0 forward but ultimately the oldest form of client side automation, javascript, is and will continue to be the preferred solution because of its entrenched usage and frameworks like AJAX and JSON. Adding to this is the recent javascript improvements from the Webkit team and the Gecko team. Soon we will be processing javascript at near native code speed.

As you can see from my previous javascript benchmarks the current leader is Gecko when it comes to performance of a stable release rendering engine. Webkit does very well in the SVN version and will be an excellent javascript performer when the current Squirrelfish implementation is stabilized. From the benchmarks released for tracemonkey we will see even greater improvement in javascript when xulrunner 1.9.1 is released. That is estimated to be about 3 months out at this point. Currently 64 bit is not supported as I figured out in an attempt to test tracemonkey performance. Opera seems a long way from Gecko and Webkit when it comes to javascript performance but the browser itself is featureful and Opera produces an excellent mobile browser which will only become increasingly important in years to come.

The next important piece to the Web 2.0 puzzle is standards compliance. This has become increasinly important as more browsers have shown up on the average users' radar. Mac users have Safari (Webkit), Opera and Firefox (Gecko), Linux users have Gecko browsers, KHTML from which Webkit is derived, and Opera, Windows users have Opera, Safari, IE, and Firefox. As you can see only IE is not cross-platfrom. The Unix version of IE died a long time ago and the Mac version more recently but there is no recent version of IE that is supported on another OS other than a Microsoft one. On mobile devices a variety a browsers are offered. Opera is offered on many different platforms. Gecko and Webkit, being open source can be ported to many different platforms and have already been ported to some. IE stands alone again as a MS only product. For this reason I think that IE has a limited future in Web 2.0. Not everything runs a MS OS and this isn't going to change in the future in fact I think this fact is only going to become more pronounced as mobile devices become more popular.

The infamous Acid tests test W3C CSS compliance. The current test is Acid3. It tests CSS3 compliance. Browsers are on their way to achieving Acid3 conformance and in fact a previous SVN build of Webkit passed the Acid3 test but current versions do not. Here is how the current Webkit scores:



Unfortunately at this point Midori (using Webkit) crashes in addition to failing Acid3. The current unstable Xulrunner (Gecko) doesn't crash and make it farther in the test but the visual results aren't quite as good:



The unstable version of Xulrunner produces better results while still not passing:



Opera does okay but still isn't there yet either:



Testing IE with Acid3 shows similary half baked solutions but I don't have the results to show because I don't have Windows on this computer. It makes it to 71 with all gray boxes before it quits, much like the current Gecko engine. Regardless I think IE is going to have very little to do with the Web 2.0 revolution compared to other browsers simply because it is not cross platform.

Taking into consideration both javascript performance and acid3 compliance only Webkit and Gecko seems ready to take on Web 2.0 full force. They both have excellent javascript speed and are getting to acid3 compliance quicker than other browsers. Opera is a good browser but I think they might be better served working on Webkit and integrating it with their web browser than continuing their own engine. This would save resources to work on other aspects of the browser while retaining an excellent browsing experience. Gecko has made huge strides and continues to do so. In the future I think we will be talking about Webkit vs Gecko as opposed to IE vs Netscape like we did over ten years ago. Ultimately both engines are going to drive Web 2.0 development much more so than the alternatives.

Opera Javascript Comparison

Since I've been posting about javascript performance lately I thougt I would compare Opera v.9.52 JS performance to the current release of Xulrunner running under Epiphany 2.22. The results are mixed. Celtic Kane's javascript speed test shows results that best Gecko but not Webkit.



In comparison I was only able to achieve a score of 255 (lower is better) for the current Xulrunner. This looks good for Opera. Let's take a look at the sunspider benchmark now. Xulrunner is show in the FROM column while Opera is shown in the TO column.



As you can see Opera's javascript perfomance is significantly slower than Xulrunner's performance. I am a bit confused by the results considering Opera did pretty well in Celtic Kane's test and I have heard that Opera's JS performance is very good. My sunspider results were so out of whack from what I was expecting that I ran them again but the results were the same. I wonder if anyone else can confirm my findings.

Tracemonkey javascript performance

I thought that my comparison of the latest Webkit vs stable Gecko wasn't fair so I decided today to create an ebuild to fetch the latest xulrunner so I could test Tracemonkey. I had to tweak the ebuild a little (other than fetching with mercurial) because xulrunner would not compile with lcms enabled. I created another USE flag that toggles lcms. With this and some modification to the patch tarball I was able to build the latest xulrunner.

Here is my ebuild:

/usr/local/portage/net-libs/xulrunner/xulrunner-9999.ebuild

This ebuild requires the xulrunner.conf from the original ebuild files.

Next I launched Epiphany and typed "about:config". I searched for "jit" and then toggled "javascript.options.jit.content" to true and restarted my browser. Then I proceeded to run Celtic Kane's javascript speed test and the Sunspider Benchmark. Celtic Kane's test didn't show much of an improvement over the old javascript engine and didn't come close to Squirrelfish's performance. The result was a paltry 23ms decrease in time.



Maybe the Sunspider Benchmark would be different. Here is the comparison of the old Gecko JS engine and Tracemonkey. The old engine is under the FROM title and Tracemonkey is under TO.



We see some significant improvements over the old engine but we see some regression too. The overall picture shows us a slight improvement. Next let's compare Webkit's results to Tracemonkey. Here we have Webkit in the TO column and Tracemonky in the FROM column. Get ready for an all out browser war!



Yikes! That's disappointing. For all the noise we've heard about Tracemonkey these were not the results I was expecting. Squirrelfish is still eating it for lunch. I guess it's a good sign that Mozilla's numbers are improving but I have yet to see the performance they claim. There could be many reasons for this and I'm sure things will only improve but as it stands now I cannot achieve the kind of numbers I've seen put up on the web about Tracemonkey.

UPDATE: Apparently JIT cannot yet be enabled in 64-bit browsers so that is why perfromance is so close between the new Tracemonkey JS engine and the old one. When that changes I'll post a real performance comparison.

Webkit vs Gecko Javascript performance

If you haven't noticed I have a fascination with bleeding edge software. One of the pieces of software I have been following since its inception is the GTK port of Webkit. It is an excellent browser engine that is just as capable if not more so than other major competing engines. One of Webkit's strong points is javascript performance. The Squirrelfish javascript engine that was introduced a few months ago offers incredible performance.

To prove my point I tested the latest Webkit from subversion againt Xulrunner-1.9 using two different javascript tests. The first test is I used was Celtic Kane's Javascript speed test. I ran the test a couple dozen times on each browser. The Webkit based Midori browser scored an 89ms (lower is better) for its best case while the best I was able to acheive with Epiphany on Xulrunner was 255ms which makes it over 2.5 times slower. In fact Midori's worst time wasn't even close to Epiphany's best time. I took screenshots of the two best times I achieved. On the left are the results for Webkit/Midori while on the right are results for Gecko/Epiphany.

The second test I ran was Webkit's own Sunspider Javascript test. It is a much more complete javascript test and takes several minutes to complete. Again Webkit outshines Gecko in the Sunspider tests but the results are much closer. Webkit's javascript engine, Squirrelfish averages about 1.5 times faster than Gecko's engine. Webkit results are shown in the FROM column while Gecko results are in the TO column.



It seems this is all about to change. Firefox 3.1 is supposed to include a new and improved javascript engine called Tracemonkey that outshines Squirrelfish. It is not yet complete so it is hard to tell how much better it is actually going to be but preliminary tests show some amazing results. When I get a chance to test it I'll post another comparison.

With all the improvements going into both the Gecko engine and the Webkit engine it should make web browsing on either platfrom a much better experience. I do have to give the Mozilla foundation credit for Gecko. I was starting to prefer a Webkit based GNOME environment over embedded Gecko but Xulrunner-1.9/Firefox-3 was a very good release and the next release is only going to be better. Things are really heating up now in the browser wars again.