Graded Browser Support Update: Q4 2009

By Eric MiragliaOctober 16th, 2009

This post announces an update to Graded Browser Support. The GBS page on the YUI site always has the most current GBS table. This post includes:

GBS Changes for Q4 2009

With this update, Mac OS 10.4 Tiger drops from the A-Grade testing matrix (replaced with Mac OS 10.6 Snow Leopard) and the testing surface is reduced to 12 browsers on 4 OS platforms (down from 14 browsers on 4 platforms). Specific changes include:

  • Initiated A-Grade support for Firefox 3.5.† on Mac OS 10.6.†
  • Initiated A-Grade support for Opera 10.0.† on Windows XP
  • Discontinued A-Grade support for Firefox 3.0.† on Mac OS 10.5.†
  • Discontinued A-Grade support for Firefox 3.5.† on Mac OS 10.5.†
  • Discontinued A-Grade support for Safari 3.2.† on Mac OS 10.4.†
  • Discontinued A-Grade support for Opera 9.6.† on Windows XP
Win XP Win Vista Mac 10.5.† Mac 10.6.†
Firefox 3.0.† A-grade
Firefox 3.5.† A-grade A-grade A-grade
Opera 10.0.† A-grade
IE 8.0 A-grade A-grade
IE 7.0 A-grade A-grade
IE 6.0 A-grade
Safari 4.0.† A-grade A-grade

Notes:

  • The dagger symbol (as in “Firefox 3.5.†”) indicates that the most-current non-beta version at that branch level receives support.
  • Code that may be used on pages with unknown doctypes should be tested in IE7 quirks mode.
  • Code that may appear in IE8′s “compatibility mode,” which emulates but is not identical to IE7, should be tested explicitly in compatibility mode.

GBS Forecast

We expect to make the following changes in the Q1 2010 GBS update:

  • Discontinue A-grade for Opera across all OSs (if current data trends continue); the latest version of Opera (currently 10.0.†) will be considered an X-grade browser as of Q1.
  • Initiate A-Grade support for the latest version of Google Chrome on Windows XP (if current data trends continue).
  • Initiate A-Grade support for IE8 on Windows 7.

Discussion

This update pares the testing surface to 12 browser/platform combinations (down from a high of 15). The most significant aspect of this update is our guidance for Q1 in which we forecast Chrome beginning to receive A-Grade support and Opera 10 moving to the X-Grade. Here are notes from the GBS committee with respect to that guidance:

  1. Chrome: The rate of growth in Chrome’s global usage has been dramatic. By some measures, including ours, it is now double that of the A-Grade Opera browser (source: StatCounter). Chrome on Windows is built around a solid WebKit core, supportive of web standards (including forward-looking HTML5 standards), and extremely performant. With Google’s backing, the project is making rapid progress both on Windows and with the not-yet-released Mac OS X version. If this growth rate continues, we conclude that Chrome will require A-Grade attention as of Q1.
  2. Opera: Opera’s marketshare, which is small and shows signs of diminishing, makes a compelling case for moving this excellent browser from the A-Grade to the X-Grade in Q1. X-Grade is a category designed to encompass modern, capable browsers with small marketshare, and Opera is squarely in that category today. Opera’s marketshare in specific Eastern European and emerging markets might argue for some developers to retain this browser on their testing matrix beyond Q1. We encourage you to watch carefully the argument presented by Opera’s Andreas Bovens and David Storey recently with respect to the “marketshare myth” and reasons why Opera’s importance transcends the global marketshare metric (source: YUI Theater).

Graded Browser Support is a QA philosophy, not a report card on the quality of popular browsers. It’s designed to provide guidance for QA teams about how best to use their limited testing resources (and to frontend engineers about how to sanely cross-check work across a finite set of browsers). The goal is to be conservative and calculating: We want to test the smallest possible subset of browser/platform combinations, leveraging implicit coverage by testing the most commonly shared core browser engines.

Inevitably (and by design), this leaves a lot of browsers out of the matrix. And, unfortunately, the percentage of users still tied to IE6 requires us to include that browser (not because we like IE6, but because we like the many tens of millions of users who rely on it).

One of the most interesting aspects of the quarterly GBS update is hearing your advice (often different than our own), and we’d love to hear your take on these issues in the comments section.

The GBS Archive

28 Comments

  1. I think Opera makes some really good points about the “marketshare myth”. If your library stops being fully compatible with Opera, you are basically telling all those countries where Opera is a major player to piss off and find another toolset.

    The bottom line criteria for A-grade on the GBS matrix seems to be “does this browser have significant global marketshare?”, but it would make much more sense if it were “does this browser have significant marketshare in one or more major internet markets?” And for Opera, the answer is a definite yes.

  2. Necessary disclaimer: I used to work for Yahoo! and now work for Opera. Drum roll… :)

    Josh makes a good point about how dropping A-grade support for the latest version of a browser might have on projects like YUI which are also relied upon outside the realms of Yahoo!.

    Testing for one more browser always takes time. But support for the core specifications implemented (HTML, CSS, JS, DOM) is increasingly becoming reliable in all browsers.
    All major browsers still have rendering bugs and probably always will, but testing for one more browser now takes less time than it used to (at least, from what I’ve seen over the past 10 years where I worked).
    And there is no doubt all vendors are now continuously at work to push updates that improve these implementations and therefore reduce the time we have to spend fixing rendering bugs.

    This being said, if you really don’t care about the markets where Opera is doing well, both for the Y! websites and projects like YUI, then feel free to drop it. It’s always worth reminding people this is Yahoo!’s internal guidelines and anyone else is free to do whatever they want.
    Although I have to say, and I had a bit of an argument about that when the last GBS revision came out, I can’t possibly understand why Yahoo! wouldn’t want to have rock solid sites in Eastern Europe and Asia in order to grab more market share.

    Now two big questions that have nothing to do with the above:
    1. any chance of seeing a C-Grade list out there in the not-too-distant future? (really important for people who want to strictly follow the GBS philosophy, in my opinion at least).
    2. any chance of seeing Yahoo! take a harsher stance against the use of IE6 and promote the installation of modern browsers? (I know there must be a big fear somewhere that it’d hurt business, but it doesn’t cost me anything to ask… :)

    Anyway, thanks for continuing to publish the GBS and sharing your work with everyone.

  3. It’s an interesting predicament Opera is in. I guess it sort of underscores how the Graded Browser Support is really just a generalized description of worldwide usage, as Josh L. says.

    A similar example of a similar predicament is IE6 in South Korea (I think I remember hearing this at the same Opera talk linked to above), which commands an estimated 60% market share due to a government decision 10+ years ago [1].

    IE6 is due to (hopefully soon) disappear from the global usage map, but at that time it’s likely to still show up as the most-used browser in South Korea. Should we keep it on the map because of that one market? I hope not.

    I think GBS is a general recommendation to a general audience of worldwide developers. If developers are working towards a specific market, then it ought to be up to them to cater to that market and know its needs.

    [1] http://mozillalinks.org/wp/2009/10/not-funny-60-still-using-ie6-in-south-korea/

  4. Cyril,

    Thanks for the thoughtful supplement to the argument in favor of keeping Opera in the A-Grade.

    Regarding C-Grade browsers, here’s our draft of that list, and feedback on that is welcome as well: http://developer.yahoo.com/yui/articles/gbs/#cgradelist

    -Eric

  5. Cyril brought up a really good point that I forgot to mention (coincidentally I did bring it up the other day): The amount of extra resources required to offer A-grade support for Opera is relatively trivial. On the other hand, the amount of Eastern European/Asian sites who would potentially have to choose another toolkit due to full Opera support is not.

  6. Awesome, thanks to the YUI team for keeping up with this. It sucks that IE6 still has such a large market share, but there are so many companies that simply will not switch away from IE6 due to outdated internal tools they use that require it. Here’s some particularly depressing statistics…

    http://en.wikipedia.org/wiki/Usage_share_of_web_browsers

  7. I have a few points to make.

    Firstly, Opera market share is not diminishing. On the source you use (StatCounter), Opera has currently a big global drop in October, but this looks to be an error on StatCounters part. There is a sudden drop in certain markets, where it looks like Opera 9.6 is dropping off, while Opera 10 is replacing it, but the growth plateaux off. This isn’t backed up by our internal metrixs where we had record numbers in September following the Opera 10 launch, and shows no sign of slowing down. It is increasing, especially in the markets that dropped on StatCounter. I believe they are not detecting Opera 10 correctly (possibly due to Opera Turbo, I’m not sure. That would make sense as they would think that many requests are from the same IP – the Opera Turbo servers.). It would be a shame if Yahoo! removes testing of Opera due to faulty stats.

    Secondly, libraries such as YUI are often used to help applications and sites be cross browser, and cross platform, due to functions that work around browser issues. YUI will be the only major library that doesn’t test in Opera (that I know of). Making sure the library runs on more engines helps quality in my honest opinion. It is less likely that you will rely on bugs in a browser engine. We use YUI at Opera on some of our web properties, and we’ll probably have to start looking at other options, as we need to know it works in Opera.

    Thirdly, Google Chrome is based off WebKit, and YUI is already testing in two different OS (both which will be largely the same) in a WebKit browser. Chrome has a different JS engine, but differences between Safari and Chrome will be relatively minor. I expect the time and effort taken to test and develop for Chrome will be small. Odf course I’m biased, but to me it makes more sense to test Safari on only one version of OS X than it does to remove Opera.

    Fourthly, Opera is the largest mobile browser, and quite likely the largest device based browser, with the Wii, DSi, and other devices. Last time I checked there was more active monthly users on Opera Mini alone than Apple had sold iPhones. It doesn’t make too much sense, especially as mobile is growing at a great rate, to remove testing for an incredibly popular mobile browser. Opera uses the same engine for our mobile browsers as we do for our desktop browser, so ensuring YUI works on Opera 10 will go a long way to getting it to work on Mobile. See for example StatCounters mobile data.

  8. If something works on all the A-Grade browsers, and that same thing doesn’t work on an X-Grade browser, it seems to me that is a good indication that the X-Grade browser in question has something wrong with it.

    Everyone wants IE6 off the market so that browsers can converge on a set of standards, but if any browser bug that comes along in an X-Grade browser is worked around hidden from folks who use YUI, then that encourages these bugs to persist.

    It will be sad to see Opera drop off the A-Grade browser list, but the thing that Opera really has going for it is adherence to the standards. I rarely ever have to worry about working around something Opera throws at me. It’s been said before, and I’ll say it again, it’s all versions of IE that muck things up.

    Opera has nothing to worry about by not being listed as A-Grade. I think most folks who care about cross browser compatibility will fix any issues that come up with their sites working well on Opera, and so long as Opera adheres to the standards, YUI should work fine on it.

  9. I would say that the Opera dip is definitely a glitch of some sort. If you switch the StatCounter view to Bar, it still shows Opera at 2.84%, compared to Chrome’s 2.41%.

    Obviously there are always trade-offs to be made when selecting for the A-Grade list. More browsers mean more development time which mean less time for new features. I don’t think market share should be the only criterion applied when choosing the browser grading. There are many arguments (some already made) for keeping a capable and modern browser such as Opera in the list.

    The biggest for me would be the coverage of mobile and home entertainment devices afforded to you by supporting Opera. On top of this, you are creating a very positive message that supporting standards-compliant browsers is worthwhile. This chart is looked to by a lot of professional developers as a touchstone for what is “right” when it comes to supporting browsers. You have been brave (compared to many) by resisting the urge to drop IE6 just because it takes a lot of time to fix issues with. Be just as brave by keeping Opera.

    On a side note, I also would point out to people that the GBS (theoretically) applies to all Y! sites, not just the YUI libraries even though it is published within the YUI namespace on the developer network.

  10. OPERAOPERAOPERAOPERA! ♥

  11. Petter Nilsen said:
    October 19, 2009 at 5:41 am

    The Statscounter data doesn’t appear to include Opera 10, judging from this faq entry: http://gs.statcounter.com/faq#opera10

  12. Thanks for all the thoughtful comments regarding Opera. I personally am a huge Opera fan, am and not the only YUI developer that uses Opera as the primary browser.

    We forecast expected changes not only to give developers a heads up as to the direction we will likely be going, but to foster this type of discussion. We do not make these decisions lightly, and understand that there may be some controversy. We will investigate the StatCounter glitch, and ensure that any decision is based on the best data available, including Y! traffic statistics. In general, our statistics align with StatCounter, and if we exclude October due to bad data, the trend is still downward. If there is some uptake due to Opera 10 adoption, this will definitely factor into next quarter’s discussion.

    As Norm mentions, this is first and foremost the support philosophy of Yahoo!. Some sites and developers may have more or less stringent testing standards. For example, the YUI team must test in both quirks and standards modes in order to ensure A-grade support for IE. In general, we go above and beyond the A-grade matrix to ensure the widest possible coverage.

    I feel compelled to clarify that we are by no means dropping “support” for Opera. The reason we came up with GBS was so that we could strive for 100% support of HTML agents without actively testing them all. The “A-grade” matrix is a list of browsers that QA must test and verify that they in fact receive the A-grade experience. Contrary to some of the opinions expressed, regressing a complete test cycle against a browser is by no means trivial.

    The A-grade matrix is designed to provide the widest coverage possible with the smallest test surface. As David S. mentions, testing Safari helps give coverage to other WebKit-based browsers that aren’t actively tested against. Same applies to Firefox and Gecko-based browsers. Chrome’s custom JS engine is enough to warrant calling it out explicitly for active testing.

    What we are proposing is demoting Opera to “X-grade”, meaning that QA is not required to test this browser, and bugs against this browser will be examined on a case by case basis. As Floydian points out, Opera has evolved to the point where in most cases “it just works”, which, combined with market trends, make a compelling case for demoting to X-grade.

    While an interesting conversation, mobile support is currently outside the scope of GBS, as it requires adding support for new platforms and browsers that are generally less capable than their desktop counterparts. That said, we may have some guidance to offer this space in the future.

    As I mentioned, Opera is my favorite browser, and I would love to continue A-grade support. Practically speaking however, we need to make sure that QA resources are not spread too thinly, and that that precious testing cycles are spent ensuring the best experience for the most users.

  13. I realize the intent behind supporting IE 6, but A-grade? Really? I like to think of the YUI team as one of the leaders in UI web development. With that in mind, why you don’t practice graceful degradation is a mystery. Shut out IE 6 users? Of course not. But wasting valuable time and resources to give it first-class treatment is not a choice I would make.

    If IE 6 holds 20% market share for two more years( a distinct possibility due to corporate dependence ), will you still give it A-grade? IE 6 will be over 10 years old at that point, and using a rendering engine that dates to 1997.

    It isn’t a crime to serve basic HTML to ancient clients.

  14. El Guapo,

    Not only do we practice graceful degradation, we came up with GBS as a practical way to support it.

    Yes, unfortunately if in 2 years IE 6 holds 20% of the market, it will likely still be A-grade. However, we will do our best to make sure that this isn’t the case.

    I’ll reiterate that this is not a list of “capable” browsers, but of browsers that QA must test against to ensure the A-grade experience. A 20% percent market share demands this status.

  15. While everyone seems to be focusing on the impact to Opera, I have to say I am very excited to hear that Chrome will finally be joining the A-Grade list. Kudos to the YUI team for putting that on the roadmap.

    With regards to testing, has the YUI team ever considered making it’s A-Grade browser test cases public? If there is a way to automate the test process such that someone could run the tests in the browser of their choice, then developers could run the tests on the browsers they wish to support (skipping the ones that are A-Grade). I would think that would be a win/win situation. Yahoo could drop Opera off the A-Grade list, and perhaps a forum could be set up explicitly for developers to post test case results. An informal A-Grade list if you wish. Would Yahoo ever consider doing this?

  16. Thanks for the reply, Matt.

    I understand that you don’t base things on capability( which if it was the case, IE 7 would be a target for deprecation too ). I guess I view things differently, in that progress must continue despite how many people refuse( corporations ), or are unable( for whatever reason ) to move beyond very old software. If it was the case that you had actual customers who were paying for support, well, the client gets what they want to pay for. Otherwise, a reduced experience is encouragement for people to upgrade. Browsers are free, and so is the content. There is no obligation to support something for *decades* in this context. My opinion, obviously. :-)

    By providing access to content, reduced in features and wow factor, no one is deprived, and limited resources can be directed toward building the future.

    Again, thank you for taking time to reply!

  17. I am curious as why Firefox 3.5 is dropped on Mac OS X 10.5 if you’re planning to support Safari 4?

    Seems to me that 10.5 isn’t going to drop of the radar just because of Snow Leopard.
    Snow Leopard is after all Intel-only, and as such I feel that there will be a lot of people running PPC Mac’s using Firefox 3.5 that simply can’t switch to 10.6.

    Is there any data that decision is based on?

  18. Hi Peter,

    We include all of our unit tests with each YUI release. Additionally we regression test all of the examples we have on the site against the A-grade matrix.

    We are working on automating these, however keep in mind that while the YUI team is the owner and caretaker for this list, it is the baseline matrix for Yahoo! as a whole. As with other sites, depending on specific traffic, available QA resources, and other business concerns, various Y! properties may have more or less strict testing requirements. For example, even though its considered X-grade, we have been testing Chrome against most of our core components for the past couple of release cycles. Ditto for Opera 10 beta.

    Unit tests are fairly easy to automate against a wide range of browsers, however deep functional testing against a site or “application” requires manual QA cycles, which need to be allocated as strategically as possible.

  19. Nicky,

    We have every expectation that FF 3.5 on 10.5 and 10.6 will perform similarly. By testing on the latest the OS version, we providing implicit coverage of the browser on other Mac OS X versions.

    The idea here is to support all capable browsers by testing the right cross-section of capable browsers. A site that’s been tested against this A-Grade matrix is likely to do very well with the best X-Grade browsers, including FF 3.5.x on 10.5.

    -Eric

  20. I know this is like Emacs vs. vi, but I really don’t see the point of using Opera these days. The choice is really between Chrome (performance and security) and Firefox (extensibility).

  21. Opera 10 is my browser of choice. It behaves well, supports a wide range of modern standards (without bothering with a few that have yet to achieve wide cross-browser support), supports a very wide range of platforms and devices that no other browser quite manages to, provides accessibility support that no other browser quite manages to either. My particular favorite feature is its support for link rel, which I wish other browsers (pardon me SeaMonkey you do too) would out of the box too – it’s just so useful to be able to page through a site (using “prev” or “next”), rather than click on navigation in the site.

    I do use other browsers, Firefox, in particular, because of its range of extensions; sometimes Safari and Chrome, simply because their recent versions are so cool and beautiful. Of course I also use IE, because sometime you just have to, but it’s just not so fun.

    I look forward to the day that you drop IE 6 support (and actually IE 7 as well, as it is also a mangy cur). Looking forward actually to IE 9 catching up in 2010 with where other browsers are in 2009!

    I think it would be a mistake to drop Opera 10 – it is actually the main browser in some countries, particularly in Northern and Eastern Europe. Even taking Europe overall it still accounts for about 5-10% of all browsing, by most measures.

    Drop IE 6 before Opera 10 from the A list. It is possible to provide progressive enhancement in a way that guarantees that IE 6 users have access without wasting resources giving them the full sunshine experience – that I think is fair. Running major lumps of javascript on IE 6 must be a nightmare anyway, as it is sooooo slow.

    Anyway, that’s just my tuppenceworth. I have a feeling that you will end up supporting both Chrome and Opera. Chrome for the desktop, Opera for the SSD and for some European markets. (I know you don’t yet include SSD’s in your supported platforms, but you know you will have to, the market is going that way.)

    Sorry for droning on so long ;-)

  22. Well, Yahoo Search Engine together with Bing/MSN have less than 0.5% market share in Russia or Ukraine. While Opera has around 40%.

    Indeed, USA should not be a global metric for Internet stats anymore. Sorry to say, but this country has stuck in legacy of Web 1.0 trash (just like South Korea with IE6, in linked example), and now its only slowing the global adoption of modern standards, browsers and practices.

    on topic: Opera 10 is quite compliant browser, as are FF3.5+, Chrome 3+ or Safari 4+ . They have only a couple of minor inconsistencies between each other, which are not hard to track down and fix. And i hope IE9 will catch up with them, so all the major browsers will become interoperable ;)

  23. I have a question for the creators of the browser matrix and to the other developers in this community. I was in a conversation with Microsoft and they verified that the core rendering engine for IE8 and IE7 are the same for the different operating systems. Also, you can install the same installation file of Firefox on multiple OSes and have the same rendering results. This appears to also be supported by my company’s development team as they have also noticed that bugs in one OS usually are present in another with the same browser type (ex. IE8 on XP and Vista usually have the same bug). Along with this the same fix usually works on the different OSes. My question is, if there is no code that may have calls to applications that interact directly with the OS (ex. Java) would it suffice to test the browser in just one of the Operating Systems?

  24. Hi KTX10,

    In the past, we have seen platform specific issues arise with both IE and Firefox. With IE this is generally in the area of activeX/directX controls. For Firefox the issues were generally rendering related.

    As you mention, these browsers generally have the same bugs, but not always. This is the rationale for requiring testing on multiple platforms.

    This assumption is worth challenging from time to time, and will be re-considered prior to the next quarterly update.

  25. @KTX10
    You can see here an IE8 bug that happens only in Vista: https://connect.microsoft.com/IE/feedback/ViewFeedback.aspx?FeedbackID=425465&wa=wsignin1.0

    It’s not that hard to find also Firefox bugs that only happen in one OS but not the others (in fact it’s easier thanks to bugzilla).

  26. [...] update implements the guidance we provided in Q4 2009. That update generated significant discussion, and high-quality input from Opera is included in the [...]

  27. The Statscounter data doesn’t appear to include Opera 10, judging from this faq entry: http://gs.statcounter.com/faq#opera10