Performance Research, Part 1: What the 80/20 Rule Tells Us about Reducing HTTP Requests
By Tenni TheurerNovember 28th, 2006
This is the first in a series of articles describing experiments conducted to learn more about optimizing web page performance. You may be wondering why you’re reading a performance article on the YUI Blog. It turns out that most of web page performance is affected by front-end engineering, that is, the user interface design and development.
It’s no secret that users prefer faster web sites. I work in a dedicated team focused on quantifying and improving the performance of Yahoo! products worldwide. As part of our work, we conduct experiments related to web page performance. We are sharing our findings so that other front-end engineers join us in accelerating the user experience on the web.
The 80/20 Performance Rule
Vilfredo Pareto, an economist in the early 1900s, made a famous observation where 80% of the nation’s wealth belonged to 20% of the population. This was later generalized into what’s commonly referred to as the Pareto principle (also known as the 80-20 rule), which states for any phenomenon, 80% of the consequences come from 20% of the causes. We see this phenomenon in software engineering where 80% of the time is spent in only 20% of the code. When we optimize our applications, we know to focus on that 20% of the code. This same technique should also be applied when optimizing web pages. Most performance optimization today are made on the parts that generate the HTML document (apache, C++, databases, etc.), but those parts only contribute to about 20% of the user’s response time. It’s better to focus on optimizing the parts that contribute to the other 80%.
Using a packet sniffer, we discover what takes place in that other 80%. Figure 1 is a graphical view of where the time is spent loading http://www.yahoo.com with an empty cache. Each bar represents a specific component and is shown in the order started by the browser. The first bar is the time spent for the browser to retrieve just the HTML document. Notice only 10% of the time is spent here for the browser to request the HTML page, and for apache to stitch together the HTML and return the response back to the browser. The other 90% of the time is spent fetching other components in the page including images, scripts and stylesheets.
Figure 1. Loading http://www.yahoo.com
Table 1 shows popular web sites spending between 5% and 38% of the time downloading the HTML document. The other 62% to 95% of the time is spent making HTTP requests to fetch all the components in that HTML document (i.e. images, scripts, and stylesheets). The impact of having many components in the page is exacerbated by the fact that browsers download only two or four components in parallel per hostname, depending on the HTTP version of the response and the user’s browser. Our experience shows that reducing the number of HTTP requests has the biggest impact on reducing response time and is often the easiest performance improvement to make.
| Time Retrieving HTML | Time Elsewhere | |
|---|---|---|
| Yahoo! | 10% | 90% |
| 25% | 75% | |
| MySpace | 9% | 91% |
| MSN | 5% | 95% |
| ebay | 5% | 95% |
| Amazon | 38% | 62% |
| YouTube | 9% | 91% |
| CNN | 15% | 85% |
Note: Times are for page loads with an empty cache over Comcast cable modem (~2.5 mbps).
Shouldn’t everything be saved in the browser’s cache anyway?
The conclusion is the same: Reducing the number of HTTP requests has the biggest impact on reducing response time and is often the easiest performance improvement to make. In the next article we’ll look at the impact of caching, and some surprising real-world findings.
Disclaimer: Design imperatives dictating visual richness need to be weighed against this request-reduction goal. When you need visual richness, additional steps can be taken — aggregating JS files, using CSS sprites, etc. — but visual richness does tend to run counter to a slender HTTP request pipeline.
December 18, 2006 at 10:12 pm
Good post on Yahoo! User Interface blog on web site performance…
There’s a good post on web site performance on the Yahoo! User Interface blog. It says it’s part one…
December 18, 2006 at 11:38 pm
Good post on Yahoo! User Interface blog on web site performance…
There’s a good post on web site performance on the Yahoo! User Interface blog. It says it’s part one…
January 5, 2007 at 1:50 pm
[...] An entertaining pair of posts from the Yahoo! User Interface group on optimizing performance (via Noah Lockwood). [...]
January 7, 2007 at 4:37 am
Thanks for the nice article.
We should use HttpWatch in IE for considering the response time.Fiddler Tool is the another one.But Fiddler works as a proxy.
January 8, 2007 at 11:07 am
[...] Tenni Theurer, a performance engineer at Yahoo!, has written a post on What the 80/20 Rule Tells Us about Reducing HTTP Requests. [...]
January 15, 2007 at 10:09 pm
[...] Performance Research, Part 1: What the 80/20 Rule Tells Us about Reducing HTTP Requests » Yahoo! User Interface Blog The conclusion is the same: Reducing the number of HTTP requests has the biggest impact on reducing response time and is often the easiest performance improvement to make. In the next article we’ll look at the impact of caching, and some surprising real (tags: performance optimization http yahoo) « links for 2006-12-12 | links for 2006-12-15 » [...]
January 25, 2007 at 1:39 pm
[...] The arch enemy of performance are HTTP requests. Many browser still can’t handle more than two or four requests at a time. Keep the number of files down, your website will be faster. [...]
January 25, 2007 at 10:54 pm
[...] Martin Kliehm hat in einem interessanten Beitrag einen Teil seiner Erkenntnisse über Webseitenoptimierung zusammengetragen. Ich finde, für fortgeschrittene Webentwickler und für Profis ist dieses Thema ein Muß und der Artikel ein wirklich guter Einstieg. Interessanterweise erscheinen immer mehr Artikel zu diesem Thema und gerade Yahoo! hat in seinem YUI-Blog ein paar interessante Beiträge dazu gehabt. Die beiden Artikel über den Cache [1,2] fand ich sehr bemerkenswert. Und so manche Erkenntnis rüttelt mittlerweile an liebgewonnenem Best-Practice. Ich denke, wir haben in dieser Richtung noch einiges zu lernen. Viel Umdenken wird dafür vonnöten sein. [...]
February 8, 2007 at 2:10 pm
[...] Wer mag kann inzwischen auch mal bei meinem Bruder, CSSImport, CSSBeauty, CSSGuy ,F-LOG-GE oder Alistapart vorbeigucken, für Zwecke der Seitenoptimierung empfehlen sich noch diese, diese und diese Seite – seeehr interessant. ersteres bei Interesse an Bergen, schönen Bildern und Tourberichten, letztere für CSS/Webwork-Interessierte. Und weil es immer iweder Spass macht: Beetlebum. [...]
February 12, 2007 at 12:42 am
[...] Minimize Client Requests: Yahoo! pre-compiles some of their YUI libraries so that developers can just include one or two files in their apps and get the functionality they need, and we do the same thing with most of our libraries / applications. Separating thousands of lines of JS into separate files can really help with organizational purposes, but serving those multiple files to the client can really be a drag to visitors, especially in light in of studies like these. What we’ve done to help fix this is create a quick server-side app that reads an XML config file to determine which JS files to include in the single request. We can then ask for something like /CompressedScripts/Prototype.js to get our Prototype.js file, the Event.onDOMReady file, Justin Palmer’s awesome EventSelectors extension, something we wrote for generating SOAP requests, and Sylvain Zimmer’s impressive $$ addon. That’s five potential requests that we’ve merged into one request, saving an incredible amount of unnecessary requests over the lifetime of our app. I should note that we don’t use versioning (point 4) on our libraries just because it would probably degrade performance most of the time since the libraries rarely change. [...]
February 13, 2007 at 12:03 am
[...] A JavaScript library is definitely the way to go for things like Animation effects, rich controls and wrapping the XMLHttpRequest object. Sure, you can do this yourself but someone else has already resolved the cross-browser issues, tested in terms of performance and stability and the library probably has several people constantly improving it. For these scenarios the slight overhead of an extra HTTP request is worth it, but only if the library is modular so you are only including the code you need! [...]
February 14, 2007 at 7:21 am
[...] One small additional little .js.php only for the data you need? Look, I said “no”! .js.php will require a new database connection, since it’s a seperate request. Performance will suffer. In addition it’s a new HTTP request and those are expensive. [...]
March 7, 2007 at 12:53 pm
[...] fonte Dove viene speso il tempo durante il download di una pagina? Nello scaricare le immagini ;-) Battute a parte… [...]
March 8, 2007 at 7:52 am
[...] So, I was reading a blog post on the Yahoo UI blog, Performance Research, Part 1: What the 80/20 Rule Tells Us about Reducing HTTP Requests which basically argues that you can significantly improve the download time of your pages by reducing the number of HTTP requests that are made to your web servers. [...]
March 8, 2007 at 12:28 pm
Where can I find a program/site that will give me the information about my site? I believe you referred to it as a “packet sniffer”. I would like to create a similar chart as in Figure 1, as a diagnostic for different pages on my site. Are there any free tools out there or was that chart using some proprietary yahoo diagnostic?
March 13, 2007 at 2:03 pm
@Brian: I typically use IBM Page Detailer as my “packet sniffer” of choice. It’s the best tool for showing the components of an HTML page and you can get a 90-day trial version. There are many other similar tools out there such as LiveHTTPHeaders and Microsoft Fiddler, and the latest Firebug “Net” tab.
April 19, 2007 at 12:16 pm
Web 2.0 Expo group on slideshare has cool stuff!…
The web 2.0 Expo group on slideshare already has some nice stuff in it. Some highlights: Casual privacy from kellan (at Yahoo!) High performance web sites from Steve Souders and Tenni Theurer (also at Yahoo!). Are agile projects doomed to……
April 19, 2007 at 12:42 pm
[...] Some highlights: Casual privacy from kellan (at Yahoo!) High performance web sites from Steve Souders and Tenni Theurer (also at Yahoo!). Are agile projects doomed to half-baked design? from Alex Chaffee and Leslie Chicoine. Usability testing in the wild from Andre Charland Posted by jon Filed in Uncategorized [...]
April 30, 2007 at 11:31 am
[...] Il numero dei componenti da scaricare in una pagina è il parametro che più influenza i tempi di scaricamento della pagina. [...]
May 9, 2007 at 12:25 pm
Ask10 Performance-Other Improvement Tasks…
To Do Utilize the same images for the top, left, right and bottom shadow effects on suggests, right rail sa and others. Defer the loading of the compete js and compress the compete js. Use the same button image for cross and close…….
May 12, 2007 at 5:00 am
[...] YUIBlog: http://yuiblog.com/blog/2006/11/28/performance-research-part-1/ [...]
May 12, 2007 at 2:06 pm
[...] For example, the a November post made me think about my techniques a bit differently — they pointed out that having a low-weight page is nice, but the speed experience is much more greatly affected by the number of elements on the page. An understanding of latency vs. bandwith makes this point even clearer, once you consider it. [...]
May 14, 2007 at 5:02 am
[...] Their four-part article on reducing HTTP requests is lengthy, and may contain more details than you you’ll care about, so to break it down, here’s the key point in regards to this write-up: “Reducing the number of HTTP requests has the biggest impact on reducing response time and is often the easiest performance improvement to make.” In the context of Evolved CSS, we can make a large impact by using PHP/ASP/JSP et al to take all of our style sheets and combine them into a single document. [...]
May 15, 2007 at 2:02 pm
[...] Also, Tenni Theurer has written a great series of articles about some of our research on the Yahoo! User Interface Blog. I’m presenting at OSCon in July. This high level of interest in improving web performance is [...]
May 23, 2007 at 9:05 am
[...] YUIBlog: http://yuiblog.com/blog/2006/11/28/performance-research-part-1/ [...]
May 25, 2007 at 6:55 am
[...] and scriptaculous javascript libraries which also consists of a couple of different files. Now that an article about the same problem featured on the Yahoo! User Interface blog, I decided to make my solution [...]
May 25, 2007 at 8:52 pm
I made a post on my blog about the increase in page load speeds I saw while concatenating YUI scripts…
http://rob.cogit8.org/blog/2007/may/08/reducing-http-requests-using-make/
July 16, 2007 at 10:20 pm
[...] includes. At times one plugin will even depend on another adding even more server requests. A revealing YUI post recently discussed the negative performance impact of so many [...]
July 17, 2007 at 12:06 am
Enabling keepalive on the server side will reduce HTTP overhead as it will allow keepalive supporting browsers (pretty much every modern browser) to reuse an open HTTP socket to send multiple assets instead of closing and reopening the connection for each asset being downloaded. Of course this doesn’t reduce the bandwidth requirements of the page but something else that might is the use of gzip compression of the text files being sent. Apache has a module to do this dynamicly and you oculd also staticly gzip content that doesn’t change.
July 23, 2007 at 8:02 pm
[...] it can dramatically improve the performance of a site by reducing the number of HTTP requests (see this article for more information). Alpha transparency is also quickly becoming a favorite feature among web [...]
July 25, 2007 at 2:15 pm
[...] tags using the Apache server setttings. According to research at Yahoo they have come up with the 80/20 rule of reducing HTTP traffic. Since browsers spend 80% of the time fetching external components [...]
August 1, 2007 at 10:33 am
[...] Performance Research, Part 1: What the 80/20 Rule Tells Us about Reducing HTTP Requests [...]
August 6, 2007 at 12:04 am
[...] at Yahoo, Google and elsewhere have found that as much as 80% of the user’s time can be spent waiting for things other than the main HTML to …. Furthermore, browsers may not be caching as many files as you [...]
September 4, 2007 at 7:36 am
[...] year, Steve has worked with Tenni Theurer (author of a four-part YUIBlog series on performance: part one | part two | part three | part four) and their Exceptional Performance team to hone the message. [...]
September 6, 2007 at 12:44 pm
The chart posted about shows that 3 images and a script are taking the lion’s share of the time. While it is not specified, I would be willing to bet that some if not all the slow loading images are ad related. Fetching banners takes longer than get a simple image call. Also, a script at the bottom of the page tends to be tracking related.
How much of the that excess time is loading content that is not the real content of the site but the ads? My experience, sorry no graphs, suggests that ads are a major contributor to user perceivable slowness. Try loading a page in Firefox with AdBlock and without and see the difference for yourself.
September 17, 2007 at 7:31 am
There’s a good post on web site performance on the Yahoo! User Interface blog. It says it’s part one…
September 19, 2007 at 9:13 pm
[...] What the 80/20 Rule Tells Us about Reducing HTTP Requests [...]
September 24, 2007 at 10:49 pm
[...] Those organizations that are spending hundreds of thousands of dollars on speeding up their sites are probably looking in the wrong spots for the speed improvements. They are probably spending their time with expensive efforts to increase their server and database speed while ignoring the reality of Yahoo’s 80/20 Rule that the majority of the savings come from frontend design decisions. [...]
September 29, 2007 at 9:11 pm
[...] is consistent with Yahoo’s 80/20 rule and indicates that the biggest benefit will come from optimizing front-end [...]
October 2, 2007 at 6:53 am
[...] for Improving Web Site Performance 7 Easy-to-Apply Tips to Improve Your Web Site Performance Performance Research, Part 1: What the 80/20 Rule Tells Us about Reducing HTTP Requests Firebug [...]
October 6, 2007 at 7:24 am
I think these blog is really useful for new comers and Excellent resource list.
It´s a very interesting Blog and simple answer of many questions.
Keep up the good work!
October 9, 2007 at 1:22 pm
[...] properties. It is part art and part science. Let’s look at the science. He writes in his Yahoo! User Interface Blog: “When we optimize our applications, we know to focus on that 20% of the code. This same [...]
October 10, 2007 at 3:02 pm
[...] good folks over at the YUI blog posted this: What the 80/20 Rule Tells Us about Reducing HTTP Requests a while ago. I bookmarked it, but wanted to point it out to other folks–it’s a nice bit [...]
October 16, 2007 at 10:07 pm
[...] got an HTML page that includes a bunch of Javascript files, which makes development easy but which hurts performance in [...]
October 25, 2007 at 8:19 am
[...] fewer HTTP requests turns out to be the most important optimization technique, with the biggest impact. If your time is limited, and you can only complete one optimization task, pick this one. HTTP [...]
October 26, 2007 at 1:01 pm
Where can I find a program/site that will give me the information about my site? I believe you referred to it as a “packet sniffer”. I would like to create a similar chart as in Figure 1, as a diagnostic for different pages on my site. Are there any free tools out there or was that chart using some proprietary yahoo diagnostic..
November 7, 2007 at 12:37 pm
The web 2.0 Expo group on slideshare already has some nice stuff in it. Some highlights: Casual privacy from kellan (at Yahoo!) High performance web sites from Steve Souders and Tenni Theurer (also at Yahoo!).
November 15, 2007 at 8:41 am
[...] look at a quote from the article “Performance Research, Part 1: What the 80/20 Rule Tells Us about Reducing HTTP Requests” by Tenni Theurer on the Yahoo! User Interface Blog. Table 1 shows popular web sites [...]
November 29, 2007 at 5:04 am
[...] Использование множества отдельных изображений довольно привычный способ и наиболее часто используется в процессе создания сайта. Вырезали из дизайна огромное количество иконок и вставили во все необходимые элементы. На первый взгляд создается впечатление, что страница грузится быстрее; графические элементы появляются один за другим, постоянно создавая дополнительные HTTP запросы, что далеко не улучшает эффективность вашей страницы (Оптимизация веб сайтов Web Site Optimization: 13 Simple Steps и Performance Research, Part 1: What the 80/20 Rule Tells Us about Reducing HTTP Requests). [...]
December 9, 2007 at 4:35 am
[...] and scriptaculous javascript libraries which also consists of a couple of different files. Now that an article about the same problem featured on the Yahoo! User Interface blog, I decided to make my solution [...]