Yahoo! User Interface Blog (YUIBlog)

Analyzing Bandwidth and Latency on YUIBlog.com

April 8, 2010 at 7:38 am by Philip Tellis | In Development | 8 Comments

A few months ago, Eric helped me run some analysis on the types of network connections that were coming in to YUIBlog. In this article, I’ll analyze the data we collected and look for insights about the tools and infrastructure the YUIBlog readership uses to browse the web. Being the performance guy at Yahoo!, my interest focuses how fast our readers connections are. In particular, with what kind of network bandwidth and latency do YUI geeks experience the web? Although your mileage will vary, YUIBlog is likely to be representative of US tech blogs read by people in the software industry, and the data here may prove useful for those of you catering to a similar audience.

Caveats

Before I go any further, I have a few caveats to list. The tests and analysis we ran were based in the US (since that’s where YUIBlog is hosted), so if you view this site from outside the US, your measured experience will probably be a little slower than what you’d see with local sites. Secondly, while we see traffic from all over the world, I’ll only look at countries with a statistically relevant sample size. Lastly, each individual IP was considered only once. If the same IP accessed the site more than once, we consider the arithmetic mean of values from each access. For overall summaries, we use the median. No statistical filtering was applied to the data before analysis. The total sample size was approximately 27,000 points.

The tests

The test measured HTTP bandwidth and HTTP latency between an end user’s computer (or a proxy somewhere in between) and YUIBlog. Note that I say HTTP bandwidth and latency because these may be different (i.e., worse) from the actual bandwidth that your ISP provides you, but it’s what we, as frontend engineers, care about more. Latency is the time between a request going out and the first byte of the response coming back. It depends on how far you are from the server and the number and quality of hops between you and the server, so if you’re far away, you’ll have high latency. HTTP latency also depends on the overhead of HTTP headers, but we’ve tried to keep that down, and on how long it takes for the web server to respond to a request. We think that’s fairly fast.

Bandwidth depends on the bandwidth at every step between you and the server, and it’s somewhat affected by latency as well. The measured bandwidth is also affected by other applications or devices using the network in parallel. If you’re streaming movies or loading an image heavy site while a measurement is taken, your bandwidth will appear lower than it actually is. However, this is the effective bandwidth that you browse YUIBlog with, so it’s what we care about.

So, what kind of networks do YUI geeks browse through?

Quick summary answer: The bandwidth of YUI geeks is 1 Mbps (that’s Megabits) and the latency is 262 ms. Most of you are either on Broadband, DSL or Cable connections and come in from the US, Canada, Great Britain, Germany and India (ordered by distance from our server). For more detailed results, read on…

Connection types

The type of network connection you have with your ISP determines the primary bandwidth limit and secondary latency limit (the first being distance divided by the 66% speed of light). Not surprisingly most of you are very closely associated with the internet, and use Broadband data connections. Only 2% of you used your mobile phones to reach the site, but that’s still more than the number of people on dial-up.

Of all the connection types we’ve seen, Cable and T1 offer the best bandwidth and latency, going over four times as fast as Broadband with half the latency. This could possibly mean that most of these connections were within the US while the others span the globe. The low latency seems to suggest that, but drilling down reduces the sample size too much to make an accurate assessment. From the data we see a clear inverse relationship between bandwidth and latency.

International data

We expected connections from outside the US to appear slower because the test has an intrinsic geographical bias, but what does the data really tell us?

The US and Canada top the list as expected. All other countries with a bandwidth over 1 Mbps are from Europe. Looking at the other end of the spectrum, we see a few surprises.

I know from recent experience that Korea and Hong Kong have really high bandwidth network connections. Faster than most of the US in fact. The data puts them at less than one sixth the bandwidth of the US. A possible explanation is the latency. As we already know, there’s a correlation between bandwidth and latency. Looking at the latency numbers for these countries gives us a possible answer.

We get hits from many more countries (147 in all). They either fell between these two blocks or had too few data points to be a statistically sound sample.

ISPs

As the final piece of analysis, I looked at ISP information. I only looked at a few countries since there were too many ISPs world-wide to put on a bar chart. Presented in the chart below is data for US and Canadian ISPs.

I’m not sure what the difference between AT&T Worldnet and AT&T Internet is, but I’ll leave that for you to figure out. We saw more hits from Comcast than from any other ISP on the planet.

Users of different browsers

The browser you use doesn’t affect your bandwidth (at least hasn’t in lab tests) and has a very negligible effect on latency, so I didn’t expect to see any differences here. I was curious though, to see what the data said, and this is what I found. I’ve included iPhones and iPods as well as they made up a sizeable fraction of all data points. Opera’s mobile browsers are included under Opera. The “other” category only included 22 data points.

The iPhone’s latency is close to other mobile phones. All other browsers have the same latency. This is as expected. Bandwidth tells a different story though. It appears that those of you with the fastest internet connections use a Webkit based browser like Safari, Chrome or Konqueror, while users with the slowest internet connections use Internet Explorer.

Summary

• The overall bandwidth of YUIBlog readers is 1 Mbps and the overall latency is 262ms
• We see a definite correlation between latency and bandwidth, so it would be interesting to see what the measured bandwidth is after eliminating delays caused by latency. Perhaps in a future test I’ll do that.
• Putting a mirror in Asia probably makes sense for a US-based engineering blog like this one. Putting one in Europe probably doesn’t.
• Mobile users have terrible latency, so a site optimised for mobile should really reduce the number of HTTP requests it makes. Note that mobile users that surf the web over wifi experience far lower latency.

Find conversations: bloglines | technorati

Share and extend: Bookmark with del.icio.us | digg it! | reddit!