Thursday, September 20, 2007

Testing 3G/EVDO Device Speeds

The speed or throughput of every EVDO device ever released has been called into question by countless people. Everyone wants there to be a "overall fastest device", and they want us to tell them which one it is and then back it up claims with irrefutable evidence.

If we felt that we could, we would stick our necks out there and establish our opinion on what device is the fastest... but even we cannot come up with any repeatable evidence of any device being capable of consistently faster throughput than any other device.

Each device has to go through many carrier certification processes to justify their claim as being a Rev-0 or Rev-A device. Once a device is certified, it has proven itself capable of attaining the theoretical top speed of either specification. Even the FCC gets involved in these certification processes since EVDO is radio/cellular technology.

Speed of any device is most often a question of signal strength, cell tower load, and overall carrier network load. Rarely (if ever) is realized speed a result of the brand or model of device.

The only true way anyone could pass judgment on which device is superior to another would be by using third-party "closed loop" testing equipment which would eliminate all other potential traffic from affecting results. This kind of equipment and testing environment is out of reach of most people, and instead, everyone has their favorite online speed test website.

Our favorite speed test sites are BroadBandReports.com and Speedtest.net, but they aren't favorites because we think they are the most accurate... We like them because they are simple to use, provide historical archives and comparisons, and provide end users with a graphic that includes test results and can be passed along as evidence/proof of test results.

These and all other online test sites operate in similar ways. They run a script that times how long it takes to download a specific amount of non-compressable data from their server to your computer and calculates the speed of your connection based on the amount of time it takes for the test to complete. During that test, your connection could easily be passing through 10-20-30 or more 'hops', each one suffering their own network load, and each one contributing to the overall speed and latency of your individual speed test.

Because of the nature of what happens during these online speed tests, even multiple consecutive tests will show varied results. Sometimes incredulously.

Using these online speed tests, there has never been any concrete evidence that one model of 3G device is consistently and repeatedly faster than another similar device. Even speedtests recorded on EVDOmaps by the over 10,000 members of EVDOforums provide only one conclusion...

One person's fast device, is just as easily another person's dog