Comparing speed tests
We get a lot of questions from people who want to know how a SamKnows’ speed test differs from a standard online speed test – so, we're here to tell you!
Firstly, SamKnows has many different types of speed tests that are designed to measure different parts of network infrastructure, including some that are the same as standard speed tests. A big difference between these tests is purely location! Where you measure is as important as what you measure. Let’s take a look at 3 different types of speed tests and explain what each of them is designed to measure, as well as what we can learn by comparing them against each other.
This diagram shows the differences between the 3 speed tests which we’re going to compare. The tests themselves are exactly the same, but the difference is the start and end points of each test.
- - - - - -
Standard speed test
A standard speed test runs from a browser/device over the user's home network out to the nearest test server. This type of test is designed to measure the available speed on the test device.
Some of limitations of this type of test are:
- If the device is connected via Wi-Fi, performance will be variable depending on how close the test device is to the router.
- Even if the device is hardwired to the router, there could be network activity from other devices connected to the router which would distort the results.
- On-net test servers are typically located inside the ISPs network and optimised for speed, performance to a on-net test server isn’t representative of real-world internet performance.
- If the broadband service being measured is 1Gbps or more, even if the device is hardwired, the fastest speed it could measure is 940Mbps due to the limits of the network interface port.
- Standard speed tests are self initiated, meaning the results are a self-selecting sample. Users will often only run a test when they think they have a problem, which can distort results when looking at an aggregated data set.
- - - - - -
Router speed test
The Router test is SamKnows most commonly-used test. It tests from the router to an off-net test server on the internet. This type of test is designed to measure the service provided to the home. Rather than being self-initiated, SamKnows schedules router tests throughout the day to capture a granular understanding of performance to the home.
Some advantages of this type of test are:
- Testing at the router – the broadband access point inside the home – eliminates any unknown variables inside the home.
- The router test listens for background traffic and only tests when the connection is idle, this ensures the measurements aren’t obscured any other network activity.
- Testing against off-net test servers in a neutral location on the internet is more representative of real-world internet performance. Also this allows us to fairly compare different ISPs performance for our national studies.
- With a SamKnows router agent we are able to measure speeds beyond 1Gbps.
- - - - - -
RealSpeed test
RealSpeed runs two concurrent tests, the first from the router to an off-net test server, the second from the user's device over the home network and out to the test server. This type of test is designed to disambiguate between a service provided and home network performance.
Some advantages of the this type of test are:
- By comparing the two results you can calculate how much performance is lost within the home, this can be useful for troubleshooting and optimising home Wi-Fi.
- Many broadband users don’t understand that Wi-Fi performance is variable around their homes, this test helps educate them by visualising the difference.
- The test device itself will have a fixed speed limit due to hardware and software configurations. Realspeed can alert the user when a result has been limited by the device and isn’t a true reflection of the available speed.
- - - - - -
Different strokes for different folks
In summary, different test types are designed to measure different things, there’s no one test that’s better than all the others. It’s important to choose the appropriate test for the intended purpose. There’s a common misconception that a standard speed test is measuring the service provided to the home. Under ideal conditions it potentially could, but conditions are rarely ideal, that’s why we developed the router test to more accurately measure service provided to the home, and the RealSpeed test to disambiguate between the available speed inside the home and the service provided to the router.