using that testing to learn and improve your knowledge of the various operating systems and browsers,
gaining empathy by seeing how users really experience your work.
Having the lab open to the community, and sharing knowledge, means you'll find fixes for your bugs more quickly, and that you'll know what to look out for next time.
If you'd like to come and do some testing, please make an appointment by dropping a line to [email protected].
Why test on actual devices?
More realistic testing
Testing on real devices gives the clearest picture of how the site will work “in the wild.” It allows for testing in a way that’s closest to how real users will interact with the site. One of the most important benefits of using real devices are the physical characteristics of them:
input methods such as fingers on a touch screen rather (are link targets big enough?);
the weight, size, form factor, and portability of a device (is the text well-sized with the screen up close?);
the quality and condition of the hardware (how well does the site work if the touch screen is not very responsive, is cracked, or dirty?);
capabilities: touch, GPS, accelerometer, etc. (can we use the device’s extra capabilities to add extra value for the user?)
More realistic testing isn’t the only benefit of using real devices. You learn more each time you use them, and become more efficient at knowing what to look out for next time. Exposure to more platforms and design approaches, Operating Systems, and browsers is also beneficial. It serves as a reminder of the wide range of use cases and that the site needs to be as lean and robust as possible to survive in whatever environment it is used in. It reminds us of a key fact: we are not our users.
Testing on emulators can be useful, and is certainly better than not testing for mobile at all, but is not as good as testing on real devices. Emulators are essentially ports of an Operating System / browser, so there is the chance of errors or differences in performance and behaviour. They also add a layer of abstraction from the experience that the user has, and another potential place for bugs to occur. They can be a good first test of layout, but can’t replicate a real user experience.
It would be impossible to test your site on every device that exists today, and the range and variation is only increasing. Testing on as many devices as possible can quickly become impractical. A pragmatic approach is to test on a representative sample of devices, covering as broad a range as possible.
The focus of development should be on long term functional support for as many devices as possible, followed by optimisation for higher-end devices. Achieving perfection for specific devices or platforms will only be a short term success, given the remarkable pace of change.
The devices in the lab are selected using high-level device categories: the specific devices in the lab aren’t as important as the categories that each belongs to. Some of the categories used are:
For now, testing is done via WiFi internet connections. Testing on cellular networks would be ideal, but isn't financially possible at the moment. If you have SIM cards that you'd like to use for testing purposes, you are very welcome to bring them along.