Building a device lab

At the Money Advice Service we’ve been working on a series of web improvement initiatives. This has included reviewing our existing IA, site search strategy and also building a new responsive front end. A lot of this work has been live in ‘beta’ for a few months now which has allowed us to continually improve based on actual usage.

One of our design principles was a ‘mobile first’ approach and to support this we decided to invest in a physical device lab to aid development and testing. A while ago I wrote a blog post about this, and it seemed fitting to also include it here.

Building a device lab

For the past few months the tools we’ve developed in-house have been built in a responsive manner, however
once deployed they’ve lived within an unresponsive frame. The enhancements from the web improvements initiative
will change this, and along with our goal of being ‘mobile first’ we’ve been thinking about the tools we
need to help design, develop and test across small screen devices.

One of the things we’re doing is building a physical device lab that’ll complement the use of online emulators. This is at a fairly early stage, but over the course of the next few weeks we’ll be moving forward and I’ll update with progress. For now I thought it might be useful to write some notes around why we’re building a device lab, how we decided on the devices to start with, and what our aspirations are for the future.

Why

There are a number of reasons why we’ve opted to invest in a physical device lab over relying purely on emulators. Emulators definitely have a place and will be a part of our workflow. They provide good coverage and can be quick and easy to use, especially on static, content based sites. However having access to the physical device provides a number of advantages:

1 – Firstly, it’ll be no great surprise that the number of visits to the site on small screen devices has increased rapidly. Looking at analytics data* there’s been an average increase of 398% year-on-year. The increased usage meant that optimising the site for small screen devices became a growing priority, and with that the need for more effective testing.

2 – an emulator can replicate how a site will look given a device’s particular capabilities (screen resolution, browser support etc) but some of the subtilise around how the site will behaviour on a device can be lost, for example the responsiveness of a touch screen or having to navigate using a trackball or keys.

3 – the third advantage is that having physical devices available on site really re-enforces the mobile-first mindset right across the team, from product owners, to designers, to dev and QA. It means that everyone can look at the latest product developments across a variety of devices rather than just the thing that they have in their pocket or on their desk.

Getting the device lab set-up

Having decided that we wanted to a physical device lab the next question was what devices to get. Thankfully the interwebs came to the rescue and we were able to look at plenty of useful resources from the likes of Jeremy Keith, PPK and Stephanie Rieger. They’ve all written a bunch of stuff on the topic, and are much more knowledgable than me, but I figured I’d summarise a few of the things we found useful.

1 – in the first instance we looked at existing traffic to the site to see what the most common devices, OS, and browsers were within our existing audience. The data showed us that when it came to small screen devices the vast majority of traffic came via Apple iPad and Apple iPhone devices (36% and 33% respectively) with a further 7 devices making up 80% of our total mobile traffic. Looking the operating systems, 80% of our mobile traffic came from 12 devices, of which 9 were different variants of Apple’s iOS. The browsers hitting the site also told a very similar story.

2 – looking at out own data analytics provided a great snapshot of our immediate audience, however we also wanted to consider the wider device/browser landscape. At the Money Advice Service our target audience is the whole of the UK and our goal is to make everything we produce available to as many people, in as many ways, as possible. Previous experience has shown that in making a site optimised for mobile then traffic from mobile devices is likely to increase and broaden. To help get a wider view we looked at data from GDS and data.gov.uk.

3 – with a view of device traffic to both our site and our target audience we were confronted with a fairly daunting list of potential devices. Through looking at existing device labs we were able to take some inspiration as to the best combination of devices for us to start out with. Combing this information from the data we’d captured from the first two points gave us a good starting point.

4 – finally, we looked at the devices we already had in the office. Having a set budget we wanted to be frugal with our spending. The most common devices we wanted to form part of a dedicate lab, but some of the less popular devices we didn’t want to duplicate what we already had in the office knowing they could probably be called on for testing purposes as and when needed.

Where we are now, and where we want to be

Right now we’re at the point of starting to collect devices. Newer ones have been ordered, and for some of the older models we’re scouring eBay and 2nd-hand electronic shops. By Christmas we should have the basis of a good device lab.

We’re also starting to figure out a way of making testing as easy as possible. We’ll probably using Adobe Edge Inspect and Forward to help with this, but very keen to get any tips or suggestions! There’s also the logistics of how to manage devices, store them, and maintain the right version of operating system. As we figure some of this stuff out I plan to write a follow-up post on some lessons learnt.

Our aspiration for the future is to be able to open up our device lab so that others can also benefit from it too. The potential of being able to share access to devices and with that learn from each other about problems we’re facing and ways to overcome them is hugely exciting. We’re in the process of starting an office refurbishment that’s due to be completed in the spring of 2014, so hopefully that’ll give us the opportunity to set something up for then.

The original post was written at Matt Lucht’s blog.

* data taken from April 2010 through November 2013


About mattlucht

Tech BA at the Money Advice Service

Leave a Reply

Your email address will not be published. Required fields are marked *