One of the things a UX researcher dreams about is a ‘usability lab’. As a student, I learned there are many ways to enhance a test by adding more dimensions to it in order to (objectively) measure the user experience. Formal labs are usually equipped with fancy (and expensive) tools to do just that, from eye-tracking to bio-feedback sensors and monitoring systems. Depending on what you need to test, you can have labs that look like plain rooms, perhaps just having a table inside, or you can also have themed rooms – for example a cosy living room or a sterile hospital room. And that can be very useful if you want to test, say, a new television (in its context of intended use, a living room), or a medical device (in a hospital surgery). Many of these facilities are like those interrogation rooms that we see in CSI: one main room where the interviewer and the interviewee sit (no good cop/bad cop play though), with a two-way mirror allowing anonymous observers to see the process from the adjacent room.
But what about testing mobile apps?
If you have ever visited MOBGEN, then you would know that we do various types of usability testing, but… there is no defined ‘lab’ space. This is because we believe a mobile app is not a static, autonomous product to be tested within the boundaries of a room (like a television). As Erika Hall mentions in her book:
- “We live in the future. There is no reason to test in anything called a ‘usability lab’, unless there’s a danger your experiment will escape and start wreaking havoc. A usability lab gives you the illusion of control when what you are trying to find out is how well your ideas work in the wild.”
Apps have their own ecosystem, and the context in which they will be used depends not only on the purpose of the app itself, but also on the people that hold that mobile device in their hands. And that context is always unpredictable. We may design for a purpose and an intended use, but it is a messy world. Just because a mobile is… mobile, people might be anywhere and under any circumstance. It might be that your app will be used by someone that has been trying to find a parking spot for an hour, holding a baby that is annoyed and crying, pushing a trolley along a supermarket corridor while manipulating a mobile ‘to buy list’ so that they don’t forget anything that needs to be bought for that week. Sure, you can test how clearly information is perceived within a formal lab, but what have you discovered about the context and the unintended usage of that product?
So what we prefer doing here at MOBGEN is to evaluate every time what kind of context to include in our tests, and what kind of experience the user might need to go through.
With that in mind, some of our tests still take place in our office using phones and prototypes, with cameras to document the interview. Other tests take place outside, in a car, a music studio, or in a gas station; wherever they need to be in order to be realistic. This way we can see how the digital artefact that we designed merges with the real world. Since the time that phones became smart, interactions are not just those that happen when tapping at your screen, so we want to observe the whole range of them and discover mental patterns in how people think.
What we also want to get right is the experience the participants will go through. So besides designing the usability of an application, we also design the tests themselves. Once, I was observing a usability testing within a formal lab (yes, behind that fancy mirror), the participant walked in, looked around, saw the desk loaded with an eye tracking device and said “Wow, what is that? What are you gonna do to me?” By avoiding this kind of lab environment we can create a more casual approach, making the participants feel relaxed and far less stressed. Although this strategy can sometimes be perceived as ‘unscientific’, it is much more natural, and it lowers the barriers people usually have about being really, really, really honest with giving feedback for your designs. ‘Scientific’ labs and formal places (with big names) tend to influence people to think that they have to act in a certain way, to say only good things about your designs. But we love and need critical feedback!
In another situation, we had to activate the internal (audio) communication between rooms in order to ask the participant to elaborate on an answer. The user hears the question, smiles, and says “thank you voice from above!”. It seems that many people are not fully aware of the capabilities of such rooms, and it is always tricky: do I let my participant know that they are being observed on the ‘other side’ (so that they are aware of the conditions), or do I let them think they aren’t being watched (so that they behave naturally)? Sure, this can create funny moments, but how does the participant actually feel? It is not an everyday situation, so it can potentially lead to additional stress which can colour your test results.
In short, formal usability labs have their place and can be useful, but they need to be considered as a tool, not a rule. The best way to test a product is contextually, in the environment that it will eventually be used. If that product is an app, chances are that it is not designed to be used within an interrogation room!
P.S. Did you know that you can become a participant in MOBGEN’s usability lab without a lab? Check it out!
Reporting live, from our lab without a lab, over.