One of my favorite usability analysis activities to do is the Field Study. Talking to real users of your products in any capacity… on the phone, at a party, in a formal or informal test session… anywhere, is always interesting and insightful, but nothing is more revealing than watching them in their natural habitat.
In a field study (similar in theory to ethnographic observation), you will go where your user uses the product, and simply observe them while they use it.
Your goal is to try to remain as much in the background as possible, not to interrupt or direct the process at all.
You might occasionally ask a question, but I would recommend using a lot of restraint and instead of asking every question that comes to mind, write them down and ask later. You may get the question answered anyway or decide it’s not important based on what else you observe them doing.
Here’s how I approach doing a field study:
- Choose a user and workplace to focus on that is most representative of your end user audience. You don’t need to observe more than one user to begin with, unlike formal user testing where we do 3-5 test sessions. I will explain that following this list.
- Ask the user to choose a time that works for them, about an hour or two, when their workplace will be busy enough to be normal, but not so hectic that you are in the way. Explain that you just want to observe them using the product naturally, with whatever interruptions and other tasks they do at the same time, as normally as possible given you are watching. Always emphasize that the product’s usability and intuitive flow is what you are trying to determine… this is not a “user test” of their skills using the product and you should not view it that way. Tell them you know the product isn’t perfect and you’re trying to see where any areas of stoppages occur, or any technical breakdowns happen.
- Make sure they have permission from their boss for you to come on-site and observe them.
- I have done half-day studies before but that is a bit long. If you go an hour or two before lunch, you could plan to observe them and then take the user to lunch with your notes, where you can ask any questions and have candid conversations at that time. You could ask to record (with audio) the lunch conversation so you don’t have to worry with notes.
- Don’t worry about recording anything – just take notes in the fastest way you can do so, which might be on your phone, on a tablet, or on paper. (I use paper and pen or pencil.)
- If you can’t take them to lunch, plan a few minutes to wrap up the session where you can ask your questions. In formal usability testing it is common to give a stipend (cash or something like a gift card) and you should do the same here, making sure they have no conflict with the business they are employed by. If accepting a personal gift is a problem, take something like a fruit basket or box of candy or something the whole office/workplace can enjoy together.
- Ensure the user that anything you observe or talk about with them regarding their workplace will remain 100% confidential. You may see real problems in the organization that impact the user’s interaction with your software or hardware – but you are not there to do an expose on the company. If you witness problems try to discuss them more vaguely back with your co-workers and make sure you don’t wind up spilling another company’s secrets.
- A field study is not like a formal user test – this is not to be scripted or guided. If your product contains many features you may discuss only observing something specific, like installation for example, and go when the user is doing that. With enterprise software, I have often observed installations which can take more time, and then on another day users inside the product working on tasks. An installation turning into a half-day or all-day observation has happened to me before, and indicates some uxp and technical work remains to be done.
- Be sure and send a thank you email or note following the study and let the user know they have helped the product design process significantly by allowing you to observe them. You need to keep their information to follow up later.
- Use your notes and observations to have discussions with your team about any changes that need to be made and create new UXP Requirements for the next product release.
Now, here is the reason I feel you should only observe 1 (new) user at a time in a field study. In my experience field studies yield much more human factor data, like potential pitfalls or noticing tiny non-intuitive details, than a formal user test does. So taking all this information and processing it into actionable tasks by your larger team at work (which may involve customer service, developers, managers, distribution, or even sales and marketing) is going to be enough to help you improve the experience.
When you have processed this information and improved the product using the expertise of your team, go back to the first user and do another field study to see if these changes helped (or have a phone call with them to discuss the latest release and any areas that were of concern to see if they got better) and then observe a second user you have not observed yet, with this improved product. Repeat this process of iterative improvement, checking the original user and a new user, as long as you have concerns.
It has become almost a trend for software organizations to employ no user experience or human factors employees, and yet attempt to do usability processes, so they don’t have the security and confidence in their design and think they can test every little thing and the users will essentially help them design the product the right way. Though that might be better than nothing, that is a bit of a myth. Users do not design world-class products. A world-class designer and developers who have technical mastery do. So you need people on your team who are qualified to do this type of observation, or to hire a consultant (like me) to help you do this type of study. If you are small team it makes total sense to hire user experience help but not have an employee on staff every day.
I value direct observation of real people over popular A/B testing, heat maps and other digital methods of analysis some user experience focused vendors sell, that provide some helpful data but not truly the actual experience someone has using your product. There is no substitute for seeing a real user, with your product, in their natural environment at work, at home, in a coffee shop, in the car, on a boat –wherever they use it – actually using it. Their experience will include things out of your control, but all of that factors into their overall impression. If the computer’s internet connection times out and they lose a ton of data they put into fields, for example, that is a giant pain. Or they get interrupted too often while trying to focus. Or, and this happens to me constantly with sites, they fill out registration information and something goes wrong somewhere, and now they can’t login and do what they were trying to do (that happens in ecommerce buying flows if you don’t allow a guest checkout with no registration, too often.)
Observing people is the only way you can truly understand their experience of what you have created, so don’t be afraid to go deep into the trenches.
Special Note for UXP Practitioners: The purpose of the field studies I do is NOT to conduct “contextual observation.” In contextual observation the user talks about what they are doing while they are doing it, and that significantly alters the “natural” aspect of a field study. Instead I have them do that during a formal user test, but not when in their own environment. I try to strictly observe without questioning or guiding if possible (until after the observation.) If the user asks a question you do want to answer it though.