Managing stakeholders and observers during user testing
No. 6— How a UX Researcher can be their own Project Manager
This is the sixth in my series of 7 things UX Researchers (or Project Managers) can do to keep user testing on track.
These are 7 things that SOMEONE has to do. Whether or not it’s officially your responsibility as Researcher, depends on where you work and who you have around you. Nevertheless, someone has to get it done.
It might be you.
If you don’t take ownership, or you sit around waiting for someone to give you permission to do each and every task.. well just ask yourself this — if the research fails, who’s on the hook? Hint: it’s not the PM. There you go.
If you need other reasons why this is important — here’s the explanation.
Step 6 — Managing clients & observers (or, who can de-rail my test days)
In some roles you are a solo UX Researcher, left to your own devices. In which case this article is probably not for you, as you can quietly and calmly run sessions to your own schedule.
However, at some point you will have to manage a scenario where you have observers who are:
- Other UX people or product squad members
- Clients or stakeholders who are funding the project
Here are some things to think about to ensure that those two groups have a great experience on test days, and don’t disrupt or even accidentally ruin your test sessions.
Let’s take them one at a time.
1. Your team
Your team can be made up of other UX people, product squad team members — basically anyone who is working with you to interpret the findings and move the design project forward.
Here are some things to think about:
Where will they be observing from?
First up, what is their observer experience going to be? Where will they be observing from?
If remote, ensure the tech works in advance by running rehearsals.
If in person, make sure they have a room booked.
Just be sure no one suggests anything daft like having them in the same room with you. Nothing will intimidate a user more than seeing a load of people face-to-face noting everything they say and do.
What is their role on the day?
They should in theory be on your side, and wanting the research to have a successful outcome. They should be helpful. This means that they don’t just get to turn up on the day and enjoy the show. This is work; not entertainment.
These colleagues should be actively engaged in watching, note taking, post-it’ing and/or capturing verbatims and time stamps.
You could even see who you can recruit to help you with at least some of the PM tasks in this series!
Whatever the plan, ensure that you’ve discussed their role in advance, and been clear on your expectations from them.
How, if ever do you want them to input into the process?
Sometimes colleague observers can see something you don’t, or know about some other political thing going on within the business that suddenly becomes important while watching testing.
Obviously you don’t want them interrupting the live session, and I strongly advise against allowing them to WhatsApp or IM you during sessions as it is hugely distracting.
However, you could design the session to have a break mid-way through (see more on this below with stakeholders) or ensure that you have enough time between sessions to get feedback on the method.
If you have written and communicated your plan, rehearsed and set expectations for the day however, interruption really should not be needed.
Have a cadence to the day
I recommend having a plan for the day and sharing it in advance. This is even more important if testing across multiple days.
Ensure that colleagues know when they should be observing/taking notes, when is discussion time, when is break time. Example — if you only have 15 mins between sessions, that’s not enough time to have a bathroom break and also discuss anything urgent.
I also recommend that at the end of each day, even if testing for several days, you have a “what we think we have seen” session. This also helps you to understand where your colleagues are coming from, and starts everyone thinking at a very high level about the shape of your findings. (More coming up on that in Part 7 of this series)
Why this is important: Your team are critical to research success. They are at the same time your client, and your support. You need to ensure they have a role and assign responsibilities that will help you deliver the best results for them.
What can go wrong: If you have team observers who turn up but don’t really listen, treat it as ‘netflix’, don’t contribute to findings or worse- misunderstand and disrupt testing, then you are wasting everyone’s time and money. Never assume that a designer or PM has even observed testing before. Always communicate clearly and repeatedly and set expectations.
2. Your clients & stakeholders
If you’ve got clients or stakeholders attending who are not part of your day-to-day design team.. well done. You want to make sure these people have a great experience of testing to ensure future research opportunities however, you also don’t want them disrupting your tests.
The best way to manage this, as ever, is to prepare in advance and communicate your plans.
Where will they be observing from?
Where your stakeholders observe from can massively impact their experience, and the amount of work you have to do.
In a lab or office
If you’ve got a research lab, then you probably have an observation room already — that’s a great set up if you’re lucky enough to have one.
In this case, it’s just a matter of having someone coordinate arrival and get them settled. If you are hiring a lab, they will usually do this for you.
Also if in a lab, it can be a great idea to have someone with the clients/stakeholders in the observation room — setting expectations, answering questions and so forth. Use someone from your team if you can.
When I ran a research lab, I would often take this role in order to protect the time and focus of the researcher working in another part of the lab.
And of course you’ll need someone to make sure the tech is working for them — just remember that the most important thing is that the test is completed. I would never recommend stopping a test just because the observer tech broke.
Remote
If you have stakeholders dialling in remotely, you need to make sure that they have the correct links and know how and when to join the call.
Remote observation can be very convenient for stakeholders however it doesn’t really give them the optimal first experience of testing, nor the opportunity to ask questions if they don’t understand something about the process.
Again, testing the tech in advance is a good idea — and you can do this even if you don’t have access to the stakeholders themselves.
What is their role on the day?
Depending on location (as above) you might have stakeholders in front of you face-to-face, or dialling in. Either way, their role is mainly to observe and understand.
It can be stakeholders’ first time watching testing, so some advanced communication of what to expect is a good idea. Try to pre-empt questions they might have by referring to your well-written research plan.
You can use colleagues to help facilitate stakeholder discussion but just as a word of warning — having lengthy discussions while testing is happening means you’ll miss things, and possibly disturb others in the room who are trying to listen. Try to keep these conversations to the times between sessions.
How, if ever do you want them to input into the process?
Stakeholder input is great. However, sometimes stakeholders turn up to one session only, get fixated on one particular thing they see and then fail to be open to the wider findings. This is not their fault — they are human and they are busy.
Have a cadence to the day
Your primary cadence is concerned with yourself, your participants, your team and then your stakeholders (unless they are also attending all day, but in my experience they tend to drop in and out).
Ensure that your team observers (if you have some) know when a client or stakeholder may join, and have them ready to engage and help those observers understand. Use your colleagues and wider team wherever possible to help facilitate and look after your stakeholders. You will not have time on the day for a wide-ranging strategy discussion.
If there are discussions of findings in the room, but the stakeholders only stay for one or two sessions across many, consider sending updates after testing including overall how it went, and any bullets of key, very very high level themes that you saw to help client and stakeholder observers feel part of the process, and whet their appetite for any subsequent report or summary of findings.
Overall, the best approach is the following:
- Communicate your research plan before testing
- Provide a short bullet list of what to expect on the day
- Provide a schedule of tests — but do not invite stakeholder observers to the first test (the tech always breaks)
- Invite discussion, but focus on the session when it’s live
- Do not allow stakeholders to interrupt live sessions
- Provide summaries or play backs of key themes
- Set expectations for next steps
As part of your final steps here you need to set clear expectations around timelines for analysis and reporting — assuming you are producing final documentation. If stakeholders think they have seen “the answer” from one session — acknowledge their observations and insights, but let them know that there are
Why this is important: Stakeholders and clients usually control your research budget. Therefore it is entirely your responsibility to ensure that they get what they need from their spend. This includes the final report, but if they actually attend sessions, it includes their experience and understanding of those sessions.
What can go wrong: Stakeholders who don’t feel welcomed into the process will not understand the value and benefit of research. And they will cut your budget.
I’ve had stakeholders who turn up to a first session, then cancel their next several days of meetings so that they could watch every session. That’s because they were helped to understand very quickly the value of what they were seeing to themselves and their business.
They also felt like part of the team, without having to take on yet more direct responsibility for outcomes.
These are all things you can achieve as a researcher with the right planning, and the right boundaries around your work.
In the final part of this series I will be looking at analysis and findings and how that can be communicated and delivered. If there is no link here 👉 sign up to be notified when it drops.
7 steps to being your own User Testing Project Manager:
👉 How a UX Researcher can be their own Project Manager
👉 №5 — Admin, legals & due diligence
👉 №6 — Managing stakeholders and observers
👉 №7 — Analysis and findings
Hello 👋
Did you know, you can 👉 subscribe for free to get notified of any new articles I write. Woot.
I’ve even made a ton of helpful lists, depending on what you’re most interested in.