Building Research into every Iteration

SCRUM & LEAN UX | Episode 5

LEAN UX promotes continuous inspections and adaptations; guiding teams in minimizing risk and maximizing value by validating riskiest assumptions early and often throughout each Sprint. Scrum requires those inspections to be performed diligently.

To achieve this, teams inject evidence-based decisions based on customer and user data into development. This enables the team to journey from doubt to certainty during a Sprint. These insights will act a a guide. It will tell the team if they are on the right track and timely tell them to change direction. Guided these insights a team can make steps more confidently.

Connecting the team more closely with end users requires significantly more than just having interactions with Stakeholders during a Sprint Review.

Performing research each iteration is a team activity in Lean UX. This creates transparency. The whole team is learning and developing a shared understanding which enables them to adapt early.

“Shared understanding is the currency of Lean UX. The more a team collectively understands what they are doing and why, the less they need to debate what happened and can quickly move to how to solve for the new learning. In addition, it reduces the team’s dependencies on second-hand reports and detailed documents to continue its work.” — Lean UX, by Jeff Gothelf and Josh Seiden.

Research Activities

A suggested valuable question / workshop to run with your Scrum Team is to answer:

What activities will enable us to build research into each Sprint, so we can get the earliest possible validations/learnings about what we are developing?

Examples of these activities which help the team develop a better understanding of the space, product and its usage:

• On-site feedback surveys.
• Recordings of users interacting with the latest increment.
• Creating interview guides and run these interviews with customers about the latest increment or what is on the radar to develop/improve next.
• Developing A/B tests.
• Include feature/interaction counters.
• Sketching and running through paper prototypes.
• Analysing Heat Maps.
• Before/after comparisons.
• Adding a 5 star rating review (or smiley/happiness buttons) about the latest additions to the increment.
• Run through search logs and analytics.

It is essential teams self-manage how they perform research as they are the subject matter experts. If they have autonomy over the way they perform research, they will also develop a higher sense of accountability over what they research. The team will be more connected to the outcomes. They will learn first handed and this enables them to adapt fast!

LEAN UX suggests teams be broken up into research pairs, frequently mixing up individuals with different disciplines to run small research activities. Naturally, they’ll require space to experiment and improve their research skills over time.

Evidence-Based Research

Scrum.org provides an excellent companion called the ‘Evidence Based Management Guide for Scrum’.

“Evidence-Based Management (EBM) is an empirical approach that provides organizations with the ability to measure the value they deliver to customers and the means by which they deliver that value, and to use those measures to guide improvements in both.”

The guide provides metrics and questions Scrum Teams can zoom in on. A way to start this journey could be to select a metric or question each Sprint as a goal to research.

Making sense of the research

With cross-skilled individuals, each person will interpret the data differently. Aligning on how the data should be interpreted and deciding what to do next will be a challenge for the team. Specialists might disagree on how they interpret the data, or what to do next. That is why it is important to define hypotheses that require further validation. We learned […], so we now we need to validate […].

Making sense of the research is not a team-only activity. Stakeholders should be involved as it helps them and the team better understand the market. These insights will truly level-up your Sprint Reviews.

When team members (and stakeholders) strongly disagree this is a sign more needs to be learned and validated. However, the team should not be impeded from progressing on the increment. After all, there is always more to be learned. Often an anonymous vote or dot-voting can be useful to disagree and commit on the first next steps.

Don’t Deliver and Forget

Many development teams focus almost only on delivering new stuff, better, faster, cheaper: feature factories. With each increment more is added to the product. But does this really increase the value of the product?

How much of the stuff a Scrum Team delivers is actually being revisited?

Is your team really inspecting if it indeed delivered the value it expected? Yeah, the results can be scary and disappointing. What if it becomes transparent the team is not delivering the expected value? 😲 Would such transparency make the environment unsafe for the team and is this why it keeps up its success theater? If this is the case, organizations might suffer from the ‘Emperor’s New Clothes’ effect (learn more about this effect in this article about Transparency).

If your team believes “done” equals “final” they might not yet understand what Incremental Development is supposed to be about. They are instead caught in a ‘Deliver and Forget’ habit moving on to the next new thing as quickly as possible.

Even building research into each Sprint is not sufficient to get out of the Deliver and Forget mindset if the team is not adapting to what it is learning.

With larger departmental organizations it can very well be that insight collected from customer service employees never make it directly to a Development Team. In other cases research is outsourced, and conclusions are only shared with a steering committee in a quarterly review (or something along that line).

I experienced a situation where an organization almost always valued forecasting new stuff over improving stuff. What’s worse was, when this was made transparent, the conclusion drawn by the Product Owner was something along the line of:

“All these suggested improvements are just ‘nice-to-haves’. We’ll never get round to them. They just clutter the Product Backlog. Research would be a wasteful activity and slow the team down in delivering the new features…”.

In this case, the Product Owner assumed that customers valued new features over improved features. It was about what the Product Owner wanted next, not about what users really valued. The Product Owner’s assumptions were not validated. What if a healthier balance between delivering new features and improving existing features would result in a better user experience and a more valuable product?

Building research into each Sprint will enable the Product Owner to focus on what is most likely to maximize the user experience and value of the Product.

So….

Thanks to Maarten Dalmijn for reviewing this episode!

Next Episode