Retrospectives for Data Science Teams

Data science work doesn’t come in sprints. It is traditionally demand-driven or problem-driven. Problems, as you know, do not come in fixed cycles and mostly, there is an urgency to solve said problem — as the opportunity cost is counted daily (if there is no urgency, you’re not tackling the right problems OR aren’t framing them properly). There have been attempts to sprintify DS work, but it never works. Ask any data scientist what they’re working on, they’ll talk about how the majority of their time goes in ‘cleaning’ the data — a concept, that is not what you think it means. Regardless, there is a lot of work that goes before building a model, so many assumptions that are taken and factors discarded — all in service of clean, viable output.

Hence, there is a mental debt that a team carries. Such a mental debt, as product managers, we’re well aware of… one of the key reasons we have a sprint retrospective is to unburden the teams’ overall frustrations & aspirations. So why not implement the same for Data Science teams? Given that the work is unusually ad-hoc and the pressure on the delivery is high, it makes sense that they get an avenue to vent out their frustrations OR express their aspirations regularly.

A General Framework of a Data Science Team

I’ll explain this in short:

1. Work is found/created from various business areas like Marketing, Business Operations, Digital etc.
2. This work is first translated to a problem statement. There is a problem that needs solving, hence the work is generated. This is defined by Data Product/Delivery managers/Data Scientists that formulate the statement. Usually, this problem statement is defined for real-world problems that can be solved.
3. From there, this is translated to a solution design — this requires exploring various data sources, putting together requirements for problem-solution fit and defining the output. In simple cases, it can be model that outputs values that is consumed by the team. Timelines are ascertained at this point.
4. Solution delivery is done here. The team has a back & forth with the layer above.
5. Finally, the solution is checked if it actually fits the problem and how it impacts the chosen performance metrics. It is then delivered to the business unit with further potential problems identified, repeating the cycle.

The importance of Retrospectives

We’ve been doing DS Retrospectives for a while now. I can boil down the importance of this ritual to the following:

Process Improvements

A good way to get collective feedback on the ways of working is through discussion. You get to expose both sides of the coin — from you, as a work bringer/outcome translator and them as work bearers. As a classic example, ‘not enough data’ at the time of workload creation is common to complain about. This leads to a discussion that which workloads require what level of detail — may be an intermediate layer of technical translation is needed to identify if the work is viable or not? Remember, not every feedback has to be taken for its face value. You are to facilitate a discussion. Experiment! Yes, even within your process design!

Building camaraderie

There is no better way to humanize fellow workers than to have them face the same frustrations as you. If you both agree on the same thing, you’ll find yourself more empathetic to the other persons’ situation — which can lead to better team morale, which leads to better efficiency. Remember, the output is a reflection of your team — if your team is happy, the output will be great.

The 20%s

You’ll be surprised at the kind of ideas the team generates. Hell, you can push in some of your ideas and see how the team reacts. Some of the great work we’ve done in the last 6 months has been through the 20%s we’ve captured as action items. I feel that everyone within the team wants to do great. They can only do so with the opportunity provided. Retro’s are a great place to bring them out.

The above are the most important reasons to conduct Retros for data science teams, but you can always find more as you go down that path.

Conducting A Retrospective

Throw out whatever you’ve known about retros — the sailboat technique, the need for prime directive, etc. — these are complications that add unnecessary load on the ritual. All you need is:

A simple board with 4 columns

These are the 4 columns you need: “Kudos”, “Needs Improvement”, “Went Well”, and “Action Items”. The team adds the cards beforehand — reminders are sent out at intervals of T — 3 days, T — 1 day, T-1 hours, T — 1 hour. The board is open for any user to add cards, either anonymously or by name. We use https://easyretro.io/.

A time-bound, quick process

Here is the process:

1. We went with a round-robin, alphabetical, 1 card per turn approach. Everyone gets a turn to speak about 1 card. Cards written by others are hidden, so you do not see what someone else has written — you only see your card.
2. The choice of card is random (all except the ‘Action Items’ card). The names are called out in alphabetical order. After every turn, the participant removes the card to the done column. There are no naming conventions or additional ritualistic constraints. If you as a participant agree with what is being said or would like to add to it, you can. As long as it doesn’t derail the conversation, it’s acceptable. The team enforces the limit on itself. No additional policing is required.
3. In the end, all the cards are revealed — ideally, there won’t be any cards in the first 3 columns. If there are anonymous cards, they’ll be read out by the retro-lead. Agreements or disagreements are discussed.
4. Once the first 3 columns are empty, 3 mins are given to vote on the Action Items. They’re sorted by votes are each action item owner talks in detail about the item and what they expect — it is debated and clear outcomes are defined.
5. The last 5 mins are left for any other points people want to bring up.
6. After this, a synopsis is sent out as an email thread or a slack post to keep a track of the action items.

Simple. Easy. No Complications.

Conducting Retros in the Work From Home World

It is easier to conduct retros in the remote working era. Everyone joins on MS Teams (or your preferred team communication tool). The information is digitized and easy to extract — easyretro.io is a really good tool for this. Within MS Teams, we use the ‘raise hand’ feature to indicate when there are no more cards for you to speak.

Statistics for the yearning soul

With our team of around 10–12 participants, we spend around 30 mins on the first 3 columns, 15 mins on the action items and 5 mins on the final debrief. Progress is tracked independently in separate threads. Each one is responsible for their thread — as outlined above.

Points to remember

From my experience here are a few things you need to remember

1. The retro is for the team. Keep it free and loose. Let conversations flow, but prevent them from derailing.
2. It is important to show outcomes. The time that is spent in discussions has to have a return value for the participants. Effect change or provide a timeline for the change. If a change cannot take place for whatever reason, explain why. It helps the team bond — even if 90% of what they suggest is not possible within the scope. Transparency above bureaucracy.
3. The team is the most important thing here. Taking a page from the SCRUM guidelines, remember that the team is the one that delivers — you are subservient to the team, within reason. The prime directive here is that “Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.”
4. Avoid extremely detailed processes. Our retro guideline is just a simple 300 words post that describes the outcomes from each step. The team can choose to do things differently, but the outcomes remain.
5. Keep a decent cadence. It shouldn’t be too soon or too late. Experiment with the timeframe so that the participants don’t forget what happened between the timeframes or that the retro was so soon that it there isn’t anything to talk about. I recommend a 2–3 weeks cadence.
6. Do not force anyone to partake— if a participant has nothing to talk about, that’s ok. Sometimes, people are not tuned in. The peer pressure will force even the most quietest of members to talk, eventually. Have faith.

In conclusion

We’re on the path to achieving great things by keeping this fortnightly retro. One of the things I’ve observed is the greatness of the retro process for other teams that do not follow a sprint model. The prime directive, after all, asks us to believe in the best of people’s intentions. This ritual has led to great ideas and outcomes that we wouldn’t have achieved or thought of without it.

Thank you for reading. I would love to hear how you conduct retros within your organization for non-development teams. Does it work? If yes, what insights can you offer? If not, how would you improve it? Let’s do a retrospective of the retrospective!