Bridging the Gap: Integrating Data Science and Decision Science through Six Essential Questions
Data Science is the discipline of making data useful — But How?
Intro
It has been now more than one decade since Thomas H. Davenport and DJ Patilthree wrote their famous Harvard Business Review article:
“Data Scientist: The Sexiest Job of the 21st Century”
The article made many discussions, and now, after a decade, we have thousands of job profiles titled “Data Scientist.”
Many organizations have embraced the idea of having an analytic team in their structure.
Yet, it is a little bit surprising to see that according to Gartner report estimated that:
“ 60 percent of big data projects will fail to go beyond piloting and experimentation, and will be abandoned.”
In this blog, I reflect on how the field of Data Science/analytics can meet expectations and fulfill the anticipated value.
Main Hurdle for Analytics in the Next Decade?
I believe the main challenge facing the analytics field in the upcoming decade will involve reducing the gap between “Time Spent Analyzing Data” and the actual “Value Creation” with data within organizations.
Decision Intelligence
Then, how can data be used for “value creation”? Well, the field of “Decision Analysis,” or a more refined one proposed by Cassie Kozyrkov's “Decision Intelligence” [1], has some answers on how to “Create Value.”
When we go to the literature on Descion Anlaysis [2] , we see that value can be created through high-quality decisions. Well, you may say that “chance” plays a role too.
Sure, but the “High Quality” decision is the only way you can “purposefully” create value.
Role of Data Scientist: Making Data Useful for Decision-Making
Thus, my suggestion is that the role of data scientist/analyst should be centered around framing the decision context within organizations.
Then, that enables business leaders/managers to make informed, “high-quality” decisions.
It is important to note that often, data scientists/analysts do not make final decisions. But, by using their analytic capability, they can function as context providers to make high-quality decisions.
Then, What is a “High-Quality” Decision?
To answer the question of what constitutes a “High-Quality” decision, we first understand the main six elements of any “decision.”
- The decision-maker;
- A Frame;
- Alternatives from which to choose;
- Preferences;
- Information; and
- The logic by which the decision is made
The six elements of decision make a mental framework for “high-quality” decisions with six questions, and the quality of the decision depends on the quality of your answer to these six questions.
I will explore these six questions. This approach will help analytic professionals make decisions within organizations that optimize for value creation.
Six Questions for Achieving High-Quality Decisions
1. Decision Maker: Who is the Decision Maker?
The first thing that is crucial for data scientists to know is who the decision maker is. Often, data scientists themselves are not the decision-makers. Instead, they provide decision context to business managers and leaders.
In many cases, the decision maker is not one person but rather multiple. In many decisions, you may have a partner who needs to make decisions together, or you have a customer who is a stakeholder.
Every decision needs a decision maker—the person who makes the decision and also commits to act.
2. Right Frame: What am I Deciding?
The way you view a decision is “Frame.” Depending on how you frame the decision, you may address different decision contexts, hence making different choices.
Example:
Imagine that you are part of the data science team at a retail company. Your company is considering enhancing its customer experience through personalized recommendations. Two frames for this decision can be:
Frame A: The decision is whether to build an in-house recommendation system or buy a machine learning platform from external resources.
Frame B: The decision has already been made to buy a recommendation system from external resources (service provider); now, the choice is whether to use the service of provider X or Y.
As you can see, each framing provides a different view of the decision problem to be addressed.
3. Right Alternatives: What Are My Choices?
When you build your frame, that frame will lead to the creation of alternatives appropriate to the frame.
Making any high-quality decision involves considering substantially different alternatives.
Sometimes, you have only one alternative to choose from. In that case, it means that you do not have a choice to make, and you do not have a decision to make.
Also, you may face some cases where the decision involves a small, limited set of alternatives. In this case, skilled scientists and analysts can be very beneficial. The data science team can provide a new “creative” alternative to the leaders, one that was not initially considered.
Within any organization, teams that come up with new “alternatives” are valuable assets. This is where the data science team can truly shine.
4. Right Values: What Consequences Do I Care?
Every decision-maker will have “preferences” for the future come from various alternatives. The preference is “What do you want to achieve?”.
If you are indifferent to the possible outcomes of the decision, then there would be no need to make a decision.
In the words of Lewis Carroll:
“If you don’t know where you are going, any road will get you there”.
If you don’t discuss your values and what you want to achieve, the decision-making process is just wasteful.
5. Right Information: What do I Need to Know?
Information serves as a link between what we can do and what we want. Simply put, our alternatives lead to outcomes. But there is uncertainty in the outcome of each option we make in decisions.
Often, we need to make decisions with incomplete information, meaning making decisions in the face of uncertainty.
However, we are always tempted to get more information, yet information costs resources. A high-quality decision ensures gathering (buying) information is neither overdone nor underdone.
6. Right Reasoning: Am I Thinking Straight About This?
Finally, we need to derive some process to make a decision. The final decision is based on what we know, what the choices are, and what we know.
A Key Lesson:
A data-driven organization would embrace the value of good data-driven decision-making.
Yet, data-driven decisions are not only about data. Organizations also need awareness about the science and practice of decision-making to maximize value creation.
A starting point to understand the decision-making practice can be: What is a high-quality decision? And what are the elements of high-quality decision?
This blog offers an overview of the essential questions that must be addressed to facilitate the high-quality decision making.
Takeaway Messages:
- Cassie Kozyrkov, in her LinkedIn “Decision Intelligence” course[3], defines“Data Science is the Discipline of Making Data Useful.” But How?
- The primary focus of the data team should be on how to facilitate making “high-quality” decisions, irrespective of the potential outcome.
- Achieving a high-quality decision requires ensuring that the six questions raised earlier have been answered correctly.
- These six questions form a decision-making chain. It is a chain of six questions as links. The notion here is that the “Chain” is only as strong as its weakest link.
- The quality of a decision is as strong as its weakest answer to each question (each link).
References:
- [1]: Introduction to Decision Intelligence, Cassie Kozyrkov, 2019
- [2]: Foundations of Decision Analysis, Ronald Howard (Author), Ali Abbas (Author)
- [3]: Decision Intelligence Linkedin Course, Cassie Kozyrkov, 2023