Unlocking Decision-Making: AI Bridges Theoretical Frameworks with Technological Advancements

A brief overview of how data science and AI help decision-making

Our lives consist of constant decisions and choices. Because the consequences of our decisions can lead to considerable economic and social impacts, the research on decision-making was highly interdisciplinary from the beginning. Scientists from mathematics, sociology, psychology, economics, political science, and computer science have been actively studying how to make better decisions since the mid-20th century. Among the many achievements in these fields, the most well-known are the expected utility, prospect, and game theories. These theories are driven by mathematical models but often face challenges from empirical tests of real-world scenarios.

With the tremendous increase in computation power and the blooming of cloud technologies, decision support systems (DSS) have advanced in parallel with the decision theories to assist humans in making decisions, particularly in the business and organizational context. A typical DSS includes scalable knowledge databases to collect and store large amounts of information, statistical and analytical algorithms for forecasting and projection, and user interfaces (including charts and dashboards) for human decision-makers to visualize and interact with the decision-making process.

Most human decisions, however, have resulted from incremental learning of trial and error. The iterative approach is particularly effective in dealing with unknowns in a novel environment. It requires explorations for new information and assessments of errors to refine decision-making. Remarkably, deep reinforcement learning has emulated the trial-and-error nature of human decision-making and surpassed human players in specific games.

Reinforcement learning (RL) was one of those machine learning fields that have been around from the beginning. Its significant breakthrough happened after the deep neural network was applied to the model. The deep RL is just one of those examples of AI and deep learning revolutionizing the decision-making field. We now find ourselves in an era where AI provides the foundation to bring all decision-making disciplines together, fastens the pace of understanding the human decision-making process, and further empowers humans to make data-driven rational decisions.

Decision-Making Process and Decision Theories

Decision-making is a process ending with selecting a course of action. Simply put, this process has four consecutive stages: Assessment, Options, Evaluation, and Action Selection. Each stage relies on the successful completion of the preceding one.

Stage 1: Assessment

The first decision-making stage is to identify a problem and call for a decision. To properly assess a problem, we should ideally collect all the relevant facts, identify the gaps between the current state and the desired outcome, and finally, confirm the need for a decision to be made.

Problem assessment is essential to understand the environment, conditions, and possible constraints. The more complete and accurate information gathered will make the following decision-making stages smoother and ensure better choices are made. Skipping this stage can lead to entirely wrong decisions with detrimental consequences. Plenty of stories in the literature and our daily lives have taught us the lesson that people rush to make bad decisions due to incomplete information and previous unfounded beliefs.

Stage 2: Options

Once a problem is confirmed and all the pre-requisite information has been gathered, the next stage is to develop different options before reaching the final decision. This step involves forecasting future outcomes based on previous knowledge and experience. It includes inventing, developing, and analyzing decision alternatives and testing the feasibility of each solution if possible.

This stage requires skills and expertise to make accurate predictions. It may require tools or calculations to formulate the options. Without those tools, the human brain uses imagination to visualize future actions and potential consequences based on memories. A temporal factor is also attributable to this phase: the further away in time, the more difficult to predict accurately.

Stage 3: Evaluation

When faced with multiple options with uncertainties, most established decision-making theories have focused on how the alternatives are evaluated and how humans make the final choice, based on the assumption that each option should have an expected value for a purpose. In the biological world, the value could be to reduce hunger, thirst, or other basic survival needs. In many different situations, it could be a reward or punishment. The most straightforward value for a financial or economic decision is the money earned or lost.

The expected utility theory was first proposed by Swiss mathematician Nicolaus Bernoulli back in 1713 and was later finalized by other economists and mathematicians in the 20th century. The theory proposes that a rational agent should choose the option with the maximum expected utility. The expected utility is the weighted average of all the probable utility levels an action is expected to reach under specific circumstances. In essence, the utility reflects both the expected value and the risk a person will get for a choice.

Let’s use a simple example to see how the expected utility theory works. Suppose there are two scenarios, each has two options, and we ask participants to choose the best option for each scenario:

Scenario 1:

• Option 1: 100% chance to gain $450
• Option 2: 50% chance to gain $1000

Scenario 2:

• Option 1: 100% chance to lose $500
• Option 2: 50% chance to lose $1100

The utility is derived for each option by multiplying the probability with the expected dollar value. Based on the expected utility theory, the participant should choose the options of the highest utility in each scenario: the 2nd option for scenario 1 and the 1st option for scenario 2.

The expected utility theory was developed from an economic point of view to ensure people make choices consistent with their goals. It also takes into account diminishing marginality, in which when the value of money increases, the extent of utility increase slows down (e.g., a wealthy person has a lower desire for an extra hundred dollars than a poor person).

In 1979, psychologists Daniel Kahneman and Amos Tversky challenged the expected utility theory with their experimental findings. Their research revealed that people do not consistently choose the maximum utility but have the propensity for more risk-aversive or risk-seeking options depending on the relativity of the value and risk. The discoveries led to the development of the prospect theory.

Given the same example above, most participants choose option 1 for scenario 1 because $450 is guaranteed with 100% probability. For scenario 2, most people choose the risker option 2 even though its utility value ($550) indicates more loss than the other option. In other words, people prefer smaller utility gains to avoid risks but are willing to take more risks when facing losses.

Therefore, in contrast with the symmetrical calculation of the maximum utility for both gains and losses in the expected utility theory, the prospect theory has an asymmetrical S-shaped curve with the negative utility curve significantly steeper than the positive part of the curve (see the picture below). In a nutshell, the expected utility theory is about what people should do to make a rational choice of the maximum utility. In contrast, the prospect theory explains why people tend to make worse decisions due to their preference for avoiding immediate losses instead of achieving long-term gains.

In real life, however, the expected value may be obscure because it is not intrinsic to our human mind. The value could depend on well-being or other psychological or social factors. Furthermore, in many cases, we have yet to determine the probability of each outcome. A general challenge for the decision theories is that they all have assumptions only suitable to the particular field and have been tested using small samples. In addition, the expected value and probability are not static but depend on many factors and could be relative to other options. Decision-makers must re-evaluate the utilities every time those factors or options change.

Stage 4: Action Selection

Making the final call and selecting the course of action can sometimes be inseparable from phase 3. Humans can assign the expected value, find the maximum utility, and act immediately. In many other situations, however, the best option is not apparent or available after the evaluations. We often encounter situations where the two choices provide equal value in a complementary or competing way, and the cost seems comparable when choosing one versus the other. The typical dilemma could make it challenging to decide or make the decision-making process stand still. It is when people seek help from others (e.g., family members or friends) or leverage social groups to decide. When there are multiple decision-makers for a specific problem, game theory has been developed to address this type of dilemma in decision-making (note: game theory is a vital field in social decision-making, but beyond the scope of the current article.)

Decision-Support Systems

Deliberate decision-making is an essential function of human intelligence. When facing a novel problem, humans must go through the typical four decision-making phases described above. However, our human mind often struggles to rationalize a complex situation due to our brain capacity limit and potential biases. Since the mid-20th century, decision support systems (DDS) emerged to extend the human capacity to support decision-makers in making better decisions. The main features of DDS include its knowledge system and data-driven algorithms to make predictions and evaluations.

Knowledge Systems to Extend Humans’ Memory

We need to know the world to solve a new problem. An accurate and complete grasp of the relevant facts is crucial to making predictions and forecasts in the subsequent step of decision-making. For simple life problems, it may be good enough to rely on our recall of our experience and knowledge. For more complex problems, however, our memory is not entirely trustworthy primarily due to two reasons:

First, human memory does not store everything. Humans take information for only the aspects that they pay attention to while ignoring many details deemed irrelevant in the first place. The acquired short-term memories then gradually decay if without use, and only a small portion is stored for the long term. Sometimes, even when a fact is stored in the brain, humans may experience difficulty in finding it (e.g., a search problem.)

Second, our memories are constantly modified and updated. Our brains do not have a separate place to store different instances of our experience. It uses the same brain areas that initially recorded and processed the information to store the memory. Whenever we recall or replay it mentally, the same neural network is activated, and the synapses’ weights are adjusted. In addition, because of the neural network’s pattern recognition and associative nature, our brain can automatically fill in details that were not recorded or observed initially. It explains the framing effects of how initial questions asked and implicit suggestions during problem-solving can distort people’s memory and impact their decision-making.

On the contrary, computer knowledge systems and databases are the tools to store historical facts faithfully without misses or distortions. They keep the raw facts and aggregated information organized for easy accessibility and searchability. With the advances in cloud computing, various types of databases for storing different kinds of information can hold vast amounts of information with high scalability, reliability, and performance.

Advanced Algorithms to Make Better Predictions

For deliberate decision-making, the human brain uses imagination to predict the future. While it is very powerful, our mind does have several limitations:

1. Human imagination has limited prediction power because it tends to reconstruct and fill in the present information.
2. Many predictions require non-linear calculations (e.g., power function), which people need external tools to help with.
3. The human brain uses working memory to hold multiple scenarios to compare pros and cons simultaneously. The working memory has limited capacity and requires high concentration for intense computation, which is not sustainable for extended periods.

On the contrary, DDS helps humans forecast and predict using mathematical, statistical, and machine-learning algorithms executed with powerful computation resources. Numerous algorithms and methodologies have been developed, implemented, and matured in the past few decades, which help humans make better decisions tremendously in various industries and organizations.

Help Human to Make Rational Decisions

We all know emotion has a substantial impact on our decision-making, in particular, the option evaluation process. For example, the risk-aversion emotion leads to biased decisions, as illustrated by the prospect theory. Human cognitive biases and heuristics also result in subjective evaluations and sub-optimal or wrong choices. Given this, data analytics can help alleviate the impact of emotions and overcome potential biases by leveraging utility theories and the associated utility functions to allow people to make optimal choices. In addition, DSS often has user interfaces to present the choices and replay the scenarios. It makes the decision-making process interactive and enables decision-makers to produce unbiased, data-driven decisions.

Unknown Unknowns and Reinforcement Learning

“…there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say, we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know, we don’t know.”

– Donald Rumsfeld

The challenge of decision-making is the uncertainty of the future, specifically related to those types of “unknowns” in this famous line stated by the former US Secretary of Defense Donald Rumsfeld (at a press conference in 2002). If everything is known (known knowns), no more decisions are necessary since the outcome is already determined. The “known unknowns” are represented by the probabilities and risks for the options, which are used in the utility functions to identify the optimal choice with the maximum utility. However, the “unknown unknowns” underscore the uncertainty of the outcomes that cannot be predicted or forecasted given the existing knowledge and experience. They can only be identified or discovered after specific actions occur.

Trial-and-error is the solution to deal with the unknown unknowns, which humans are, in fact, good at. They explore the environment, make the best move, and learn from the outcome. This process iterates until they can make the perfect decision. It is how humans become the masters of anything they do, including decision-making. Because of this mode of iterative learning, most of our choices are automated: some are fully wired by nature, such as reflexes (e.g., withdrawing hands from a fire, blinking to avoid an intense light beam), and others are the results of learning from training (e.g., win a game, fly an airplane).

One of the oldest machine learning algorithms, Reinforcement Learning (RL), learns using the same trial-and-error method as humans. Although RL was initially designed in the 1950s, the true revolution happened in 2013 when Deep Mind applied deep neural networks to RL, which enabled the model to learn a game from scratch and eventually outperform humans in those games. The most famous one was the AlphaGo that defeated human world champions.

The deep RL model has a similar decision-making process previously outlined in this article. Within the model, an agent makes observations, evaluates optimal paths, takes actions, and receives feedback through rewards from the environment. Subsequently, the agent compares its anticipated rewards with the actual rewards and uses the difference to fine-tune its next round of actions. With the iterative adjustments over time, it will act in a way that maximizes the rewards. It’s worth noting that RL uses the term “reward” instead of “utility” because the computation of rewards diverges from the utility calculations in the utility theories.

The advantage of deep RL over DSS systems is that the agent can explore the environment and gather information by itself through a set of sensors, such as cameras and touch senses, and be able to act, such as sending signals to activate motors. In other words, the deep RL mimics human and animal brains as an integrated system to encompass the decision-making process end-to-end, with the ability to learn and improve continuously.

So far, many use cases have successfully leveraged deep RL, including games, self-driving cars, recommending systems, web ads placements and delivery, financial market forecasts, etc. Furthermore, AI and deep learning have opened a new avenue to connect the fragmented decision-making disciplines at every level. Here are some of the trends, which are certainly not exhaustive:

1. Researchers use deep learning algorithms to evaluate, test, and advance decision theories with large-scale experimental data.
2. Decision support systems leverage deep reinforcement learning to replace or enhance previous algorithms to make more accurate predictions and optimal decisions.
3. New decision-making frameworks, such as dynamic decision-making and robust decision-making, have emerged, breaking the traditional one-time bulky decision-making process into smaller iterative cycles and leveraging AI to learn from previous decisions and tackle uncertainties.
4. AI and deep neural networks have become indispensable tools for neuroscientists to study the circuity and neural mechanisms underlying human decision-making and to look for ways to prevent humans from making bad decisions such as addictions.
5. Advances in decision theories and new brain research findings will help AI enable more automatic decision-making.

Conclusion

In their diverse capacities, individuals, organizations, and societies aim to make optimal decisions that align with their goals and objectives. Given the criticality of decision-making across various contexts, it has emerged as a subject of study spanning many key disciplines over the decades. Five Nobel winners have been recognized for their significant contributions to decision-making and the associated expected utility and prospect theories. Concurrently, data scientists have successfully assisted human decision-makers by implementing scalable decision support and forecasting systems. The recent breakthroughs in deep reinforcement learning in mimicking and even surpassing human decision-making signifies a transformative phase where AI plays a pivotal role in converging the decision-making disciplines for a unified understanding of the human decision-making process and bringing various decision-making systems to an unprecedented level of automation.