Summary

The article discusses the balance between human decision-making and AI automation in intelligent system design, using the example of Paul, a manufacturing engineer, to illustrate how AI can support human workers without fully automating their tasks.

Abstract

The blog post, part of the SAP Design series on intelligent system design, addresses the misconception that AI equates to complete automation, akin to a "Skynet" scenario. It presents a case study of Paul, a manufacturing engineer, who faces numerous decisions in his work. The article contrasts a full automation approach, which could lead to unintended negative consequences, with a more nuanced system that assists Paul by providing recommendations and learning from his decisions. This system fosters trust and efficiency by offering transparency and control, allowing Paul to make informed decisions and gradually delegate trivial tasks to the AI. The article advocates for a stepwise introduction of AI, with varying levels of automation tailored to user trust and company needs, ultimately aiming for a harmonious collaboration between humans and AI.

Opinions

The article suggests that the fear of AI taking over human jobs is unfounded and that AI should be seen as a tool to enhance human capabilities rather than replace them.
It emphasizes the importance of maintaining human control and oversight in AI systems to prevent negative outcomes and ensure that decisions align with human values and objectives.
The author believes that building trust between the user and the AI system is crucial for successful integration of AI into the workplace.
The article posits that AI can learn from human decisions, both explicit and implicit, leading to improved recommendations and decision support over time.
It argues for a gradual increase in AI complexity and autonomy, starting with support roles and moving towards greater automation as trust and system capabilities grow.
The article concludes that the goal of AI in the workplace should be to achieve greater efficiency by complementing human decision-making with automated processes, rather than seeking full automation.

Human in Control or Automate Everything?

This blog belongs to the SAP Design series about intelligent system design. You might also be interested in the first blog, Principles of Intelligent System Design.

It is a common misconception that artificial intelligence inevitably means 100% automation. Movie producers would have us believe that AI is going to control absolutely everything — a Skynet scenario à la Terminator. So, if you design and implement an AI system, should you fear this or just ignore it? Is there a roadmap for intelligent automation of a specific system?

Let’s take Paul, for example. Paul is a manufacturing engineer of aircraft turbines. Each production order may take up to several weeks to fulfill, and many changes are made on the way. There is a product engineer, who constantly tunes the design for quality and cost reasons, there are dozens of workers required to be routed correctly through the production hall, and, of course, there is a Big Demanding Customer setting deadlines, threatening penalties, and changing requirements.

During his day, Paul makes hundreds of micro-management decisions concerning changes to the production order: apply them or ignore them? If he applies them, should they be downstream, upstream, or in both directions? Everything seems to be under Paul’s control, but, at the end of the day, Paul is very tired. Is there a way to help him?

The Skynet shadow

Imagine what would happen with full automation. Paul’s company just implemented a new software system that can “learn” from historical and environmental data, and recommend a decision with a certain level of confidence.

It is early morning. Paul checks the system to get the status of his production orders.

*No-go: A manufacturing engineer is notified of changes to a production order. The system is “smart” and takes over.*

Wow. This is probably the typical scenario that every customer we talked to is afraid of. The system delays the production of a complex aircraft engine and rolls back all operations, due to the changes received overnight from the production team.

The system is 99.8% confident in its decision and redirects the workers to other orders. The lights are off. This evening is Paul’s regular sync with the Big Demanding Customer… Good luck, Paul!

If a customer called you the next morning in this situation, what would you tell him? We delayed your production order because of… 99.8%?

We can do it better

Imagine the same morning in a parallel universe. Paul checks his system for changes.

*Better: The engineer is supported in his decision. The trust relationship starts building. Later the system starts to take over in trivial cases.*

What we see here is the same algorithm with a different UX. Paul is notified about the change. He sees the operations affected and understands the reasoning behind the change. He also has access to similar cases and decisions made by him or his peers in the past. We can proudly say that Paul has a complete overview of the situation and can make faster and better decisions.

The system can also learn from Paul explicitly or implicitly by recording new cases and decisions. As time goes by, one morning, Paul may find himself in front of the following screen.

Now, Paul is confident enough to delegate trivial decisions to the system. He still wants to approve them. But, as his trust in the system grows, he will probably only need to review them from time to time, in the future.

Levels of automation

What we have done here? We have introduced two different levels of automation, which were applied depending on the use case and the user’s trust in the system. They helped us introduce and grow the AI complexity sequentially, without excluding the user and losing his trust.

Of course, in other scenarios, there could be more levels. The amount and the final level to be reached is always defined by the company.

In the end, it comes down to the right balance between human control and the autonomy of an intelligent system. The correct mix fosters efficient collaboration and builds trust. Where full automation is not feasible, we should aim for greater efficiency. By combining automation with better use of existing information, transparency, and learning effects, we can help users to obtain the same result with fewer steps.

But, we’re just scratching the surface. Stay tuned for more interesting topics and add your thoughts in the comments section below!

Several illustrations in this article were created with Scenes™ by SAP AppHaus

Originally published at experience.sap.com on January 29, 2018.