Reducing noise in decision-making: Insights from Daniel Kahneman's latest book

Have you ever wondered why different judges can hand out wildly varying sentences for the same crime, or why doctors can give different diagnoses for the same patient? The answer lies in the concept of "noise," as explained by Nobel economist and psychologist Daniel Kahneman in his best-selling book "Noise: A Flaw in Human Judgment." 'His analysis challenges the conventional knowledge and provides practical strategies for reducing noise and improving the consistency and reliability of our judgments'.

Judgment, according to Nobel Laureate Daniel Kahneman, is like measurement. It involves the assignment of a value to an object on a scale, be it a probability, a size, or a decision made. But what distinguishes judgments from other forms of measurement is the inherent uncertainty involved. Noise refers to the inconsistency in human judgement that can arise even when people are presented with the same information.

“In order to distinguish noise from bias, you really have to define judgment,” he says.

“I should add that there's a real problem with the word bias and there is a real problem with the word noise,” he continues. “They’re used in many different meanings. So, for example, when we talk about a psychological bias, we sometimes think of the mechanism inside the mind that produces the errors, and that's not the same thing as an average error. Similarly, noise is a terrible word in that we use the word noise for any kind of uncertainty in addition to using it for physical noise.”

According to Kahneman, noise can be just as damaging as bias or systematic errors in judgment. While bias is often easier to detect and address, noise can go unnoticed and lead to unpredictable and inconsistent outcomes. That’s why it’s crucial to be aware of the role of noise in decision-making and take steps to lessen it.

Kahneman gives several examples of noise in action, such as judges handing out widely varying sentences for similar crimes or doctors making different diagnoses for the same patient. He also highlights the role of noise in performance appraisals, where different managers can give vastly different ratings to employees based on the same criteria. But the most dramatic example to him was the justice example.

“There has been a lot of research, research that I call noise audits, where you present multiple judges with the same case and you look at the variability of their judgment,” says Kahneman. “The variability is really appalling. I mean, for the same case, you can get sentences that go from 15 days to 15 years.”

This is cited from an experiment done several years ago with 208 American federal judges. The average sentence given by the judges presented with the same case was seven years. When two judges’ sentences were compared at random, the average difference between them was four years.

“So that means that a defendant coming before a judge is really facing a lottery, and I would say an intolerable lottery,” says Kahneman.

This problem extends beyond the courtroom and in fact is also as prevalent in hospitals and emergency rooms. Kahneman says there are systematic differences in the behavior of physicians when they’re fresh and rested early in the morning versus late in the afternoon. In the afternoon, physicians are more likely to prescribe opioids, they give more antibiotics, and they order more tests.

“They do what’s easier,” he says. “And that is a source of occasional noise, because from the point of view of the patient, unless you know that you want to get the physician at the physician's best, you normally don't care what hour you get the appointment. But the hour at which you get the appointment is a sort of lottery because the physician is going to be in different states. Depending on the physician that you see in the ER, many different things are going to happen to people.”

The root cause of noise, according to Kahneman, is the natural variability in human judgment. While people may believe they’re making objective and rational decisions, their judgments are often influenced by a range of factors such as mood, context, and personal biases. When asked if it’s possible to eliminate noise, Kahneman says it’s possible in principle, but that’s turning the problem from a judgment problem into a problem where the decision is made by a rule or by an algorithm.

“As long as it's judgment, you can reduce noise but you cannot eliminate it,” he says.

To mitigate noise however, Kahneman does have some suggested strategies. One approach is to standardize decision-making processes by using algorithms or checklists that reduce the influence of individual judgment. Another approach is to use multiple judges or evaluators to average out individual variations in judgment.

“When you’re going to hire a candidate for example, what is the best way of doing that? It turns out we know the way,” he says. “Instead of running an interview in which you try to understand the other person in a general way and form an impression, you do something entirely different. You look at what are the relevant attributes, what are the relevant dimensions that they want to learn about. And you interview about those dimensions one at a time and independently of the others, and you score them each. When you're finished with a section of the interview, you'll give a score to that dimension, and then you move to the next. Try not to think of the final conclusion, we call that delaying intuition. And this we know is the optimal procedure for selecting candidates.”

The general idea here is what’s referred to as the “principle of independence” and it works in several scenarios. By looking at one measurement independently of the others, it delays the formation of an overall or general opinion from happening too early.

“Just like if you're going to collect witnesses to a crime, you don't want them to talk to each other,” says Kahneman. “You want to talk to each of them separately and not allow them to communicate. And the reason is that you maximize information when you make different aspects of the information independent of each other, and in assessing the different attributes of a problem separately, making them independent of each other is very useful when you are going to have multiple people looking at the same candidate. You want the people to be independent of each other. And there are many companies that do it right.”

Kahneman also emphasizes the importance of feedback and calibration, which can help people become more aware of their own biases and variability in judgment. By providing feedback on their own performance, people can adjust their decision-making processes to reduce noise and improve the consistency and reliability of their judgments.

While the justice example highlights fairness in a starker manner, Kahneman says the issue of fairness is everywhere that noise exists, and this can have huge economic costs.

“Even in insurance companies, it’s not only inappropriate, it’s unfair for somebody to get a different premium depending on who the underwriter is,” he says. “That violates fairness. An organization is expected to speak in one voice and that can be hurt by noise. So, reducing noise improves credibility.”

“There's no question that when you have a system, if the system is not talking in one voice, that is simply introducing noise, and that directly interferes with the accuracy of the quality of the judgments that the system produces,” he continues.

Kahneman thinks that organizations have a much better chance of improving the way they reach judgements and decisions versus individuals. For most organizations, the ways in which decisions are made develop organically over time. He hopes that more emphasis is put on the design of decision making and judgements. This is more possible for organizations because a standardized approach often already exists.

“I would say the first step in dealing with noise is to measure it, identify how much noise there is,” says Kahneman. “In order to measure noise, you don't need to know the correct answer, but you do know that if people disagree with each other, this is making decisions less accurate. So that’s the first step. Conduct your own noise audit. Construct some realistic problems and have a number of people make the same judgment about these problems. Then look at the variability and that variability will tell you how much noise there is in those judgements.”