Machines can outperform humans
Everybody makes mistakes: Humans do, and machines do. Nevertheless, by now Artificial Intelligence (AI) is able to outperform humans in many fields, such as speech and image recognition. The authors Brynjolfsson and Mcaffee (2017) show that over the past 10 years image recognition through machine learning has improved significantly. The authors conducted a test where machines and humans had to identify a picture as a muffin or puppy picture. The human error rate stayed constant over their six year time frame of research (2010-2016). On average, people failed in 5% of all cases to label an image correctly. While in the year 2010 machines failed at roughly 27% of all cases, in 2015 they bet humans by undercutting their error rate. In 2016, the most recent test year, machines only failed to identify 3% of all images. So does that mean that we should trust machines more than humans with doing our work?
Opting for the lowest error rate?
While in the case of identifying a muffin or a puppy a failure is not that tragic, think of other scenarios where an error would be fatal. Common examples are driving a car or a medical treatment – if a mistake is being made the consequences can often be quite extreme and in some cases even deadly. Naturally we would conclude that the medium with the lower error rate should be used for a task so that as many tragedies as possible can be avoided.
Who do we trust more – Human or Machine?
Prahl & Van Swol (2016) from the University of Wisconsin have conducted a research identifying whether we trust computers or human advisors more when we need to make a decision. In their experiment they gave participants the task of making an estimate of the duration of an orthopedic surgery, a task that none of the participants had ever performed before or pre-knowledge about, with the help of either a computer or a human experienced in the field. Before the experiment started, participants were shown some recent hospital data to get an idea for a reasonable estimate. Each participant had to complete 14 rounds of making a prediction. In each round, they would first enter an estimate, then see the estimate of their advisor displayed and then be allowed to change their initial estimate. After each round they received feedback on how they performed. In the 6th round, participants were given a bad advice by either the human advisor or the machine, depending on which group they were in. If they followed the advice, their accuracy decreased extremely much. The following rounds, the advice was back to very good estimates (Prahl & Van Swol, 2016).
Their results of Prahl & Van Swol (2016) show that in the beginning of the experiment, the trust in a human or machine advisor was almost the same. Nevertheless, after the human or machine advisor made a big mistake in the 6th round, participants used the human and machine advice less frequently in the upcoming rounds. Surprisingly, the trust in machine advisors dropped significantly more than the trust in human advisors (see the graph below).
Source: Prahl, A., & Van Swol, L. (2017).
So this leaves us with the question: Why do we find machine errors “worse” than human errors?
The authors tried to find multiple explanations for this phenomenon. The authors state, that a “perfection schema” exists. Initially, people simply expect an application or machine to work perfectly and be run by an accurate algorithm; therefore they are losing more trust in a machine if it makes a mistake compared to a human (Madhavan & Wiegmann, 2007). For human advisors, participants could have had more empathy and might have kept a higher trust in them after their error because “every person can make a mistake”. Furthermore the authors suggest that participants might trust human advisors more than machines since they have more experiences with human advisors such as e.g. doctors or consultants as compared to machine advisors (Prahl & Van Swol, 2016).
What do you think? Would you trust a machine more than a human advisor? Can you think of further reasons why we are seeing an “algorithm aversion” in this or other examples?
References
Brynjolfsson, E., & Mcafee, A. (2017). The business of artificial intelligence. Harvard Business Review.
Madhavan, P., & Wiegmann, D. A. (2007). Similarities and differences between human–human and human–automation trust: An integrative review. Theoretical Issues in Ergonomics Science, 8(4), 277–301.
Prahl, A., & Van Swol, L. (2017). Understanding algorithm aversion: When is advice from automation discounted?. Journal of Forecasting, 36(6), 691-702.
Further/Related Readings
www.sciencedaily.com/releases/2016/05/160525132559.htm
https://rmresults.com/blog/why-do-we-tolerate-human-over-machine-error
https://www.digitaltrends.com/cool-tech/the-challenges-of-driverless-shuttles-in-smart-cities/
https://phys.org/news/2016-05-humans-automated-advisor-bad-advice.html#jCp
I think that the fact that humans trust more other humans stems from the fact that we are naturally afraid of the unknown and tend to take it with a certain extent of reserve. In this example, the machine is the unknown so human subjects in the experiment were more distrusting to the advice given to them. Once their prediction that machine cannot fully be trusted has been confirmed, they automatically start trusting machines even less. To me this is an example of confirmation bias.
Personally, my trusting (or distrusting) machines would depend on the context. I would be willing to assume that computer would be better in solving quantitative questions, however when it comes to any social problems, I would take AI- driven advice with a bit of reserve.
I think that the fact that humans trust more other humans stems from the fact that we are naturally afraid of the unknown and tend to take it with a certain extent of reserve. In this example, the machine is the unknown so human subjects in the experiment were more distrusting to the advice given to them. Once their prediction that machine cannot fully be trusted has been confirmed, they automatically start trusting machines even less. To me this is an example of confirmation bias.
Personally, my trusting (or distrusting) machines would depend on the context. I would be willing to assume that computer would be better in solving quantitative questions, however when it comes to any social problems, I would take AI- driven advice with a bit of reserve.
Hi Anna, thank you for your comment. I agree that we are probably indeed scared of machines since they are something new and we are only gathering our first experiences with e.g. AI right now. I think the media also plays a large part in scaring us away from AI by reporting accidents involving AI and error rates more frequently when they are high. Then again, I was surprised that, as you can see in the graph, the participants initially trusted the machine slighter more than the human, maybe due to the bias that the machine wouldn’t be used or existing if it didn’t outperform a human. But I do indeed find it very puzzling that the participants did not have a negative bias towards AI in the beginning.