Jasmine Leonard
theRSA.org
Originally posted March 23, 2018
Here is an expert:
The trouble is, since many decisions should be explainable, it’s tempting to assume that automated decision systems should also be explainable. But as discussed earlier, automated decisions systems don’t actually make decisions; they make predictions. And when a prediction is used to guide a decision, the prediction is itself part of the explanation for that decision. It therefore doesn’t need to be explained itself, it merely needs to be justifiable.
This is a subtle but important distinction. To illustrate it, imagine you were to ask your doctor to explain her decision to prescribe you a particular drug. She could do so by saying that the drug had cured many other people with similar conditions in the past and that she therefore predicted it would cure you too. In this case, her prediction that the drug will cure you is the explanation for her decision to prescribe it. And it’s a good explanation because her prediction is justified – not on the basis of an explanation of how the drug works, but on the basis that it’s proven to be effective in previous cases. Indeed, explanations of how drugs work are often not available because the biological mechanisms by which they operate are poorly understood, even by those who produce them. Moreover, even if your doctor could explain how the drug works, unless you have considerable knowledge of pharmacology, it’s unlikely that the explanation would actually increase your understanding of her decision to prescribe the drug.
If explanations of predictions are unnecessary to justify their use in decision-making then, what else can justify the use of a prediction made by an automated decision system? The best answer, I believe, is that the system is shown to be sufficiently accurate. What “sufficiently accurate” means is obviously up for debate, but at a minimum I would suggest that it means the system’s predictions are at least as accurate as those produced by a trained human. It also means that there are no other readily available systems that produce more accurate predictions.
The article is here.
