Potemkin AI

Is your risk software as smart as it seems?

As the capacity for automating risk management grows, there’s a lot of buzz about AI. Risk managers like it because it gives credibility to areas that are harder to quantify (i.e. pretty much everything except market and credit risk) and their stakeholders like it as it promises sharper answers.

At a recent AI event https://bit.ly/2CX3DBS, a point was made about how enthusiasm for AI has raised both promises and expectations, resulting in a degree of ‘backfilling’ to keep up, with concealed, or at least downplayed, human decision making.

In risk this isn’t new. I’ve written elsewhere https://bit.ly/2JyM6kM about the ‘tweaking’ of assumptions. With AI this kind of manipulation can hide deeper in the software.

‘Potemkin AI’ can also come from undiscussed human activity between the system and the reports.

I am greatly in favour of increasing the use of AI but risk managers must be honest about their interventions and readers of risk reports have to keep up and apply an educated scepticism to what they receive.