“Prediction at risk: Value-at-Risk concept for assessing machine learning prediction risk”

Title: «Prediction at risk: Value-at-Risk concept for assessing machine learning prediction risk«
Authors: Maciej Zalwert, Kira Kempinska, Lander Ketelaars, and Adam Karwan.
Pages: 14.
Edition: 1st.
Genre: Computer science.
Date: 2020.03.30.
Language: English.
Format: Kindle Print Replica.
ASIN: B086KQJZJD.

This «book» is available for free on both Kobo and Amazon. The version I got from the Kobo store was damaged (it started on page 3, and the resolution was quite bad; impossible to read). The Kindle version, in contrast, is quite good.

This is not really a «book»; it is just a journal article introducing the Prediction-at-Risk (PaR) methodology for assessing the risk of machine learning (AI/ML) inference. It was written by people from Ernst & Young (EY) in Poland. So, my assumption is that one of them (surely the first author) found a way to publish in both bookstores, without charge, just for the value to their profiles.

The methodology is an adaptation of the Value-at-Risk (VaR) technique used in market risk management to machine learning model evaluation. The PaR estimates are based on the quantiles of a model’s output, delivering an intuitive risk figure that captures true model performance risk and supports robust backtesting. Formally, PaR is defined as «the minimum potential metric value of a model with a given probability over a certain dataset.»

This approach offers an intrinsic view of a model’s robustness and metric risk at a specified probability level. By addressing both individual and systemic risk, PaR enables the construction of a comprehensive risk management framework for AI/ML models. The paper focuses on the formal definition and real-world implementation of this concept, with experiments validating its applicability across a range of risk assessment scenarios.

Deja un comentario

Este sitio utiliza Akismet para reducir el spam. Conoce cómo se procesan los datos de tus comentarios.