# How to predict full probability distribution using machine learning Conformal Predictive Distributions

--

In my previous article “Conformal Prediction forecasting with MAPIE” we looked into how one can create probabilistic forecasts using Ensemble batch prediction intervals (EnbPI) implemented by one of the open-source libraries for conformal prediction MAPIE.

Many of the Conformal Prediction models output set predictions, in regression case set predictions *are basically Prediction Intervals that specify, with a given confidence, a lower and an upper bound on predicted values of target variable y**.*

A recent paper by DeWolf et. al “Valid prediction intervals for regression problems” was the first independent study that looked at desired qualities of a probabilistic regressor and how all four classes of approaches achieved the primary objectives of a good probabilistic predictor.

What the authors found was that Conformal Prediction was clearly *the best approach for Uncertainty Quantification in regression tasks* as, unlike other classes of approaches conformal prediction guarantees validity (lack of bias) of probabilistic predictions for **any problem, any data distribution and any dataset size and any underlying regression model whether statistical or machine learning or deep learning.**

One can agree that the validity of predictions is *the most desired quality of a probabilistic predictor*, especially in critical applications such as health, finance and self-driving cars. This is because using biased predictions can result in catastrophic outcomes such as misdiagnosing a critical disease or not stopping a self-driving car when a pedestrian is on the road or entering multi-million trades on the basis of biased predictions.

Even outside of critical applications using biased prediction results in suboptimal decisions that for any large company can result ** in very significant losses**. Take a manufacturing or retail company for example — a biased probabilistic prediction of demand will result in incorrect decisions in procurement, demand planning and operations resulting in customers either not served and leaving elsewhere or excess and slow-moving inventories resulting in write-offs and costly damage to the bottom line P&L.

Compare the…