Meaning
Mathematical metric used to measure the difference between two probability distributions based on the minimum cost of transforming one into the other. The wasserstein distance provides a nuanced view of distribution shifts by considering the actual distance data points would need to move. It is highly effective for detecting subtle drift in machine learning models.
This metric is a central part of the audit trail for automated decision systems.
Distribution Shift
Comparing a model current output to its baseline reveals how much the real world has changed. A large distance indicates that the model is no longer operating in the environment for which it was designed. This measurement helps organizations decide when to retrain their algorithms.
Transport Cost
The metric is often described as the earth mover distance because it treats the distributions like piles of soil that need to be relocated. The total cost is the amount of material multiplied by the distance it is moved. This physical analogy helps in understanding the scale of the statistical change.
Model Assessment
Regular monitoring of this distance provides a defensible reason for taking a model offline. It ensures that the performance standards required by commercial agreements are met. This assessment remains vital for high stakes automated processes.
