Following innovations in machine learning and computational statistics, a large variety of new modeling techniques are being applied to premium rating. In order to carry out model comparison and selection in this regime it is particularly valuable to develop metrics that allow us to evaluate predictive power of candidate models with respect to the insurance outcome without relying on the knowledge of their internal structure. Common diagnostics used today include calibration plots, quantile charts, double lift or loss ratio plots, Lorenz curves and the Gini index (Berry et al., 2009; Goldburd et al., 2016). The relationships between these tools and the potential economic value of the models are not necessarily well understood (Meyers, 2008; Meyers and Cummings, 2009). In this paper we establish a precise connection between the traditional diagnostics and the economic value and take advantage of the resulting intuition to motivate a new family of model-agnostic performance metrics and draw links to established literature on forecast evaluation.