Not all models are as good as they seem – new publication

Predictive distribution models can be used to predict the impact that climate change will have on the potential distribution of a species. To believe the model, you need to evaluate it and measure its goodness-of-fit using some metric. The most commonly used metrics (AUC, maxTSS) have been widely criticized in the literature - but they are still used by modelers as no better alternative exists. Imelda Somodi and Ákos Bede-Fazekas (assistant professor in our department) have now published a joint first-authored paper in the Q1-ranked scientific journal "Ecological Modelling" (IF = 3.1), which aims to remedy this unfortunate situation.
Researchers from the HUN-REN Centre for Ecological Research and the Department of Environment and Landscape Geography at ELTE have developed model goodness-of-fit metrics that they hope will replace those used so far. To facilitate this, an R package has been developed to accompany the publication. The new metrics do not classify model predictions into the usual two categories (false vs. true), but introduce a third, intermediate category reflecting uncertain predictions. This allows the new model evaluation metrics to paint a more nuanced picture of model reliability. The researchers not only presented the new metrics, but also tested them on both real data (natural vegetation types in Hungary) and virtual species, comparing them with traditional metrics. The results are convincing: the new metrics were able to reveal features of the models that the traditional metrics could not.