Forecast skill

In meteorology, more specifically in weather forecasting, skill measures the superiority of a forecast over a simple historical baseline of past observations. The same forecast methodology can result in different skill scores at different places, or even in the same place for different seasons (e.g., spring weather might be driven by erratic local conditions, whereas winter cold snaps might correlate with observable polar winds). Weather forecast skill is often presented in the form of seasonal geographical maps.

Forecasting skill for single-value forecasts (i.e., time series of a scalar quantity) is commonly represented in terms of metrics such as correlation, root mean squared error, mean absolute error, relative mean absolute error, bias, and the Brier score, among others. A number of scores associated with the concept of entropy in information theory are also being used.

The term 'forecast skill' may also be used qualitatively, in which case it could either refer to forecast performance according to a single metric or to the overall forecast performance based on multiple metrics.

Metrics

Probabilistic forecast skill scores may use metrics such as the Ranked Probabilistic Skill Score (RPSS) or the Continuous RPSS (CRPSS), among others. Categorical skill metrics such as the False Alarm Ratio (FAR), the Probability of Detection (POD), the Critical Success Index (CSI), and Equitable Threat Score (ETS) are also relevant for some forecasting applications. Skill is often, but not exclusively, expressed as the relative representation that compares the performance of a particular forecast prediction to that of a reference, benchmark prediction—a formulation called a 'Skill Score'.

Forecasting skill metric and score calculations should be made over a large enough sample of forecast-observation pairs to be statistically robust. A sample of predictions for a single predictand (e.g., temperature at one location, or a single stock value) typically includes forecasts made on a number of different dates. A sample could also pool forecast-observation pairs across space, for a prediction made on a single date, as in the forecast of a weather event that is verified at many locations.