How accurate are the Street-by-Street Air Quality maps?

Every map produced by our system goes through a rigorous validation procedure to ensure its robustness and accuracy. This validation has two objectives:

• Make sure we do not overfit on the data: Overfitting occurs when the predictions are very close to the real air quality measured by the stations in places with lots of available data (typically in the cities with dense monitoring networks), but at the cost of being possibly very wrong when there’s less available air quality data

• Measure how close our predictions are to the reality

We use a technique called cross-validation to test our mapping system: Basically this means we have the system use only 80% of the available data from monitoring stations, and then accuracy of the resulting map is evaluated using the remaining 20% of the stations. This way, our work can be be evaluated using real data and conditions at locations. 

Illustration: accuracy evaluation of our street-level maps at a given time

Fore example, let’s see how this evaluation looks like in the 100+ cities with street-level maps on a day picked randomly (the 5th of December at 11PM UTC time).

As shown below, there is a very strong correlation between the air quality measured by the monitoring stations and the air quality predicted by our models. 

This means that our models perform very well at capturing the air pollution spatial variability, and that’s what enables us to build the street-level maps.

Concentrations predicted at the monitoring stations as a function of the concentration measured by the stations

Moreover, the average error of our predictions remains low (below 5 for all pollutants in terms of Plume Index).

Average absolute difference between the concentration predicted at the monitoring stations and the concentration measured by the stations

	Average absolute error (Plume Index)
NO2	4.5
O3	2.5
PM2.5	4.7
PM10	4.6

The results of the evaluation at other times are similar to what is shown here.

In conclusion, our street-level maps capture the spatial variability of air pollution in the world’s largest cities very accurately and with a low level of error.