Evaluation analysis
Here’s a guide for interpreting these charts:
The blue dot is the average pollution level, and the gray bar is the Standard Error. The shorter the gray bar, the greater the confidence in the average – the more precise we know our average to be.
The purple dot shows the difference between the observed and projected values after congestion pricing, along with 95% confidence intervals (the purple bar), a statistical measure that shows the range of values that difference could be and still be true. If the confidence interval stretches from negative numbers to positive numbers, then the difference is not significant, meaning there is no change in pollution due to congestion pricing. If the purple bar and dot are entirely on the right or left side of zero (dotted line), then the difference is significantly different from zero and may be due to congestion pricing.
This study was designed to isolate the effect of congestion pricing on air quality. Vehicle traffic contributes roughly 14% of PM2.5 and 20% of NOx emissions in New York City. If congestion pricing changed traffic volumes in the CRZ, adjacent neighborhoods, or neighborhoods near major routes around the CRZ, it could therefore change the level of air pollution in those communities.
Determining the extent that air quality changes due to any policy is challenging, since many things impact air quality from year to year, such as average temperatures, rainfall, the presence of wildfires (both here and in other parts of the U.S. and Canada), the electrification of buildings and vehicles, the price of gasoline, and changes in regulations here and in other states. A simple before-and-after comparison would not tell us how much of the change is due to congestion pricing. So instead, we studied the difference between the air quality levels we measured after congestion pricing and estimates of what the air quality would have been if there were no congestion pricing. We did this at each of the EJ neighborhood sites, for the CRZ as a whole and the rest of NYC.
We did this by comparing the air quality results at locations where traffic volumes may have changed due to congestion pricing to a location where there was no expected change in traffic volumes. This study selected a site on the Van Wyck Expressway as its control for two reasons: it is a high-traffic highway without parallel, alternative routes, so any changes in traffic should reflect general background trends, and modeling results from the environmental review process predicted that there would be no change in traffic due to congestion pricing. We used before-and-after data from the control site to help predict what the air quality would have been elsewhere if congestion pricing had not been implemented.
This gives us projected “business as usual” air pollution values. We then use statistics to compare the projected to measured observed air quality data after congestion pricing was implemented. If the difference between the projected and observed values is statistically different than zero, then air pollution levels were affected by congestion pricing.
This is called an interrupted time series study with control. In addition, the model takes into account temperature, wind and precipitation, and both long term and seasonal trends in air pollution. To date, the Health Department’s analysis is the most comprehensive evaluation of congestion pricing’s impact on air quality.
To understand the results from our air pollution analysis, we also looked at the change in total traffic and truck traffic counts on the highways in the EJ communities. Like air quality, traffic is impacted by many factors beyond just congestion pricing. Weather, road closures, construction, regular maintenance, crashes and disabled vehicles, emergency response, and other events can all impact vehicle volumes. The periodic traffic counts collected were seasonally adjusted and compared before and after congestion pricing to see if changes aligned with the patterns we saw in the air pollution levels. More detail on traffic data collection and analysis can be found in the Detailed Traffic Analysis Methodology Appendix.