Estimated Validity and Reliability of On-Board Diagnostics for Older Vehicles: Comparison with Remote Sensing Observations

Based on requirements under the Clean Air Act Amendments of 1990, most state vehicle inspection and maintenance (I/M) programs have, since 2002, replaced the tailpipe emission testing with the on-board diagnostic (OBD) II testing for 1996 model and newer vehicles. This test relies on the OBD II system to give the pass or fail result, depending on certain conditions that might cause the vehicle to emit pollution 1.5 times higher than the regulated standard. The OBD II system is a computer and sensors installed in the vehicle to monitor the emission control units and signal if there is any malfunction. As a vehicle ages, its engine, pollution control units, and OBD II system deteriorate. Because the OBD II system's durability directly influences the test outcome, it is important to examine the fleetwide trend in the OBD II test results in comparison with an alternative measure of identifying high emitting vehicles. This study investigates whether the validity and reliability of the OBD II test is related to the age of the OBD II system installed in the fleet. Using Atlanta's I/M testing records and remote sensing device (RSD) data collected during 2002–2005, this research establishes the convergent validity and interobserver reliability criteria for the OBD II test based on on-road emissions measured by RSDs. The study results show that older vehicles exhibit significantly lower RSD–OBD II outcome agreement than newer vehicles. This suggests that the validity and reliability of the OBD II test may decline in the older vehicle fleets. Explanations and possible confounding factors for these findings are discussed. IMPLICATIONS This research demonstrates the potential worsening validity and reliability of the on-board diagnostic (OBD) II test in old vehicles. If the main source of low validity and reliability comes from the OBD II system malfunction, we expect this malfunctioning OBD II fleet will continue to grow in the future. If unchecked, the deterioration of OBD II system may impair the effort of the inspection and maintenance (I/M) program to identify high-emitting vehicles and the ultimate objective of reducing the air pollution from automobiles. This result is especially important in a regulatory context where technological and emissions standards dominate environmental policy and yet little attention is paid to the possible degradation of environmental monitors themselves.