Any scientific measurement has some error associated with it. The concept of “measurement uncertainty” means that for even the most carefully performed measurement, the value of the thing being measured can never be known exactly – only an estimated value can be given. Measurement uncertainty and error rates can come into play in forensic science whenever a numerical measurement is made, for example with blood or breath alcohol levels or even in measurements such as IQ scores.
This figure can be used to understand the concept of measurement uncertainty:
Here, a BAC was reported as 0.080. The shaded blue area represents all of the possible actual BAC levels and the probability that that measurement is the actual value.
An “error rate” is often used to address systematic error and insure the results reported account for uncertainty in measurement. Without an error rate, a lab leaves the inference that the test result is an absolute or true result.
Standards and Recommendations
ISO 17025 is an international standard for testing and calibration of laboratories and is the accreditation standard that the NCSCL lab is currently working to achieve. ISO standard 5.4.6 requires labs to estimate uncertainty of measurements.
Additional support for requiring measurement uncertainty to be calculated comes from the 2009 Strengthening Forensic Science in the United States: A Path Forward which states “[f]ew forensic science methods have developed adequate measures of the accuracy of inferences made by forensic scientists. All results for every forensic science method should indicate the uncertainty in the measurements that are made, and studies must be conducted that enable the estimation of those values.” (See p. 184.)
The Scientific Working Group for the Analysis of Seized Drugs (SWGDRG) states that “an understanding of uncertainty [is] fundamental to the interpretation and reporting of results.” See additional SWGDRG recommendations on Quality Assurance and Uncertainty here.
Attorney Ted Vosk has authored articles and presents nationally on this topic. Below are links to several of his articles and presentations:
- DWI: Trial by Numbers – Nov 2010 Champion Magazine article by Ted Vosk – provides an explanation of measurement uncertainty that is geared toward lawyers, but explains the math and statistics behind the concept.
- Introduction to Uncertainty – a 2011 manuscript prepared by Ted Vosk which explains to attorneys the concept of measurement uncertainty.
- Forensic Metrology: A Primer for Lawyers, Judges, and Forensic Scientists – This 153-page book is based on one of Vosk’s textbooks. Teaches the concept of measurement uncertainty to an audience of lawyers and scientists.
- Metrology: A Knowledge Base for Communication and Understanding – 2010 PowerPoint by Ted Vosk which explains several concepts related to measurement including validation, calibration, uncertainty. Includes helpful diagrams and sample reports that demonstrate the concepts presented.
- Two additional articles by Ted Vosk on related topics are, DUI Defense which focuses on whether standardized field sobriety tests are within the realm of lay understanding and the significance of not administering them according to NHSTA standards and City of Fircrest v. Jensen: Due process and science by legislative decree which discusses due process challenges to the accuracy and reliability of breath alcohol testing.
- CSAFE 2020: Statistics, CSAFE
This presentation, now available to view for free, was given as a part of the Center for Statistics and Applications in Forensic Evidence’s (CSAFE) 2020 All Hands Meeting. The meeting brings together researchers, forensic partners, and interested community members to discover potential areas for collaboration, highlight the organization’s achievements, and discuss goals for the future.
The presentation, given by Dr. Danica Ommen, a professor of statistics at Iowa State University, discusses the goals research projects, both past and future, within the world of forensic evidence. The presentation focused broadly on topics that are applicable to multiple disciplines and would benefit from more in-depth statistical investigation.
79th District Court for the County of Mason, Michigan – May 6, 2011 opinion holding that “calculation of an uncertainty budget or error rate and the reporting of the same is an essential element of the scientific methodology for analyzing blood alcohol content using gas chromatography” and denying the prosecution’s Motion to Admit the blood test results.
Motions and Briefs
May 4, 2011 Order Suppressing Defendant’s Breath-Alcohol Measurements in the Absence of a Measurement for Uncertainty
Sept. 21, 2010 Order Suppressing Defendant’s Breath-Alcohol Measurements in the Absence of a Measurement for Uncertainty, District Court of King County, WA
March 23, 2010 Memorandum Decision on Motion to Suppress, Snohomish County District Court, Washington holding that to allow the numerical value of blood alcohol tests without stating a confidence level violates Rule of Evidence 403 because the probative value of the evidence is substantially outweighed by its prejudicial value.
Measurement Uncertainty in the News
- Measurement Science and Standards in Support of Forensic Science, NIST News Release, 2/16/2012
- Judge Rules DUI Blood Tests Require Error Rate Report, Forensic Magazine/Lansing State Journal, 5/25/2011