From the National Academy of Sciences, to the Washington Post, to Last Week Tonight With John Oliver, there seems to be consensus that additional research is needed to validate several fields of forensic science.
One challenge in collecting sufficient evidence to establish foundational validity is that because there is no ground truth in forensic casework, it is impossible to assess the accuracy of the work being done by crime lab analysts. So-called “open” proficiency tests, in which analysts are given mock case samples to analyze, cannot be trusted to produce reliable evidence of accuracy rates because use of known proficiency tests is subject to the Hawthorne Effect–a phenomenon in which people behave differently when they know they are being watched. In a 2018 post, we discussed a new program of “blind” testing being implemented by the Houston Forensic Science Center (HFSC).
The HFSC program uses lab managers as a “buffer” in-between clients and analysts to shield analysts from case irrelevant information (for example, a confession from a suspect/defendant) which can be a source of bias. This also allows the lab to present mock case samples to analysts in a manner that will make them indistinguishable from true casework in order to obtain a more accurate measure of the accuracy of their conclusions. A recent article from the Houston Law Review describes the HFSC program in further detail and contends that it should serve as a roadmap for other labs to collect the evidence necessary to validate forensic methods and to ensure the quality of their work.
The article describes in detail how the program was implemented in three of the lab’s six disciplines (toxicology, firearms, and latent fingerprint sections). The authors address how the HFSC has responded to various challenges that arose as they implemented their program– additional costs, obtaining cooperation from law enforcement, addressing the possibility of resistance from analysts, and creating samples that appear authentic enough to fool analysts. They also emphasize the success of the program thus far– the HSFC has already begun accumulating the evidence necessary to gauge the validity of their methods.
While more information is needed to determine whether or not these blind tests could be an appropriate solution to the problem of validating some forensic disciplines, it would seem that experts are encouraged by the results thus far. Regardless of whether or not blind proficiency tests prove to be the solution to validating some forensic disciplines, it seems that their use would offer courts important information about the accuracy of some forensic methods and improve the reliability of expert testimony.