The President’s Council of Advisors on Science and Technology (PCAST) report on Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods was released on September 20, 2016. (Available for free download here.) The PCAST report gives an in-depth look at the current state of certain forensic science disciplines. It makes recommendations as to the use of forensic science evidence in court, improvements to be made in research and improvements to be made in forensic science in general. This report will be very useful in the courtroom, as it uses a framework to assess scientific validity that mirrors the NC Rule of Evidence 702, which governs the admissibility of expert testimony.
The report evaluates whether eight areas of forensic evidence are foundationally valid and validly applied, which are equivalent to prongs two and three of Rule 702 (whether the testimony is the product of reliable principles and methods (prong 2) and whether the witness has applied the principles and methods reliably to the facts of the case (prong 3)).
The summary for each discipline examined is as follows:
- DNA analysis of single-source and simple-mixture samples: Foundationally valid, but examiners should undergo rigorous and relevant proficiency testing, should routinely disclose in reports and testimony whether he or she was exposed to any facts of the case that might have influenced the conclusion, and should disclose upon request all information about quality testing and issues in the lab.
- DNA analysis of complex-mixture samples: Objective analysis based on probabilistic genotyping is new but promising; more research by independent scientists pertaining to accuracy in different scenarios is needed. Some methods may be valid for use in certain situations (e.g., an adequate sample with up to 3 sources and at least a 20% contribution of the smallest contributor).
- Bite mark analysis: Bite mark analysis as used to identify a source of a bite mark or to establish whether an injury is due to a human bite mark is scientifically unreliable, and additional research in this area is not warranted as it is unlikely to be developed into a reliable and accurate methodology.
- Latent print analysis: Has a foundationally valid subjective methodology, but the false-positive rate is substantial and likely to be higher than expected by many jurors. Conclusions need to be accompanied by accurate information about reliability and false positive rates. Several steps should be taken to further strengthen latent finger print analysis, including blinded proficiency testing incorporated into regular casework, limiting an examiner’s access to potentially biasing information and applying a “linear” analysis (examining and documenting features of the unknown sample before comparing them to a known sample). All of these factors should be considered when determining whether the techniques were validly applied in the case at hand.
- Firearms analysis: PCAST concluded that the foundational validity of firearm analysis had not been established: only one study was found that examined accuracy and reliability (and that met the criteria for how such a study should be designed and conducted). The false positive rate was 1 in 66 examinations (upper 95th percentile confidence limit 1 in 46 cases). PCAST recommended additional studies using representative samples to establish accuracy and the creation of databases to establish prevalence of features.
- Footwear analysis: Foundational validity has not been established, as no properly designed empirical studies to evaluate accuracy in associating a shoe print to a specific shoe have been conducted. Additional research should also include creation of databases to establish prevalence of features.
The PCAST report did not do a comprehensive evaluation of microscopic hair analysis, but did review the Department of Justice’s (DOJ) recently released proposed Uniform Language for Testimony and Reports, and its Supporting Documentation, for this discipline. PCAST noted that DOJ’s statement that “microscopic hair comparison has been demonstrated to be a valid and reliable scientific methodology” is not supported by the available studies. The report specifically noted an FBI study from 2002 comparing microscopic hair analysis to DNA analysis in 170 samples, in which a false positive rate of 11% was found.
The PCAST report also gives suggestions as to how to improve research in forensics fields. To this end, it calls upon the National Institute of Standards and Technologies to take on the task of determining if new and current fields are foundationally valid as defined by the document. It also recommends that more unaffiliated scientists be consulted by the Organization of Scientific Area Committees. The report calls for the creation of a Metrology Resource Committee which it suggests be made up of independent scientists. This committee would review the OSAC’s standards with an eye towards their reliability and validity.
Also included in the report are suggestions for improving the use of forensic evidence in court. The report calls on the U.S. Attorney General to direct attorneys appearing on behalf of the USDOJ to ensure expert testimony meets the standards for scientific validity (p. 140-141). Furthermore, it provides guidance to judges in determining the scientific validity as a foundation for expert testimony. The report suggests that judges review false positive rates, the method’s sensitivity, the sufficiency of validation studies, the appropriateness of proficiency testing, adequacy of procedures used and documentation and whether appropriate limits on reporting language were employed. It also recommends that best practices materials and training on scientific evidence be made available for judges (p. 145).
The PCAST report also recommends that for each field, the proponent of the forensic evidence
- Demonstrate the capability of the analyst through routine, blinded proficiency testing.
- Demonstrate that the techniques were reliably applied in the case by providing a complete description of the procedures, results and laboratory notes.
- Utilize comprehensive and accurate reporting and testimony, including information from the empirical studies of false positive rates and sensitivity, information on the comparability between the types of samples used in these empirical studies and the sample(s) available in the particular case, and an accurate portrayal of the probative value of the observed features (i.e., how common or rare the features are, based on empirical studies).
This report will be useful for attorneys considering a challenge to testimony by forensic experts, as it explains in an accessible way what must be shown to demonstrate reliable principles and methods and reliable application of those principles and methods in a particular case. It is a worthwhile read for every criminal advocate as it suggests a framework for making a rigorous challenge to the admissibility of expert testimony.
The National Association for Public Defense is offering a webinar entitled, Denuding the Emperor: Understanding and Using the PCAST Forensic Science Report at 1:00 pm on Dec. 2, 2016. Andrew Northrup, Assistant Public Defender in the Maryland Office of the Public Defender will discuss how the PCAST report can be used in the courtroom. The program is free of charge for NAPD members and $20 for non-members.
I would like to thank Glinda Cooper, Director of Science and Research for the Innocence Project for her contribution to this post.