Wisconsin Lawyer
Vol. 83, No. 9, September 2010

by Amelia L. Bizzaro

Not only does forensic science play a role in most criminal cases, it also is now part of mainstream American culture, thanks in part to the prolific and popular “CSI” television series and several other programs that depict crime solving. Prosecutors may be faced with jurors who expect forensic evidence to be as clear-cut as it is on television, where it is easy to understand, interesting, and 100 percent accurate. Jurors faced with lengthy expert testimony discussing complex scientific principles may disbelieve the evidence, holding prosecutors to an impossible standard. At the same time, it also is possible that given their familiarity with some forensic science terms, jurors may believe the evidence to be more accurate than it really is, aligning it with their television experiences in which the evidence is always infallible. As a result, lawyers on both sides of cases are faced with the daunting task of convincing jurors that art does not always imitate life.

The task of painting an accurate picture of forensic science evidence, while still difficult, has become somewhat easier since a committee of the National Academies of Science (NAS) National Research Council published its report, Strengthening Forensic Science in the United States: A Path Forward (hereinafter, the NAS Report). The NAS Report, created by a diverse group of scientists, academics, and legal scholars, called into question the basis for several different forensic science disciplines, made sweeping recommendations for reform, and called for standardization, accreditation, and independence for the nation’s crime labs.

The report questioned the reliability of most forensic science disciplines, with the exception of DNA-evidence research, noting that such research is the only discipline that “has been rigorously shown to have the capacity to consistently and with a high degree of certainty support conclusions about individualization (more commonly known as matching of an unknown item of evidence to a specific known source).”¹ Compared to DNA-evidence research, the report opines, several forensic science disciplines fall woefully short. “The simple reality is that the interpretation of forensic evidence is not always based on scientific studies to determine its validity.”²

On a national scale, the report’s findings and recommendations are slowly being implemented through legislation. The Senate Judiciary Committee made public a draft outline for legislation in response to the NAS Report. The legislation calls for the creation of the Forensics Science Commission (FSC), which would be made up of members appointed by the President based on recommendations from the NAS and the American Academy of Forensic Sciences. If created, the FSC will be responsible for setting “rigorous standards for accreditation,” determining which disciplines require certification and the standards for such certification, developing a “comprehensive strategy for increasing and improving peer-reviewed scientific research related to the forensic science disciplines, including research addressing issues of accuracy, reliability, and validity in the various disciplines,” and establishing “standard protocols, methods, practices, quality assurance standards and reporting terminology for each applicable forensic science discipline in order to ensure the quality and integrity of the data generated.”³

Although no such legislation is pending in Wisconsin, the report is still a helpful tool for any lawyer seeking to admit or challenge forensic science evidence. Use of the report, however, depends largely on the forum, the case, and the type of evidence.

Admission of Scientific Evidence

The relationship between science and law has long been tumultuous. The legal system usually relies on the adversarial system to root out the truth, while the scientific community uses empirical analysis. Law and science often collide in the courtroom, where lawyers often find themselves exploring scientific concepts they may not fully understand. Similarly, judges, who also often lack a scientific background, are put in the untenable position of deciding whether evidence is relevant, and at least to some extent, reliable. Perhaps the Hon. Harry T. Edwards, cochair of the committee that authored the NAS Report, said it best: “I started the NAS project with no skepticism regarding the forensic science community. Rather, I assumed, as I suspect many of my judicial colleagues do, that the forensic disciplines are well grounded in scientific methodology and that crime laboratories and forensic practitioners follow proven practices that ensure the validity and reliability of forensic evidence offered in court. I was surprisingly mistaken in what I assumed.”⁴

Whether in state court or federal court, there are at least two underlying issues for every piece of forensic evidence offered for admission: 1) the extent to which the particular forensic discipline is based on reliable scientific methodology, and 2) the extent to which the expert’s conclusion depends on his or her own interpretation, which may be colored by error or bias and may lack operational and performance standards.⁵ The less science is involved, the more subjective the conclusion.

However, the impact of these issues depends largely on the forum. In federal court, the standard for the admission of scientific evidence has evolved. It began with the landmark case Frye v. United States,⁶ in which the court ultimately held that evidence was not admissible unless it was generally accepted. Fifty years later the test changed with the implementation of Federal Rule of Evidence 702, which required only that the evidence “assist the trier of fact.” What the promulgation of this rule meant for Frye was hotly debated until nearly 25 years later, in 1993, when the U.S. Supreme Court decided the landmark case Daubert v. Merrell Dow Pharmaceuticals Inc. The Court held that Rule 702, not Frye, controlled, but that the “trial judge must ensure that any and all scientific testimony or evidence admitted is not only relevant, but reliable.”⁷ The Daubert court emphasized that evidentiary reliability must be based on scientific validity and provided a list of factors to consider, including whether the scientific theory or technique had been tested, subjected to peer review, and accepted.

The evolution of requirements for the admission of evidence did not end with Daubert, however. In 2000, Rule 702 was amended to permit the admission of expert testimony so long as the testimony is based on sufficient facts or data, the testimony is the product of reliable principles and methods, and the witness has applied those principles and methods to the facts of the case.

Wisconsin courts have adopted the version of Rule 702 that was in place before Daubert as the test for the admissibility of expert testimony. Thus, so long as the expert is qualified and the testimony will assist the trier of fact and is relevant, it is admissible. Unlike federal courts, which permit the holding of pre-trial evidentiary hearings to determine the reliability of evidence before ruling on its admissibility, Wisconsin courts rely on juries to distinguish charlatans from scientists. Circuit court judges have “considerable discretion in determining the admissibility of expert testimony.”⁸ “First, the expert’s principles, methods, and tests must be ‘reliable enough to be probative’; that is, a reasonable jury must be able to find them reliable (the standard of conditional relevancy). Second, the trial judge has discretion as a ‘limited gatekeeper’ to limit or exclude expert testimony based on a number of factors, including the consumption of time and the degree to which it assists the trier of fact.”⁹

Because of the immense amount of discretion vested in both federal and state courts to admit or exclude expert testimony, appellate court decisions are confined to determining whether the lower courts abused discretion, a highly deferential standard difficult to overcome. As the NAS Report acknowledged, it is difficult to know just what is happening at the trial court level because district courts do not routinely publish decisions. “Reported opinions in criminal cases indicate that trial judges sometimes exclude or restrict expert testimony offered by prosecutors; reported opinions also indicate that appellate courts routinely deny appeals contesting trial court decisions admitting forensic evidence against criminal defendants.”¹⁰ The same is not true in civil cases. In civil cases, the NAS Report found, the parties are more equally matched in terms of their ability to introduce forensic evidence. “And, ironically, the appellate courts appear to be more willing to second-guess trial court judgments on the admissibility of purported scientific evidence in civil cases than in criminal cases.”¹¹

While federal court appellate decisions concerning the scientific underpinning of evidence are hard to come by, relevant state court decisions are virtually nonexistent. Appellate courts are not in a position to decide whether evidence from a specific forensic science discipline is admissible, given that their review is almost always limited to whether there was an erroneous exercise of discretion. That is not to say, however, that appellate courts have never considered the admissibility of scientific evidence.

Federal courts have specifically rejected comparative lead-bullet-analysis evidence, which parties used in attempts to match recovered bullets to a particular box of ammunition. Declared unreliable across the board, such evidence is per se inadmissible. Wisconsin appellate courts have directly addressed two types of expert evidence to date: polygraph evidence and psychiatric testimony of a defendant’s ability to form intent. The Wisconsin Supreme Court rejected polygraph evidence, in part because “the legal and scientific communities remain significantly divided on the reliability and the usefulness of the polygraph in a criminal case.”¹² Despite recognizing that the polygraph had some degree of validity and reliability, the court held the evidence inadmissible because it relied too much on the examiner’s subjective evaluation and on factors that could not be reliably quantified.¹³

Similarly, the Wisconsin Supreme Court held that expert psychiatric testimony regarding a defendant’s capacity to form intent was inadmissible when based on the defendant’s mental health history. “There is substantial doubt whether evidence such as was sought to be introduced here is scientifically sound, and there is substantial legal doubt that it is probative on the point for which it was asserted in this case.” ¹⁴

On the trial court level, several federal district courts have begun to question whether firearm and tool- mark identification evidence, which seeks to match recovered bullets or casings to a particular firearm, meets the test for the admissibility of expert testimony. The Wisconsin Court of Appeals may also soon weigh in on this particular topic when it decides a pending case, State v. Jones.¹⁵ Simply because the appellate courts have not reached many decisions on the admissibility of specific forensic science disciplines does not mean that trial lawyers should avoid actively litigating the issues in the circuit and district courts. If nothing else is clear, it is clear that the trial court level is the only place to actively litigate forensic science issues, given the high standard of review.

The NAS Report

The NAS Report is a powerful resource concerning the admissibility of forensic science evidence. The U.S. Supreme Court, for example, cited the prepublication version with approval, noting that it is not “evident that what respondent calls ‘neutral scientific testing’ is as neutral or as reliable as respondent suggests.”¹⁶ Similarly, the American Academy of Forensic Sciences¹⁷ and the Board of Directors of the American Statistical Association¹⁸ have adopted the report.

The most valuable part of the report is its detailed evaluation of several forensic science disciplines. The report authors pored over journal articles and studies, solicited and listened to direct testimony, and conducted independent research. The authors concluded that the majority of the disciplines they evaluated call for the examiner to declare a match using subjective methodology completely lacking in scientific validity. As a result, the disciplines have similar problems: the conclusions reached often are prone to confirmation bias (an analyst’s predisposition to confirm that the evidence supplied by law enforcement matches the identified suspect) and cannot be replicated from one examiner to the next, sometimes not even by the examiner who declared the match. The NAS Report criticized analysts for lacking supporting documentation detailing their evaluation, given their willingness to testify to a zero-error rate.

The three most frequently used types of forensic evidence are friction-ridge analysis (fingerprints, palm prints, and sole prints), pattern/impression evidence (encompassing anything that can leave an impression of a pattern, like shoeprints and tire tracks), and firearm and toolmark identification evidence. All three disciplines involve comparing a recovered item (for example, a fingerprint from a glass door) to a known sample (the suspect’s fingerprint taken by law enforcement). Analysts view the recovered sample and compare it to the known sample, determining whether a match in fact exists. Unlike DNA analysis, few tools other than a microscope and the analysts’ own vision are used.

Overall, the NAS Report’s criticisms of each of these three disciplines were similar: the methodology is subjective; analysts cannot consistently replicate the results, in part because of a lack of documentation; analysts often improperly embellish the accuracy of their findings; and there is no valid, independent research supporting the methodology.

Amelia L. Bizzaro, Marquette 2003, is the principal at Bizzaro Law LLC, Milwaukee, and practices appellate law. She is on the board of directors for the Wisconsin Association of Criminal Defense Lawyers and is chair of its Oct. 8-9 seminar, Whatever Happened to the Science in Forensic Science. She is a member of the State Bar’s Appellate Practice Section and is cochair of the Milwaukee Bar Associations Bench/Bar Court of Appeals Committee. The Wisconsin Law Journal recently named her one of 2010’s up and coming lawyers. She can be reached at abizzaro@bizzarolaw.com.

The method for identifying fingerprints, for example, “does not guard against bias; is too broad to ensure repeatability and transparency; and does not guarantee that two analysts following it will obtain the same results.”¹⁹ Fingerprint experts often testify that their methodology, when done correctly, has a zero-error rate. Such a conclusion, the NAS Report concluded, “is unrealistic, and, moreover, it does not lead to a process of method improvement.”²⁰

When it comes to pattern/impression evidence, the NAS Report concluded, “there is no consensus regarding the number of individual characteristics needed to make a positive identification.”²¹ There is no research about “the persistence of individual characteristics, the rarity of certain characteristic types, and the appropriate statistical standards to apply to the significance of individual characteristics.”²²

The NAS Report, like several recent federal district court decisions, was perhaps most critical of firearm and toolmark identification evidence. “A fundamental problem with toolmark and firearms analysis is the lack of a precisely defined process.”²³ The controlling authority for declaring a firearm identification match lies with the Association of Firearm and Toolmark Examiners (AFTE), which advises examiners to declare a match when there is “sufficient agreement” between two sets of marks.²⁴ The AFTE’s definition “does not even consider, let alone address, questions regarding variability, reliability, repeatability, or the number of correlations needed to achieve a given degree of confidence.”²⁵

Because the majority of forensic science consists merely of having an analyst look at the evidence, prosecutors should exercise caution in admitting it, defense lawyers should be prepared to challenge it, and the courts should carefully consider whether such visual examinations, with nothing more, are really helpful to the trier of fact.

Challenging the Admissibility of Forensic Evidence and Limiting Experts’ Opinions

Given Wisconsin’s assist-the-trier-of-fact standard for the admission of expert testimony, litigators must challenge forensic evidence by demonstrating that the evidence in question is not reliable enough to be probative and is not helpful to the trier of fact. If, as is true within some disciplines, the experts cannot agree with one another about what constitutes a match, or even the terminology for describing a match, then experts within those disciplines cannot possibly help jurors figure it out. Certainly, when declaration of a match depends on what the expert sees, then jurors ought to be able to also see the match for themselves with the help of pictures. Simply declaring a match is not enough. Forensic evidence from disciplines that depend on the examiners’ subjective opinion to know a match when they see it should be inadmissible for the same reasons that polygraph and defense psychological testimony on intent are inadmissible.

Although it remains to be seen whether this argument will persuade state court judges, federal district courts have begun to limit the admissibility of some types of forensic evidence. More than one federal district court has limited the type of opinion experts can express to the jury in firearm and toolmark identification cases. In one case, the court concluded that allowing the examiner “to testify that he had matched a bullet or casing to a particular gun ‘to a reasonable degree of ballistic certainty’ would seriously mislead the jury.” As a result, it permitted the examiner to state his opinion “in terms of ‘more likely than not,’ but nothing more.”²⁶ Another court noted, “there is no reliable statistical or scientific methodology which will currently permit the expert to testify that it is a ‘match’ to an absolute certainty, or to an arbitrary degree of statistical certainty.”²⁷ The court refused to allow the expert to “assert any degree of statistical certainty, 100 percent or otherwise, as to a match.”²⁸

Federal courts, while not refusing to allow experts to testify, are at least limiting the conclusions of the experts. These limitations, while certainly a step in the right direction, are not enough. Attorneys must vigorously cross-examine the experts on their conclusions, particularly relating to the two main concerns expressed by the NAS Report: confirmation bias and the lack of documentation accompanying test results.

Analysts carelessly use terms like “match,” “consistent with,” “identical,” “similar in all respects tested,” and “cannot be excluded as the source of” without any agreement or consensus within the discipline about the meaning of these terms.²⁹ “The use of such terms can have a profound effect on how the trier of fact in a criminal or civil matter perceives and evaluates the evidence.”³⁰ Attorneys should force the analysts to define these nebulous terms on the stand. If an analyst does not know what those terms mean, how can a jury rely on his or her conclusions to any degree?

Similarly, analysts should not be permitted to simply supply a one-sentence report declaring a match without explaining more. Reports should, at a minimum, describe “methods and materials, procedures, results, and conclusions, and they should identify, as appropriate, the sources of uncertainty in the procedures and conclusions along with estimates of their scale (to indicate the level of confidence in the results).”³¹ Analysts who omit these critical details should be forced to explain themselves on the stand. Again, an inability to do so will show the jury how little science is really involved.

Although most lab reports do not reveal it, most analysts also have notes from their evaluation, in addition to the report. These notes may explain what the reports do not (but also may be nothing more than the same one-line sentence declaring a match). Arguably, the state is required to provide these materials in response to a discovery demand because they are statements of a witness, and failure to do so is reversible error unless the state can prove harmless error.³² The notes may reveal further avenues of cross examination, not for what the notes say but for what they do not, particularly involving disciplines in which the analyst declares a match, like friction-ridge analysis, pattern/impression evidence, and firearm and toolmark identification evidence. If the analyst’s notes do not describe the supposedly unique marks, where they are located, or what makes them unique, then the analyst certainly will not be able to testify to those things several months later at trial.

In addition to the state’s obligation to provide relevant materials, attorneys have an obligation to become familiar with the problems within a particular discipline and to seek to challenge admissibility of the evidence and the analysts’ opinion about it. Attorneys must avail themselves of readily available information calling into question the reliability of a particular forensic discipline.

Conclusion

The NAS Report is a valuable tool that looked closely at the state of our nation’s crime labs and the scientific validity of several frequently used forensic science disciplines. Although the law often lags behind developments in science, attorneys must continue to challenge trial courts to be more active gatekeepers by challenging the underlying premise of forensic evidence, despite the fact that the evidence may have been generally accepted in the past. Such challenges are primarily fact-based and achieved through cross-examination, because very few appellate court decisions have addressed specific disciplines.

Endnotes

Wisconsin Lawyer