Pattern Analysis for Beginners

Two disciplines use the same word. Only one of them earns it.

Two Things People Mean When They Say “Pattern”

I hear the word “pattern” used two ways in this work, and only one of them means what it claims to mean.

The first is forensic. An examiner looks at a fingerprint, a bite mark, a tool mark, a shoe impression, or a strand of hair under a microscope, compares it to a known sample, and declares that the two share a common source. The language is scientific. The certainty is often absolute. “To the exclusion of all others” is a phrase that has been used in American courtrooms for decades to describe fingerprint conclusions.

The second is investigative. It’s the kind of pattern analysis that separates a single bad interaction with an institution from a documented, provable pattern of institutional conduct: repeated denials that follow the same script, outcomes that cluster in a way random chance doesn’t explain, procedural delays that only ever seem to happen to the same category of litigant. This kind of pattern claim has to be built, incident by incident, source by source, before it can hold up to scrutiny.

Both disciplines use the word “pattern.” Only one of them has to prove it.

How Forensic Pattern Matching Claims to Work

Most forensic pattern disciplines follow a version of the same basic process: an examiner looks at two samples side by side, the questioned item and the known item, and identifies features considered similar enough to indicate a common origin. Latent fingerprint examiners generally describe this as Analysis, Comparison, Evaluation, and Verification, a four-step process sometimes shortened to ACE-V. Bite mark examiners compare the pattern of an injury to a dental cast. Firearms examiners compare striations on a bullet or cartridge case to a test-fired sample from a specific weapon. Footwear examiners compare an impression at a scene to the sole of a specific shoe.

What most of these disciplines have in common is that the final judgment is subjective. There is no population database that tells an examiner how common a given set of dental features is, the way a DNA database can quantify how rare a genetic profile is. There is no agreed statistical threshold for how many matching striations on a bullet mean a positive identification. The examiner’s opinion is the method. The confidence in the result depends entirely on how well that opinion has been tested, and for most of these disciplines, it hasn’t been tested nearly as often as courtroom testimony implies.

Where the Science Falls Apart

In 2009, the National Research Council, part of the National Academy of Sciences, published a report titled “Strengthening Forensic Science in the United States: A Path Forward.” Congress had ordered the study. The conclusion was blunt: apart from nuclear DNA analysis, the committee found that no forensic method had been rigorously shown through adequately designed research to reliably and consistently connect evidence to a specific individual or source. The report’s central recommendation, an independent federal body to set and enforce forensic science standards, was never funded into existence.

Seven years later, the President’s Council of Advisors on Science and Technology took a narrower look at six specific forensic “feature-comparison” methods: DNA analysis of single-source and simple-mixture samples, DNA analysis of complex mixtures, bite mark analysis, latent fingerprint analysis, firearm and tool mark analysis, and footwear analysis. PCAST’s 2016 report found that only two of the six had what it called foundational validity: single-source and simple-mixture DNA analysis, and latent fingerprint comparison, the latter with real caveats about examiner bias. Complex-mixture DNA needed more evidence. Firearms and tool mark analysis and footwear analysis lacked adequate validation studies. Bite mark analysis fared worst of all. PCAST concluded that the discipline falls very short of meeting the scientific standards for foundational validity, and advised against spending further resources trying to fix it.

Hair microscopy comparison wasn’t one of PCAST’s six categories, but it had already been the subject of a joint review by the FBI, the Department of Justice, the Innocence Project, and the National Association of Criminal Defense Lawyers. In April 2015, that review reported that FBI examiner testimony contained erroneous statements in at least 90 percent of the trial transcripts analyzed. In the 268 cases where an examiner’s testimony was used to inculpate a defendant at trial, 257, or 96 percent, contained errors. Thirty-five of the defendants in those cases had received the death penalty. Thirty-three of those death penalty cases contained errors. Twenty-six of the twenty-eight FBI examiners involved in the review had given erroneous testimony or submitted erroneous lab reports at some point. This was a technique the FBI itself had trained state and private crime labs to use for decades.

None of this is abstract. In May 2004, the FBI arrested Oregon attorney Brandon Mayfield as a material witness in the investigation of the Madrid train bombings, after senior FBI fingerprint examiners called a partial print found on a bag of detonators a match to Mayfield, with the identification described internally as a “100 percent” match. Mayfield had no connection to the bombing. Two weeks after his arrest, Spanish National Police identified the actual source of the print, an Algerian national. The FBI publicly apologized and later paid Mayfield a $2 million settlement. A Justice Department review afterward concluded the FBI had departed from its own verification procedures, in a discipline PCAST would later rate foundationally valid.

The Innocence Project’s own data on DNA exonerations lists misapplied forensic science as a contributing factor in a substantial share of the wrongful convictions the organization has helped overturn, the second most common contributing factor behind eyewitness misidentification.

What Real Pattern Analysis Requires

Investigative pattern analysis works on the opposite principle. Instead of one examiner comparing two samples and asserting confidence, it requires multiple, independently sourced incidents that corroborate each other before anyone is allowed to call something a pattern rather than a coincidence.

That distinction matters because a single bad experience with an institution, one denied appeal, one dismissive caseworker, one inexplicable delay, doesn’t prove anything about how that institution operates generally. It might be an outlier. It might be a mistake that gets corrected the next time. A documented pattern requires more: separate incidents that didn’t originate from the same complaint or the same source, records or accounts that corroborate each other independently, evidence the institution had notice of the conduct, and a documented record of whether the institution responded to that notice or let it continue.

This is slower than declaring a match. It’s also the version that survives scrutiny. A pattern claim built this way can withstand a defense attorney’s cross-examination, an editor’s fact-check, or an insurer’s risk review, because every piece of it traces back to something documentable. A pattern claim built on a gut feeling about how these things always go cannot.

How to Tell the Difference in Your Own Case

If someone tells you a pattern exists, whether it’s a forensic examiner testifying that a bite mark matches your case, or an institution insisting that what happened to you was an isolated incident rather than part of something larger, the same questions apply. Ask what the underlying method is, and whether that method has ever been tested against known outcomes to establish an error rate. Ask whether the conclusion rests on one comparison or on multiple independent incidents. Ask who reviewed the underlying data, and whether that reviewer had any stake in the outcome. Ask what would have to be true for the claim to be wrong, and whether anyone checked.

None of these questions require a law degree or a forensic science background. They require treating certainty as something that has to be earned, not something that comes free with confident language.

Why This Distinction Matters

Overselling forensic pattern evidence and dismissing a documented institutional pattern as a series of unrelated incidents point in the same direction: both let an unproven claim of certainty stand in for actual analysis. One puts people in prison on the strength of a comparison nobody validated. The other lets accountability efforts get waved off as isolated complaints when the record says otherwise.

The word “pattern” doesn’t do the work by itself, in a lab or in an investigation. The methodology behind it does. Anyone learning to evaluate a pattern claim, in a courtroom, in their own case file, or in an institution’s public record, is really learning to ask the same question every time: is this conclusion built on tested method, or is it built on confidence that nobody has tested?