One of the first lessons you learn as a criminal defense attorney, particularly an attorney working on innocence cases, is that nothing is as it seems. Those stories you thought were too crazy to be true? Totally happened. And all that stuff you thought was fact? It’s not. This is particularly true for pretty much everything you think you know about forensic “science,” where the truth is, with the exception of DNA testing, there is very little actual “science” involved.
In fact,most forensic sciences– e.g., handwriting, bitemark, fire pattern, blood spatter, bullet and tool marking analysis–are open to subjective interpretations. Which means when we allow testimony about these “sciences” at trial, what we’re really doing is allowing juries to pit an expert’s credibility (and unreliable testimony) against that of the defendant’s. Considering the supposed “expert” is typically some combination of educated, middle- or upper- class, white and male, and that most defendants come from disadvantaged backgrounds, you can guess how the jury (which is also typically made up of white folks) will decide the case.
But it’s not just the expert’s testimony that is skewed against the defendant. Often, the very request for testing will skew the results themselves. Although state crime labs technically work for both the prosecution and the defense, requests for testing are typically submitted by the District Attorney. Included on the tests is information about the crime necessary to perform the testing. For example, if you have clothing that you want tested for trace DNA, it would be helpful to know that the victim had been strangled, and so they should look for DNA near the collar of the shirt. But along with this necessary information comes prejudicial information about the defendant, what the police suspect happened, and what they expect to find. Thus, by the time the lab technician even begins to analyze evidence, they are already looking for a particular outcome.
This isn’t to say that all lab technicians are shady, vindictive people. It just means they are human. When the same people come to you day after day, you start to treat them better. Just like a barista might be nicer to her regulars. In this same way, technicians at crime labs come to favor the prosecution–it’s most often the State that’s requesting the testing. Similarly, lab technicians and forensic experts may just be trying to help. They genuinely believe their work is necessary to catch the “bad guy.” It’s not that they are maliciously motivated.
But all of these motivations aside, what results is a system that favors fraudulent science and hinders the defense. And you don’t just have to take my word for it. The situation has become so dire that last year, the National Academy of Sciences issued a report asking for regulation and oversight in the forensic fields. And just this last month, the News & Observer ran a four-part series detailing the corrupt and fault practices of the SBI crime lab in North Carolina. According to the investigative report, SBI agents distorted the rules to yield the desired test results of the prosecution more than a dozen times when the truth threatened to undermine their cases.
When most people hear science has exonerated an incarcerated person, they tend to think of DNA evidence. But the truth is, a lot of exonerations occur after scientists testify that the forensic science presented at trial wasn’t actually science. Take for example, the two men exonerated in Mississippi after scientists explained that marks on the bodies that had previously been identified as bite marks had actually created by crawfish. Or the case of Audrey Edmunds, which highlights the problems with the very notion of Shaken Baby Syndrome. (N.B. No one has yet proven this phenomenon exists.)
Critics have called for reformation of the system to reflect these biases. And clearly something needs to be done. But until it is, it’s hard to imagine that justice can be reached when our “objective” facts are really just subjective interpretations.
********UPDATE**********
Three convictions based upon faulty SBI work in the NC labs ultimately resulted in executions.
“It just means they are human.” QFT
Great post. It’s fascinating how belief in the infallibility of science and the scientific method has taken on cult-like proportions. The fact is that scientist are people and people make mistakes. People also have biases. I hope that more people can recognize this without going backwards into the whole “scientist be lying” ridiculousness.
Another reason to not trust the suits working for the state. They are such slimy creeps.
There’s a great article by Gerd Gigerenzer that touches on this a little, but is really focused on statistical literacy in health (http://www.psychologicalscience.org/journals/pspi/pspi_8_2_article.pdf). Researchers asked people which of several kinds of tests (DNA, fingerprinting, HIV, mamography, horoscope) were “absolutely certain.” No such tests are ever absolutely certain, but 78% of people thought DNA was and 63% thought fingerprinting was.
I think we want something airtight to believe in that causes us to not have to doubt. And yet, though we may doubt religion, we would be wise to doubt science too! Not in a sort of “there-is-no-global-warming” kind of doubt, but an understanding that so much about all of reality is entirely subjective.
[...] When Science is SubjectiveFeministe (blog)When most people hear science has exonerated an incarcerated person, they tend to think of DNA evidence. But the truth is, a lot of exonerations occur after … [...]
The problem is that people don’t want to think–on all parties. Prosecutors, judges, police officers, juries all want things to go their way, and the average person wants definitive statements because that’s the step up from the usual ambiguous morass of their lives. If a test sez 65%, all nonpersipacious people merely ask if that’s good or bad and go by that–and resent you if you try to give them details and multiple choices. I always kinda chuckle when it comes to DNA tests in trials. There is almost no point in putting most high science before a random jury. I mean, DNA tests are quite reliable, but they can be (mis>aimed. There are tons of fingerprint and ballistics test gone wrong, and police/prosecutors still refuse to avoid techniques that cause false positives, from line-ups to dna tests.
It’s like scientists are wizards with that crystal ball and a majestic beard, and they can peeeeeeer into the minds of the GUILTY!
Comrade Kevin…Good luck with that, but that requires being wrong and having to learn tolerance and humbleness (oh! and plenty of history, literature, science, and math!), and plenty of people are very allergic to that.
Justice and science (and plenty of reading about history of criminology) makes me despair for humanity.
This post isn’t about science in general. It’s about forensic science in particular. The practice of forensic science is not representative of the practice of science in general. It seems to me that that’s one of the points Tricia is making, but two of the first four comments took the post as a comment on science in general.
Actually, DNA science can often be subjective as well. People often do not understand the statistics behind the figures. This New Scientist article points out some of the pitfalls. DNA testing only utilizes a few small set of markers from the entire genome (we could use more now but many oppose more accurate tests), which means that chances of a random overlap with an unrelated individual are higher. Also, as the use of large DNA database trawling grows, we will see people picked up by chance with this technology. If the statistics say there is a one in million chance that he is not the individual that did this, and you have a database of 40 million people, you will have 40 potentially guilty guys.
And to add onto what Sonia sez, one *might* think that oh! 40 guys in a country? How likely is such an evil-twin hypothesis? Contamination. Again, as Sonia sez, when more and more people are in that database, database issues will be just as paramount as the usual lab contamination/corruption issues.
I think one thing has to be clear, right here and now. Statistics is really great for giving you a picture of what you want to see, rather than what you need to see. It’s about comprehension of contexts, in other words, and the use of sophisticated statistics really should be banned outside of strictly guidelined and stipulated entered evidence. It’s one of the easiest way for those with authority, whether on the offense or defense, to strip away Justice’s blindfold and lead a jury to a conclusion. Certain classes of evidence just don’t belong in courtrooms unless it’s heavily sanitized. We really need a system of magistrate judges for pre-trial hearings on evidence for sophisticated cases, whether that be dna or money-laundering.
shah8, if you don’t mind my asking, what is your scientific background? I’m not being snarky; I read a similar comment from you on Pandagon and I’m trying to figure out where you’re coming from — the scientific side? Legal side? Etc?
@shah8: You missed the message by a mile. “40 guys in a country? How likely is such an evil-twin hypothesis?” It is not an evil-twin hypothesis, it is simply the effect of limited set of markers. The more markers you have, the fewer number of false positives you will have.
This is idiocy. Simple-minded statistics is how we ended up in this soup. People see a statement like ‘the test is 99.99% accurate’ as a guilty sentence without taking into account things like false positive rate, false negative rate, and the general rate of whatever is being tested for in the larger population. Check out Prosecutor’s fallacy. There is no way around learning statistics in this world.
Formally? Undergrad bio with emphasis on neurobiology and materials science/engineering. Beyond that? Just books
Also important to note that even if you can say that DNA comes from a given individual with sufficient certainty, it’s still just DNA. It’s not something that’s only ever exuded by criminals during the commission of the act. If you so much as speak to a murderer, it’s possible traces of your DNA might be found at the scene of the crime.
“we would be wise to doubt science too!”
And any good advocate of science will remind people that the scientific method is intended to *be* systematic doubt. When forensic experts claim that fingerprint analysis is 100% accurate, it only proves that we have *no idea* how inaccurate it is.
Judges are supposed to act as gatekeepers to make sure that the experts use sound principles and express conclusions supported by their data, but I’m not sure they call all that tight a strike zone on the prosecution’s experts in criminal cases. I don’t think they scrutinize the expert testimony the way they do in, say, high-dollar civil cases in the federal courts. And I say that as something of an inside observer.
Also, there have been a number of cases of straight-up bias where crime labs deliberately and systematically slant to the prosecution.
I’m surprised that you didn’t mention the case of Cameron Todd Willingham who was executed based largely due to the false testimony of an “expert” in fires who testified that Willingham intentionally set the fire that killed his daughters. In addition to being a tragic story from start to finish, it provides a compelling story about how and why mistakes happen and how the witnesses convinced themselves that Willingham was a killer as the “evidence” emerged.
Also, my understanding is that very few methods used in forensics have emerged from the scientific community (with the exception of DNA, really) and that they’re mostly based on anecdote.
Its worth nothing too that, once you get outside of the most well-traveled ground of basic physics and chemistry, what we call science is subject to a whole range of biases and pulls that people who have never been part of academia often aren’t aware of. A fair amount of science is produced at universities where internal politics has a strong affect on what gets studied by whom. Personal interests play into lines of inquiry even before you get into the potential problems of messy design and methodology. Grant money influences the kinds of things that get studied and the ways in which things are conceptualized. Few students are going to write a doctoral thesis that attacks the interests (much less the work) of people who are going to sit on their committees or show up at their dissertation defense. Once you get outside of the universities you run into the issue of having to satisfy whoever is footing the bill and science being used to solve very specific problems rather than to increase general knowledge or understanding of our world.
There is also the problem of people trying to extend science further than science is meant to extend. One of the major statistical methods for judging data (testing against a null hypothesis) turns out to just not work for the vast majority of social science research. We’ve known this for decades, statisticians have explained why it doesn’t work, but doing the right kinds of statistical analysis would require massive numbers of subjects and would make certain questions answerable only through more subjective kinds of research (theory and case studies) so the social sciences continue to use practically worthless statistics as if they were a gold standard.
Lurking behind all of this is the problem that the way in which we conceptualize science is vulnerable to bias. Are we using a positivist model of understanding which searches for absolute Truth? Do we understand that math derived from a base 10 numbers system is essentially illusory? Do we expect everything to submit to an elegant “yes or no” binary? Do we trust our senses? Do our machines measure what we think they do?
None of this ought to devalue science. Science is still great. It helps us eradicate diseases and gives us all of the advancements we take for granted. The problem comes when people start to use science as magic, when we want it to tell us things that it cannot. All too often we ask science questions that science isn’t designed to answer and assume that, because we used the trappings of this particular kind of inquiry, the subjective answers we come to will somehow be less subjective. Much as we’re loathe to admit it, much of what we call “science” isn’t any different from a cargo cult.
It is most often those who don’t understand science that claim that it can do the most (they claim it can do practically everything). Unfortunately science education is very thin so people get just enough to be enamored with science but not enough to get the nuances and full understanding of what science does and its limits and end up completely missing the point. As a result most people are easily exploited when something “scientific” is presented to them. They are unaware of how science functions, the problems of funding and publication bias, its limitations, especially when it comes to the numerous problems with “quantifying” or evaluating subjective social and psychological phenomena with bad methodology and even, as OP stated, the limitations and subjectivity of forensic science which is seen by many as infallible. Also because science is so highly regarded (placed up as the harbinger of absolute Truth) many assume that anyone who is called a scientist will have no bias whatsoever and that bias will affect those subjective conclusions.
However, there are even scientists who fall into this, making the whole situation that much worse.
Wasn’t there a scandal a few years back regarding the FBI crime lab, similar to what this post describes about N.C.?
@Doc Alpert — Yes! You’re exactly, right! In all my ramblings, what I was trying to say is this – With the exception of DNA, forensic sciences are not science, do not comport to the scientific method, and even if proper procedures are followed, cannot be trusted. There is NO science to behind the rest of them. We haven’t even proven that fingerprints are unique, let alone bitemarks. Because we can’t show these “sciences” are sensitive and specific, they are inherently unreliable. That said, they can be *useful* in excluding people as suspects, but never in identifying someone.
@Pacian – You’re right that if DNA exists at the scene, it doesn’t mean someone committed the crime. The mere existence of DNA is usually only helpful in sexual assault cases, but even then, there are plenty of murky waters.
Yes! This exactly!
‘Which means when we allow testimony about these “sciences” at trial, what we’re really doing is allowing juries to pit an expert’s credibility (and unreliable testimony) against that of the defendant’s.’
I may be mistaken, but I thought we allowed the defence to call expert witnesses too.
Even with DNA, it can be used badly. A few months ago, I read a scary case of a guy who was convicted of a murder based on a DNA match in a California system with thousands of records. The statistical test they used has a high risk of false positives when used on large data sets. Evidence that prosecution and investigators improperly reported the statistical confidence was excluded from the trial and denied on appeal. I’ll have to see if I can look it up.
[...] a comment » I just caught this post over at Feministe: “When Science is Subjective” by Tricia. Some highlights: In fact,most forensic sciences– e.g., handwriting, bitemark, [...]
@FashionablyEvil: The Willingham case makes me sick. Literally, sick. It is one of those things I think of often (and sometimes write about, especially when something new happens in the investigation of the investigation, trial, and execution.). It was the first thing I thought of when I read this post.
I actually think the more distressing thing about the case was that the evidence from the arson “expert” wasn’t purposefully false. He wasn’t lying. He just was bad at his job. So, while there was no negligence on his part, per se, his entire method was old and faulty. He didn’t even know that the “science” he was practicing was bad.
And now, based on that testimony (which has been refuted multiple times over), Willingham is dead. Murdered by the state. Oh, it’s so sad.
[...] is not ironclad — lab scientists in criminal trials all too often get it wrong, and real people are [...]
WHAT, the criminal “justice” system is injust? “Science” can be manipulated by people harboring biases?
Can someone please explain this to my male friends and relatives who let me know that belief in “science” is unassailable and I am an idiot to ever question someone with “scientist” or “expert” in their title? IT’S SCIENCE!!1(tm)
There is an excellent book called “Picking Cotton” about eyewitness testimony (nonfiction). The woman was raped (during the rape, she concentrated on picking out identifying characteristics of the rapist) and picked a guy out of a lineup, then identified him in court as her rapist. He spent years and years in prison.
Only he didn’t do it.
Now they both advocate for changing how eyewitness testimony and identification is used. It was a horribly sobering story. (And adds to the caution of you never talk to the cops without a lawyer. Ever. The alleged rapist actually had an alibi for the night in question, but gave the wrong date at first, so his credibility was shot immediately.)