Science AMA Series: I'm Cliff Spiegelman, let's talk about about flawed forensic science in the case of John F. Kennedy's assassination. AMA!
Hi Reddit!
I am Clifford Spiegelman. I grew up on Long Island and was a rather undistinguished High School student (rank about 80 out of 400). I graduated from the same small High School (Berner in Massapequa) that Alec Baldwin attended. I graduated with 3 majors math, stat, and economics in 1970 from SUNYAB. After a stint in the army reserves, I graduated with a PhD in Applied Mathematics/Statistics in 1976 from Northwestern U.
I spent 9 years working as a staff statistician at NBS (now NIST) and now 30 years at Texas A&M where I am Distinguished Professor of Statistics. I have a number of appointments including Official Statistician of the Texas Holocaust and Genocide Commission, and stat advisor to the Texas Forensic Science Commission. I cofounded Chemometrics and Intelligent Laboratory Systems in 1985 and am editor emeritus.
I will use the JFK comparative bullet lead analysis (CBLA) and firearm/toolmark evidence as examples of flawed forensic science. While the flaws in the JFK case are not extraordinary, people seem to care more about these flaws than 100s of other cases with less known victims and defendants. (See http://www.newsweek.com/jfk-assassination-modern-forensic-science-could-finally-solve-shooting-741292 and https://www.innocenceproject.org/flawed-forensic-science-misleads-more-than-juries/. )
I'll be back at 10 am ET to answer your question, ask me anything!
Science AMAs are posted early to give readers a chance to ask questions and vote on the questions of others before the AMA starts.
Guests of /r/science have volunteered to answer questions; please treat them with due respect. Comment rules will be strictly enforced, and uncivil or rude behavior will result in a loss of privileges in /r/science.
If you have scientific expertise, please verify this with our moderators by getting your account flaired with the appropriate title. Instructions for obtaining flair are here: reddit Science Flair Instructions (Flair is automatically synced with /r/EverythingScience as well.)
Are the flaws in this science only applicable to bullet forensics or are there errors in things like fingerprint analysis too?
The flaws are common to all pattern evidence. Some areas of pattern evidence have more flaws than others. Brandon Mayfield is a famous example of mis-IDed fingerprint. He was arrested for connection with connection to the Spanish train bombing due to a fingerprint match. He sued for false arrest and the FBI had to pay him two million dollars.
Based on your evaluation of the methods and evidence, do you believe that there was a second shooter on the grassy knoll?
This research is about counting the bullets in the JFK party. If there are in fact more than 2 bullets in the Kennedy party then it is more likely that there is another shooter. The current chemistry cannot say if there are two bullets, three bullets, or up to five bullets in the Kennedy party. If the actual answer is three or more then there is a highly increased likelihood that there is more than one shooter. If there was a second shooter, the chemistry applied can’t tell where the shooter would have been.
I don't know much about the JFK case. Could you explain, in laymen's terms and in enough words, why the combative bullet lead analysis was flawed? And what came of the flaw?
The bullets were assumed by the Dr. Gwin, the chemist hired by the House Select Committee to work on the assassinations, to be chemically unique. There were 5 fragments groups of bullets in the JFK party. Since 2 fragment groups matched chemically those fragments had to come from one bullet, the 3 other fragment groups matched chemically, since bullets are unique they had to come from one bullet- the chemical count in the JFK party was 2 bullets. It turns out that bullets had to be completely unique was completely wrong so chemistry could not be used to count those bullets due to chemistry. The correct analysis is that there were between 2-5 bullets, thus is Oswald only fired 3 shots, which is the common belief, if there are more than 2 bullets in the JFK party there is likely more than one shooter.
Hi Dr. Spiegelman,
As science grows more complicated, it becomes more and more difficult to explain it adequately to the average person (ie, the jury). Do you think this has any effect on the outcome of cases; for example, do you think juries are simply "led" by lawyers without properly understanding the scientific evidence?
Are there any other "popular" forensic tests that are likely on their way out as we raise our scientific standards?
Thank you for your time today.
Yes, this is a major issue. There is a group called OSAC, Organization of Science Area Committees that is trying to deal with the proper presentation of scientific evidence. A lot depends on psychology which is not my area of expertise, but it is a very important area. Doing good science, and having the jury, often not college educated, understand the science is very difficult and needs to be addressed immediately. CBLA is already dead, bitemarks are mostly dead, hair and fiber is not used very much or as much as it used to be. So the answer is yes, even some highly regarded methods, such as firearm toolmarks, are under severe pressure.
Do you see machine learning becoming a more significant part of forensic science moving forward? Will we trust an expert AI to make appropriate analyses over people? Or is that a step too far for the court system in our lifetime?
Machine learning should have a large role, as a machine can be less biased. A machine doesn’t care about the facts of the case, the race of the victim or defendant. AI can be combined using Bayes’ Rule with human examiner's opinions. I think this is an enormous room for growth in forensic science and will be one of the keys of the future to forensic science.
Hi Dr. Spiegelman,
With all due respect, machines care only about the information given to them, which is going to be the facts of each case. Because machine learning is at its core learning from data, it makes use only of the data we give to it -- and if that data is biased, it will be biased as well.
Can you elaborate on what you mean by combining AI with Bayes' Rule? I'm not sure I understand what you mean by that.
Thank you!
I disagree. Machine learning is a hype thing more than a magic thing that finds truth from "stuff". Also, due to how the algorithms are trained, they are indeed full of biases. As it's a human that decides how to train the machine in what's good and what's bad.
Furthermore, compared to other algorithms, machine learning ones are uncontrollable, so you don't know why it reaches some verdict. Truly scary
What can we do? Anything? What should the government do?
OSAC is working to improve forensic science but it’s going to be a 10-20 year effort before enough change is made so that forensic science is in a much better place. One of my mentors had a sign “science is timeless,.” and unfortunately it runs on a different time scale than a defendant's.
What was the most memorable case you worked on with the Innocence Project? thanks
There was a case in Dallas where the FBI toolmark examiner, at that time the head of the toolmark division, said he was delighted to testify against myself and another expert witness. Regardless of that, a plea bargain was reached and an innocent person was set free for time served.
Alright, it’s time for me to sign off. Thanks for all the great questions, it’s a been a pleasure!
How dire is the situation as it pertains to bad forensic science and the criminal justice system? From fingerprints to bite analysis to hair analysis and beyond... where is your level of concern?
On a 10 point scale, where 0 is no concern and 10 is extremely concerned, my level of concern is 9. The Hollywood presentation of forensics is very far from the truth. Examiners too often overstate their conclusions from the evidence on their cases. About half of the Innocence Project exonerations are in part due to bad forensics. That is a lot of innocent people going to jail for flawed forensics.
I'd like to follow up on this response because you've touched on something I find disconcerting. How much of our perception of the legitimacy of current forensic science is biased in part because of how the field is presented in film and television: CSI, NCIS, Bones, etc.?
I’m not an expert on this but both defense and district attorneys complain the the CSI effect is real. Defense attorneys complain that forensic science is flawless when it’s presented on TV and it’s not. District attorneys complain that juries have unrealistic expectations because of the capabilities of forensic science and budgets. So both sides of the legal profession complain bitterly about the CSI effect.
Dr. Spiegelman,
Are you telling me that most jurisdictions don't have $10 million dollar laboratories stocked with top of the line research equipment that major research universities would envy?
Haha
I am here to answer any questions
Hello Dr. Spiegelman,
I expect a lot of people will be asking about JFK, so I'll leave those questions to them. Rather, I'm more curious about your appointment as Official Statistician of the Texas Holocaust and Genocide Commission. What is this role and how does your background in forensic science factor into the duties and responsibilities associated with it?
The Texas Holocaust and Genocide Commission’s main function is to education K-12 students about the Holocaust and other genocides. In order to do that surveys need to be generated and analyzed. So my role is to guide survey development and data analysis.
As a follow-up, what kinds of surveys are involved? I would guess that one of the metrics is general knowledge/awareness of the Holocaust and other genocides, but what are some of the other metrics considered in these surveys?
The surveys deal with where the Holocaust and genocide are taught, what classes. For example, most people might think they are taught in History but they’re actually taught in English through the use of books Diary of Anne Frank. We also want to know what material is used to teach the K-12 students.
I see! I grew up in Texas and don't recall much material in History regarding the Holocaust. As you said though, there was a lot of discussion in English based on literature relating to the Holocaust and genocide (one that comes immediately to mind being Night by Elie Wiesel). Thank you for taking the time to answer!
Which state innocence project is the most effective?
I don’t know. I’ve worked with several. They seem to all be to be very capable and dedicated. The main one I work with is out of New York, and it was founded by Barry Scheck and Peter Neufeld. The one out of NYC only presents cases in court where their sure the convict was innocent of the crime for which they were convicted, hence their name.
How difficult is it to "throw out" something like bullet lead analysis from scientific practice as a whole? How do legal teams find out that a certain test should no longer be used?
It is extremely different. The usual path is court challenges. Once a procedure has been accepted for many years (CBLA was used for decades) it’s hard to get rid of. To get rid of CBLA took court challenges, an NRC panel report, and a joint Washington Post 60 minute expose that embarrassed the FBI into stopping CBLA. Bitemarks got stopped largely due to DNA, when it was found out that what was being testified as positive matches were not. And research from the University of Buffalo and Canisius College showed that there was no foundation for bitemark analysis. The Innocence Project out of NY was extremely helpful in ending the use of bitemarks in court.
Dr. Spiegelman,
Can you tell me a little about how you got started on this? How does a statistician get involved in forensic bullet analysis? Thank you for your time!
I was an editor of a interface journal between math and chemistry. Due to court challenges the NRC created a panel to look at CBLA and two staticians were asked, myself and Dr. Kafadar from University of Virginia. We were asked to help about 12 other scientists to review the procedure. The FBI said CBLA was first used in the JFK assassination. While the JFK assassination was not the focus, when I read what was done in that case it stood out as a horrendous use of science. I was able to verify that a few later.
Hi, Dr. Spiegelman! What would you say is the most interesting project you're working on right now? Thank you for your time this morning.
This AMA is being permanently archived by The Winnower, a publishing platform that offers traditional scholarly publishing tools to traditional and non-traditional scholarly outputs—because scholarly communication doesn’t just happen in journals.
To cite this AMA please use: https://doi.org/10.15200/winn.151585.55046
You can learn more and start contributing at authorea.com
What sort of hurdles do you experience in your role at the Texas Holocaust and Genocide Commission?
I find response rates are the big issue. Getting schools to respond to surveys. The Commission could not be more supportive of my role.
Do you know of scientific and statistical methods that have the most promise for being used effectively and reliably in the modern courtroom? DNA evidence appears to be the most reliable now, but I am wondering what you think about other methods such as microbiome signatures or mass spectrometry. Thank you for your time!
DNA is considered the most reliable but has problems with mixtures- particularly if it’s a mixture of three or more people. For example gang rapes cannot use DNA to find the . Mass Spectrometry is used widely in toxicology. Using bacteria as a forensic signature is at best in its infancy.
If i am on a grand jury being presented with evidence in drug cases, what questions should I be asking?
How close is the amount of the drug at trial to the sentencing guideline borders. What is the error rate or the typical standard deviation of the measurement error of the drugs. What quality control does the lab have to assure the grand jury that the evidence is presenting is reliable. How many measurements were used to report the drug amounts to the grand jury. Finally, you can find lectures online about this topic that go beyond the scope of this AMA.
Was... was there actually a second shooter on the grassy knoll (or anywhere else)?
Answered elsewhere
As you point out, many cases suffered from flawed forensic science- what couple of cases concern you the most, and what about their forensics do you find most worrying/flawed?
Pineapple on pizza or no?
What caused the delay in recognition of the fact that a secret service member misfired after the first bullet killed Kennedy? If there was no delay, how did the populace with gun knowledge not recognize the trajectory of the second bullet?
I’m sorry I missed this earlier.
I grew up in North Texas and heard many things, as you might imagine. One was my high school civics teacher showing a slide that he had of what appeared to be a bullet fired from the front.
This is just a note of all sorts of theories and not my question.
My question is I saw a gun TV show a few years ago where they tried to discover how the Russian made rifle with its unique bullet shape wouldn’t tumble like what we expect.
The theory is that the bullet shot straight and didn’t deviate course after hitting JFK, the seat in front of him or John Connolly.
Do you have any input on that?
What about the Droid attack on the Wookies?
Where does magic fit into forensic science?
2]1]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]513
T54ET6 64TT6FTrt~WWT3
19.1m
Subscribers
13.7k
Online
This community is a place to share and discuss new scientific research. Read about the latest advances in astronomy, biology, medicine, physics, social science, and more. Find and submit new publications and popular science coverage of current research.
Do you have a college degree in a scientific field? Get flair in /r/science to indicate your expertise!
