Welcome to the Nexus of Ethics, Psychology, Morality, Philosophy and Health Care

Welcome to the nexus of ethics, psychology, morality, technology, health care, and philosophy
Showing posts with label Junk Science. Show all posts
Showing posts with label Junk Science. Show all posts

Sunday, April 21, 2024

An Expert Who Has Testified in Foster Care Cases Across Colorado Admits Her Evaluations Are Unscientific

Eli Hager
Originally posted 18 March 24

Diane Baird had spent four decades evaluating the relationships of poor families with their children. But last May, in a downtown Denver conference room, with lawyers surrounding her and a court reporter transcribing, she was the one under the microscope.

Baird, a social worker and professional expert witness, has routinely advocated in juvenile court cases across Colorado that foster children be adopted by or remain in the custody of their foster parents rather than being reunified with their typically lower-income birth parents or other family members.

In the conference room, Baird was questioned for nine hours by a lawyer representing a birth family in a case out of rural Huerfano County, according to a recently released transcript of the deposition obtained by ProPublica.

Was Baird’s method for evaluating these foster and birth families empirically tested? No, Baird answered: Her method is unpublished and unstandardized, and has remained “pretty much unchanged” since the 1980s. It doesn’t have those “standard validity and reliability things,” she admitted. “It’s not a scientific instrument.”

Who hired and was paying her in the case that she was being deposed about? The foster parents, she answered. They wanted to adopt, she said, and had heard about her from other foster parents.

Had she considered or was she even aware of the cultural background of the birth family and child whom she was recommending permanently separating? (The case involved a baby girl of multiracial heritage.) Baird answered that babies have “never possessed” a cultural identity, and therefore are “not losing anything,” at their age, by being adopted. Although when such children grow up, she acknowledged, they might say to their now-adoptive parents, “Oh, I didn’t know we were related to the, you know, Pima tribe in northern California, or whatever the circumstances are.”

The Pima tribe is located in the Phoenix metropolitan area.


Here is my summary:

The article discusses Diane Baird, an expert who has testified in foster care cases across Colorado, admitting that her evaluations are unscientific. Baird, who has spent four decades evaluating the relationships of poor families with their children, labeled her method for assessing families as the "Kempe Protocol." This revelation raises concerns about the validity of her evaluations in foster care cases and highlights the need for more rigorous and scientific approaches in such critical assessments.

Thursday, October 6, 2022

Defining Their Own Ethics, Online Creators Are De Facto Therapists for Millions—Explosive Demand & Few Safeguards

Tantum Hunter
The Washington Post
Originally posted 29 AUG 22

Here are two excerpts:

In real life, mental health information and care are sparse. In the United States, 1 in 3 counties do not have a single licensed psychologist, according to the American Psychological Association, and Americans say cost is a top barrier to seeking mental health help. On the internet, however, mental health tips are everywhere: TikTok videos with #mentalhealth in the caption have earned more than 43.9 billion views, according to the analytics company Sprout Social, and mentions of mental health on social media are increasing year by year.

The growing popularity of the subject means that creators of mental health content are filling a health-care gap. But social media apps are not designed to prioritize accurate, helpful information, critics say, just whatever content draws the biggest reaction. Young people could see their deepest struggles become fodder for advertisers and self-promoters. With no road map even for licensed professionals, mental health creators are defining their own ethics.

“I don’t want to give anyone the wrong advice,” Moloney says. “I’ve met some [followers] who’ve just started crying and saying ‘thank you’ and stuff like that. Even though it seems small, to someone else, it can have a really big impact.”

As rates of depression and anxiety spiked during the pandemic and options for accessible care dwindled, creators shared an array of content including first-person accounts of life with mental illness and videos listing symptoms of bipolar disorder. In many cases, their follower counts ballooned.

(cut)

Ideally, social media apps should be one item in a collection of mental health resources, said Jodi Miller, a researcher at Johns Hopkins University School of Education who studies the relationships among young people, technology and stress.

“Young people need evidence-based sources of information outside the internet, from parents and schools,” Miller said.

Often, those resources are unavailable. So it’s up to consumers to decide what mental health advice they put stock in, Fisher-Quann said. For her, condescending health-care providers and the warped incentives of social media platforms haven’t made that easy. But she thinks she can get better — and that her followers can, too.

“It all has to come from a place of self-awareness and desire to get better. Communities can be extremely helpful for that, but they can also be extremely harmful for that,” she said.

Friday, March 20, 2020

Flawed science? Two efforts launched to improve scientific validity of psychological test evidence in court

Karen Franklin
forensicpsychologist Blog
Originally posted 15 Feb 20

Here is an excerpt:

New report slams "junk science” psychological assessments

In one of two significant developments, a group of researchers today released evidence of systematic problems with the state of psychological test admissibility in court. The researchers' comprehensive survey found that only about two-thirds of the tools used by clinicians in forensic settings were generally accepted in the field, while even fewer -- only about four in ten -- were favorably reviewed in authoritative sources such as the Mental Measurements Yearbook.

Despite this, psychological tests are rarely challenged when they are introduced in court, Tess M.S. Neal and her colleagues found. Even when they are, the challenges fail about two-thirds of the time. Worse yet, there is little relationship between a tool’s psychometric quality and the likelihood of it being challenged.

“Some of the weakest tools tend to get a pass from the courts,” write the authors of the newly issued report, "Psychological Assessments in Legal Contexts: Are Courts Keeping 'Junk Science' Out of the Courtroom?”

The report, currently in press in the journal Psychological Science in the Public Interest, proposes that standard batteries be developed for forensic use, based on the consensus of experts in the field as to which tests are the most reliable and valid for assessing a given psycho-legal issue. It further cautions against forensic deployment of newly developed tests that are being marketed by for-profit corporations before adequate research or review by independent professionals.

The info is here.

Monday, March 9, 2020

The dangerous veneer of ‘science’

Regini Rini
Times Literary Supplement
Originally posted Feb 20

"Race science” seems to be the hideous new trend of 2020. Last month, an article in the journal Philosophical Psychology calling for greater investigation of purported genetic bases for racial IQ differences became the focus of intense academic controversy. Then came a new book, Human Diversity, from Charles Murray, prompting the New York Times columnist Jamelle Bouie to tweet: “i guess we’re gonna spend february arguing with phrenologists”. And then just this week, a twenty-seven-year-old consultant to the British government quickly resigned following media reports of his apparent musings on eugenics.

What’s going on? Why are we suddenly talking about this nonsense again? Donald Trump, and the winks he tosses to torch-wielding “blood and soil” marchers, may have something to do with it. Clearly there is a market for white coats who cater to white hoods. But the “race science” racket is growing, and we needn’t assume that all its practitioners have such transparently bigoted motives. Rather, I suspect that some are in it for the iconoclastic thrill of prodding at bien pensant pieties from behind the intellectual shield of capital-S Science.

There has always been a certain sort of mind that delights in saying whatever is verboten, from the Marquis de Sade to Christopher Hitchens. The writer George Packer worries that, in the high-stakes moral atmosphere of the Trump era, we no longer have cultural space for such fearless exploration of opinion. But I think this gets things exactly backwards. Trumpism is partly a result of the fact that it is now much easier to acquire an audience for heterodoxy. You don’t have to be a gifted essayist; you need only a Twitter account and lack of moral inhibition. Thoughtful iconoclasts aren’t silenced, they’re merely lost amid the din of so many icons being artlessly shattered.

The info is here.

Friday, October 18, 2019

The Koch-backed right-to-try law has been a bust, but still threatens our health

Michael Hiltzik
The Los Angeles Times
Originally posted September 17, 2019

The federal right-to-try law, signed by President Trump in May 2018 as a sop to right-wing interests, including the Koch brothers network, always was a cruel sham perpetrated on sufferers of intractably fatal diseases.

As we’ve reported, the law was promoted as a compassionate path to experimental treatments for those patients — but in fact was a cynical ploy aimed at emasculating the Food and Drug Administration in a way that would undermine public health and harm all patients.

Now that a year has passed since the law’s enactment, the assessments of how it has functioned are beginning to flow in. As NYU bioethicist Arthur Caplan observed to Ed Silverman’s Pharmalot blog, “the right to try remains a bust.”

His judgment is seconded by the veteran pseudoscience debunker David Gorski, who writes: “Right-to-try has been a spectacular failure thus far at getting terminally ill patients access to experimental drugs.”

That should come as no surprise, Gorski adds, because “right-to-try was never about helping terminally ill patients. ... It was always about ideology more than anything else. It was always about weakening the FDA’s ability to regulate drug approval.”

The info is here.

Friday, October 4, 2019

Google bans ads for unproven medical treatments

Megan Graham
www.cnbc.com
Originally posted September 6, 2019

Google on Friday announced a new health care and medicines policy that bans advertising for “unproven or experimental medical techniques,” which it says includes most stem cell, cellular and gene therapies.

A blog post from Google policy advisor Adrienne Biddings said the company will prohibit ads selling treatments “that have no established biomedical or scientific basis.” It will also extend the policy to treatments that are rooted in scientific findings and preliminary clinical experience “but currently have insufficient formal clinical testing to justify widespread clinical use.” The change was first reported by The Washington Post.

The new Google ads policy may put the heat on for the stem cell clinic industry, which has until recently been largely unregulated and has some players who have been accused of taking advantage of seriously ill patients, The Washington Post reported.

“We know that important medical discoveries often start as unproven ideas — and we believe that monitored, regulated clinical trials are the most reliable way to test and prove important medical advances,” Biddings said. “At the same time, we have seen a rise in bad actors attempting to take advantage of individuals by offering untested, deceptive treatments. Often times, these treatments can lead to dangerous health outcomes and we feel they have no place on our platforms.”

The Google post included a quote from the president of the International Society for Stem Cell Research, Deepak Srivastava, who said the new policy is a “much-needed and welcome step to curb the marketing of unscrupulous medical products such as unproven stem cell therapies.”

The info is here.

Thursday, February 1, 2018

How to Counter the Circus of Pseudoscience

Lisa Pryor
The New York Times
Originally published January 5, 2018

Here are two excerpts:

In the face of such doubt, it is not surprising that some individuals, even those who are intelligent and well educated, are swept away by the breezy confidence of health gurus, who are full of passionate intensity while the qualified lack all conviction, to borrow from Yeats.

It is a cognitive bias known in psychology as the Dunning-Kruger Effect. In short, the less you know, the less able you are to recognize how little you know, so the less likely you are to recognize your errors and shortcomings. For the highly skilled, like trained scientists, the opposite is true: The more you know, the more likely you are to see how little you know. This is truly a cognitive bias for our time.

(cut)

Engaging is difficult when the alternative-health proponents are on such a different astral plane that it is a challenge even to find common language for a conversation, especially when they promote spurious concepts such as “pyrrole disease,” which they can speak about in great, false detail, drawing the well-informed physician, dietitian or scientist into a vortex of personal anecdote and ancient wisdom, with quips about big pharma thrown in for good measure.

The information is here.

Sunday, December 17, 2017

The Impenetrable Program Transforming How Courts Treat DNA

Jessica Pishko
wired.com
Originally posted November 29, 2017

Here is an excerpt:

But now legal experts, along with Johnson’s advocates, are joining forces to argue to a California court that TrueAllele—the seemingly magic software that helped law enforcement analyze the evidence that tied Johnson to the crimes—should be forced to reveal the code that sent Johnson to prison. This code, they say, is necessary in order to properly evaluate the technology. In fact, they say, justice from an unknown algorithm is no justice at all.

As technology progresses forward, the law lags behind. As John Oliver commented last month, law enforcement and lawyers rarely understand the science behind detective work. Over the years, various types of “junk science” have been discredited. Arson burn patterns, bite marks, hair analysis, and even fingerprints have all been found to be more inaccurate than previously thought. A September 2016 report by President Obama’s Council of Advisors on Science and Technology found that many of the common techniques law enforcement historically rely on lack common standards.

In this climate, DNA evidence has been a modern miracle. DNA remains the gold standard for solving crimes, bolstered by academics, verified scientific studies, and experts around the world. Since the advent of DNA testing, nearly 200 people have been exonerated using newly tested evidence; in some places, courts will only consider exonerations with DNA evidence. Juries, too, have become more trusting of DNA, a response known popularly as the “CSI Effect.” A number of studies suggest that the presence of DNA evidence increases the likelihood of conviction or a plea agreement.

The article is here.

Monday, November 13, 2017

Medical Evidence Debated

Ralph Bartholdt
Coeur d’Alene Press 
Originally posted October 27, 2017

Here is an excerpt:

“The point of this is not that he had a choice,” he said. “But what’s been loaded into his system, what’s he’s making the choices with.”

Thursday’s expert witness, psychologist Richard Adler, further developed the argument that Renfro suffered from a brain disorder evidenced by a series of photograph-like images of Renfro’s brain that showed points of trauma. He pointed out degeneration of white matter responsible for transmitting information from the front to the back of the brain, and shrunken portions on one side of the brain that were not symmetrical with their mirror images on the other side.

Physical evidence coinciding with the findings include Renfro’s choppy speech patterns and mannerisms as well inabilities to make cognitive connections, and his lack of social skills, Adler said.

Defense attorney Jay Logsdon asked if the images were obtained through a discredited method, one that has “been attacked as junk science?”

The method, called QEEG, for quantitative electroencephalogram, which uses electrical patterns that show electrical activity inside the brain’s cortex to determine impairment, was attacked in an article in 1997. The article’s criticism still stands today, Adler said.

Throughout the morning and into the afternoon, Adler reiterated findings, linking them to the defendant’s actions, and dovetailing them into other test results, psychological and cognitive, that have been conducted while Renfro has been incarcerated in the Kootenai County Jail.

The article is here.

Friday, November 10, 2017

Court ruling on expert testimony could open door to junk science

Andis Robeznieks
AMA Wire
Originally posted October 20, 2017

The New Jersey Supreme Court is expected to issue a ruling soon that may affect more than 2,000 cases before the state’s courts. The court’s decision could have an even more far-reaching impact and might eventually undermine medical research, patient-physician decision making and informed consent.

The issue is whether scientific testimony expressing refuted theories that have not been subjected to peer review and do not follow the traditional hierarchy of scientific evidence should be admissible in litigation involving plaintiffs who claim their inflammatory bowel disease (IBD) was caused by the drug isotretinoin, marketed as Accutane by Hoffmann-La Roche, formerly headquartered in Nutley, N.J.

The first lawsuit on this matter was filed in July 2003. A hearing on whether the plaintiffs’ witnesses would be allowed to testify was held in February 2015, after which trial Judge Nelson C. Johnson barred their testimony. In May 2015, Johnson dismissed 2,076 related cases based on his ruling about the evidence.

This past July, a three-judge panel of the state appellate court reversed both the ruling barring the testimony and the dismissal of the cases. Hoffman-La Roche has requested the state Supreme Court to review this ruling.

The pressor is here.

Sunday, November 5, 2017

Inside the CIA's Black Site Torture Room

Larry Siems
The Guardian
Originally posted October 9, 2017

Here is an excerpt:

Jessen, who interrogated Rahman six times over a two-week period, and Mitchell, who met with him once, claimed throughout the lawsuit that they tried to mitigate the harsh conditions of Rahman’s confinement. But cables show it was Jessen who debated whether to subject Rahman to enhanced interrogations techniques with CIA headquarters, and it was Jessen whose advice held sway when he and Zirbel plotted Rahman’s interrogation. “He could tell that [the site manager] was running all of his suggestions through his ‘bullshit filter,’” the investigator notes from his interview with the psychologist, but “Jessen said he was the guy with all the tricks”.

Zirbel accepted Jessen’s suggestion that when Rahman complained that he was cold, he was using a sophisticated al-Qaida resistance technique. When Rahman “claimed inability to think due to conditions (cold),” “complained about poor treatment,” and “complained about the violation of his human rights”, as a cable recorded after one of Jessen’s interrogations, these were evidence, Jessen said, of a “health and welfare” resistance strategy.

The article is here.

Friday, October 6, 2017

AI Research Is in Desperate Need of an Ethical Watchdog

Sophia Chen
Wired Science
Originally published September 18, 2017

About a week ago, Stanford University researchers posted online a study on the latest dystopian AI: They'd made a machine learning algorithm that essentially works as gaydar. After training the algorithm with tens of thousands of photographs from a dating site, the algorithm could, for example, guess if a white man in a photograph was gay with 81 percent accuracy. The researchers’ motives? They wanted to protect gay people. “[Our] findings expose a threat to the privacy and safety of gay men and women,” wrote Michal Kosinski and Yilun Wang in the paper. They built the bomb so they could alert the public about its dangers.

Alas, their good intentions fell on deaf ears. In a joint statement, LGBT advocacy groups Human Rights Campaign and GLAAD condemned the work, writing that the researchers had built a tool based on “junk science” that governments could use to identify and persecute gay people. AI expert Kate Crawford of Microsoft Research called it “AI phrenology” on Twitter. The American Psychological Association, whose journal was readying their work for publication, now says the study is under “ethical review.” Kosinski has received e-mail death threats.

But the controversy illuminates a problem in AI bigger than any single algorithm. More social scientists are using AI intending to solve society’s ills, but they don’t have clear ethical guidelines to prevent them from accidentally harming people, says ethicist Jake Metcalf of Data & Society. “There aren’t consistent standards or transparent review practices,” he says. The guidelines governing social experiments are outdated and often irrelevant—meaning researchers have to make ad hoc rules as they go.

Right now, if government-funded scientists want to research humans for a study, the law requires them to get the approval of an ethics committee known as an institutional review board, or IRB. Stanford’s review board approved Kosinski and Wang’s study. But these boards use rules developed 40 years ago for protecting people during real-life interactions, such as drawing blood or conducting interviews. “The regulations were designed for a very specific type of research harm and a specific set of research methods that simply don’t hold for data science,” says Metcalf.

The article is here.

Tuesday, September 26, 2017

Drug company faked cancer patients to sell drug

Aaron M. Kessler
CNN.com
Originally published September 6, 2017

When Insys Therapeutics got approval to sell an ultra-powerful opioid for cancer patients with acute pain in 2012, it soon discovered a problem: finding enough cancer patients to use the drug.

To boost sales, the company allegedly took patients who didn't have cancer and made it look like they did.

The drug maker used a combination of tactics, such as falsifying medical records, misleading insurance companies and providing kickbacks to doctors in league with the company, according to a federal indictment and ongoing congressional investigation by Sen. Claire McCaskill, a Democrat from Missouri.

The new report by McCaskill's office released Wednesday includes allegations about just how far the company went to push prescriptions of its sprayable form of fentanyl, Subsys.

Because of the high cost associated with Subsys, most insurers wouldn't pay for it unless it was approved in advance. That process, likely familiar to anyone who's taken an expensive medication, is called "prior-authorization."

The article is here.

Wednesday, August 23, 2017

Procedural ruling sets higher bar for expert-witness testimony

Brendan Murphy
AMA Wire
Originally posted August 9, 2017

In a procedural decision that could keep so-called junk science out of the courtroom, the District of Columbia Court of Appeals adopted an evidentiary standard that places additional scrutiny on testimony from expert witnesses.

The case at the center of the ruling—Motorola v. Murray—raises the issue of whether cellphones cause brain cancer. In total, 29 cases on the subject matter were brought before the Superior Court for the District of Columbia.

The court did acknowledge isolated strands of scientific data that suggest a possible causal connection between cellphone use and brain cancer. But the court ultimately ruled that based on the research to date, there was inadequate data for any scientist to opine on a causal connection between cellphone use and cancer to any degree of scientific certainty.

In spite of this, the plaintiffs offered their own expert testimony to the contrary, arguing that the jury should determine the validity of the testimony.

The article is here.

Thursday, August 10, 2017

Predatory Journals Hit By ‘Star Wars’ Sting

By Neuroskeptic
discovermagazine.com
Originally published July 19, 2017

A number of so-called scientific journals have accepted a Star Wars-themed spoof paper. The manuscript is an absurd mess of factual errors, plagiarism and movie quotes. I know because I wrote it.

Inspired by previous publishing “stings”, I wanted to test whether ‘predatory‘ journals would publish an obviously absurd paper. So I created a spoof manuscript about “midi-chlorians” – the fictional entities which live inside cells and give Jedi their powers in Star Wars. I filled it with other references to the galaxy far, far away, and submitted it to nine journals under the names of Dr Lucas McGeorge and Dr Annette Kin.

Four journals fell for the sting. The American Journal of Medical and Biological Research (SciEP) accepted the paper, but asked for a $360 fee, which I didn’t pay. Amazingly, three other journals not only accepted but actually published the spoof. Here’s the paper from the International Journal of Molecular Biology: Open Access (MedCrave), Austin Journal of Pharmacology and Therapeutics (Austin) and American Research Journal of Biosciences (ARJ) I hadn’t expected this, as all those journals charge publication fees, but I never paid them a penny.

The blog post is here.

Thursday, February 23, 2017

How To Spot A Fake Science News Story

Alex Berezow
American Council on Science and Health
Originally published January 31, 2017

Here is an excerpt:

How to Detect a Fake Science News Story

Often, I have been asked, "How can you tell if a science story isn't legitimate?" Here are some red flags:

1) The article is very similar to the press release on which it was based. This indicates whether the article is science journalism or just public relations.

2) The article makes no attempt to explain methodology or avoids using any technical terminology. (This indicates the author may be incapable of understanding the original paper.)

3) The article does not indicate any limitations on the conclusions of the research. (For example, a study conducted entirely in mice cannot be used to draw firm conclusions about humans.)

4) The article treats established scientific facts and fringe ideas on equal terms.

5) The article is sensationalized; i.e., it draws huge, sweeping conclusions from a single study. (This is particularly common in stories on scary chemicals and miracle vegetables.)

6) The article fails to separate scientific evidence from science policy. Reasonable people should be able to agree on the former while debating the latter. (This arises from the fact that people ascribe to different values and priorities.)

The article is here.

Tuesday, January 17, 2017

Fake news invades science and science journalism as well as politics

By Ivan Oransky and Adam Marcus
STAT News
Originally published December 30, 2016

Here is an excerpt:

Science itself never falls victim to this sort of distortion though, does it? We wish that was so. Take, for example, a conspiracy theory about cloud trails from jet planes that was published in a peer-reviewed journal. How about a study linking vaccines to autism long after such a connection had been thoroughly debunked? That one was published in a public health journal. Or this fake news whopper: HIV doesn’t cause AIDS. A peer-reviewed paper made that claim until it was retracted.

We could go on. But as you’ve gathered by now, science in its current state isn’t exactly keeping us safe from bogus research. Predatory publishers continue to churn out papers for a price, with minimal peer review — or very often no peer review — to vet the results. Unscrupulous researchers use those and other soft spots in the scientific publishing system to get away with presenting wild theories or cooking their data.

Journalists who don’t fact-check deserve criticism, whether the topic is politics, entertainment, or science. But the real trouble with fake news is when there’s a kernel of truth in the pile of garbage. That’s especially problematic in science: scientists continue to dress up weak findings in flashy clothes, all the better to publish with. Then their universities often bolster this flimsy work with frothy press releases that journalists fall for.

The article is here.

Friday, July 3, 2015

AMA is finally taking a stand on quacks like Dr. Oz

By Julia Belluz
Vox.com
Originally posted June 13, 2015

Medical students and residents frustrated with bogus advice from doctors on TV have, for more than a year, been asking the American Medical Association to clamp down and "defend the integrity of the profession."

Now the AMA is finally taking a stand on quack MDs who spread pseudoscience in the media.

"This is a turning point where the AMA is willing to go out in public and actively defend the profession," Benjamin Mazer, a medical student at the University of Rochester who was involved in crafting the resolution, said. "This is one of the most proactive steps that the AMA has taken [on mass media issues]."

The entire story is here.

Friday, June 26, 2015

Rein It In, Dr Oz

By Art Caplan
MedScape
Originally published April 30, 2015

Dr Mehmet Oz is in trouble again. He was accused by 10 physicians in a letter of promoting quackery. They demanded that Columbia University Medical Center fire Dr Oz. Now, I can say with some authority that as "America's Doctor"—the person who, for many Americans, is the voice of medicine—he is not going to be fired. His show is not going to end. That isn't going to happen.

Dr Oz has evoked this response from these 10 physicians because he continues to push the border of legitimacy on his shows with respect to touting things for which there isn't much evidence. And that is a problem. Many doctors tell me that when Dr Oz endorses something—green coffee beans, some neti pot to cure the common cold—whatever it is, they are going to be asked about it, and their patients run out and buy it. He has enormous power when it comes to the platform he has built. And let's face it: He is an effective communicator. His show is fun to watch. I understand why the American people are paying attention to Dr Oz.

The entire article is here.

Wednesday, June 10, 2015

I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here's How.

By John Bohannon
i09
Originally published May 27, 2015

Here is an excerpt:

Here’s a dirty little science secret: If you measure a large number of things about a small number of people, you are almost guaranteed to get a “statistically significant” result. Our study included 18 different measurements—weight, cholesterol, sodium, blood protein levels, sleep quality, well-being, etc.—from 15 people. (One subject was dropped.) That study design is a recipe for false positives.

Think of the measurements as lottery tickets. Each one has a small chance of paying off in the form of a “significant” result that we can spin a story around and sell to the media. The more tickets you buy, the more likely you are to win. We didn’t know exactly what would pan out—the headline could have been that chocolate improves sleep or lowers blood pressure—but we knew our chances of getting at least one “statistically significant” result were pretty good.

Whenever you hear that phrase, it means that some result has a small p value. The letter p seems to have totemic power, but it’s just a way to gauge the signal-to-noise ratio in the data. The conventional cutoff for being “significant” is 0.05, which means that there is just a 5 percent chance that your result is a random fluctuation. The more lottery tickets, the better your chances of getting a false positive. So how many tickets do you need to buy?

The whole article on the scam research that fooled millions is here.