Home run hitters

Image source

Signal Podcast and Stat authors Luke Timmerman and Meg Tirrell authored/recorded Should we believe published scientific research? on December 3rd, writing among other things:

"In the world of biotech and pharmaceuticals, investors know the odds are stacked against them. Only about one out of 10 drugs that enters clinical trials ends up as an FDA-approved medicine. Drugs that appear safe in the first few patients begin to appear unsafe after they are taken by a lot of people. Molecules sometimes fail for mysterious reasons. 

Is science suffering from a reproducibility crisis? How do we really know what we know?"

Their article led me to The Watchdogs and Stat authors Adam Marcus and Ivan Oransky, who authored A lot of science isn’t groundbreaking, and that’s a good thing on December 2nd, writing among other things:

"But there’s another way to look at the findings. The publish-or-perish model may push some to be very conservative in their work, but at the other extreme, it creates an incentive to cut corners and commit misconduct in the service of a home-run article in a sexy journal like Cell, Science, or Nature. Even a single paper in one of those titles can mean a fast track to a job at a prestigious institution. Evidence indicates that these journals are more likely to retract articles than other, less splashy titles – partly because more people scrutinize these journals, but partly, in all likelihood, because their high-stakes appeal is a magnet for cheats... 

Cleaving closely to the past therefore may not be as bad for science as the UCLA researchers suggest, and even they acknowledge in the paper that “not all scientists should pursue risky strategies.” Incremental advances in knowledge can be critically important for future research. Indeed, it should be reassuring to know that most of the time researchers aren’t swinging for the fences but are content to hit for average. Here’s to Slow Science."

Timmerman and Tirell's article/podcast discuss a topic regularly revisited but seemingly mostly ignored by much of the biopharmaceutical industry and its ecosystem (e.g., media, investors, etc.). Consider, for example, the December 2011 Wall Street Journal article Scientists' Elusive Goal: Reproducing Study Results, which highlighted the lack of reproducibility in biomedical research and noted:

“This is one of medicine's dirty secrets: Most results, including those that appear in top-flight peer-reviewed journals, can't be reproduced.”

Also consider the March 2012 Reuters story In cancer science, many "discoveries" don't hold up, which expanded on two studies mentioned in the Journal article and underscored the widespread lack of reproducibility in research:

“During a decade as head of global cancer research at Amgen, C. Glenn Begley identified 53 "landmark" publications – papers in top journals, from reputable labs – for his team to reproduce. Begley sought to double-check the findings before trying to build on them for drug development. Result: 47 of the 53 could not be replicated”…Scientists at Bayer did not have much more success. In a 2011 paper titled, "Believe it or not," they analyzed in-house projects that built on "exciting published data" from basic science studies. "Often, key data could not be reproduced," wrote Khusru Asadullah, vice president and head of target discovery at Bayer HealthCare in Berlin, and colleagues. Of 47 cancer projects at Bayer during 2011, less than one-quarter could reproduce previously reported findings, despite the efforts of three or four scientists working full time for up to a year. Bayer dropped the projects.” (see this link and this link)

Timmerman and Tirell write of Marcus and Oransky:

"But what if the science that laid the groundwork for new discoveries … was wrong? In this week’s episode, we explore what some have come to call the “crisis of reproducibility” plaguing the world of scientific research. Way more than you’d expect, scientific papers that were once presented as fact are taken back. The blog Retraction Watch, created by STAT contributors Ivan Oransky and Adam Marcus, exists solely to shine the spotlight on these retractions."

The articles seem at odds with each other. On the one hand, Timmerman and Tirell write of a crisis of reproducibility. On the other, Marcus and Oransky herald incremental scientific work. Taken together, is there a crisis of reproducibility of incrementalism?

Provectus' PV-10 (Rose Bengal) has had pre-clinical and clinical data and outcomes reproduced by multiple parties in multiple cancer indications in multiple species. See June 13, 2015 blog post Reproducibility, the Hallmark of Western Science:

"Provectus Biopharmaceuticals Chairman, CEO and co-founder Dr. Craig Dees, PhD frames outcomes of PV-10 use — preclinical or clinical, study or trial, generated by the company, a clinical investigator or a third-party medical researcher, affiliated or unaffiliated with Provectus — in the context of reproducibility, which he calls the hallmark of Western science. 

He says (paraphrasing) if a scientist cannot repeat an experiment’s outcome and another cannot replicate the result, one should be skeptical about the veracity of the original work and claims made from it. PV-10’s track record of reproducible therapeutic features as well as preclinical and clinical results by multiple parties in multiple cancer indications in multiple species is noteworthy."

Examples of Rose Bengal's reproducibility include:

• Moffitt Cancer Center's Intralesional injection with PV-10 induces a systemic anti-tumor immune response in murine models of breast cancer and melanoma, and

• University of Illinois at Chicago's Intralesional Injection of Rose Bengal Induces an Anti-tumor Immune Response and Potent Tumor Regressions in a Murine Model of Colon Cancer.

UIC's The Potential of Intralesional Rose Bengal to Stimulate T-Cell Mediated Anti-Tumor Responses reviewed Rose Bengal's anti-tumor cytotoxicity and induction of anti-tumor immunity (pre-clinically and clinically) in multiple solid tumor cancers.

We know the co-founders of Provectus did not set out to generate incremental scientific work. The essence of Craig's vision in regards to PV-10 (Rose Bengal) has been:

• The necessity of specificity in treating disease,

• The relationship between the response of the immune system, and

• The importance of the reproducibility of PV-10 results.

Provectus CTO and co-founder Dr. Eric Wachter, PhD's vision in regards to Rose Bengal (conjoined with Craig's) is to see the compound used as pervasively aspirin. In an ironic twist, see As pervasive as aspirin (October 11, 2015) on the blog's Current News page.

Provectus needs to generate data...randomized data. Then, and presumably only then, notions of reproducibility and home runs will sink in to Marcus, Oransky, Timmerman, Tirell, and the greater biopharmaceutical industry ecosystem.

Management seemingly thinks their key constituents get it:

• The FDA, by virtue of the Agency's acceptance or allowance of the pivotal Stage III melanoma Phase 3 and Stage IV melanoma combination Phase 1b trial,

• Moffitt, by virtue of arriving at PV-10's mechanism of action

• Pfizer, perhaps starting to, by virtue of the combination therapy patent, and

• Boehringer Ingelheim, starting to, by virtue of the collaboration in China.

Now everyone else needs to get it or start getting it.