John Bohannon is at it again. In 2013 he published the results of a sting operation in which he submitted terrible papers with fake credentials to 304 open access journals. Over half of the journals accepted the paper for publication. He published his results in Science magazine, and it caused a bit of a stir, although arguably not as much as it should have.
Bohannon was asked to repeat this feat, this time to expose the schlocky science of the diet industry. He was asked to do this for a documentary film which will be release shortly, but he has already published his reveal. You can read his full account for details, but here is the quick summary.
He collaborated with others to perform a real (although crappy) scientific study. His researchers recruited 16 people, with one drop out, the remaining 15 were divided into three groups: low carb diet for three week, low carb diet plus daily chocolate for three weeks, and no change in diet. The results were no surprising in that the two diet groups lost 5 pounds on average, while the no diet group did not. However, they also found that the chocolate group lost 10% more weight. He explains:
Here’s a dirty little science secret: If you measure a large number of things about a small number of people, you are almost guaranteed to get a “statistically significant” result. Our study included 18 different measurements—weight, cholesterol, sodium, blood protein levels, sleep quality, well-being, etc.—from 15 people. (One subject was dropped.) That study design is a recipe for false positives.
Think of the measurements as lottery tickets. Each one has a small chance of paying off in the form of a “significant” result that we can spin a story around and sell to the media. The more tickets you buy, the more likely you are to win. We didn’t know exactly what would pan out—the headline could have been that chocolate improves sleep or lowers blood pressure—but we knew our chances of getting at least one “statistically significant” result were pretty good.
Yes, our old friend p-hacking. Bohannon is simply demonstrating effects that we have been discussing here and on science-based medicine for years – published small studies are likely to be false positives because they are more susceptible to quirky results and manipulation through p-hacking. Throw in a little publication bias and we have a literature flooded with worthless positive studies.
I have argued that the press should not even be reporting such preliminary studies. They are nothing but noise in the background of science. At best they serve as a way to guide future research (by giving some indication of which questions are worthwhile and helping design more rigorous studies). They are useful for scientists, but really should not be presented to the public as if their results were reliable.
Here Bohannon is using one common method of p-hacking – multiple analysis. He looked at 18 variables, but did not adjust the statistic to reflect this. You can make an adjustment for multiple analysis so that the p-values reflect the “multiple lottery tickets.” At least then the p-value is legitimate. Otherwise the p-value is meaningless.
Even when legitimate, p-values are problematic. Some scientific journals are discouraging or even banning their use. The essence of the problem is that p-values are being over-used as a single measure of reliability of scientific results. The p-value was never meant to be used this way. It is only really a quick assessment of how seriously to take the data, or the signal to noise ratio in the data. But because p-value became the one measure of a study’s results, that lead to methods that essentially amount to p-hacking – tweaking the methods and massaging the data until you get across the magical 0.05 p-value.
Mutliple analysis is just one of those methods (sometimes called researcher degrees of freedom). Other methods include collecting data until you get the result you want, making multiple comparisons among the variables, and looking at multiple types of statistical analysis then using the one that works the “best.” These tricks are often done innocently, and sometimes not-so-innocently.
It is great that more attention is being paid to these problems with published scientific studies, and Bohannon’s efforts should be applauded.
This is a good time to point out, however, that these problems do not mean that science is broken or that no results can be trusted. It just means you have to understand the structure of the scientific literature and how to interpret results. Science progresses through eventually performing high quality studies with clear results that can be independently replicated. Until we get to this level of evidence I am suspicious of any claims.
Preliminary, small, soft, one-off studies are just noise. I pay no attention to them (although I do pay attention to the sloppy press they often receive). More robust results start to get interesting, and then you have to pay attention to plausibility. There is no sharp demarcation line, but at some point the combination of plausibility and direct evidence is sufficient to conclude that a phenomenon is more likely to be true than untrue. Of course, scientific conclusions are always tentative and subject to revision, but probabilities can reach so high that it is reasonable to treat certain conclusions as if they were facts. At the very least, overturning them would require a mountain of evidence equal to the mountain of evidence that establishes them as true.
I can’t give you a formula for this. It takes scientific knowledge and judgment. That is why we further rely on the consensus of scientific opinion to indicate when a claim has crossed the threshold and should be generally accepted. The judgment of any individual scientist can be quirky or mistaken, but the consensus of many scientists has a greater chance of being valid because individual quirkiness should average out. It is like crowdsourcing, but within an expert population.
In any case, this is the best we can do. It has proven very effective overall, as the amazing progress in science and technology attests. So we are clearly doing something right.
But we can do better. I look at this in terms of efficiency. Science is grinding forward, and eventually most claims work themselves out. Bad ideas do not get long term traction within science, while good ideas eventually do. The real question is, how rapid and efficient is this progress. Sloppy research techniques and poor journal filters slow progress by making the system inefficient.
I would argue that sloppy science journalism does as well by contributing to the scientific illiteracy of the population. The elaborate and expensive institutions of science depend upon public support. Public beliefs also tend to drive funding and therefore research, and sometimes scientists have to waste resources addressing popular, but not very scientifically valid, ideas. Think of the money wasted researching highly implausible alternative medicine treatments, or proving yet again that vaccines do not cause autism.
How much more rapid would our progress be if these inefficiencies were worked out of the system, or at least minimized?
You can pick your friends. You can pick your nose. But you can’t pick your friends nose. Unless you practice Nasal Cranial Release.
There are so many pseudo-medicines, it is hard to keep track. New variations appear, new combinations of old SCAMs occur, old pseudo-medicines wax and wane, although no pseudo-medicine ever dies. Except phenology? Maybe? I find a few phrenology sites on the web, but I cannot tell if they are real or satire.
There is an ongoing discussion in my family as to the greatest band of all time. The Who is THE correct answer, but really, bands do not have to follow the Pauli exclusion principle. There can be multiple greatest bands of all time, all existing simultaneously at the same place. Except that the Who is a little bit more greater than the others. Yes. More greater.
The same concept applies to SCAM’s. Whose goofiness reigns supreme? Homeopathy? Reiki? Epigenetic Birth Control? They are all equally goofy, each in their own special way.
In my feeds I saw the announcement that Anderson Chiropractic Announces Nasal Cranial Release Therapy. I had never heard of Nasal Cranial Release Therapy. It sounds bad, turning humans into a PEZ dispenser, popping off the skull by way of the nose, a particularly horrific form of rhinotillexomania. It’s not that goofy. But close.
Nasal Cranial Release appears to be the mutant offspring of chiropractic and cranio-sacral therapy and goes by a variety of other names including
Endonasal Technique, Bilateral Nasal Specific Technique, Functional Cranial Release, and Neurocranial Release
Balloons, or hobbit condoms, I can’t tell from the photographs, are inserted in the nose and inflated to realign cranial bones. Really. Watch the video. Judging from the patient it is not a pleasant experience. And I always watch these videos with the eye of an infection control doctor. No gloves or hand hygiene to be seen and given the propensity for MRSA to be harbored in the nose it gives me the willies. The proccedure appears to be booger compression, an intervention I suspect no one really needs. Compressing snot is unlikely to have any significant anatomic or physiologic effects.
The method is different than Balloon sinuplasty where the sinuses are opened up by expanding a balloon in the sinus ostia. In contrast to Nasal Cranial Release the balloon is directed into place with a endoscope and look! they use local anesthesia and wear gloves in a procedure room. Watch the videos sequentially and compare and contrast real medicine with cargo cult medicine.
Nasal Cranial Release was developed in the 1930’s by a chiropractor and naturopath in Portland Oregon. to treat concussion and traumatic brain injury. Really. It gives me the heebie-jeebies to think of inflating a balloon in the nose after traumatic brain injury. The balloon should do nothing to normal skulls, but if there were an occult basilar skull fracture?
Chiropractic manipulation and soft tissue manipulation administered 2–6 times per month for approximately 1 yr had minimal long-term effect on the patient’s head pain.
Really? The patient returned 6 times a month for a year with no effect from the intervention? That’s salesmanship.
Then the patient had resolution of the headache after several months (!) of nasal realignment. The patient probably reporting improvement so they would never have to return for more treatments. It would appear to be the only way to escape the clutches of that particular chiropractor.
Even chiropractors admit that
No articles appear available in the scientific literature that examine the efficacy of the nasal specific technique for treating any pathology. Searching for such literature uncovered one unpublished study by Nyiendo and Goldeen. Their study concluded that claims for improved vision and hearing following nasal specific treatment could neither be supported nor refuted. They did find, however, changes in craniofacial measurements that did not reach significance when compared to a control (sham-treated) group.
No known efficacy and no effect on the skull, yet like all pseudo-medicines, the ‘indications’ are broad. Mostly suggested for concussion and head trauma, it is also suggested for a variety of neurologic diseases and symptoms.
the connective tissues that surround your brain and spinal cord called the Dura Mater are specifically released using endonasal balloon inflations.
and the nasal balloons
Restore the brains ability to oxygenate itself through both improving air flow into higher area’s through the nasal passage
Which made me giggle as it seems to imply that oxygen gets to the brain directly across the base of the brain rather than from the arteries by way of the lung.
Chiropractors like to brag that their education is on par with MD’s. Then they do bone-headed pseudo-science like this, actually thinking that by inflating a balloon in the nose they can place
… the bones in better alignment allows for optimal blood flow to the brain and spinal cord and neuro-transmitter activity throughout the central nervous system.
After a shorter version of this essay a chiropractor commented
Please don’t generalize. Most chiropractors don’t do this ridiculous crap and we definitely do not learn this in chiropractic school.
As if correcting spinal subluxations, the raison d’être of chiropractic, is any less goofy. And maybe the other ridiculous crap in chiropractic school will be no longer be taught:
Six chiropractic schools signed a position statement that says
The teaching of vertebral subluxation complex as a vitalistic construct that claims that it is the cause of disease is unsupported by evidence. Its inclusion in a modern chiropractic curriculum in anything other than an historical context is therefore inappropriate and unnecessary.
which is similar to the General Chiropractic Council in the United Kingdom, which has a Guidance that says
The chiropractic vertebral subluxation complex is an historical concept but it remains a theoretical model. It is not supported by any clinical research evidence that would allow claims to be made that it is the cause of disease or health concerns.
They are referred to “Subluxation Deniers.” Like Vaccine and Global warming deniers, one of these is not like the others.
I wonder if all the ridiculous crap is removed from chiropractic practice, what will be left?
Neuroscientists are making progress mapping out the cortical pathways that allow us to know where we are and navigate to a desired location. A recent study adds another bit of information to this growing picture.
The Nobel Prize for Physiology or Medicine for 2014 was given to several scientists, Dr. John M. O’Keefe, Dr. May-Britt Moser and Dr. Edvard I. Moser, for their collective research in working out the basic neurological function that underlies our ability to place ourselves in our environment and to navigate around. O’Keefe identified place cells. A specific place cell will fire when we are in a specific location. Different patterns of place cells firing represent different locations.
These place cells are found in the hippocampus, specifically area CA1. O’Keefe also found that the place cells have memory function, and are therefore critical to our ability to remember specific locations.
Moser and Moser extended this work by finding grid cells in the entorhinal cortex. This area connects heavily with CA1, and contains cells that behave like the place cells. However, the grid cells are arranged in a hexagonal grid, and they fire in sequence as rats move through their environments. The grid cells therefore seem to be a literal map of the environment, and track our movement through the environment, while the place cells tell us where we are.
The new study adds yet another piece to this circuit. Hiroshi Ito and others, including Moser and Moser who discovered grid cells, did further studies looking at rat brains as they navigate their environment. In this case they were specifically interested in how the rats decide which direction to go in when confronted with a T in a maze. They used a maze designed like a figure 8 so that as they rats when around they would endlessly come to right-left branchings.
They found a number of things. First, the intensity of firing in the CA1 area predicted in which direction the rat would go. The neurons were all firing, but would fire more intensely in one pattern when the rat went right, and in another pattern when the rat went left.
They also identified a circuit leading from the prefrontal cortex to the nucleus reuniens in the thalamus and then to the CA1 area of the hippocampus. The prefrontal cortex is essentially part of the decision-making part of the brain. The thalamus is the main relay center in the brain. This circuit, therefore, makes sense based upon our current models of brain circuitry.
The researchers found that the differential firing of the CA1 neurons predicting navigation direction stopped when they lesioned or blocked signals from either the prefrontal cortex or the nucleus reuniens.
This circuit, from prefrontal cortex to nucleus reuniens, and then to the place cells in CA1, which itself connects to the grid cells in the entorhinal cortex nearby, allows us to think about where we are, where we want to go, and how to get there. This may not be the complete circuit, but it seems like the core portion of it at least.
These types of circuits in the brain likely represent the low-hanging-fruit for neuroscientists because they display some version of what is called somatotopic organization. In other word, the layout of the neurons reflect their function. In this case, grid cells are literally laid out in a grid that reflects the outside world. Another part of the brain that reflects a physical lay out that relates to external reality is the visual cortex. Neurons light up in a pattern that reflects the actual image that is being viewed.
These core physical functions are a great opportunity for scientists to explore the circuitry of the brain. However, there are many other circuits in the brain that subsume more abstract functions that will likely prove more challenging to map out. Imagine the complexity of the circuitry that represents language, for example.
Another advantage of basic functions like navigation is that we can study rats. We can’t study rats to map out the circuitry of language, however.
We might not even have the tools yet to explore brain circuitry in sufficient real-time resolution to map out the more complex and abstract functions. Technological advancements here, however, are making steady progress, and so our ability to map out brain circuits will likely continue to improve with the technology. I also think that efforts to reproduce brain function in computers will greatly advance our efforts to map the brain’s circuits (and vice versa).
It is always difficult to predict how long a certain scientific endeavor will take. It tends to be slower at first then we hope or imagine, but then quickly becomes faster than anyone expected as technology advances. Think of the human genome project, which accelerated by orders of magnitude over the length of the project. We are now engaged in the human connectome project. As new tools come online the pace of progress will increase. It’s hard to say if progress will be as dramatic as it was with the genome project, but it might be.
Brian Clement is a charlatan. Unfortunately, that doesn’t seem to be a problem for the State of Florida. I made two (which turned into three) attempts to get the state to take action against Clement or the Hippocrates Health Institute, where he serves, with his wife Anna Maria Gahns-Clement, as co-director. All of them failed. Brian Clement slithered through the cracks in Florida law each time.
Before we get into the details of Florida’s failure to act, a bit of history (and there is plenty of it) is in order.
In recent months, Clement’s sordid cancer quackery has been well-documented in the media as well as in the science “blogosphere.” (I’ve listed what I hope is a — but almost certainly isn’t — complete blog archive at the end of this post. Many of the Canadian Broadcasting Corporation [CBC] and other news reports are linked in these posts.) Most of the coverage has centered on two Canadian girls suffering from lymphoblastic leukemia whose parents pulled them from conventional cancer therapies, which gave them an excellent chance of survival, in favor of treatment at the Hippocrates Health Institute (HHI), a sprawling spa in West Palm Beach, Florida, licensed as a massage establishment by the state.
Clement gave a talk in Canada, in 2014, claiming “we’ve had more people reverse cancer than any institute in the history of health care.” (“We” is the operative word here, because it later served as Clement’s ticket to avoid prosecution by the Florida Board of Medicine, as you shall soon find out.) The girls’ families were impressed.
Sadly, one of the girls, Makayla Sault, died earlier this year. The other, identified only as “JJ” in the media because of a publication ban, has returned to conventional treatment. However, her mother apparently remains under the influence of Clement: JJ is restricted to a raw foods diet and is still being followed, if that is the right word, by HHI.
Other cancer patients have been lured by the false hope Clement and Hippocrates offer: Stephanie O’Halloran, a young Irish woman, the parents of Anael L’Esperance-Nascimentol, a three-year old Canadian boy who went for treatment there, Laurie Ann Prince, Kathyrn Tachell and Kim Curry, also Canadians and all fairly young, and Lajos Tringer. (Tringer is also considering treatment by another cancer quack, Stanislaw Burzynski.) Most of these either drained their own resources or raised funds,or both, to pay the considerable expense of going to Hipppocrates. Unfortunately, we know that Stephanie, Kathryn and Kim have died. I’ve been unable to discover the fate of Aneal and Laurie Ann. Lajos is, as far as I know, still living, but very ill.
Orac, in his inimitable way, has described the services (for lack of a better term) offered at Hippocrates as “cancer quackery on steroids.”
Let’s take a look again at the sorts of treatments offered by Brian Clement as part of HHI’s “Life Transformation Program“. They include:
Laughably, Clement claims to be a researcher who is “constantly in a position where I’m addressing medical conferences and universities,” and has “a body of evidential science ” which demonstrates why he’s observed “tens of thousands of people recovering from catastrophic disease.” Oddly enough, even though HHI has “the number one ratio of having people reverse the aging process and reverse disease than any other organization in the history of man,” a search of PubMed does not disclose a single journal article published by Clement. Really, Mr./Dr. Clement, we implore you, let the rest of the world in on your ground-breaking discoveries. Publish in a top-flight, peer-reviewed science journal, so more lives can be saved. He also claims that “we’re in the middle of a study with the University of California to disprove genetics.” Maybe that research will show up soon in the medical literature. We can only hope.
With that background, we’ll turn to the State of Florida and its failure to stop Brian Clement, who, as we shall see, has embarked on yet another international speaking tour.Strike One: Operating a healthcare facility without the appropriate state license
I filed a complaint against Hippocrates Health Institute with the Florida Agency for Health Care Administration (AHCA) for operating a health care facility without the proper state license. (Regrettably, none of the AHCA or Florida Department of Health documents I mention in this post are available online.) AHCA licenses health care facilities, such as health clinics and hospitals. As I did with my complaint against Clement, which we’ll get to in a minute, I informed AHCA that these allegations were not based on personal knowledge, but rather upon Canadian media reports. However, in light of what I had read, I felt an obligation to report Clement and Hippocrates to the state. It appeared to me that the health of these girls and others was at stake and that it was appropriate for the state investigate what was going on.
According to a January 28th email from AHCA:
After some additional Agency research, unfortunately this “Health Center” does not meet the definition of a Health Care Clinic, see [Florida Statutes] Section 400.9905(4),
“Clinic” means an entity where health care services are provided to individuals and which tenders charges for reimbursement for such services, including a mobile clinic and a portable equipment provider.
Since this institute is a cash-only business, it does not fall under AHCA’s regulation. The only recommendation I have, is to file complaints with the Department of Health (Board of Medicine) against the individual doctors, thanks.
In other words, since Hippocrates doesn’t get reimbursed by third party payers, such as health insurance companies, it does not need a license and AHCA had no jurisdiction to do anything about the clinic.
A Senate bill was introduced this year in the Legislature to eliminate this loophole in the health care licensing law.
According to Sen. Eleanor Sobel, one of the bill’s sponsors,
flimflam artists and snake-oil salesmen have escaped state scrutiny by running clinics that accept only cash.
Like Clement and Hippocrates, for example.
It wouldn’t matter anyway. Hippocrates would have been exempted if the bill had become law (it died in committee). The licensing law exempts non-profits. The IRS has granted Hippocrates tax-exempt status as a Section 501(c)(3) charitable organization. A non-profit that, according to its 2013 tax return, paid Brian Clement and his wife, Anna Maria, together about $1 million dollars in salary, bonuses and other unspecified income, has $30 million in assets, and took in over $17 million in revenue.
What Hippocrates does have is a massage establishment license issued by the Board of Massage Therapy, for which it pays the paltry sum of $100 per year. The law seems designed to prevent houses of prostitution from operating as fictitious massage parlors, but rank quackery doesn’t appear to be a problem as far as licensing goes.Strike Two: Department of Health v. Brian Clement, Case No. 2014-19139
On November 13, 2014, I filed a complaint with the Florida Department of Health against Clement for practicing medicine without a license. (The Medical Quality Assurance [MQA] division of the Department investigates unlicensed practice complaints as well as complaints against licensed health care practitioners.) I explained to the MQA that I didn’t have any personal knowledge about Clement’s conduct, but was reporting him based on news stories covering Makayla’s and JJ’s treatment at Hippocrates, including the report that his degrees were from diploma mills and the fact that he is not a state licensed health care practitioner, although he says he’s an NMD on HHI’s website. I specifically included his statements about “reversing cancer,” Makayla’s grave medical condition at the time (I later reported that she had died) and the fact that HHI was continuing to provide care by analyzing the child’s blood test results which are sent by mail. Later, I spoke to an investigator with MQA and gave her links to the HHI website and to a number of Canadian media reports.
On February 10, 2015, the Department of Health issued a “Notice to Cease and Desist” in the case, stating that
the Department has probable cause to believe that Brian Clement of Hippocrates Health Institute . . . is not licensed by the Department or the Board of Medicine and is practicing medicine without a license. . . .
Wherefore, . . . Brian Clement . . . is hereby notified to cease and desist from practicing medicine in the State of Florida unless and until Brian Clement is appropriately licensed by the Department.
The Department also ordered Clement to pay a fine of $2500 and costs of $1238.
On February 16th, an “Unlicensed Activity Investigative Report” was issued by the Department. As investigations go, it seems pretty cursory. It contains copies of only two of the many news reports and one of the many blog posts. Also attached are printouts from the HHI website detailing the appalling quackery offered there, which seems to have bothered exactly no one at the Department.
An investigator went undercover to an appointment with Clement, which elicited, as might be expected, only quack recommendations (including the sale of merchandise from HHI) but no smoking gun statements to the effect that Clement is an MD or specific medical diagnoses or treatments.
The report says that the investigator did not attempt to obtain a release of patient information authorization from the Canadian girls because “their names were not provided or noted in the Canadian News Release,” even though Makayla’s name is clearly given in one of the news stories attached to the investigative report itself, as well as in links to other information provided by me. And it seems that the Department could have at least attempted to discover the identity of JJ through contacting the Canadian government. After all, the ban is on publication of her name, not giving it to the authorities in another jurisdiction doing an investigation of alleged criminal activity. (Practicing medicine without a license is a felony in Florida and, according to the Department, the Sheriff’s Office was notified, although I have a feeling that one will go nowhere too.)
Another person not contacted was Steven Pugh, RN, who is suing HHI and Clement over his being fired for refusing to follow Clement’s orders. According to a CBC news report, published well before the investigative report was issued, Pugh maintains he could not legally do what Clement told him because the orders were not issued by an MD. Pugh also reported to the CBC that Clement told patients he could “cure” them, prescribed treatments for them, would overrule what their physicians told them and used the term “Dr.” with patients (as he formerly did on the HHI website). Most disturbingly,
They (Clement and his wife) also placed restrictions on when staff could call an ambulance to take patients for emergency medical care at local hospitals, according to Pugh and other former staff.
According to this same news story, the CBC has interviewed both past and present employees who are disturbed by the Clements’ treatment of cancer patients. It also reports that a former HHI physician is suing for breach of contract, alleging he was terminated after he documented concerns that what he was being asked to do at HHI could be illegal.
A search of the public records (in this case, the of the 15th Judicial Circuit Court) would have revealed Pugh’s suit, along with two others, against HHI. You’d think a decent investigation would, at the very least, include a search of the public records of the state doing the investigating.
All of this leads me to conclude that the Department should hire the CBC to do its investigations. They do a much better job.
Much to my surprise, I learned from – you’ll not be surprised to learn – the CBC, that the Department, citing insufficient evidence, had dropped all charges against Clement. I called the Department for more information but my call was never returned. Later, when I called about the Department’s second complaint against Clement (see “Strike Three,” below), I was told by a Department official why they couldn’t prosecute Clement for the unlicensed practice of medicine. He explained that HHI has a physician on its staff and, by Clement always using the collective “we” ( as in, “we can reverse cancer”), he personally was not practicing medicine.
Since we’re on the subject, HHI has a new physician as Medical Director, Tina Discepola, MD. I wondered what sort of physician would associate herself with an operation like HHI and a guy like Clement, especially in light of their well-publicized troubles. Even if you fully believed in what they were doing, you’d have to be wary of stepping into the hornet’s nest of litigation and on-going investigations by the Department of Health. (Pugh has filed his own complaint against Clement.)
In any event, Dr. Discepola is certainly well acquainted with, shall we say, unconventional medical practices. According to her HHI bio, although board-certified as an emergency medicine doctor, she formerly practiced Functional Medicine in New York. She is also into acupuncture and cranial sacral manipulation. She is a Diplomate of the American Board of Integrative Holistic Medicine, Andrew Weil’s brainchild, a Diplomate of the American Board of Anti-Aging Medicine (also, here) which is not recognized as a specialty board by any of the medical specialty-credentialing organizations, and is an “active member” of the American College for the Advancement of Medicine, ditto.
Curiously, while she holds an active license to practice medicine in Florida, according to the Department of Health’s Practitioner Profile, she still lists New York as her residence and states that she does not practice in Florida. I understand that Dr. Discepola is new to Florida, and is likely unfamiliar with the laws and regulations governing medical practice in this state, but she may want to review them, including the state’s rules on physician advertising.
Dr. Paul Kotturan, the former Medical Director, is still on staff, running the “Vida Building” operation, which must surely contain more quackery per square foot than any other building in the state, including, for example, the Aqua Chi ionic detox footbath, which, if it works as advertised,
Your meridians are permeated and re-aligned back to their original strength and placement.
If there is a medical device equivalent of homeopathy, the detox footbath is it.Strike Three: Dietetics & Nutrition Practice Council v. Brian R. Clement, Case No. 2014-19150
In a letter dated March 24, I learned that this complaint had been reviewed by legal staff and recommended for submission to the next Probable Cause Panel. It was another surprise from the Department for me, because I didn’t even know this particular complaint had been filed. This time, according to the complaint, it was because Clement illegally used the initials “NMD,” which stands for “Naturopathic Medical Doctor.”
Clement is a licensed nutritional counselor in Florida, but is not an NMD. (Florida used to licensed naturopaths, but stopped issuing licenses in the 1950s, although the state allowed those with ND licenses to continue their practices. To my knowledge, none of these licensees are still practicing.) As with all types of health care practitioner licensing, nutritional counselors are required to follow certain laws and administrative rules. The complaint alleged Clement violated three separate provisions in advertising himself as an NMD.
Based on this, one might say that the Department of Health is charging Clement with practicing quackery without a license, but that’s not how they see it.
“Nutrition counselors” are no longer licensed in Florida either, but those, like Clement and his wife, who hold nutrition counselor licenses, can still practice. The state now licenses only registered dietician/nutritionists, who must meet more rigorous requirements than nutrition counselors. Neither of the Clements have the qualifications for the newer license, at the least because they didn’t graduate from an accredited school nor have they taken the current licensing exam.
Nutrition counselors are, by law, limited in their practices to advising people on appropriate nutrition intake by integrating information from an evaluation of nutritional needs, using appropriate data to determine those needs, and making appropriate nutrition recommendations.
It is striking to me that, given the reams of material documenting Clement’s inappropriate nutrition advice, it never occurred to the Department to add a count or two to the complaint for substandard practice as a nutritional counselor. What about wheatgrass, which, as Orac points out, Clement seems to regard as a virtual panacea, and raw foods diets? And what about wheatgrass enemas? is that an “appropriate” nutrition recommendation?
In the Department’s investigative report, mentioned earlier, we learn that the investigator reviewed the Hippocrates website and confirmed that Clement was using the title NMD. (Gee, notice anything else, Mr. Investigator?) The report notes that Clement had been issued an informal Notice to Cease and Desist in the past for advertising and using the title NMD on the website. Later, when the undercover investigator revealed his true identity, Clement denied using the title “Naturopathic Medical Doctor” on the Hippocrates website, despite all the evidence to the contrary.
Brian Clement no longer uses the honorific “Dr.” nor does he list the initials “NMD” on the website. He still uses PhD on his bio page, and the covers of book advertised at the bottom of the page identify him as being an NMD. He says nothing about where he earned any of the claimed degrees. He has been referred to as “a naturopath” or “Naturopathic Medical Doctor” in a couple of news reports, information they must have gotten from somewhere. (Also: “a formally trained biochemist.”)
Clement has variously claimed his degrees came from University of Science, Arts, and Technology (USAT), where he says he earned his PhD and NMD, and Lady Malina Memorial Medical College, where he says he earned a post-grad degree. (Interestingly, for both Brian and Anna Maria, no undergraduate degree is ever mentioned.) But those institutions have been called out as diploma mills in both a CBC report and academic literature. The president of USAT denies Clement was granted an NMD degree by his school.
Anna Maria Ghans-Clement’s HHI bio say she has a PhD in nutrition from “Denmark University” but I could not find that any such institution exists. It refers to her as “working on her nursing degree,” implying that she has one, but does not say whether the degree requirements were completed. Her book covers also show her as having an NMD and one of her book covers shows the book as being authored by “Dr. Anna Maria Clement.”
Nevertheless, once again, the Department backed off. According to the Department’s letter to me of May 20, the probable cause panel determined that probable cause existed to support prosecution of Clement for allegations set forth in the complaint. However,
after the finding of probable cause, the Department discovered correspondence from [Clement’s] counsel that was not available to the probably cause panel during consideration of the case. As a result of this new information, the Department dismissed the Administrative Complaint due to insufficient evidence to prosecute.
They don’t say what information was “discovered” in this letter. A Department official did tell me in a phone conversation that Clement does, in fact, have an NMD degree and that, while he can’t practice as a naturopath, he is not prevented from using those initials after his name. I informed the official that, to my knowledge, he does not have an NMD degree, information that seemed to surprise him. In fact, the denial was reported by the CBC (yes, them again) in 2014, well before the Department acted.
So, where in the world is Brian now? According to his busy speaking schedule, he is, a this moment, at the “Real Truth About Health” conference in Orlando, where he is identified as an NMD. Then he is off to New York to lecture. (At the Manhattan lecture, attendees will get 10% off HHI’s 3-week “Life Transformation Program,” a “$600 value.” Which means, if you do the math, the 3 weeks normally costs $6,000.) After that, it’s Europe: Norway, Sweden, the UK (he’s still a naturopath according to one of the London announcements), Ireland, Germany (“Dr. Clement”), France, Switzerland and Holland. The announcements indicate Clement is being circumspect about what he says, but who knows what he’ll do when he actually gets there. Clement doesn’t seem like the type who can resist grandiose pronouncements regarding his many self-proclaimed talents. And, of course, once within the confines of the vast acreage of the privately owned (but taxpayer supported) Hippocrates campus, he is virtually free from scrutiny.
The Brian Clement/Hippocrates Health Institute Blog Archive
Society for Science-Based Medicine
One of the persistent themes of SBM is that CAM (complementary and alternative medicine, or integrative medicine) is nothing more than a marketing brand. Its recent popularity is not based upon new evidence or a changing paradigm of medicine as its proponents claim. Its popularity is increasing despite the lack of evidence for specific CAM treatments and despite a dedication to evidence-based medicine within the profession.
CAM is also modern mythology, which I guess all really effective advertising and branding is. It floats atop a number of demonstrably false marketing claims. One is that the popularity and use of CAM is surging. This is partly a self-fulfilling prophesy, and no doubt it is increasing, but the degree to which CAM is popular has been consistently exaggerated by proponents (largely as a way to justify its existence).
This myth is largely perpetuated by redefining CAM as needed, including things like prayer, massage, and taking vitamins. I suspect that praying for a sick loved-one has always been popular and doesn’t represent a trend toward CAM. When unequivocal alternative modalities are considered, their use is still tiny and not increasing. The most recent NIH survey found:
Use of acupuncture (1.1%), homeopathic treatment (1.7%) naturopathy (0.2%), and energy healing (0.5%) was miniscule.
Other myths of CAM marketing include that the treatments are “natural,” that they are entirely safe, and in many cases that they are ancient. Collectively these claims create an emotionally compelling narrative or image – we can almost see the “noble savage,” one with nature, ancient wisdom evident in their gentle gaze, as they lay their healing hands on the afflicted. Imagine a boardroom scene from Mad Men as they pitch this idea.
Perhaps the biggest CAM myth, however, is the notion that people turn to CAM primarily because they are dissatisfied with mainstream medicine. I think this is partly the default assumption that most people make, but it is also perpetuated by CAM proponents. So popular is this myth that I find it is common even among CAM critics and proponents of SBM.
The problem with this assumption is that it is at odds with the actual evidence that we have. Studies show that satisfaction with mainstream medicine is not an important factor in deciding to use CAM, that CAM users are generally satisfied with their mainstream care, and they use CAM because it aligns with their philosophy, and they simply want to expand their options.
A 2007 review found that people who seek CAM hold a number of beliefs and motivations. These include a desire for control, a belief in “natural” remedies and “holistic” medicine, and a belief that psychological factors are important to health.
Overall world view is also highly predictive of CAM use. Specifically, a world view researchers have called the “cultural creatives:”
This group is said to represent unconventionality and is characterized by commitment to causes such as feminism, environmentalism, spirituality, personal growth and a love of the foreign and exotic.
The evidence also suggests that the same factors are involved in maintaining CAM treatment. While CAM practitioners are quick to cite the alleged effectiveness of their treatments as the reason their customers keep coming back, CAM users cite empowerment, communication, and beliefs about treatment as their main reasons – even when they perceive that the treatment is not working. They also do not cite dissatisfaction with conventional medicine.
A 2015 survey of CAM use among cancer patients had similar findings. They found that attitudes toward CAM is what predicted CAM use. Positive attitudes toward CAM also correlated with being young, female, and college educated, factors which also correlate with the “cultural creatives.”
The evidence clearly shows that CAM is a cultural phenomenon, not a result of change in scientific thinking, and not an emergence of new evidence. It is a brand that taps into a particular subculture, one that values the natural, ancient, and exotic. Surveys also show that CAM use has leveled off in the last 10 years, perhaps because it has reached full market penetration in the target demographic.
However, there is still significant spill-over into the general culture. The dedicate minority of CAM enthusiasts have been successful in spreading their brand, and infiltrating specific markets, such as hospitals, universities, and pharmacies. They have been politically successful as well, securing dedicate research funding (which in practice is just another method of marketing) and favorable regulations.
They have also, quite disappointingly, put academics off their game. Most academics are completely unfamiliar with the CAM phenomenon and give it little thought (what we call “shruggies”). A few see it for what it is (an attack on the scientific basis of medicine) but feel constrained by the forces of political correctness, which are powerful within the halls of academia. This is a generational failure. How this is looked upon in the future will likely depend upon which philosophical outlook emerges as dominant.
At the very least we need to expose the facts as they are. The facts tell a very different story from the CAM marketing mythology.
This is very cool – programmers have created a process with which they scour the internet for photographs. They then categorize them by subject matter, and then select groups of photos that are essentially of the same subject over different periods of time. They crop, color correct, and adjust each photo so that it matches a master, and put them together to create a time-lapse video.
The result is thousands of time-lapse videos that might have taken years to otherwise create.
This is a fun demonstration of two technological trends that are worth pondering. The first is the absolute explosion in digital data, including photographs and video. One estimate is that there were 880 billion photos uploaded in 2014. There are 27,800 photos uploaded to Instagram alone every minute. This is partly due to the smartphone revolution – a large portion of the population in developed nations walk around with a camera on them at all times.
Those cameras are getting better, which is good because they are becoming the primary camera that people use to photograph their lives. I have been going to school events, for example for the last 10 years. Over that time there has been a clear shift from dedicated cameras to cellphone and iPads. It is not unusual now to be in an auditorium with hundreds of parents, and for there to be only two or three “real” cameras in the crowd; the rest are cell phones.
As this trend continues it is interesting to ponder where it will level off. We may not be close to the ultimate extent to which our lives will be photographed and videoed. Further, we are largely doing it to ourselves. Combine this with the trend toward public cameras. Traffic cams, security cams, and other public cameras are also dramatically increasing. There is a move in this country (which, all things considered, seems like a good idea) for every police officer to wear a go-pro type camera. Drones with cameras are also becoming more common. What other trends will be added to the mix? How long before significant numbers of people are walking around with some Glass equivalent, video recording every moment of their lives.
The other technological trend is the power of computers to data mine – in this case to sift through billions of photographs to find ones of interest. This is a powerful combination: mountains of data with the power to sift, sort, locate, and analyze that data.
What I find interesting to consider is where this trend will get us. It does not seem unreasonable that eventually the density of recording devices will be so high that virtually any moment in any public space will be recorded and accessible.
This, of course, is a boon to law enforcement and security. It is also potentially a threat to our privacy.
As an example, I have already had to shift my perspective in that, whenever I am in a public space or communicating with someone electronically I assume that whatever I say or do can be made public and broadcast to the world. Some people who have not made this shift have had to pay the consequences. We pretty much have to assume a complete loss of any privacy except in our own sanctuaries, and even then electronic devices may be spying on us.
One specific aspect to consider is, who controls and has access to the data? I am not as worried about public data on the internet. The public has access to this, and therefore this data could potentially be used to protect the public. We have already seen abusive cops come to justice because of a cell phone video. (I won’t get into specific controversial cases, but you can see the potential.)
There is reason to be concerned about data that is only accessible by large corporations or the government. This, of course, is the focus of the NSA controversy. The government is sifting through billions of phone calls looking for the telltale signs of terrorists. Whether this program as currently enacted makes the optimal balance between protecting the public and spying on the public is a matter of legitimate debate. I do think this requires open and transparent discussion and oversight. This is an incredibly powerful tool to put in the hands of any government, and the potential for abuse is massive.
The online time-lapse algorithm is a cool use of this technology, but it highlights the power of big data and powerful computer algorithms. This power can be used for good, for fun, or for evil.
The technology is great, but it may be getting away from us a little bit. Perhaps we are at a crossroads where we have to think carefully about how big data will be regulated. We have to look at the big picture. Part of the problem is that individual small decisions are adding up to a big unintended effect. Taken by itself, it may be a great idea for all cops to have personal video recorders. It may make sense to have traffic cams everywhere.
But taken as a whole, all these individual uses of recording technology are eating away at our privacy. What net effect will this have on our lives?
I think the existence of ubiquitous cameras is a done deal. It is probably not worth fighting this trend. We may need to focus on who controls the data, and how to protect privacy given the existence of massive recordings.
Cervical dysplasia is a precancerous condition picked up by Pap smears. It is most often caused by human papillomavirus (HPV) infection. Mild cases may resolve spontaneously and can be followed by observation with frequent Pap smears, but cervical dysplasia can progress to cancer. The standard treatment is to remove the abnormal cells with a cone biopsy (using a knife) or a Loop Electrosurgical Excision Procedure (LEEP) using a wire loop heated by electricity. Those procedures not only treat the disease, but they provide a pathology specimen that can be examined to rule out more serious or invasive disease. Both LEEP and cone biopsy are 85-90% effective in removing all the abnormal cells. If cancer is suspected, a cone biopsy is preferable because LEEP may damage the edges of the specimen and make it more difficult to interpret. Otherwise, LEEP is often preferred because it is less expensive and doesn’t require anesthesia or an operating room.
Surgery is often perceived as scary and not “natural,” so it’s not surprising that a “natural” treatment has been devised to replace surgery. Escharotics are corrosive salves that get their name from the thick dry scab that they can produce called an eschar. The “natural” escharotic treatment alternative for cervical dysplasia involves applying a solution of bloodroot (Sanguinaria canadensis) and zinc chloride. They claim that the solution selectively kills abnormal cells of the cervix while leaving healthy cells unaffected. That claim is almost certainly false, and the efficacy and safety of escharotic treatment has not been properly tested or compared to conventional treatment.
Who is using escharotic treatments?
Not mainstream doctors. Neither the American Congress of Obstetricians and Gynecologists (ACOG) nor the American Cancer Society (ACS) even mention escharotic treatments on their websites. The only proponents I could find were naturopaths and chiropractors.
Dr. Nicholas LeRoy bills himself as a “holistic medicine physician” but he is actually a chiropractor who is also trained in acupuncture and oriental medicine. His website says he “designs unique therapies” for the treatment of cervical dysplasia and other conditions, and he integrates chiropractic, diet, nutrition, acupuncture, and herbal supplementation. He also offers breast thermography and makes misleading claims for it, and he spreads misinformation about “90 deaths and thousands of injuries” from HPV vaccines. He has a book coming out on HPV and cervical dysplasia. It is ironic that someone who is putting himself forward as an expert on cervical dysplasia is discouraging the HPV vaccine, the one thing that offers an effective way to prevent the disease.
Naturopathic treatment of cervical dysplasia
A naturopathic review of cervical dysplasia recommends this approach to treatment:
They admit that critics of natural medicine say there is no published evidence that these options work or are backed by science, but they say several recent published articles explain the science and patient outcomes. They don’t list those “several” articles; they only cite a single published case report of one patient who was treated with 10 escharotic treatments along with an oral vitamin and botanical protocol.
Another naturopathic website makes similar recommendations and adds detoxification, exercise, stress management, homeopathy, and flower essences. It never ceases to amaze me how naturopaths will throw everything but the kitchen sink at a disease hoping something will stick. If the treatment works, they have no way of knowing what part of the treatment worked; so they convince themselves that the whole schmear was necessary and that they have some kind of holistic wisdom that guided them to know what particular mix of treatments would work.
Their reasons to prefer escharotic treatment
They claim escharotic treatment is superior to LEEP because it is natural, doesn’t damage normal cells or leave a scar, doesn’t cause complications like infection or reduced fertility, stimulates regrowth of normal tissue, is preferred by most women, and is less expensive. Every one of those claims is questionable. They only admit one drawback: escharotic treatment requires more visits (ten versus one). They don’t mention that the tissue removed in LEEP can be examined for invasive cancer, which is not possible with escharotic treatment. They also don’t mention that LEEP is covered by medical insurance, while escharotic treatment is not.
What is the evidence?
A PubMed search produced only the same single case report mentioned above.
And the only other evidence I could find anywhere online consisted of references to two studies reportedly published in an obscure, unobtainable naturopathy journal. A 2003 review article on cervical dysplasia in the Alternative Medicine Review describes those two studies. The references for those studies list the same author, Hudson TS, and a journal called J Naturopathic Med. That journal is not listed in PubMed and I have been unable to even find any reference to such a journal on the Internet except in a very few citations on naturopathic websites. I even tried to locate it by asking one of the proponents of the therapy for help, but my query went unanswered. Note: I did find an International Journal of Naturopathic Medicine, but it was founded in 2004, long after the 1991 and 1993 dates of the studies in question.
So I can’t evaluate the studies for myself; I can only copy the description of the two studies provided in the review article:
A small study using local escharotic treatment was conducted on seven women with carcinoma in situ of the cervix. Three levels of treatment were employed: local treatment to the cervix, systemic treatment, and constitutional treatment. Local escharotic treatment utilized preparations of ZnCl, bloodroot, bromelain, and Calendula succus. Local treatment was repeated twice weekly for five weeks with treatments 2-3 days apart. Topical vitamin A was applied following each local treatment. Systemic treatment was comprised of ascorbic acid 6-10 g per day, beta carotene 120,000-180,000 IU per day, and selenium 400 mcg per day. Patients were also prescribed a vegan diet to eliminate animal fats and two botanical compounds to enhance immune function – Taraxacum officinalis and Arctium lappa. Systemic treatment was continued for not less than three months. Constitutional treatment consisted of a homeopathic remedy prescribed on an individual basis. All seven patients received one year of follow-up. Four of the women remained disease free after the one-year period. One woman improved to atypia and then reverted to mild dysplasia. One woman had resolution of the cells of the endocervix and not the ectocervix, and one woman had resolution of the cells of the ectocervix and not the endocervix. The latter patients appeared to be non-compliant with regard to diet and lifestyle changes, suggesting the influence of synergistic effects of this multi-modality approach.
Comment: That dose of beta carotene is 60 times the RDA, and The Medical Letter says no one should take high dose beta carotene. The addition of homeopathic remedies to the protocol speaks volumes about the author; it does not inspire confidence. This was a very small uncontrolled preliminary study with only 7 subjects, and the results were not promising. The success rate of 4 out of 7 is far inferior to the 85-90% success rate with conventional treatment. And then they have the gall to blame the treatment failures on the patients for non-compliance with peripheral recommendations that were highly unlikely to have affected the study results. They say it suggests the influence of synergistic effects of their multi-modality approach. No, it doesn’t. It suggests that the escharotic treatment is less effective than LEEP or cone biopsy.
A follow-up study was performed by the same group on 43 cases, including cervical atypia (n=7), cervical dysplasia (n=26), and carcinoma in situ (n=10) during the following two years. A similar protocol was used, with the addition of 10 mg folic acid daily. The results of the study were encouraging as 38 of the women had complete regression to normal, while three of the women had partial regression and two had persistent lesions. The two women with persistent lesions had lowgrade dysplasia. These studies yield promising results, suggesting the need for a multi-faceted approach to preventing cervical cancer. Table 3 summarizes the escharotic treatment.
You can read the details of Table 3 here. It describes the protocol for applying bromelain and washing it off with a solution of calendula succus, then applying a zinc/bloodroot mixture, and washing that off. Finally, two vaginal suppositories of vitamin A are left in the vagina for 24 hours.
Comment: This study mixed in cases of atypia, dysplasia, and carcinoma in situ, and without access to the study, we can’t determine the success rate for the various diagnoses. What if the seven cases of atypia were all in the group that had complete regression to normal? What if they were all mild cases of atypia that would have resolved spontaneously without any treatment? While larger than the first study, it is still small and preliminary, and there is no control group, so we have no way of knowing if it is less, more, or as effective as conventional treatments like LEEP. I am left wondering why they chose to add folic acid, and why they used such a large dose, a whopping 25,000 times the RDA, a level that has been shown to increase the risk of cancer and heart problems. One of the instructions, “avoid contact of the zinc/sanguinaria mixture with the vaginal wall,” is a tacit acknowledgement that it does damage normal cells, contradicting their claim that it only affects the abnormal cells.
I found the same description of these two studies almost exactly word-for-word on a naturopath’s website, making me wonder whether the naturopath had read the studies herself or merely copied the description from the review article.
Bloodroot and Black Salve
Bloodroot is one of the main ingredients in black salve, a dangerous topical quack treatment for cancer. In the form of Cansema, black salve containing zinc chloride, bloodroot and other ingredients has been banned by the FDA for use as a cancer cure. But it is still readily available online.
Steven Barrett says “The idea that bloodroot kills only cancer cells and spares normal ones is preposterous.” There is good evidence that bloodroot does damage normal cells. A woman who used it for skin blemishes developed a large eschar on her neck and was left with a scar. Black salve products containing bloodroot are known to cause necrosis of normal tissue along with scarring, granulomatous inflammation, implanted foreign material, reactive stromal atypia, suppuration, and residual cancer. A 2012 review of bloodroot pointed out that it can cause significant tissue destruction, scars, and keloid formation.
Quackwatch warns against using escharotics, and shows a picture of a woman whose naturopath treated her with a black herbal salve for a bump on her nose. It ate away a large part of her face and her nose sloughed off. It took 3 years and 17 plastic surgeries to reconstruct her face. The picture is not pretty; click on the link only if you don’t mind being grossed out.
Andrew Weil used a bloodroot paste to treat skin lesions on his dog and on a friend (neither had biopsies to establish a diagnosis) and thought the results were satisfactory, so he started recommending it. According to Quackwatch, his advice persuaded a woman to use it and it destroyed her nose.
In 2014, a letter in The Medical Journal of Australia reported the case of a 55-year-old man who used a bloodroot-containing black salve on a lesion he suspected might be cancerous. It left a 1-inch-wide hole in his head. There is a picture of the resulting hole here. Viewer discretion advised.
What about bromelain?
The only reference to bromelain and cervical dysplasia in PubMed is that same single case report. Bromelain is a mixture of proteolytic enzymes derived from pineapple; it is used as a meat tenderizer in cooking. It is approved in Europe for the removal of dead and damaged tissue in severe burn wounds. It is not harmless to normal cells: the healthy surrounding skin has to be protected with a sterile paraffin ointment.
The escharotic treatment of cervical dysplasia is not evidence-based, is not approved by any professional organization, and is used only by a few chiropractors and naturopaths. The claim that it doesn’t harm normal tissue is false: there is clear evidence that both bromelain and bloodroot damage normal cells. Sure, they can kill cancer cells, but as Rose Shapiro pointed out in her book Suckers: How Alternative Medicine Makes Fools of Us All, you can destroy cancer cells in a petri dish with a flamethrower or bleach. I suppose that anything that destroys tissue ought to be effective for cervical dysplasia (sulfuric acid? a cigarette lighter?). I suspect that they have used escharotics very cautiously and have been lucky to avoid adverse effects so far. The effectiveness and safety of escharotic treatment has not been established; and until it has, I think it is irresponsible to offer it as an alternative to evidence-based conventional treatments. There is reason to think it may be less effective than conventional treatments and there is reason to think it may be dangerous.
When I wrote a week ago about the sham that is “right-to-try,” , one criticism (among many) that I made of these misguided, profoundly patient-unfriendly laws was that I have as yet been unable to find a single example of a patient who has managed to obtain access to an experimental therapeutic through such a law, much less been helped by it. So-called “right-to-try” laws, of course, claim to provide a mechanism by which patients with terminal illnesses can obtain access to experimental therapeutics not yet approved by the FDA but still in clinical trials. They are, as I’ve pointed out, a cruel sham, placebo legislation that makes lawmakers feel as though they’ve done something good but do nothing of substance for patients while providing them with false hope. The federal government through the FDA controls drug approval, which means that states can’t compel a drug company to provide a drug to a patient, and most drug companies would not want to risk jeopardizing approval of their drug, which is what could happen if they grant access to an investigational drug under right-to-try and the patient suffers an adverse event. After all, the success rate for drugs that have passed phase 1 (which is all that right-to-try requires) in phase 3 trials is only on the order of 9-12%, meaning that that’s the most optimistic probability that such drugs would benefit a patient. In reality, it’s almost certainly much, much lower.
Basically, the whole right-to-try movement is built on a delusion, namely that there are scads and scads of cures out there that are only being kept from the people by the cautious bureaucracy of the FDA. If the people could just get the FDA out of the way—or so the delusions go—cures would flow to the people. Add to that the libertarian delusion of the Goldwater Institute, upon whose model legislation nearly all of these bills and laws are based, that the free market will take care of safety issues, and it’s possible to see the long game being played, a strategy designed to drastically weaken the FDA’s control over drug approval. Then add to that the way that right-to-try legislation strips virtually every protection away from patients in clinical trials. For instance, no IRB oversight is required in right-to-try. There is nothing in the laws that provide for paying for the experimental therapy being used. Indeed, not only don’t insurance companies have to pay for right-to-try, but they don’t have to pay for any care that results from complications resulting from right-to-try. If a patient chooses right-to-try and it goes on, he’s completely on his own.
All of this brings me to a story I saw after I wrote last week’s post. The story is about an unfortunate man named Bob Bardone who last summer got one of the worst pieces of news a human being can get. He found out that he had amyotrophic lateral sclerosis (ALS), and it was estimated that he probably had about two or three years to live. His story is, as is the case for most ALS patients, heartbreaking. ALS, as many of you know, is a relentlessly degenerative neurologic disease in which the patient gradually loses motor control. Ultimately, patients lose the ability to walk, to move, to speak, and, ultimately, to breathe. Indeed, if there is a disease that I fear more than cancer, it’s something like ALS, which produces a nightmarish, unstoppable deterioration. There’s currently only one drug approved for ALS that I’m aware of, riluzole, but it only adds at best months to life expectancy. If ever there were a disease that needs new treatments, it’s ALS.
As you might recall, Missouri was one of the first states to pass right-to-try legislation. As such, and because of his past connection to the state, Mr. Bardone and his wife decided to move back there in order to be closer to family and, not coincidentally, to try to take advantage of right-to-try:
The move to St. Louis happened to bring them hope. They learned Missouri had become the third state to enact “right to try” legislation, which went into effect in August and allows terminally ill patients to try promising drugs yet to be approved by the FDA.
The family had been closely following news about a drug, GM604. Its maker reported that it significantly slowed the progression of ALS in 12 patients over 12 weeks with no dangerous side effects, though some scientists question the findings. The drugmaker, Genervon Biopharmaceuticals, is seeking fast-track approval from the FDA to sell the drug before spending years studying it in larger numbers.
“My dad had hope for the first time since diagnosed with ALS when he found out Missouri was a right-to-try state,” Connors said. “He thought that because he was moving back to Missouri, that he could try this drug.”
Bardone’s neurologist tried to get more information about the drug, and the family made emotional requests to the company. Genervon, however, refused to provide it.
After months of trying, the family is disappointed and frustrated, and Bob Bardone’s health is worsening.
“Can you imagine the hope we felt?” Connors said. “This entire process puts the most emotional time in our lives on even a more emotional roller coaster.”
The article further notes:
But right-to-try laws may be meaningless because companies can refuse to provide their experimental drugs. They may only create a false sense of hope among desperate families.
“It gives people a false impression that somehow because it’s a state law, it mandates this, but really it just gets everyone up in arms,” said Dr. John DiPersio, the deputy director of the Barnes-Jewish Hospital Siteman Cancer Center. “All it’s doing is causing commotion and confusion.”
This is, of course, exactly what I’ve been saying about these laws for over a year now. They are placebo legislation. They do absolutely nothing to help a man like Bob Bardone, but they do give patients like him false hope, only to cruelly take it away.
I can anticipate right now one objection. Why did Genervon Biopharmaceuticals refuse to give Mr. Bardone access to the drug? There are quite a few reasons why companies might not want to do this. This story, for instance, reports that Genervon has been overwhelmed by requests for its drug from ALS patients. Therefore, it chose to concentrate on getting fast track approval from the FDA, which “would give immediate access to all ALS patients, require doctors to prescribe and monitor patients’ progress, allow us to continue gathering data … and the cost would be supported by health insurance.” This is, of course, a perfectly reasonable justification, particularly given that if a patient had an adverse reaction to its drug outside of its clinical trials, something that is more likely if the drug is given outside of the controlled setting of a clinical trial, it could delay or even scuttle approval of the drug by the FDA. Also, some companies are small venture companies with inadequate resources to dedicate to fulfilling right-to-try requests. Indeed, some of these smaller companies might just barely have raised enough capital to make enough drug to do the clinical trials necessary to gain FDA approval. These are, of course, exactly the sorts of companies that we want to encourage, not impede, because they aren’t the big pharma behemoths and tend to be more imaginative and daring.
The Goldwater Institute’s response to these problems with right-to-try shows just how clueless or disingenuous it is. Basically its flacks argue that the FDA should assure companies that outcomes of patients who receive drugs for compassionate use will not negatively affect a drug’s approval. Seriously? That response basically indicates to me that the Goldwater Institute knows that right-to-try is a sham and has no power over the FDA, given that it’s reduced to suggesting that the FDA do something that it knows the FDA cannot do legally or ethically. Besides, the FDA already has a compassionate use program that rarely turns down a request. Also, as I’ve described, even the problem with how onerous the forms are for a physician to fill out is being dealt with. As this story notes, most applications are dealt with in four days, and the FDA has drafted a new form for such requests that, when finalized, should take a physician 45 minutes or less to complete.
There’s an excellent summary of the problems with right-to-try laws that was published on Friday on the Health Affairs Blog that I missed before but won’t now. It’s by David Farber, Preeya Noronha Pinto, Arthur Caplan, and Alison Bateman-House. They are quite blunt, noting that, contrary to the name given these laws, they provide no new rights to patients. They also point out that these laws have created an expectation that terminally ill patients will be able to quickly access life-saving experimental drugs by being exempted from FDA oversight but that that expectation is “quite simply, false.” They also note that legally:
If RTT laws are understood as actually providing a right to direct access to an investigational product without FDA approval or oversight, they would be “preempted” by federal law, meaning the federal laws are so powerful as to effectively nullify their state RTT counterparts. Indeed, several federal courts have already concluded that FDA’s comprehensive regulatory regime governing the manufacturing, approval, labeling, and distribution of drug products preempts state laws designed to legislate in this area. If challenged in court, we anticipate RTT laws will be similarly treated.
They also note:
It is only a matter of time before courts nullify RTT laws. In the meantime, these laws offer vulnerable patients misplaced hope and do not attempt to minimize the serious health and safety risks inherent in treatment with unproven treatments. State legislatures still evaluating the merits of RTT laws should think twice. Patients facing terminal illnesses have enough issues to navigate without being given false hope through legislation that is ethically, legally, and clinically flawed.
I don’t know if it’s definitely a matter of time before the courts nullify right-to-try laws, but that would seem to be a likely outcome. Of course, as I’ve discussed before, the purpose of right-to-try laws is not what is ostensibly claimed. Rather, it’s a long game in which—or so it is thought—the buildup of pressure through the states will force the FDA to loosen its grip on drug approval. Some of the more “open” libertarians will even argue for the neutering or even outright abolition of the FDA, claiming that the free market will take care of drug safety and efficacy.
I do find one thing hopeful. Over a year ago, when the very first of these laws were wending their way through state legislatures, few, if any, medical groups spoke out against them or openly opposed them. Indeed, as I described in my very own state, when our right-to-try bill came before the legislature, none of the major medical societies or cancer centers openly opposed it, and I was definitely discouraged from doing so. Again, as I’ve said so many times before, opposing right-to-try can easily be painted to be akin to opposing mom, apple pie, and the American flag and having a propensity to drop kick puppies through flaming goal posts. Never mind that right-to-try laws are profoundly anti-patient laws.
That time seems to be coming to an end. After a year, people are starting to realize that, as far as it goes to right-to-try, the emperor has no clothes. The story about Bob Bardone is one indication. I don’t recall seeing such a skeptical story about right-to-try in a long time, if ever, complete with a human interest story that demonstrates quite effectively what a sham these laws are. Then there’s this story about the right-to-try law introduced into the California legislature:
The California bills have drawn opposition from the California Medical Association and from groups representing nurses and oncologists. A group representing pharmaceutical manufacturers has sent a letter warning against skirting the FDA’s established channels.
“We have serious concerns with any approach to make investigational medicines available that seeks to bypass the oversight of the Food and Drug Administration and clinical trial process, which is not in the best interest of patients and public health,” a letter from Pharmaceutical Research and Manufacturers of America says.
The California Medical Association states that they have significant patient safety concerns with allowing access to unproven drugs outside of the FDA’s clinical trials and compassionate use programs. They further argue that offering unapproved therapies without credible scientific rationale or controlled monitoring could lead to not only endangering terminally ill patients further but potentially exploiting their hopes and circumstances. The Association of Northern California Oncologists cite the difficulty of identifying a terminally-ill patient, the danger a “right to try” policy outside the context of a clinical trial would present to adult clinical trial enrollment and the lack of an informed consent process to protect patients seeking investigational drugs as reasons for establishing their oppose position on the bill. The California Nurses Association/National Nurses United (CNA) maintain that this bill does nothing to address the real barriers to “compassionate use” revealed in a recent article in the NEJM entitled Practical, Legal, and Ethical Issues in Expanded Access to Investigational Drugs, specifically stating that nothing in this legislation impacts the availability of drugs to terminally ill patients unless the manufacturer of the drugs allow it to be used in advance of FDA approval. The CNA goes on to state instead of taking on the cost of drugs and challenging drug manufacturers that charge excessive prices for all drugs sold in California, this bill reinforces the status quo for investigational drug costs.
These are all good points. However, it’s probably too little, too late, given that the CMA refused to take a stand on this legislation initially. Indeed, there remain editorials trotting out the same Goldwater Institute talking points. Contrary to these dubious arguments, right-to-try laws sell terminally ill patients and legislators a bill of goods. they promise access to life-saving experimental drugs while leaving vulnerable, desperate patients open to false hope, lack of the oversight that patients in clinical trials receive, huge medical bills, and loss of medical coverage for any complications that might result from trying unproven drugs. People seem to be finally waking up to the sham that is right-to-try, but too late.
Alone of all the regular contributors to this blog, I am a surgeon. Specifically, I’m a surgical oncologist specializing in breast cancer surgery, which makes me one of those hyper-specialized docs that are sometimes mocked as not being “real” doctors. Of course, the road to my current practice and research focus was long and involved quite a few years doing general surgery; so it is not as though I am unfamiliar with a wide variety of surgical procedures. Heck, I’m sure I could do an old-fashioned appendectomy, bowel resection, or cholecystectomy if I had to. Just don’t ask me to use the da Vinci robot or, with the exception of the case of a cholecystectomy, a laparoscope, although, given the popularity of robotic surgery, I sometimes joke that I really, really need to figure out how to do breast surgery with the robot. After all, if plastic surgeons are using it for breast reconstruction, surely the cancer surgeon should get in on the action.
I have, however, from time to time addressed the issue of science-based surgery, and this weekend seems like as good a time to do so again, given that just last week the BMJ published a systematic review of the use of placebos in surgical trials. Before I get to discussing the nitty gritty of this particular trial, let me just note that the evaluation of surgical procedures for efficacy and safety tends to be more difficult to accomplish than it is for medications, mainly because it’s much harder to do the gold standard clinical trial for surgical procedures, the double-blind, placebo-controlled randomized clinical trial. The two most problematic aspects of designing such an RCT in surgery, as you might imagine, are the blinding, particularly if it’s a trial of a surgical procedure versus no surgical procedure, and persuading patients to agree. I’ll deal with the latter first, because I have direct personal experience with it.
Way, way back in the Stone Age of my career (around 15 years ago), there was a surgical trial going on known as the NSABP-B32 trial. This was a trial designed to test whether a procedure called sentinel lymph node (SLN) biopsy in breast cancer, which involves using dye (either radioactive or blue or both) to identify the lymph node(s) under the arm most likely to contain cancer. The idea was to determine if a patient had positive lymph nodes without doing the old standard of care, removing all the axillary lymph nodes, a procedure with a high risk of lymphedema (arm swelling due to interruption of lymphatic drainage) and other problems that made evaluating the lymph nodes more complication-prone than the surgical excision of the actual breast cancer. The problem was that we didn’t know how accurate SLN biopsy was or how often it misclassified a woman with node-positive disease as node-negative. So a trial was undertaken in which women were randomized either to (1) SLN biopsy followed by axillary lymph node dissection (removal of all the axillary lymph nodes to compare the results of the SLN biopsy) or (2) SLN biopsy alone, with an axillary lymph node dissection only if the SLN was positive for cancer. The results ultimately demonstrated that overall survival, disease-free survival, and regional control (control of the disease in the axilla) were the same in all groups, and SLN biopsy became the new standard of care in women with early stage, clinically node negative breast cancer, replacing axillary lymph node dissection.
Here’s the issue. Persuading women to be randomized was difficult. Very difficult. Most wanted the newer and less-invasive procedure, the SLN biopsy, because they feared the complication of lymph edema that axillary lymph node dissection produced. A few were risk-averse and not willing to be randomized if they might get just the “new” procedure (at the time, the SLN biopsy). And this was with a trial that had no placebo (i.e., “sham” surgery) arm and was not blinded. Of course, the B-32 trial didn’t need to be blinded because we were examining “hard” outcomes: overall survival (the “hardest” endpoint of all), disease-free survival, and recurrence in the axilla. What about trials that are not examining endpoints like this? I’ve discussed one such example before, vertebroplasty for vertebral fractures resulting from osteoporosis. Vertebroplasty, a widely used procedure to reduce pain from these fractures, basically involves injecting a glue into the fracture to stabilize it. Because it wasn’t a full surgical procedure and only involved injecting glue (or not injecting glue), the two vertebroplasty trials I discussed were relatively easy to blind compared to trials of more invasive surgical procedures, and the vertebroplasty trials ultimately showed that “real” vertebroplasty is no better than placebo vertebroplasty.
Then there are the ethical and practical issues as well. For instance, in the case of a surgical procedure involving more extensive rearrangement of anatomy (as I like to put it), a blinded trial would require that at least an incision be made on the patient identical to that of the surgery, but that nothing be done. This presents an ethical dilemma, since clinical trials rely on the principle of clinical equipoise (not knowing which treatment is better or whether the treatment being tested is better than no treatment) and beneficence, which means that risk must be minimized for clinical trial subjects. Indeed, clinical equipoise is often in conflict with scientific rigor in clinical trial design. Also, from a practical standpoint, in a placebo-controlled surgical trial, the surgeons would know which subjects were in each group, which leads to the question of whether the doctors taking care of the patient postoperatively would have to e different than the ones doing the surgery. Still, as the example of the vertebroplasty trials shows, for surgical procedures designed to relieve pain or other symptoms with subjective components, it’s important to include placebo controls in clinical trials.
This latest systematic review just emphasizes this even more. This is particularly true given how prone surgeons are to adopt new procedures before they have been adequately tested, as the example of laparoscopic cholecystectomy demonstrated in the late 1980s and early 1990s and the widespread adoption of robotic surgery for various conditions does today. (I’m sure that by saying that I’ll tick off a surgeon who swears by robotic surgery, but I don’t care.) Unfortunately, placebo-controlled trials of surgery for such conditions remain relatively rare.The need for more placebo-controlled trials of surgery
The systematic review that spurred this post was conducted by a group that included mainly researchers from the UK (University of Oxford, the University of Southhampton, and the National Institute of Health Research), but also a researcher from the Harvard Stem Cell Institute. The authors first note:
In considering any scientific evaluation, it is important to remember that an outcome of a surgical treatment is a cumulative effect of the three main elements: critical surgical element, placebo effects, and non-specific effects. The critical or crucial surgical element is the component of the surgical procedure that is believed to provide the therapeutic effect and is distinct from aspects of the procedures that are diagnostic or required to access the disease being treated.8 The placebo effects are related to the patients’ expectation and the “meaning of surgery,” whereas the non-specific effects are caused by fluctuations in symptoms, the clinical course of the disease, regression to the mean, report bias, and consequences of taking part in the trial, including interaction with the surgeons, nurses, and medical staff. It is reasonable to assume that surgery is associated with a placebo effect. Firstly, because invasive procedures have a stronger placebo effect than non-invasive ones and, secondly, because a confident diagnosis and a decisive approach to treatment, typical for surgery, usually results in a strong placebo effect.
I’ve discussed the necessity of a surgeon projecting a “decisive” and confident air to patients before and the elements of placebo effects. In any case, it does appear to be true that the more invasive the treatment, the stronger the placebo effects, which would appear to make consideration of these nonspecific effects even more important in surgery than in medical interventions. Indeed, given that surgical procedures and associated devices tend not to be as strictly regulated as drugs (in fact, surgical procedures themselves are scarcely regulated at all other than by the professional societies encompassing the relevant surgical specialties), rigorous surgical trials are critical, but unfortunately they are uncommon, as noted by the authors:
Placebo controlled randomised clinical trials of surgical interventions are relatively uncommon. Studies published so far have often led to fierce debates on the ethics, feasibility, and role of placebo in surgery. One reason for the poor uptake is that many surgeons, as well as ethicists, have voiced concerns about the safety of patients in the placebo group. Many of the concerns are based on personal opinion, with little supporting evidence. In the absence of any comprehensive information on the use of placebo controls in surgery, and the lack of evidence for harm or benefit of incorporating a placebo intervention, a systematic review of placebo use in surgical trials is warranted.
So the authors performed their systematic review. They defined surgical trials as trials testing an interventional procedure that changes the anatomy and requires a skin incision or use of endoscopic techniques. Placebo was defined as a sham surgery or imitation procedure intended to mimic the active intervention in a manner such that the patient could not tell the difference, which generally required that the patients be sedated or under general anesthesia. Examples include inserting an endoscope but doing nothing or making a skin incision. Relevant studies were identified through a search of Medline, Embase, and the Cochrane Central Register of Controlled Trials, followed by independent screening by three of the investigators for the eligibility of each trial to be included. The references were also screened to identify any recently completed or ongoing studies. Duplicates were handled by only examining the publication that reported the main outcome for the trial.
Here is the summary for the selection process:
Overall, 39 of the 53 (74%) included studies were published after 2000. One aspect of these trials is that most of them investigated relatively minor conditions that were not directly life-threatening, such as gastro-esophageal reflux (n=6; 11%). Indeed, nearly half (43%) of the included studies were trials involving endoscopy as part of the investigated procedure. Thirteen trials (25%) used some exogenous material, implant, or tissue, and a further six used balloons. Also:
Most studies reported subjective outcomes such as pain (n=13; 25%), improvement in symptoms or function (n=17; 32%), or quality of life (n=8; 15%). Less than half of the trials (n=22; 42%) reported an objective primary outcome—that is, measures that did not depend on judgment of patients or assessors. The majority of trials were small; the number of randomised participants ranged between 10 and 298, with a median of 60. No placebo controlled surgical trials investigating more invasive surgical procedures such as laparotomy, thoracotomy, craniotomy, or extensive tissue dissection were identified.
So right there we see a bias in the study towards less invasive procedures. This is not surprising, given that it is likely less ethically problematic to propose, for instance, sticking an endoscope into the rectum or stomach of a subject in the placebo-control group and looking around while doing nothing than it is to propose doing a non-therapeutic thoracotomy or laparotomy on a control subject. Be that as it may, it is useful to point out here one of the most famous surgical trials of all that utilized sham surgery, arguably the first that ever did, a randomized clinical trial of internal mammary artery ligation for angina performed in the 1950s. Beginning in the 1930s and continuing until this study, angina pectoris was sometimes treated with a surgical procedure known as mammary artery ligation. The idea was that tying off these arteries would divert more blood to the heart. The operation became popular on the basis of relatively small, uncontrolled case series. Then, two randomized, sham surgery-controlled clinical trials were published in 1959 and 1960. Both of these trials showed no difference between bilateral internal mammary artery ligation and sham surgery. Very rapidly, surgeons stopped doing this operation. More recently, a randomized, double-blind, sham-controlled trial of arthroscopic partial meniscectomy showed the same thing: Arthroscopic repair for a degenerative tear of the medial meniscus produced results indistinguishable from sham surgery. Only time will tell whether this procedure is abandoned.
So what did this review find? First, as one would expect, the authors noted that in 39 of 53 (74%) of the trials, there was improvement in the placebo arm. No surprise there. In 27 (51%) of the trials, there was no difference between the sham surgery and real surgery arms. In the remaining 26 (49%) of the trials, the authors reported that “surgery was superior to placebo but the magnitude of the effect of the surgical intervention over that of the placebo was generally small.” Overall, in 43% of the trials, the authors stated that there were no serious adverse events, although one trial didn’t report anything about adverse events. In the remaining 27 (51%) of trials, serious adverse events occurred in at least one study arm. They also noted that serious adverse events were reported in the placebo arm in 18 trials (34%) but that in many of these studies the adverse events were associated with the severity of the condition rather than the intervention itself, such as rebleeding after an attempt to stop bleeding using endoscopy, which happened in both treatment and placebo groups.
One key weakness of the study is that the chosen existing studies did not allow assessment of the magnitude of placebo effects, as only one study included an observational, “no treatment” group. The authors specifically did not include studies with wait list controls, for reasons that I find unclear. The authors state that “analysis of the placebo effect without controlling for non-specific effects of being in the trial would be flawed,” but that could be said about any analysis seeking to estimate the magnitude of placebo effects. You go with what you have, and for some reason these investigators chose not to, citing two Ted Kaptchuk papers as justification. (Yes, this Ted Kaptchuk.) I wish they had included such trials, even if the analysis ended up being somewhat “flawed.”
The authors note, as I have noted before, that the record of how physicians react to negative RCTs is mixed. (I myself have discussed this issue before.) This is how they put it:
The results of the placebo controlled surgical trials performed so far have had a varied impact on clinical care. With the exception of the trials on debridement for osteoarthritis or internal mammary artery ligation for angina, most of the trials did not result in a major change in practice. Moseley’s study on debridement for knee osteoarthritis was well received and resulted in limiting the recommendations for debridement and lavage for osteoarthritis of the knee to patients only with clear mechanical symptoms such as locking.82 The reaction of the medical community to the trials on tissue transplantation to treat Parkinson’s disease was also favourable. Although this treatment is not currently recommended, the need for more studies on mechanisms of disease and on tissue transplantation is recognised. These studies also provoked a discussion about ethical aspects of randomised clinical trials and placebo.
In contrast, the results of the trials on the efficacy of vertebroplasty were challenged and their authors were criticised for undermining the evidence supporting this commonly used procedure. The critics acknowledged that the injection of cement might be associated with many side effects, some of them potentially dangerous, but they argued that the treatment was justified because earlier unblinded trials had shown superiority of vertebroplasty over medical treatment. This argument against the validity of placebo controlled trials neglected any potential placebo effect of vertebroplasty.
It’s true, too. Just last fall, I noted that, five years after the resoundingly negative randomized, double-blinded clinical trials of vertebroplasty for osteoporotic vertebral fractures and nearly ten years after earlier trials suggesting that vertebroplasty does no better than placebo for such fractures, those with a vested interest in performing vertebroplasty continue to attack the studies their investigators. Indeed, I pointed out that, even though use of this procedure has decreased in the wake of negative RCTs and ultimately recommendations against it by the American Academy of Orthopaedic Surgeons, it hasn’t gone away by any means. I also pointed out how arguments for the procedure by certain radiologists were scientifically dubious and uncomfortably close to the sorts of arguments used by practitioners of “complementary and alternative medicine” who justify use of their modalities because they provoke placebo effects.
The authors conclude with a powerful argument in favor of more sham-controlled randomized trials of surgical procedures, arguing that they are as important in surgery as they are in medicine:
Placebo controlled trials in surgery are as important as they are in medicine, and they are justified in the same way. They are powerful, feasible way of showing the efficacy of surgical procedures. They are necessary to protect the welfare of present and future patients as well as to conduct proper cost effectiveness analyses. Only then may publicly funded surgical interventions be distributed fairly and justly. Without such studies ineffective treatment may continue unchallenged.
As surgery is inherently associated with some risk, it is important that the surgical treatment is truly effective and that the benefits outweigh the risks. Placebo controlled surgical trials are not free from adverse events but risks are generally minimal in well designed trials and the control arm is much safer than the active treatment. However, this review highlighted the need for better reporting of trials, including serious adverse events and their relation to a particular element of surgical procedure.
A need exists to “demystify” and extend the use of the surgical placebo in clinical trials. These should result in a greater acceptance of this type of trial by the surgical community, ethics committees, funding bodies, and patients. In turn, this would lead to more studies, better guidelines on the design and reporting of studies, and a larger body of evidence about efficacy and the risks of surgical interventions. Placebo controlled surgical trials are highly informative and should be considered for selected procedures.
There is no doubt that carrying out randomized, sham-controlled clinical trials of surgical procedures presents challenges greater than the already formidable challenges of doing such trials in medicine. Neither is there any doubt that there will be a lot of procedures and interventions for which carrying out a sham-controlled RCT will be problematic from a standpoint of ethics and/or practicality. For these interventions, we will continue to have to rely on a combination of unblinded randomized trials and/or observational data. However, this review presents a conclusion that should not be a surprise to any surgeon: That there are a lot of procedures out there for which the evidence base is weak and that might not even do better than sham surgery. The only way to correct that is to carry out sham-controlled blinded randomized clinical trials whenever there is doubt for as many surgical procedures as is feasible. Then we surgeons have to pay attention to the results and act upon them.
And there are quite a few surgical interventions that could use this approach.
Yesterday I wrote about our struggle to promote and defend the teaching of evolution, and good science in general, in the public school science classroom. My overall point was that, while we are winning on the legal battleground, we are not making much headway in the broader cultural context, and perhaps we need to step back and think about our strategy.
To my delight, Michael Egnor made an appearance in the comments, and it seemed he truly wanted to engage (at least for a while). Dr. Egnor, if you recall, is a neurosurgeon who rejects what he calls “Darwinism.” He blogs on his own blog and for the Discovery Institute, and we have occasionally crossed swords on our respective blogs.
I was also pleased that the conversation remained polite and civil, allowing us to drill down to the core issues. I want to summarize our exchange here and expand on my responses in the comments.
Dr. Egnor summarized his opinion of evolution as:
I use Darwinism because I don’t disagree with evolution, understood as a natural process by which populations change with time. I see the process as teleological–secondary causation in theological jargon.
So I make it a point to clarify just what it is that I disagree with–I disagree that evolution is adequately explained by random heritable variation and natural selection. RHV+NS is most succinctly written as “Darwinism”.
He is upset that this view is not getting a fair hearing in the scientific community, and he feels scientists are trying to censor debate using the courts. He writes:
As for curriculum in public schools, I support the normal channels for curricular development. Teachers, school boards, state education officials etc properly make curricular decisions, with input from parents, scientists, scholars of various sorts, etc. But federal courtrooms should not be a part of curricular development. If the folks in Augusta Georgia want to teach the kids about ID, and the folks in Cambridge Mass don’t, both are fine.
He further argues:
I don’t believe that belief in God can be taught as fact in school from a Constitutional standpoint, because that would be an establishment of religion. Same goes for teaching disbelief in God. So teaching creationism–meaning the view that the God of the Bible created heaven and earth–in public schools is unconstitutional. Also, teaching that atheism is true is unconstitutional for the same reason.
However, teaching students various critiques of evolutionary theory, including the critique that Darwinian evolutionary theory neglects the evidence for design or teleology in biology–is perfectly constitutional, and is good science education. It is an obvious inference, and even if you believe that it is not scientifically true, it is good education for students to be acquainted with it and to understand the scientific arguments advanced for and against it.
On the bright side, we have some common ground here. We both agree that the teaching of religion in the public schools is unconstitutional. I also largely agree with his summary of how school curricula should be determined (the devil, of course, is in the details of implementation, but the broad brushstrokes are fine). I specifically agree that the courtroom is not the venue for hammering out school curricula.
I further agree with the premise that science should be open, that it depends greatly on open debate, and that ideas should not be censored. I have specifically in the past expressed my support for minority opinions in science, even ones with which I disagree. They serve a vital purpose of shaking things up, keeping the process honest, and forcing scientists to continuously challenge their ideas and be forced to defend them. It’s all good.
I have also specifically endorsed teaching science by (ironically) “teaching the controversy.” That is essentially what skeptical outreach is. We teach science and critical thinking by confronting pseudoscience head on. I think it is a great way to learn science – how do we actually know what we know, and how do we deal with competing ideas? How has our thinking changed over historical time? Why are the ideas that have been rejected by science probably wrong?
So where is the disconnect? As is often the case, the devil is in the details.
First, and most obviously, we disagree about the science of evolution. That actually was not the focus of the discussion. It was taken as a given that we disagree. For the record, I think that the evidence strongly supports the conclusion evolution occurred, that natural selection acting upon random mutations is a major (not the only) driving force in evolution, and that there is no evidence for nor plausible mechanism for teleology or direction in evolution. This is the scientific consensus. I do not expect my personal opinions to be taught in the public science classroom, but I do expect an accurate reflection of the current scientific consensus.
I will also point out that this battle was fought within the scientific community in the 19th century. Lamarck, for example, started out believing in direction in evolution, but after a career of looking at the fossil evidence he changed his mind, and felt that evolution was directionless and involved simply adapting to the local environment. Teleology had its hearing in the arena of science, and it lost. Creationists (I’m using that term in the broadest sense, for convenience) want to refight that battle over and over again, endlessly. That’s fine; knock yourself out. But don’t expect time in the classroom.
The notion that there is evidence for design has also been examined by scientists, and utterly refuted. There was no censorship. The idea, the arguments, and the evidence were given the hearing they deserve, and found to be utterly wanting. The “design inference” is getting no traction among scientists because their arguments are terrible. Of course, proponents won’t see it that way.
Let’s get to the core issue – the use of courts to determine what does and does not get taught in the public school science classroom. Dr. Egnor’s main argument is that scientists are using the courts to silence the teaching of criticisms of evolution. He cites academic freedom, teaching the controversy, the debate is never over, and science is never settled – all the current standard arguments that are being used to force creationist ideas into the classroom.
He is demonstrably wrong, however. Let me quickly review the history here to show that scientists have not used the courts to determine the teaching of science.
The confrontation between the teaching of evolution vs creationism began with a ban on the teaching of evolution (remember the Scopes “Monkey” trial?). That was censorship – an outright ban on teaching an accepted scientific theory. Laws banning evolution were deemed unconstitutional in 1968 in Epperson vs Arkansas.
Creationists then moved on to their “creation science” strategy, requiring “equal time” for the teaching of “creation science” along science “evolution science.” This strategy was struck down in 1982, in McLean v. Arkansas Board of Education. Other cases supported this ruling, specifically that schools cannot require the teaching of creation science or prohibit the teaching of evolution, and that this does not violate a teacher’s free speech or free exercise of religion.
In 1997 the Supreme Court struck down a law requiring a disclaimer be read aloud to student prior to teaching evolution. The court ruled that by singling out evolution they were giving special treatment to creation, which is a religious belief. Further, such a disclaimer was not necessary to promote critical thinking, which is already a part of science education.
In 2000, in Rodney LeVake v Independent School District 656, et al., the court ruled that a teacher could be prohibited from teaching criticisms of evolution in the context that they went against the established curriculum. It was established that a teacher does not have the right to personally override the official curriculum.
And then finally in 2005, the Kitzmiller vs Dover case regarding Intelligent design. The NCSE give a good summary of the findings:
In his 139-page ruling, Judge Jones wrote it was “abundantly clear that the Board’s ID Policy violates the Establishment Clause”. Furthermore, Judge Jones ruled that “ID cannot uncouple itself from its creationist, and thus religious, antecedents”. In reference to whether Intelligent Design is science Judge Jones wrote ID “is not science and cannot be adjudged a valid, accepted scientific theory as it has failed to publish in peer-reviewed journals, engage in research and testing, and gain acceptance in the scientific community”.
Judge Jones, I will note, is a conservative Bush-appointed judge.
Dr. Egnor characterizes this legal history as scientists censoring criticisms of evolutionary theory. This conclusion is demonstrably false. In each case, attempts to teach creationism or hamper the teaching of evolution were clearly demonstrated to be unconstitutional on the grounds that they violated the establishment clause in the First Amendment. Jones’ ruling made that very clear – ID is not science, the connection between ID and creationist religion is absolutely clear, and that is the reason it is not proper for the public school science classroom.
There is no way you can legitimately get from this history that scientists are using the courtroom to censor debate. That exact claim, however, is the current talking point of the creationist political movement (which Dr. Egnor reflects well).
To be clear, it is creationists who have consistently over the last century tried to censor a scientific idea because it conflicts with their faith. They have lost this fight in the arena of science, soundly, over and over again. This is not a scientific controversy.
Evolution deniers, like other deniers, will try to criticize this conclusion by criticizing the notion of “settled science.” Their appeals to openness and critical thinking sound superficially appealing, but they are applying them in an incorrect and biased fashion. Nothing in science is ever 100%, but the evidence can reach a point where there is such a solid consensus about a particular conclusion in science that we can treat it as an established fact. DNA is the primary molecule of heritable information. That fact is “settled science” and we don’t have to waste time and resources answering that question over and over again. We don’t have to confuse students by teaching any false controversies over whether or not DNA is really the mechanism of inherited traits.
There is a long list of scientific theories that are established facts, and it is not censorship to teach them as facts.
What the scientific community opposes is the teaching of religious ideas in the science classroom. You can disagree with the court rulings, but the fact is that thorough and elaborate cases were put on by both sides in each of the cases cited above. The creationists and ID proponents have literally had their day in court. The outcome, in every case, was not even close. It was clearly established that what creationists are doing is not promoting critical thinking or teaching scientific controversies, but rather teaching their own religious beliefs in place of science.
They can try to dress it up any way they want – but in a controlled setting where there are rules of logic and evidence, and there is adequate time to thoroughly review the evidence and arguments, it has been absolutely clear that this has all been one long attempt to ban a scientific theory that conflicts with a religious belief.
Tree of Life – the first-known sketch by Charles Darwin of an evolutionary tree describing the relationships among groups of organisms. Cambridge University Library
The Merriam-Webster Dictionary defines science as:
– Knowledge about or study of the natural world based on facts learned through experiments and observation.
– Knowledge as distinguished from ignorance or misunderstanding.
While this should distinguish science from pseudoscience, those who practice the latter often lay claim to the same definition. But one of the major differences between science and pseudoscience is that science advances through constant rejection and revision of prior models and hypotheses as new evidence is produced; it evolves. This is the antithesis of pseudoscience. At the heart of pseudoscience-based medicine (PBM) is dogma and belief. It clings to its preconceptions and never changes in order to improve. It thrives on the intransigence of its belief system, and rejects threats to its dogma. Despite the constant claims by peddlers of pseudoscience that SBM practitioners are closed-minded, we know that, in fact, PBM is the ultimate in closed-minded belief. Of course, those of us who claim to practice SBM aren’t always quick to adopt new evidence. We sometimes continue practices that may once have been the standard of care but are no longer supported by the best available evidence, or perhaps may even be contradicted by the latest evidence. Often this is a byproduct of habituated practice and a failure to keep current with the literature. While this is certainly a failure of modern medicine, it is not an inevitable outcome. It is not emblematic of the practice of medicine, as it is with PBM. When medicine is science-based, it strives for continual improvement based on modifications around emerging evidence.
I would like to highlight three recent examples of how science-based pediatric medicine has made dramatic changes based on emerging evidence. One interesting aspect of these particular changes to best-practice recommendations is that they all lead to a more conservative approach to care; that is, they lead to less or even no treatment compared to prior recommendations. This is yet another significant distinction between SBM and PBM; PBM never does nothing. It always claims to have a treatment or a cure. It is another hallmark of PBM, and often what draws people to it. It is an endless fount of hope when “traditional” medicine has nothing much to offer.
Before I discuss these examples, I would like to diverge briefly to illustrate the hierarchy of evidence we use to come to evidence-based decisions or recommendations. I do this because I will be referring below to guidelines and recommendations published by academic organizations. As can be seen on the pyramid of evidence below, “expert opinion” is located at the bottom due to its high susceptibility to bias. It is after all, just an opinion. Published guidelines and consensus statements vary greatly in their location on the pyramid of evidence. They can also be highly susceptible to the bias and opinions of the committee or panel members who author them. Even then, when good studies are few and far between, these forms of expert opinion may be the best evidence we have to go on. Fortunately, many professional medical organizations, such as the American Academy of Pediatrics (AAP), use an EBM approach to their guidelines and recommendations that is stricter and more transparent than they used to be, and include standardized documentation of the levels of evidence used and the strengths of recommendations made, as I alluded to above. So while these guidelines and recommendations are not meant to be used in cook-book fashion and applied absolutely to all cases, they can be quite helpful in expanding our understanding of issue that may be quite complex.
Ear infections – primum non nocere
A middle ear infection (acute otitis media) is one of the most common, treatable conditions seen by pediatricians. Not long ago, the standard of care for any child diagnosed with an ear infection would be treatment with antibiotics. In 2013, a growing, cumulative body of evidence led the American Academy of Pediatrics (AAP) to revise the evidence-based clinical practice guideline it had previously published in 2009. For the first time in North America, the AAP presented pediatricians with good evidence to support an “observation option” with no antibiotic treatment for selected children with confirmed ear infections. The select groups for whom this option applies are children 6 months or older with one-sided ear infections and children 2 years and older with one or two sided ear infections provided, they do not have severe signs or symptoms (ie. they have mild ear pain and fever < 102.2 F). It also applies only to children without health problems that might predispose them to more serious infection. The Guideline states (the action statement for the older group is given here as an example):
The clinician should either prescribe antibiotic therapy or offer observation with close follow-up based on joint decision-making with the parent(s)/caregiver for AOM (bilateral or unilateral) in children 24 months or older without severe signs or symptoms (ie, mild otalgia for less than 48 hours, temperature less than 39°C [102.2°F]). When observation is used, a mechanism must be in place to ensure follow-up and begin antibiotic therapy if the child worsens or fails to improve within 48 to 72 hours of onset of symptoms. (Evidence Quality: Grade B, Rec Strength: Recommendation).
>> Note the statement in parentheses above. All EBM recommendations and guidelines should have a standardized framework with which to evaluate the level of evidence used and the strength of the recommendation made.
The addition of this “watchful-waiting” option was based on several new, well-conducted placebo-controlled RCTs, as well as systematic reviews of the timing of antibiotic therapy and the outcome of untreated ear infections. What these studies demonstrated is that the vast majority of ear infections are self-limited, resolving spontaneously without complications whether treated or not. In other words, “primum non nocere”, the oft-cited tenet of good medical practice to “first, do no harm.”
Though I strive to practice EBM/SBM, I can tell you it isn’t always easy doing nothing. While some parents are grateful for my explanation of the watchful-waiting approach and for sparing their children from potentially unnecessary antibiotics, others find it difficult to understand this approach or are simply not comfortable allowing their children to remain untreated. For these parents treatment may be the preferred course, but at least I have been honest in my explanation of the SBM evidence for and against the different approaches. Widespread use of these EBM guidelines will result in fewer antibiotic-related complications and fewer antibiotic-resistant bacterial infections.
Bronchiolitis – Don’t just stand there, do nothing!
Bronchiolitis is a common respiratory tract illness in infants and young children. Caused by several different viruses, the illness begins like any cold with runny nose, low-grade fever, and cough. Inflammation and spasm of the small airways of the lung produce worsening cough, wheezing, and sometimes difficulty breathing. Bronchiolitis can be relatively minor, similar to a typical cold, or more severe, requiring supplemental oxygen and even hospitalization. Because the illness typically occurs in infants and children under 2 years of age, it can be alarming to parents and even to pediatricians. And because wheezing (a result of swelling and spasm of the medium and small airways) is a common component of the illness, as it is in asthma, an asthma mindset to evaluation and treatment has been the standard of care for these children. Cumulative and emerging evidence, however, has resulted in a new AAP practice guideline that should greatly simplify our approach to this common illness. The new guideline, published in October 2014, includes significant changes to the previous guideline published in 2006. It recommends eliminating evaluative and management strategies that have long been the mainstay of treatment. Recommended for the scrap heap are:
These procedures were removed from the recommendations because the best available evidence shows no improvement in outcome with the use of any of them. In short, despite the sometimes serious nature of this illness and the distress it can cause to both children and their parents, there is little to nothing science-based medicine can do to treat it. This can make for some anxious and difficult moments in the exam room or the emergency department when trying to reassure parents and help them through this illness. The understandably strong parental need to do something is often not satisfactorily addressed by our rationale for doing nothing. Yet honest and ethical practice dictates that the use of compassion, reassurance, and support is all we can offer much of the time. Again, this is something no peddler of quackery or sCAM treatments will understand, because there is always something in their bag of tricks, even though tricks are all they are.
Introducing food to babies – the changing tides of advice
When I was a pediatric resident in training, there was a decidedly non-evidenced based dogma to the introduction of solids in infancy. I was taught to introduce iron-fortified cereal at 4 months, followed by the slow introduction of foods between 4 and 6 months of age as follows: vegetables (preferably yellow and orange, then green), then fruits, then a mixture of vegetables and fruits, then meats. Peanut and tree nut proteins, as well as egg, milk, and of course honey were to be strictly avoided until at least 1 year of age. It turns out that, except for the honey, which can lead to botulism in infants, none of this is based in good science.
While I have no intention of covering the topic of peanut allergy here, recent studies exploring the effect of introducing peanut to the diet at different ages has the potential to change the way we think about the introduction of this, and other foods, to infants. It was previously felt that the early introduction of foods could predispose children to the development of allergies, eczema, and asthma. In 2003, the AAP recommended delaying cow’s milk until 1 year of age, egg until 2 years of age, and peanut, tree nut, and fish until 3 years of age. In 2008, they withdrew specific recommendations on the timing of food introduction, recognizing the lack of good supporting evidence for any particular recommendation. Since then, evidence has emerged that suggests earlier dietary introduction of peanut may actually decrease the risk of developing peanut allergy. This observational study, published in 2008, showed that young, Israeli infants who were fed a popular soft peanut-puff as a first food snack had a significantly lower prevalence of peanut allergy than a cohort of children of similar heritage being raised in the UK, where peanut protein is not typically given until after 1 year of age. This observation was followed by a larger, prospective study known as the LEAP study (Learning Early About Peanut Allergy). This study was designed to look specifically at the effect of introducing peanut during early infancy compared to waiting until later. The results of the LEAP study were published in March of this year in the New England Journal of Medicine. The study demonstrated an 81% reduction of peanut allergy in children given peanut-containing foods early in infancy, and who had continuous consumption of peanut, compared to infants who avoided peanut. While this is an important study with potentially significant impact, it is also important to understand that the infants studied were part of a group already at high risk for developing allergy due to having either documented eczema or egg allergy. It isn’t clear that these results will be generalizable to a broader population at average risk, though it is reasonable to hypothesize that children with less of a risk might respond at least as well. In any case, given the lack of evidence for delaying peanut, the results of these studies will likely cause many pediatricians to change the way they counsel parents about the introduction of foods to their babies. Still, without additional data, specific answers to questions such has how much peanut to give, at what age, and for how long will be allusive until further studies are conducted and more data is available. As with any intriguing piece of data, it is always only one piece of a much larger, more complicated puzzle. While the emerging data is fascinating, it is too early to know the answer to these questions just yet. And that is science.
My intended goal for providing the above examples of medicine-in-evolution is just that; to show that science-based medicine evolves. It rejects and tosses out old, inadequately, or incorrectly supported ideas and replaces them with newer ones supported by more or better evidence. It grows and improves. It admits its errors and strives to better itself. When done right, it constantly fights against bias and preconception, and strives, ultimately, to reach toward the truth. This is the essence of science, and distinguishes it from belief and quackery.
If science-based medicine is unaffordable, then your care won’t be science-based. Prescription drug costs are one of the biggest concerns in health care today. There seems to be no upper limit on prices, with some new treatments costing over $1,000 per day. The arrival of new drugs to treat (and cure) hepatitis C has created a perfect pharmaceutical storm: highly effective treatments, a large population of potential patients, and huge per-patient costs. It’s renewing the debate about whether important medical treatments are being priced out of the reach of the patients that need them. It’s not just hepatitis. Cancer drug costs are rising as well, driven by more patients and new drugs that in some cases are transforming our expectations about what cancer drugs can do. And while many of us rely on some form of drug insurance to protect us from high drug costs, insurers are struggling with balancing coverage and premiums: A report by Express Scripts paints a grim picture:
An estimated 576,000 Americans spent more than the median household income on prescription medications in 2014. This population of patients grew an astounding 63% from 2013. Further, the population of patients with costs of $100,000 or more nearly tripled during the same time period, to nearly 140,000 people. The total cost impact to payers from both patient populations is an unsustainable $52 billion a year.
This isn’t just an issue in the United States. Prescription drug costs are climbing around the world, because we’re effectively all in this together: We all rely on private companies to bring new drugs to market, and we’re largely buying the same drugs from the same small group of companies. Because ready access to safe and effective prescription drugs is so important to the practice of medicine and the delivery of health care, the pharmaceutical industry is heavily regulated – not just by the FDA, but by regulators worldwide. Yet despite the dual requirements of regulatory disclosure and the financial obligation to be transparent (as many pharmaceutical companies are publicly-held), little is known about how much it costs to bring drugs to market, and how manufacturers arrive at their selling prices. Pharmaceutical manufacturers claim that high drug costs reflect the high costs of research and development (R&D), and provide the incentives for companies to invest heavily and take risks, when many drugs may never make it to market. Are they correct?
I’ve written before about different attempts to estimate how much it costs to develop a drug. This is obvious a point of contention, with pharmaceutical companies interested in profiling the risk and significant investments they make, while using high prices as the justification to support continued risky R&D. Detractors and critics believe that these estimates are overstated by industry. The most well-known analysis of drug development costs is by Joe DiMasi at the Tufts Center for the Study of Drug Development. DiMasi collected R&D data for a basket of drugs that were volunteered by pharmaceutical companies, in order to come up with average drug development estimates. His original calculation was $802 million per drug, back in 2003. Today his updated estimate is $2.6 billion per drug. But there have been multiple critiques of DiMasi’s methodology and estimates over the years. Even Andrew Witty, the head of GlaxoSmithKine, one of the largest pharmaceutical companies, has called the $1 billion price tag “a myth” of industry. A critique of the latest DiMasi cost estimate appeared the New England Journal of Medicine earlier this month. In a commentary, Jerry Avorn made the following observations:
Avorn highlights that not all costs are climbing: FDA approval times are shortening (compared to prior analyses), reducing the overall estimated costs of development. Given the collective costs of the “failures” are higher than the costs of the successful drugs, the DiMasi analysis suggests that it is the development process itself, rather than regulation, that may be driving the pricing of drugs. Avorn concludes with a call for more transparency on development costs as a first step in a dialogue on how to support research, while appropriately rewarding innovation. Some American state legislators aren’t waiting for industry reaction – and they’re proposing legislation to force the discussion.Forcing R&D cost disclosure
Responding to repeated calls for transparency, and struggling with their own prescription drug cost bills, some states are looking to legislation to help manage prescription drug costs, perhaps by publicly pressuring manufacturers into reconsidering their pricing. The latest strategy was announced recently by New York Senator Ruben Diaz:
New York is the latest state to introduce a “pharmaceutical cost transparency act,” following five others—California, Oregon, Massachusetts, North Carolina, and Pennsylvania. Like the bills before it, New York would require pharmaceutical manufacturers to submit a report to the state outlining the total costs of the production of certain expensive drugs, with the information to be published on a public website.
The bill would force pharmaceutical manufacturer to disclose specific information for a drug that costs more than $10,000 per year – a detailed disclosure of production and sales data:
While this sounds like it might be an attractive approach to forcing transparency, it ignores a very real cost that the DiMasi analysis acknowledges: Most new drugs don’t succeed, and companies depend on the “hits” to stay profitable. Looking only at the cost of the “hits” ignore much of the real cost of drug development. Hundreds of millions of dollars, if not billions, have been invested in studying drugs to treat Alzheimer’s disease with strikeout after strikeout. Pfizer had invested about $800 million in torcetrapib when it discovered the drug was killing patients instead of preventing cardiovascular disease. The road to blockbuster drugs, like the new hepatitis treatments, is paved in part by well-documented, very costly, pharmaceutical development failures.
Even if we acknowledge (and I think we must) that companies have to cover the costs of their failures as well as their hits, is there any value in forcing disclosure about the hits alone? This approach is almost certainly going to underestimate the “real” cost of development, as not every cost can be traced to a specific drug. DiMasi estimated the costs of some of the earliest development stages of drug development, recognizing that you can’t attribute costs to any specific chemical. How else can you allocate the costs of pre-clinical research that may generate dozens or even hundreds of possible drug candidates? The development stage includes screening promising chemicals, validating them, and then modifying the chemical structure repeatedly to find the compound that represents the best possible candidate for future study. Only a fraction of drugs ever proceed beyond this step. Derek Lowe goes over this in much more detail in one of his old posts – he’s in the business of drug development, and knows of what he speaks. This is a tedious, time-consuming, often unproductive process.
There are also the ongoing costs of making a drug available – the costs of meeting regulatory requirements, and the overhead cost of running a business with tens of thousands of employees. The legislation ignores all of those costs as well, focusing only on that can be directly linked to a chemical. If this type of legislation succeeds, while it may be more transparent (in part) than the DiMasi paper, it will be no more helpful in helping us understand pharmaceutical R&D costs and pricing.Another approach to estimating costs
Matthew Herper writing in Forbes in 2013, took a crude yet simple approach to estimating costs. He added up the R&D expenses disclosed by pharmaceutical companies on their financial statements and divided it by the number of drugs produced over time. What he concluded was that some companies were spending astonishing amounts to produce new drugs. Abbott (now called AbbVie) was spending $13 billion per drug (and device) produced. Sanofi appeared to be spending $10 billion per drug. And so on, down to small companies which appear to be bringing new products to market for well under $100 million. Herper noted that 66 companies launched only one drug in decade, with a median cost of $350 million for that single product. Herper’s analysis adds some credibility to the high estimates from Tufts, and only seems to suggest how unproductive the R&D process appears to be for the bigger companies. But his analysis misses a lot of context: It doesn’t adjust for new uses for old drugs. Nor does it clarify the effects of mergers and acquisitions on costs. And while these are real limitations, the analysis looks like a reasonable approximation of the challenge. As I’ve noted before, while there have been some remarkable new and effective drugs developed, the industry as a whole appears to be suffering from a productivity problem, as illustrated by this 2012 image from a paper by Scannell et al. (now paywalled), showing the number of drugs produced per billion of dollars in R&D:Will breaking patents help?
Companies can only maintain high prices in the market when they have patent protection. Another approach that’s been proposed to deal with high prices has been to allow patents on high-cost drugs to be broken, to allow early generic competition. Driven in part by skepticism about development costs, advocates for this approach argue that drug costs that bankrupt individuals are immoral and unethical. Better to change the rules, they argue, rather than let patient suffer. And this kind of proposal isn’t restricted to poor countries. Senator Bernie Sanders has proposed the same approach to help the Department of Veterans Affairs afford the cost of the new hepatitis drugs:
Our nation’s veterans cannot, and should not, be denied treatment while drug companies rake in billions of dollars in profits.
In some cases, pharmaceutical companies are breaking their own patents. Gilead is the patent-holder and manufacturer of the hepatitis blockbuster sofosbuvir (Sovaldi) and it is allowing generic companies to produce and sell their product into developing countries. Realistically, I don’t see arbitrary patent-breaking as gaining much traction, as most realize that it would possibly do more to undermine R&D investment in new drugs than any other strategy. Changing the rules of the game, when drug development can take decades, could jeopardize any investment in the areas where we need new treatments, particularly where the rewards seem modest at best. Antibiotics are an example of the disconnect between medical need (high) and drugs in development (few). Growing concerns about antibiotic resistance are heightened by the reality that there are few new antibiotics in development. If we don’t have effective antibiotics, even everyday infections can be deadly. What seems necessary, perhaps is a better linkage between the value of new treatments and the incentives that in place to develop them. Breaking the patents on drugs we don’t want to pay for wouldn’t address this problem, or the overall productivity challenge in development.Is “cost of development” even the right question to ask?
To continue to improve science-based medicine, we need to find a way to introduce new innovative treatments at prices that that are affordable. Looking at R&D costs for solutions may be the wrong question to ask. What really matters is if a particular treatment delivers value. A new treatment may be worth the high cost if it avoids other costs, such as the potential for a vaccine to prevent cancer (e.g., HPV) or for a drug to eliminate the need for a liver transplant (e.g., the new hepatitis treatments). From that perspective, the R&D costs aren’t something we should focus on. We don’t ask (or care) what it costs to develop a new car, or the latest phone. What matters is whether we value the benefits of the “new” versus the status quo. The challenge with drugs (and with healthcare in general) is that (1) assessing value is difficult when it comes to health treatments and (2) we are often shielded (via insurance) from directly bearing the costs ourselves. New drugs need to show value in the calculation of their cost. And we also need to do what we can to bring the costs of development (and innovation) down. Otherwise we run the risk of less innovation and fewer new treatments that don’t offer value for the price.Conclusion
As health professionals, consumers and as patients, it’s perhaps relevant to question what it is we expect in terms of new medicine. We’ve become accustomed to new treatments that transform diseases or even prevent them. Yet productivity is dropping and costs are rising. Is there a common ground we can agree on? There would likely be a strong consensus that we want new treatments that are truly innovative (safer and more effective) compared to current treatments. And these innovations have to be valued by patients as delivering meaningful benefits. To get there, we can likely all agree that we must support measures that foster R&D, particularly those that help identify promising drug candidates sooner, thereby reducing the costs of failures. We need to balance rigorous regulation during development with the imperative to ensure patient safety during trials, and after the drug is released. And we cannot overlook the need to understand how much better a new treatment is compared to what we’re doing now. In doing so, we should get a better sense of what it should cost to develop a new drug. And the price we eventually pay needs to support a R&D culture that continues to advance, in an affordable way, improvements in the way we deliver medical care today, and into the future.
One of the major ambitions of my life is to promote science and critical thinking, which I do under the related banners of scientific skepticism and science-based medicine. This is a huge endeavor, with many layers of complexity. For that reason it is tempting to keep one’s head down, focus on small manageable problems and goals, and not worry too much about the big picture. Worrying about the big picture causes stress and anxiety.
I have been doing this too long to keep my head down, however. I have to worry about the big picture: are we making progress, are we doing it right, how should we alter our strategy, is there anything we are missing?
The answers to these questions are different for each topic we face. While we are involved in one large meta-goal, it is comprised of hundreds of sub-goals, each of which may pose their own challenges. Creationism, for example, is one specific topic that we confront within our broader mission or promoting science.
Over my life the defenders of science have won every major battle against creationism, in the form of major court battles, many at the supreme court level. The most recent was Kitzmiller vs Dover, which effectively killed Intelligent Design as a strategy for pushing creationism into public schools. The courts are a great venue for the side of science, because of the separation clause in the constitution and the way it has been interpreted by the courts. Creationism is a religious belief, pure and simple, and it has no place in a science classroom. Evolution, meanwhile, is an established scientific theory with overwhelming support in the scientific community. It is the exact kind of consensus science that should be taught in the classroom. When we have this debate in the courtroom, where there are rules of evidence and logic, it’s no contest. Logic, facts, and the law are clearly on the side of evolution.
Despite the consistent legal defeat of creationism, over the last 30 years Gallup’s poll of American public belief in creationism has not changed. In 1982 44% of Americans endorsed the statement: “God created humans in their present form.” In 2014 the figure was 42%; in between the figure fluctuated from 40-47% without any trend.
There has been a trend in the number of people willing to endorse the statement that humans evolved without any involvement from God, with an increase from 9 to 19%. This likely reflects a general trend, especially in younger people, away from religious affiliation – but apparently not penetrating the fundamentalist Christian segments of society.
Meanwhile creationism has become, if anything, more of an issue for the Republican party. It seems that any Republican primary candidate must either endorse creationism or at least pander with evasive answers such as, “I am not a scientist” or “teach the controversy” or something similar.
Further, in many parts of the country with a strong fundamentalist Christian population, they are simply ignoring the law with impunity and teaching outright creationism, or at least the made-up “weaknesses” of evolutionary theory. They are receiving cover from pandering or believing politicians. This is the latest creationist strategy – use “academic freedom” laws to provide cover for teachers who want to introduce creationist propaganda into their science classrooms.
Louisiana is the model for this. Zack Kopplin, who was a high school student when Bobby Jindal signed the law that allows teachers to introduce creationist material into Louisiana classrooms. He has since made it his mission to oppose such laws, and he writes about his frustrations in trying to make any progress. Creationists are simply too politically powerful in the Bible belt.
This brings me back to my core question – how are we doing (at least with respect to the creationism issue)? The battles we have fought needed to be fought and it is definitely a good thing that science and reason won. There are now powerful legal precedents defending the teaching of evolution and opposing the teaching of creationism in public schools, and I don’t mean to diminish the meaning of these victories.
But we have not penetrated in the slightest the creationist culture and political power, which remains solid at around 42% of the US population. It seems to me that the problem is self-perpetuating. Students raised in schools that teach creationism or watered-down evolution and live in families and go to churches that preach creationism are very likely to grow up to be creationists. Some of them will be teachers and politicians.
From one perspective we might say that we held the line defensively against a creationist offense, but that is all – we held the line. Perhaps we need to now figure out a way to go on offense, rather than just waiting to defend against the next creationist offense. The creationists have think tanks who spend their time thinking about the next strategy. At best we have people and organizations (like the excellent National Center for Science Education) who spend their time trying to anticipate the next strategy.
The NCSE’s own description of their mission is, “Defending the teaching of evolution and climate science.” They are in a defensive posture. Again, to be clear, they do excellent and much needed work and I have nothing but praise for them. But looking at the big picture, perhaps we need to add some offensive strategies to our defensive strategies.
I don’t know exactly what form those offensive strategies would take. This would be a great conversation for skeptics to have, however. Rather than just fighting against creationist laws, for example, perhaps we could craft a model pro-science law that will make it more difficult for science teachers to hide their teaching of creationism. Perhaps we need a federal law to trump any pro-creationist state laws. It’s worth thinking about.
I also think we need a cultural change within the fundamentalist Christian community. This will be a tougher nut to crack. We should, however, be having a conversation with them about how Christian faith can be compatible with science. Faith does not have to directly conflict with the current findings of science. Modeling ways in which Christians can accommodate their faith to science may be helpful. And to be clear – I am not saying that science should accommodate itself to faith, that is exactly what we are fighting against.
As the skeptical movement grows and evolves, I would like to see it mature in the direction where high-level strategizing on major issues can occur. It is still very much a grassroots movement without any real organization. At best there is networking going on, and perhaps that is enough. At the very least we should parlay those networks into goal-oriented strategies on specific issues.
Creationism is one such issue that needs some high-level think tank attention.
Nootropics are an emerging class of drugs that are designed to enhance cognitive function. They are part of a broader category of drugs known as performance and image enhancing drugs (PIED) which are used for enhancement of memory and cognition, sexual performance, athletic performance or musculature (also called “lifestyle” drugs).
It will probably come as no surprise to regular readers of SBM that nootropics and PIED are being abused and hyped without adequate evidence. One of the primary problems is that they are sold as supplements or as drugs, often over the internet without adequate regulation. One simple fix is to properly classify these drugs as drugs, and to properly regulate them as drugs.
Many of the cognitive enhancing “supplements” on the market make all the usual claims about “natural” enhancement” – meanwhile they predictably contain just vitamins, herbs which have not been shown effective, perhaps nootropics (see below), and often a stimulant, like caffeine. The only drug in the mix which is likely to have a noticeable effect by the user is the stimulant.
The risk of overuse of stimulants in “supplements” is well known.
Meanwhile, what does the science say about the safety and effectiveness of specific nootropics for specific indications? In short, sufficient high-quality studies to make definitive determinations are lacking. For many, however, there is some preliminary and pre-clinical data.
The concept behind nootropics is to target some metabolic or nutritional aspect of brain function, especially a function involved with memory or attention, and then to provide a nutritional precursor to that metabolic pathway, or a drug that enhances the activity of a neurotransmitter, enzyme, or other metabolic factor.
This is not an unreasonable like of research, but tweaking metabolic pathways is not a simple business, and our ability to extrapolate from the petri dish to animals and finally to net clinical effects in humans is very limited. No matter how promising a treatment looks in theory, we need clinical data to see if it has a measurable and lasting clinical effect.
The primary plausibility problem with the PIED category of drugs is that they are further extrapolating from data in disease states to the enhancement of otherwise healthy function. Just because a drug may enhance memory function in someone with Alzheimer’s disease, that does not mean it will enhance memory function to supernormal in a healthy individual.
It is true and likely relevant that the brain is a very metabolically hungry organ. Optimal brain function depends upon most metabolic and physiological parameters functioning well. For this reason the brain is often the canary-in-a-coalmine of biological function – the first thing to go when something is off. Sick hospital patients, for example, are often sleepy or confused because their bodies are simply under stress, or their metabolic parameters are off. It can often be challenging to figure out exactly what is making a patient confused, because so many things can affect brain function.
Further, in everyday healthy life our brain function can be off simply from having insufficient sleep, not eating well, being depressed, or being physically or mentally exhausted.
Given our brain’s sensitivity it does make superficial sense that giving it a specific metabolic boost might help cognitive function. This is essentially what stimulants do. It still needs to be determined, however, that a specific metabolic tweak will have the desired effect. One potential problem with predicting whether a specific intervention will help is determining if that metabolic factor is the limiting factor in brain function.
Enhancing acetycholine (the primary neurotransmitter involved in memory formation) may enhance memory, but what if there is already plenty of acetylcholine and the limiting factor in memory function is something else entirely? This is probably why we cannot extrapolate from a disease state to a healthy state – the limiting factors on function are likely different.
Therefore, from a plausibility standpoint, the idea of nootropics is possible, but very complicated, and we cannot make confident predictions from pre-clinical data alone or extrapolate from one condition to another. We need high quality clinical studies for each specific clinical claim.
The Clinical Evidence
One of the most popular nootropics is piracetam. This is actually one of a large and growing class of drugs with many neurological indications, including seizure control and treatment of anxiety and depression. Piracetam has been studied specifically for memory and cognitive enhancement in dementia or cognitive impairment.
At this stage the evidence available from the published literature does not support the use of piracetam in the treatment of people with dementia or cognitive impairment. Although effects were found on global impression of change, no benefit was shown by any of the more specific measures.
They concluded that the data supports further research, but not current clinical claims or use.
In healthy individuals there are only a couple small studies, and they do not show any significant benefit. Therefore, even though piracetam is a popular “smart drug” the evidence for any significant effect is lacking, especially in healthy individuals.
Another highly popular cognitive enhancer is modafinil. This is a stimulant drug used for narcolepsy, ADHD, and cognitive fatigue resulting from neurological disease or injury. It is a controlled prescription drug, but has still managed to become highly popular as a cognitive enhancer, especially among students.
A recent study, however, highlights the difficulty in extrapolating from disease to healthy states. The researchers compared test performance on healthy subjects taking modafinil vs placebo. They found that the responses were delayed from subjects taking modafinil, but were not more accurate. In hindsight, this result makes sense.
As a stimulant modafinil increases brain function, including frontal lobe executive function. When performing a cognitive task there is often a tradeoff between speed and complexity of processing. Executive function sacrifices speed for complex higher cognitive function. If you have impaired executive function, this trade off is worth it. Modafinil can shift the balance back towards a more optimal state.
In the healthy individual, however, you are shifting the balance away from an optimal state, slowing things down without any further gains in accuracy. Therefore modafinil can be a net gain in individuals with ADHD, but a net loss in healthy individuals, and that is what the preliminary clinical data shows.
There are other nootropics but with a similar story – basic science research which is used for marketing hype, but does not establish that the drugs (often sold as supplements) actually work.
Nootropics are an interesting and potentially very useful class of drugs which deserve further research. Their greatest potential is likely in various states of neurological disease or injury. Their use as smart pills, however, is dubious from a plausibility standpoint and lacks sufficient evidence.
Many “smart pill” products on the market cheat by including a regular stimulant, like caffeine. The effects of stimulants on healthy individuals is interesting – while they create the subjective experience of being more alert and higher performance, it is unclear if they actually improve performance, and in fact they may even decrease performance. Even if they do work, their effects typically develop tolerance quickly. Daily use of caffeine, for example, develops tolerance within three weeks, after which regular users are just staving off caffeine withdrawal and experience an overall net negative effect on their alertness.
Stimulant use is therefore potentially useful for alertness in the short term or for intermittent use, but it is unclear if it has a positive effect on cognitive performance. They are good for creating the illusion of cognitive enhancement, however.
Current non-stimulant nootropics are likely useless in healthy individuals.
If you are otherwise healthy and want to optimize your cognitive function, there are some things you can do that are likely to have a dramatically greater benefit than any smart pills on the market.
– Get enough sleep.
– Get regular exercise.
– Keep mentally active.
– Do not skip meals.
– Avoid alcohol or other recreational drugs.
Americans cherish our free speech, enshrined in the very first amendment to the Constitution. SLAPP suits (strategic lawsuit against public participation) are a serious threat to that freedom of speech. We desperately need libel reform in the form of effective anti-SLAPP laws.
What I learned when I became the target of a SLAPP suit (that is still ongoing) is that anyone with money can take away your free speech at will. It works like this: if you express an opinion publicly that someone else doesn’t like because it is critical of them, their beliefs, their business, etc. then they can hire a lawyer and send you a cease and desist letter. You are now faced with a dilemma – take down your blog, article, podcast, video, or whatever and allow your free speech to be suppressed, or potentially face tens and perhaps hundreds of thousands of dollars in legal fees.
Except for those few states with effective anti-SLAPP laws (California, Washington, Oregon, Nevada, Texas and the District of Columbia – Florida just passed one which has not yet gone into effect), if you refuse to remove your free speech and you get sued, then expect to spend large sums of money and years of your life defending your rights. Here’s the thing – even if the case against you has zero merit and no chance of winning in the end, the lawsuit is a financial game of chicken. There is no way to shut the case down early. There is no bar for meritless cases.
The net effect of this is that if someone has money they can shut down your free speech at will. This, of course, has a chilling effect on free speech that can go way beyond the one instance of speech being targeted.
This is an important issue for all Americans. It is especially important for media companies and anyone whose business or activism requires speech. This includes the skeptical movement – we are basically involved in critical analysis of claims and practices. People don’t like critical analysis when they find themselves on the negative side.
Of course, they are free to answer the criticism, to level criticisms of their own, to make their case, present their evidence, and let open discourse sort things out. This is the public marketplace of ideas, and the framers of the Constitution knew how important it was. Some people, however, would rather just sue you into silence.
Here is another recent example that just came to my attention – four researchers, including Harvard Professor Pieter A. Cohen, were just sued by Hi-Tech Pharmaceuticals, Inc, for an article they published in the journal Drug Testing and Analysis: An amphetamine isomer whose efficacy and safety in humans has never been studied, β-methylphenylethylamine (BMPEA), is found in multiple dietary supplements. In the article they concluded:
Consumers of Acacia rigidula supplements may be exposed to pharmacological dosages of an amphetamine isomer that lacks evidence of safety in humans. The FDA should immediately warn consumers about BMPEA and take aggressive enforcement action to eliminate BMPEA in dietary supplements.
The company is suing researchers for publishing a scholarly article in a journal in which they warn that supplements may contain significant doses of a stimulant drug. Think about the implications of this case – the multi-billion dollar supplement industry can sue any critics into silence (even researchers publishing in the scientific literature) with harassing SLAPP suits.
Another recent case is Stephanie Guttormson, the Operations Director for the Richard Dawkins Foundation, who published a video criticizing a faith healer, Adam Miller, for presenting himself as a “medical professional.” Miller’s response was to sue Guttormson, who must now defend herself at great expense.
There is something very specific that we can do to fix this situation. Right now the libel laws in most states and at the federal level are broken. They allow for meritless lawsuits that effectively chill public participation in the marketplace of ideas. The fix is effective anti-SLAPP statutes. They can change the process to meritless cases are stopped early on, before ruinous expense is incurred. Effective anti-SLAPP laws may require the plaintiff to demonstrate that they are likely to win on the merits before they can force the defendant to spend money defending themselves. They can require the plaintiff to demonstrate merit prior to discovery. They can allow the defendant to have instant appeal, without having to wait until the end of the trial. They can also allow for the defendant to ask the judge to have the plaintiff pay their legal fees if the case is without merit.
Keep in mind – these statutes will not prevent or hamper legitimate libel lawsuits. If the case has demonstrable merit, meaning that the plaintiff was genuinely the victim of libel or slander, then they can still sue for damages and have the defamatory speech removed.
What anti-SLAPP laws do is make the system work like it is supposed to. Libel laws in the US are good in that the plaintiff has the burden of proof, and the defendant is innocent until proven guilty. However, in jurisdictions without effective anti-SLAPP laws the law doesn’t actually work that way – you are not innocent until proven guilty if you are forced to spend two hundred thousand dollars, with no chance of recuperation, just to defend your free speech.
There are efforts to pass effective anti-SLAPP laws in every state and at the federal level. The Public Participation Project is one such effort. I and the SGU are working with my attorney, Marc Randazza, to help this effort. We have a model anti-SLAPP statute we can share, and we have begun efforts to promote anti-SLAPP laws in the remaining states. We need people in every state, especially those with connections to legislators, to help us get anti-SLAPP laws sponsored.
One very interesting update is that (as reported by the PPP):
On May 13, 2015, bi-partisan co-sponsors in the House introduced the SPEAK FREE Act of 2015, a law designed to protect Americans from meritless lawsuits that target their First Amendment rights. Representatives Blake Farenthold (R-TX) and Anna Eshoo (D-CA) introduced the bill in a bi-partisan effort aimed at supporting the rights of all Americans affected by meritless SLAPP suits.
So, here is something else you can do. Write to your congressperson and senator and tell them you support this very important bill. Now is the time to put the pressure on. Let them know we are paying attention and we demand our free speech.
The bill is not perfect. Here is a thorough analysis by an expert. It is definitely a good bill that should be passed, even as is. It will get the job done.
There is one interesting aspect of the bill, however, that may make it difficult to pass and might be difficult to enforce. The statue would allow any defendant in a SLAPP case sued in state court to remove the case to federal court if they want to apply this federal anti-SLAPP law. That would effectively federalize all state SLAPP cases. While this would be good for defendants (at least initially) it might also overwhelm the federal courts, and dilute the advantage of being in federal court.
If the statute, however, succeeds in providing a disincentive to filing a weak or meritless libel case, then this may not result in the federal courts being overwhelmed. That would be the best case scenario – if meritless SLAPP cases never occurred because the law prevents them from harassing people out of their free speech, and puts the financial risk on the plaintiff.
We urgently need anti-SLAPP laws in the US, at the federal level and in every state. This should be one of the top priorities of the skeptical movement, as our very existence is threatened by frivolous SLAPP suits.
You know what to do. Do it.
A cycling enthusiast asked me about helmets. It seems compellingly obvious to me that a head impacting the pavement without a helmet is likely to sustain more damage than a head protected by a helmet. He challenged that, citing a BMJ article by Ben Goldacre that questioned whether the evidence showed that helmets do any good. He said I was making a non-evidence-based assumption and challenged me to actually look at the evidence, so I did.
Goldacre says there is a:
complex contradictory mess of evidence on the impact of bicycle helmets. Like most places where there’s controversy and disagreement, this is a great opportunity to walk through the benefits and shortcomings of different epidemiological techniques, from case control studies to modeling.
He proceeds to give a lesson in epidemiology. He points out that there are a lot of emotion involved, and that epidemiologic studies, because of their inherent imperfections, are probably not capable of resolving the debate.
There are basically two questions:
He shows why research has difficulty answering both of those questions, and covers some of the possible confounding factors:
Some other problems: studies focus on lives saved rather than injuries mitigated, and the documented effect of legislation is admittedly small. There is probably under-reporting of cycling accidents. Helmets may not prevent concussions; the scalp is protected from a direct impact but the brain is still vigorously jostled. Other measures including training may do more to mitigate injuries than helmets.
I reviewed relevant studies in PubMed and systematic reviews.
Impact simulations have verified that helmets reduce impact to the head.
Individual studies reach varying conclusions; but overall, I interpret the existing body of evidence as showing that the science of protection is clear: helmets offer a significant benefit. The advisability of helmet laws is an entirely different question. The impact of legislation has not been as impressive as hoped, and decisions about laws are not made simply on the basis of science. Those decisions can (and should) be informed by science, but ultimately they depend on societal attitudes and other factors.Physics and common sense
We don’t need to do randomized controlled studies to know that parachutes save lives or that it’s a good idea to suture wounds and immobilize broken bones, and we shouldn’t need randomized controlled studies to know that helmets are protective. The principles of physics and a little common sense tell us that helmets must provide some protection, even if the degree of that protection can be disputed. If you personally had the choice of hitting your head on the pavement with or without the protection of a helmet, which would you choose? It seems to me to be a no-brainer. (If nothing else, a helmet would prevent skin abrasions and contamination of wounds with dirt and gravel.) We have long accepted the wisdom of using helmets in a variety of sports, from show jumping to football. Construction workers use hard hats. Soldiers wear helmets in combat. Eggs are sold in protective containers rather than dropped individually into shopping bags; the principle is the same. It seems to me that the reaction against helmets for cycling is more emotional than evidence-based. Helmets clearly offer at least some degree of insurance for the individual. Whether individuals should be compelled by legislation to protect themselves is another matter entirely, a decision that I am willing to leave to others.
Last year, I did several posts on what I consider to be a profoundly misguided and potentially harmful type of law known as “right-to-try.” Beginning about a year and a half ago, promoted by the libertarian think tank known as the Goldwater Institute, right-to-try laws began popping up in state legislatures. Both Jann Bellamy and I wrote about how these laws are far more likely to do harm than good, and that is a position that I maintain today. The idea behind these laws is to give terminally ill patients access to experimental drugs—in some cases drugs that have only passed phase I testing—that might help them. It’s an understandable, albeit flawed argument. After all, it’s perfectly understandable why terminally ill patients would fight for drugs that give them hope, and it’s just as understandable why politicians and the public would see such a goal as a good thing. In practice, as I will explain again in the context of this update, such laws are far more likely to harm patients than help them. Indeed, as you will see, in the year since the first wave of right-to-try laws have passed, not a single patient that I can find has obtained access to experimental drugs under a right-to=try law, much less been helped by them.
Unfortunately, given how effectively “right to try” has been sold on grounds of providing terminally ill patients hope and as a matter of personal freedom, it’s clear that this wave is not going to abate. Since Colorado passed the very first right-to-try law almost exactly a year ago today, a total of 17 more states now have passed passed similar legislation, the most recent being Tennessee, and 22 others have introduced legislation. It’s a good bet that right-to-try will pass in all of those states, because, as I’ve explained many times before and in many interviews, if you don’t understand clinical trial ethics and science, opposing the concept of right-to-try comes across like opposing Mom, apple pie, and the American flag, and leaves opponents open to false—but seemingly convincing—charges of callousness towards the terminally ill on the order of enjoying drop kicking puppies through flaming goalposts.
As I’ve pointed out many times before, opposing right-to-try is actually pro-patient, because these right-to-try laws that are passing all follow an explicit template written by the Goldwater Institute, and that template is very much a sham. For one thing, states do not control drug approval; the federal government through the FDA does. Consequently, state-level right to try laws, while giving the appearance of giving access to experimental drugs to patients, do nothing to actually achieve that goal. Worse, even if a patient were to get an experimental drug through right-to-try, these laws very much reflect the think tank that created them in that they leave the patient basically on his own. He can be charged the full cost of the investigational drug, which means that the only people who might be able to take advantage of these laws are those who are already rich or who are very good at fundraising. Worse, these laws explicitly indemnify drug companies and physicians from any liability arising from adverse outcomes, which means that even if a physician committed malpractice in advising or administering right-to-try drugs he probably couldn’t be sued. Moreover, such laws explicitly bar state employees from blocking or attempting to block an eligible patient’s access to an investigational drug or treatment. Even though there is a clause that says “counseling, advice, or a recommendation consistent with medical standards of care from a licensed health care provider is not a violation of this section,” where does “advice” end and “blocking” begin? Certainly as a physician in what is now a right-to-try state whose law has identical language, I wonder.
Even worse still, many of these laws, such as the one in Michigan, are written such that if a patient suffers a complication from a right-to-try drug or treatment his insurance company can argue that it doesn’t have to pay for the resulting care to treat that complication. Indeed, Colorado’s law requires that informed consent for right-to-try must explicitly make clear that, “the patient’s health insurer and provider are not obligated to pay for any care or treatments consequent to the use of the investigational drug, biologic product, or device” and that “in-home health care may be denied.” Elsewhere, the Colorado right-to-try law states:
An insurer may deny coverage to an eligible patient from the time the eligible patient begins use of the investigational drug, biologic product, or device through a period not to exceed six months from the time the investigational drug, biologic product, or device is no longer used by the patient; except that coverage may not be denied for a preexisting condition and for coverage of benefits which commenced prior tot the time the eligible patient begins use of such drug, biologic product, or device.
In other words, if you access an experimental treatment under right-to-try, and you suffer a complication from the investigational treatment, you could well be out of luck getting your insurance company to pay for the medical and/or surgical treatment necessary to treat that complication. If your insurance company so decides, you’ll be on the hook for everything subsequent to that treatment. Indeed, to me the language in some of these bills implies that insurance companies can deny coverage for any new problems that come up after the patient starts using experimental therapy, whether caused by that therapy or not. These issues have largely been ignored in the news coverage of these laws, but the oncology community is finally waking up to them, as demonstrated by an article published a couple of weeks ago in HemeOnc Today in an article entitled Expansion of ‘Right to Try’ legislation raises ethical, safety concerns:
“There are a lot of issues not addressed in the bill that make the feasibility more challenging,” W. Thomas Purcell, MD, MBA, associate director for clinical services at the University of Colorado Cancer Center and executive medical director of oncology services at University of Colorado Hospital, told HemOnc Today. “If the drug is made available, who is going to administer it? Who is going to pay for any side effects related to the treatment? Are insurance companies going to cover any treatment-related complications? There are a lot of practical things that come into play with the introduction of the law, although the law doesn’t address any of those things.”
Actually, although he raises the same concerns I’ve been raising for over a year now, Dr. Purcell is about as wrong as wrong can be about one thing. The problem is not that the laws don’t address these things. It’s that the laws do address these things rather explicitly. Dr. Purcell seems blind to how these laws address these things, which is in a way likely to be highly detrimental to patients. In fact, I wonder if Dr. Purcell has even read his own state’s law! It says right there in black and white that insurance companies don’t have to pay for care or complications related to such drugs! That means that either the patient does, or we taxpayers do when patients suffering such complications are forced to go on Medicaid because they can no longer afford their medical care.
Another example of someone echoing what I’ve said many times appears here:
“People do not actually read the bills,” Alison Bateman-House, PhD, MPH, MA, Rudin postdoctoral fellow in the division of medical ethics at New York University Langone Medical Center, told HemOnc Today. “They think ‘Right to Try’ sounds fantastic and allows access to treat terminally ill patients. How could you not support that? For the most part, the response that we’ve seen is that these laws don’t do much, aside from giving people hope.”
Dr. Bateman-House is also partially wrong. The problem is not that people don’t read the bills. It’s that people don’t understand the anti-patient implications of the bills in terms of insurance coverage, eliminating the right to sue, and the like; don’t understand clinical trials; and don’t understand that it is the federal government, not state governments, who control drug approval and access to experimental drugs. Indeed, when I’m in a cynical mood, I wonder if Goldwater Institute flacks do understand all these things but don’t care because they’re playing a long game designed to weaken the FDA in the name of an ideology that, against all evidence to the contrary, preaches that the free market can assure drug safety and efficacy better than any government agency. Indeed, this intent can be seen in one version of the Goldwater Institute template (and in the Michigan right-to-try law), which changed the term “terminal illness” to “advanced illness,” without really changing the definition. I would not be surprised if, a few years from now, after the majority of states have passed right-to-try, there is a push to open up “right-to-try” to serious medical conditions that are not terminal on the basis of “fairness” and “compassion.” It’s coming. Mark my words.
Finally, there are the issues of safety and false hope. As I’ve said before, as hard as it is to believe, there are things worse than suffering a terminal illness; for instance, suffering a terminal illness and then bankrupting yourself before you die or suffering a terminal illness and then suffering a major complication of an experimental treatment that you have to pay for, thus bankrupting yourself before you die because you paid for the treatment and now have to pay for the treatment of the complications. Now, consider this. The only requirement for an investigational drug or treatment to be made available under right-to-try is that it has to have passed phase I testing and still be in clinical trials (i.e., phase II or III studies). Again, as I’ve described before, this is an incredibly low bar for safety, given that most phase I trials only include less than 30 patients and are meant mainly as screening trials to identify the worst side effects and an appropriate dose to use in phase II trials. Yet, as I described before, the Goldwater Institute blatantly describes drugs that have passed phase I testing as having had their safety adequately established. It’s a lie.
Indeed, as is pointed out in the HemeOnc Today article, the risk for toxicity is actually higher in patients who would exercise right-to-try because by definition such patients have to be ineligible for a clinical trial. Remember, clinical trials are designed to minimize risks and maximize potential benefits because it is an ethical imperative in human experimentation, codified in the Common Rule and the Helsinki Declaration. One way they do that is to design the inclusion and exclusion criteria to achieve that end. Moreover, there have been cases where the use of an experimental therapy has become popular based on public pressure related to early evidence.
An excellent example of just what I’ve been warning about for a long time shows up in the HemeOnc Today article citing Dr. Yoram T. Unguru, a pediatric oncologist at the Herman and Walter Samuelson Children’s Hospital at Sinai Hospital in Baltimore and bioethicist at The Johns Hopkins Berman Institute of Bioethics:
In multiple cases, access to an experimental therapy has been expedited due to an early benefit observed in early studies that was not substantiated in subsequent research, Unguru said. One example was the use of autologous bone marrow transplantation in women with metastatic breast cancer, based on preliminary data published in 1995 in the Journal of Clinical Oncology.
“This completely backfired,” Unguru said. “In addition to being poorly designed, the study raises serious ethical concerns and, ultimately, people did much worse, including some who even died. This is why we go through the laborious, lengthy and, at times, seemingly maddening trial phases.”
Although physicians and drug companies are required to report data attained from patients treated through the FDA’s compassionate use program, Right to Try laws do not carry such stipulations.
And, as Colin Begg, PhD, chairman and attending biostatistician in the department of epidemiology and biostatistics at Memorial Sloan Kettering Cancer Center, adds, echoing (again) things I’ve been warning about for over a year:
Eliminating the FDA from the equation may have many significant consequences.
“There is good reason we have the FDA,” Begg said. “The rigorous testing with scientific methods of drugs that come through the pipeline is absolutely essential. Without it, the market would be flooded with drugs that do not work. You would have a cupboard full of drugs that you would want to try, and you would have no idea which one to try because there would be no reliable evidence about the efficacy of any one of them.”
Right to Try laws may create a precedence for additional changes to drug R&D.
“If we go down this road, there might be a loosening of the standards of drug approval in the first place, and that would be bad for public health,” Begg said. “This movement may ultimately lead to situations where … drugs, in general, would no longer have to go through such rigorous testing to see if they work.”
Unfortunately, what Dr. Unguru and Begg apparently fail to realize is that the libertarian Goldwater Institute cares little or nothing about the difficult balancing of patient rights versus patient safety and societal good that our current clinical trial system tries to maintain, with varying degrees of success depending on the specific situation. An argument, for example, that right-to-try might make it more difficult to recruit patients to clinical trials holds exactly zero water with the Goldwater Institute and most other supporters of right-to-try. Indeed, associated as it is with the “health freedom” movement, the goals of the Goldwater Institute’s right-to-try push appear to align quite nicely with the goal of some libertarians to eliminate most of the FDA’s authority because of an ideology, which views any government intrusion into the free market with a jaundiced eye and even believes that the unfettered free market will take care of sorting out safety and efficacy of drugs. We all know how that worked out before the passage of the original act creating the FDA in 1906 and the Kefauver Harris Amendment of 1962 that introduced the requirement that drug manufacturers demonstrate efficacy as well as safety to the FDA before a drug is approved.
Those of us who take care of breast cancer patients remember Dr. Unguru’s cautionary tale from the 1990s. Indeed, the clinical trial publication that fueled the demand for high dose chemotherapy with bone marrow transplant for advanced breast cancer was ultimately retracted. The bandwagon effect is a powerful force affecting politics and even prominent researchers and physicians before the evidence is adequate to recommend a treatment. If this can happen with a treatment as incredibly toxic and risky as bone marrow ablation with high dose chemotherapy followed by stem cell rescue, imagine how easily it can happen with less spectacular examples.
Over a year ago, when the states on the vanguard of the right-to-try movement were first seeing such legislation introduced, I learned the hard way just how willing the movement was to paint its opponents as those proverbial puppy-kicking, American flag burning, cold-hearted “scientists.” Indeed, back then almost the only people I saw routinely speaking out against right-to-try were noted bioethicist Arthur Caplan and little ol’ me, an insignificant blogger. When right-to-try was introduced into my own state’s legislature, my interaction with my state representative, who politely thanked me for pointing out the many problems with right to try, but made it clear through his response that he was likely going to vote for it anyway, showed that, from a political standpoint at least, ethics- and science-based medicine were not likely to prevail in Michigan or anywhere over the emotional appeal of “doing something” to help terminally ill patients. It didn’t matter whether that “something” will actually do what it promises or not. Later, I met with an advocate of the biotech industry who testified against the bill. He described a scene in which he was the lone expert testifying against right-to-try versus a flack from the Goldwater Institute and patients with terminal illnesses and their families, the latter of whom all glared at him as though he were personally going to execute them or their ill family member. I learned that no one associated with a major cancer center was willing to publicly oppose right-to-try, even though they uniformly thought it was a horrible idea. If I had found out about the hearing in time to have made the trip to Lansing—as I recall, it occurred with little notice and on one of my operating room days—I honestly don’t know if I would have had the wherewithal to do so myself, given the pushback I had received from my previous posts on the matter.
As I said, right-to-try is a con game perpetrated on desperate patients, offering them false hope but instead delivering nothing of value that can’t be obtained under FDA compassionate use programs, as is pointed out by the opinion piece written by Dr. Jeffrey M. Peppercorn, a medical oncologist specializing in breast cancer at Massachusetts General Hospital, in a point-counterpoint forum included after the HemeOnc Today article that asks: Do the changes to the FDA’s compassionate use program eliminate the need for ‘Right to Try’ laws? Dr. Peppercorn takes the “yes” position:
However, the question is not whether promotion of access to promising drugs for patients with terminal disease is justified, but how this can best be accomplished. The same imperative that drives Right to Try laws underlies the FDA’s compassionate use program. The primary difference is that access to experimental drugs through compassionate use programs is regulated in the interest of both the patient and society. Physicians must seek FDA approval before a manufacturer provides the unproven drug, and the rationale for use of the drug, absence of alternatives, informed consent and review by an independent institutional review board must be documented. In addition, toxicities and outcomes after administration must be reported. This process promotes responsible practice of evidence-based medicine — even as the evidence evolves — and provides a chance for monitoring, both of the specific intervention and of the use of experimental therapy more generally. Although the process can be burdensome, the FDA has recently streamlined the application to allow for completion in less than 45 minutes and still allows for rapid approval of emergency access by phone when this is clinically justified.
The key difference between right-to-try and FDA compassionate use programs is that right-to-try strips pretty much all protections from patients who would use it; requires them to pay for the drug and and any care related to the drug, prevents them from suing manufacturers and doctors if something goes wrong; prevents the state from taking action against the licenses of providers who give patients bad advice recommending right-to-try; tells insurance companies that, not only do they not have to pay for the investigational agent or device, but they don’t have to pay for any complications arising from use of the investigational agent or device; and makes doctors and other health care providers working for right-to-try states leery of advising too strongly against right-to-try, lest they be prosecuted for “blocking” access to experimental drugs. FDA compassionate use programs, in marked contrast, require review and oversight by an institutional review board (IRB). The other difference was that, although 99.5% of compassionate use/expanded access requests are approved by the FDA, the process was onerous. As Dr. Peppercorn points out, that is rapidly changing, arguably eliminating the “need” for right-to-try. Yet right-to-try marches on, because the reason for right-to-try is not as represented by the Goldwater Institute. Rather, it’s to weaken and ultimately neuter the FDA.Other pernicious effects of right-to-try
Patients come first, and must always come first, which is why my key objection to right-to-try remains (and will always remain) that it is bad for patients. However, the HemeOnc Today article also points out other problems with the legislation. One is that it presents a major dilemma to industry. Now, I realize that few people are particularly sympathetic to industry, and even we “pharma shills” (at least, that’s what right-to-try proponents and those supporting alternative medicine, groups that not infrequently overlap) recognize that pharma is nowhere near innocent in provoking that reaction. Now that right-to-try laws are metastasizing throughout the US, oncologists are finally taking notice of the alarms that we few have been sounding:
The fact that manufacturers are not liable if a drug obtained through Right to Try fails or causes significant harm may allow pharmaceutical companies to benefit from such legislation, Unguru said.
“The way the laws are written, pharmaceutical companies are under no commitment to release these drugs, but should patients be able to access them, there is almost no consequence in the event of a bad outcome,” Unguru said. “Bypassing the current regulatory system means that some drugs that may not be safe or ready for widespread use essentially get a ‘free pass.’ Some pharmaceutical companies might see such laws as a way to test their drugs in a way they typically would not be able to without being held accountable.”
This would pose significant safety issues to society.
“It is hard to argue that individual patients who are dying and want access to a drug should not get it,” Begg said. “But the concern I have is that the ability of drug companies to market drugs that have not been properly tested is not in the interest of the public.”
The Texas State Senate and House have passed versions of a Goldwater Institute-inspired right-to-try law. The only difference in the two bills is that the House version would allow manufacturers to provide experimental drugs to patients at cost, whereas the Senate version would require that companies that provide such drugs donate them. Assuming the two versions are reconciled and passed and the governor signs the bill, say goodbye to any chance of shutting down Stanislaw Burzynski, as the law would explicitly bar the Texas Medical Board from perusing its current action against Burzynski’s medical license, given that he could claim his antineoplastons have passed phase I clinical trials and are in phase II, which makes them eligible for right-to-try. Even if the Senate version prevails, Burzynski’s business model of providing his unproven antineoplastons for free but charging big bucks for “case management” fees would remain intact.
Most drug companies are not as unethical as Burzynski, however, which is why most do not support right-to-try and the Pharmaceutical Research and Manufacturers of America (PhRMA) is not supportive, as described in the HemeOnc Today article. Pharmaceutical companies remember incidents like this:
In the past, public pressure has forced pharmaceutical companies to grant access under compassionate use.
Last year, Chimerix denied Josh Hardy — a then 7-year-old boy who developed an adenovirus infection after undergoing a bone marrow transplant for rhabdoid tumor of the kidney — access to the experimental antiviral drug brincidofovir (CMX001). After a social media firestorm — which included death threats to the company’s leadership — Chimerix commenced an open-label phase 3 trial so Hardy could enroll and receive treatment.
“This became publicized, and everyone assumed that if Josh Hardy got brincidofovir he would survive,” Bateman-House said. “He did get brincidofovir, and he did survive. Thomas Duncan, the man who died of Ebola in Texas, got the same drug and died. Chimerix is a small drug company and brincidofovir is its only drug in development, and its stock took a nosedive after Duncan’s death. If it were not for the fact that phase 3 trials were already close to completion, this could have killed the company. If Josh Hardy had died, that could have killed the company.”
In other words:
“It can go either way — you’re damned if you do and you’re damned if you don’t,” Unguru said. “If you release the drug and it does badly, then you’re going to get bad press. If you don’t release it, then you risk being vilified.”
And if the company charges for the drug and things go badly, it will be doubly vilified, even though small pharmaceutical companies often have just enough venture capital to make enough drug to do the clinical trials necessary for approval and releasing drug jeopardizes its ability to do those trials.
Physicians are also put in a bind when a patient has exhausted everything and wants to pursue right-to-try. End-of-life discussions are very difficult to begin with, because, as I’ve pointed out before, no physician wants to be hope’s executioner. Yet, sometimes that is our job, and the best we can offer is palliation. Given that right-to-try theoretically requires less effort than an expanded access exception, there will be enormous pressure on physicians to accede to the wishes of a terminally ill patient’s wish to pursue right-to-try even if the physician thinks it won’t work. The option of right-to-try might thus actually provide a physician with an option to avoid the hard discussion that end-of-life care entails and just facilitate the patient’s getting the drug. As Dr. Charles F. Levenback, professor in the department of gynecologic oncology and reproductive medicine at The University of Texas MD Anderson Cancer Center, puts it:
“I’ve heard ethicists say physicians may go in the direction of compassionate use to avoid the difficult conversation about mortality and the limits of what we can provide,” Levenback said. “All of compassionate use is predicated on the physician’s responsibility to be candid about the purpose of treatment — palliative vs. curative — and setting the patient’s expectations correctly.”
Correct. However, some doctors are better at doing this than others.The track record of right-to-try thus far
Given that it’s now been a year, or nearly so, since the first batch of states passed their right-to-try initiatives, I asked a simple question: Has a terminally ill patient obtained an investigational agent through right-to-try yet? Note that I didn’t even ask whether a single terminally ill patient has benefitted from right-to-try, because that’s much more difficult to mention. Doing extensive Google searches, I could not find a single example, although I did come up with a lot of examples of patients expressing hope that such laws would get them access to investigational drugs (like this one). My next thought was that, if anyone would know of a case in which a terminally ill patient has been granted access to an experimental drug under right-to-try, it would be the Goldwater Institute. So I engaged Christina Sandefur, Vice President for Policy, on Twitter thusly:
— David Gorski (@gorskon) May 6, 2015
I then asked:
@gorskon these laws are just now going into effect,& it takes time to acclimate to overhaul of status quo. Offer to have a real convo stands
— Christina Sandefur (@cmsandefur) May 6, 2015
Which resulted in:
What’s irritating about this exchange is that I was trying to engage Ms. Sandefur in a “thoughtful convo”—or, at least, as thoughtful a conversation as you can have on Twitter. What is it with people like this who seem to want to try to get me on the telephone to discuss such matters when they can’t manage to engage me online, be it on Twitter, the comments of my blog posts, or even by e-mail, where you can have a really thoughtful conversation? I tend to suspect it’s because they don’t want to leave a “paper trail” (or an “electron trail” in the case of e-mail exchanges) documenting what they actually said. Perhaps I should respond to her that I’d be happy to talk to her on the phone, but only if she gives me permission to record the conversation. (In turn, I, of course, will give her permission to do the same.) In any case, I think it reasonable to assume that, if there were a patient in Colorado—or anywhere else, for that matter—who has successfully obtained an experimental drug under right-to-try, Sandefur would know about it and would not have hesitated to provide links to relevant news stories. She did not. Instead she chose to dodge the question and make excuses. Similarly, Kurt Altman, national policy adviser and general counsel for Goldwater Institute, mentioned no patients who had yet benefited from right-to-try in his “counterpoint” to Dr. Peppercorn’s opinion piece. I know he would have if such a patient existed. Certainly, if I were writing in favor of right-to-try, I would use the examples of such patients if they existed.Right-to-try laws: Bad for patients
I have little doubt that some version of right-to-try will likely pass in every state in which it has been introduced, which means we could be looking at up to 40 states, possibly more, with such laws by next year. It must be emphasized that the vast majority of legislators proposing such bills and passing them into laws and the patients lobbying for right-to-try do so with the best intentions, believing such laws will help terminally ill patients. These patients, according to mistaken popular belief, have nothing left to lose when in fact they do, even though it might not seem that way. In the idealistic desire to help the terminally ill, supporters of right-to-try either don’t pay attention to or downplay the significant negative aspects of these bills, which permit the loss of insurance coverage, stripping of IRB protections that patients receiving such medications through expanded access programs, the economic injustice in which only the rich or those capable of raising large sums of money could benefit if the drug company was unwilling to provide the investigational drug for free, and the potential additional risk of harm that can come from using experimental drugs outside the rigorous design and protections of clinical trials. Most people are similarly unaware that granting less controlled access to such medications is far more likely to cause an individual patient harm than good. Moreover, liberalization of expanded access programs to vastly decrease the burden of paperwork and effort on the part of physicians and patient has already been placed in draft guidelines, rendering right-to-try unnecessary.
So why are right-to-try laws so popular? In an editorial last year published in JAMA Internal Medicine entitled The Strange Allure of State “Right-to-Try” Laws, Patricia Zettler and Henry Greely ask the same question, noting that a recommendation from a physician is no substitute for the evidence of safety and effectiveness that comes from later-phase clinical trials:
So what is the point of these laws? A skeptic might point out that opposing experimental treatments for dying people is unpopular. Patients have publicized—and gained public support for—their efforts to obtain experimental treatments through social media. Lawmakers have little to lose politically by supporting these laws. Companies, seeing their ineffectiveness, have no powerful reasons to oppose them. And libertarians can celebrate an attack on big government. The problem is that all these efforts are unlikely to actually help the patients with life-threatening diseases. Indeed, these laws may be harmful if they draw attention and resources away from efforts to develop effective treatments, engender confusion about the FDA pathway for compassionate use of medications, or create false hopes for terminally ill patients.
All of which right-to-try laws do, and worse.
There will always be a conflict between personal freedom and protecting patients, as well as insuring maximal societal good. We see this in opposition to school vaccine mandates, support for cancer quackery and dubious stem cell clinics, and, of course, right-to-try. Unfortunately right-to-try laws represent dangerous placebo legislation. They only give the illusion of doing something given that the FDA, not the states, controls drug approval. The reason for their existence is not so much to help patients, but as part of a long game to build a groundswell of support for policies that would ultimately hobble the FDA’s ability to oversee the efficacy and safety of drugs. These laws are bad for patients, bad for doctors, bad for drug development, and bad for science. That’s why going through the states is the wrong way to attack this problem. If we as a society believe that terminally ill patients should have easier access to investigational drugs, then reforming how the FDA handles compassionate use exemptions, not a patchwork of state laws with no teeth, is how it should be done.
I saw Ex Machina this weekend. If you haven’t seen the movie yet, mild spoiler alert – I will try to avoid any major reveals, but I will be discussing major aspects of the movie.
First, it’s an excellent film. I highly recommend it. It was both entertaining and thought provoking. Writer/Director Alex Garland clearly understands the topic of artificial intelligence (AI), and is also a talented filmmaker. I was particularly impressed by how much he accomplished with such a sparse film. The majority of the film takes place in one location and with three characters, but he quickly established those characters and the primary tensions that drive the film.
He also manages to weave in a fairly deep commentary on the nature of consciousness and creativity, the nature of AI, and all with a subtext of oppression, misogyny and male-female relationships. I still feel like I am missing a lot of subtext in this film, which will require at least a second viewing and a lot more thought.
What I want to focus on for the rest of this article, however, is his handling of the topic of consciousness and AI. The central question, at least superficially, that drives the film is how to determine whether or not an AI is actually self aware or is simply a sophisticated simulation. Two analogies are presented in the film: One is that of a computer chess master. Computers can now beat the best humans at chess. It is pretty clear, however, that they are just sophisticated algorithms and do not really “understand” the strategy of chess, nor can they get into the mind of their opponent.
Another example is taken from philosophy, that of the scientist who lives in a black-and-white environment, but studies everything about color. She understands the physics and neurobiology of color completely, but she has never experienced color herself. While she can describe exactly what “blue” is, she really doesn’t know because she has never experienced it. She has never had the “qualia” or subjective experience of blue.
Obviously the film surrounds an artificially intelligent android. The brilliant and wealthy CEO of what is the equivalent in the movie of Google chooses one of his employees who is a gifted coder to help him engage in a Turing-type test to help him determine if the AI is really self-aware. What I really liked about the film is that it went deeper than just the Turing test itself.
For background, the Turing test was devised by Alan Turing. The essence of the test is that a human evaluator converses with either another human or a putative AI remotely, and then has to determine by the subject’s responses if they are human or computer. If they cannot tell the difference then the AI is found to have passed the Turing test. In practice Turing type tests involve multiple evaluators and multiple trials and some threshold is established for the number of evaluators that have to determine that the AI was a human.
In the film, however, early on the creator of the AI simply stipulates that his AI will pass the Turing test. He is way passed that point already. What he wants to know is if it can be determined that the AI is self-aware, rather than a sophisticated (even perfect) simulation. This is something the Turing test is not designed to determine.
The deeper question is this – can we devise a test that is so complex that a computer system cannot successfully pass the test by using brute force algorithms (like Watson)? Therefore, passing the test implies an actual “understanding” of the material that can only exist in a conscious entity? This is the premise of Koch and Tononi’s proposal.
Again – I think this concept is highly problematic. We are still left with the possibility that a powerful enough computer with a sophisticated enough algorithm and sufficient database can brute force its way through the problem. There are often multiple ways to solve a problem, and not all of them necessarily require consciousness.
This question is where the film begins. I like Garland’s solution. (Warning - This is probably the biggest spoiler in my discussion.) The CEO created a situation in which the AI was given a challenge in which it would have to use manipulation, cunning, and deception in a novel and creative way that implies an understanding of how someone else thinks and feels. If there is any test that can truly determine if an AI is self-aware, I think this kind of thing is it.
This relates to the theory of mind, which is a concept in psychology. As babies mature they eventually develop what is called the theory of mind, which is an understanding that other people have thoughts, feelings, and motives similar to our own. As adults we take this for granted, but this is an important and fundamental aspect of our brains – we have an intuitive understanding that other people think and feel the way we do.
In fact, if anything we have an overdeveloped sense of a theory of mind. We tend to anthropomorphize and have what is called hyperactive agency detection – we project minds and motives onto simulations, animals, and inanimate objects.
A theory of mind is critical to our ability to exist in a social structure. We need to be able to think about what other people might be thinking, to anticipate and possibly even manipulate their feelings and motivations. Different people have different amounts of inherent talent and developed skill in this area, which is why some people can be successful politicians, actors, sales people, and con artists while others would struggle in these professions.
The notion of using tests designed to emphasize the need for a theory of mind as a way of determining if an AI is truly self-aware I think is brilliant. This is probably as close as we will ever come to knowing if an AI is self-aware. We can still never really know, because only the AI will experience its self-awareness, if it has any. But this might get us close enough that it no longer matters. An AI who can pass this test should be treated as a self-aware being, in my opinion. (Which brings up the other main tension in the film, which I won’t get into.)
If you have any interest in AI or the philosophy of mind, then I highly recommend Ex Machina. It was a very intelligently written and directly film. Even for someone who has thought deeply about this topic before, the film was thought-provoking.