Study In AMA Journal Shows Bias And Misunderstanding
by Wyn Snow, Managing Editor
recently published article in the Archives of Internal Medicine
(AoIM) suggests that consumers in the US are of mixed opinion
regarding the use and benefits of dietary supplements. Entitled
"Americans' Views on the Use and Regulation of Dietary Supplements,"
the article appeared in the 26 March 2001 issue. AoIM is
published by the American Medical Association, the leading professional
society for physicians in the US.
article uses survey data developed in association with the Kaiser
Family Foundation and National Public Radio to show consumers are
confused about the current regulatory status of supplements. Based
on their findings, the article states consumers seek heightened
government regulation as a means to increased purity and safety
for newly introduced supplements.
particular concern to the authors, who in the main are associated
with the Harvard School of Public Health and Harvard's Kennedy School
of Government, is the potential for medical complications when consumers,
for whatever reason, do not reveal their use of supplements to physicians
while being treated for a variety of medical conditions.
of supplement regulation
believe a number of suppositions made by the authors based on survey
findings show a troubling misunderstanding of supplement regulation.
Among those who crafted and analyzed the two opinion surveys appearing
in the article there appears to be a lack of knowledge of supplements
and their use. Given the presence of these misperceptions in the
article and the fact that AoIM is peer reviewed, one can
assume the authors' bias and lack of knowledge is shared by those
who reviewed the article prior to publication.
authors don't have a clue about the FDA's role in regulating
-- Dr. Shari Lieberman (see full
text of her comments)
as troubling is the presence of the article in an American Medical
Association-sponsored publication. Given the historic rejection
by physicians of botanicals and supplements for preventative and
therapeutic purposes, it would seem any article published under
the imprimatur of the American Medical Association (AMA) should
observe the most rigorous technique, taking special care to guard
against negative predisposition and structural bias.
is particularly so given the almost total lack of training in nutrition
and supplementation techniques in American medical schools on one
hand and a virtual stampede on the part of consumers to the use
of supplements, on the other.
bias of perspective
there were some minor glitches in data reporting (see sidebar describing
in data reporting), the article as a whole shows a more subtle
kind of bias. Questions were framed and presented from a perspective
that views vitamins and minerals as fundamentally different from
other kinds of dietary supplements. This bias led to survey responses
that cannot be relied upon as accurate information describing the
opinions of adult Americans concerning all dietary supplements.
do not believe this was done with an intent to mislead. The difficulty
lies in the way survey questions were designed and administered.
Author bias, always a concern, was clearly evident in this case.
As scientists and healers, doctors need to understand a wide range
of points of view, regardless of whether they agree with those views
ambiguous is the need to rationalize the current state and federal
regulatory framework surrounding supplements. Physicians in particular
are especially aware of supplement risks. People experiencing side
effects and adverse interactions resulting from inappropriate supplement
use show up in their offices, clinics, and hospitals.
are less likely to see supplement success stories unless a patient
explicitly says, "This supplement is really working for me!" Also,
bogus advertisements touting supplements for weight loss, muscle
gain, hair regrowth, and breast enhancement -- all of which sound
too good to be true and all using products described as natural
-- bombard the public from radio stations and TV infomercials.
stringent regulation seen as an easy fix
such an environment, it is easy to gain the impression that most
non-nutrient supplements are modern versions of snake oil. Equally
easy is the temptation to solve the problem of specious and misleading
advertising claims with the easy fix of more stringent regulation.
But regulatory regimes are fraught with complexity and unforeseen
consequences. We believe calls for further supplement regulation
are in fact a misguided effort to achieve laudable goals through
inappropriate methodology. When considered in their various aspects,
such concerns as doctor-patient communication, undocumented or unrealistic
benefit claims, toxic reactions and others are all susceptible to
rational solutions -- such as continuing education, databases, and
practice, such an approach requires planning and education as well
as the active enforcement of current law. But with the cooperation
of supplement manufacturers, medical practitioners and support from
Congress for funding enforcement of laws already on the books (such
as the Dietary Supplement Health and Education Act), the scenario
laid out by the authors of "Americans' Views on the Use and Regulation
of Dietary Supplements" can be put into its proper perspective.
In practice, we believe the development of quality standards, a
trend with increasing support among makers and users of supplements,
holds great promise for supplement consumers and health practitioners
FDA doesn't need more power to regulate, they need to focus
on what they're supposed to do."
-- Dr. Shari Lieberman (see full
text of her comments)
critique of the two surveys follows, accompanied by a description
of the potential consequences of more stringent regulation as opposed
to the benefits of the alternative solution of enhanced quality
use of the term dietary supplement
opinion surveys in "Americans' Views on the Use and Regulation of
Dietary Supplements" used the term dietary supplement in
a way that is not wholly accurate. This has serious implications
both for the results of the surveys and for conclusions based on
authors of the article say almost half (48%) of American adults
take "some type of nonprescription vitamin, dietary, or mineral
supplement regularly." However, only one in six (16%-18%) regularly
use "dietary supplements, like echinacea, ginseng, amino acids,
or over-the-counter hormones." Similarly, the two surveys at the
center of the article's main argument defined regular users
as people who "regularly use products, such as echinacea, ginseng,
over-the-counter hormones, or amino acids."
excluding vitamins and minerals from their definition of dietary
supplements, the surveys removed almost one-third (30%) of their
respondents from the category of regular user. In all likelihood,
this definition created a bias in the respondents' minds, which
in turn may well have affected the results and invalidated either
some or all of the researchers' conclusions.
on survey results
survey's regular users of vitamins and/or minerals may well
have responded differently if these nutrients had been included
in the definition of supplement. Respondents are especially
likely to have given different answers to the questions "How important
is access to supplements?" and "Are supplements good for health
and well-being?" The percentages of regular users, sometimes
users and nonusers who answered "yes" undoubtedly would
have been quite different.
asking the question about supplement use in this way, the surveys
also establish a meaning to the term supplement that is inaccurate
insofar as legal and regulatory definitions are concerned. As a
result, opinions expressed about "supplements" are not relevant
to the current regulatory environment. This is like asking people
about apples and then applying these opinions to discussions about
better design would be to ask respondents which supplements they
use and how often. Data from such a survey can be analyzed in a
wider context and would produce more accurate and relevant results.
means of generating more accurate information would have been to
ask each question again for each kind of supplement: vitamins/minerals/nutrients,
botanicals, sports nutrition, and so on. Americans may believe that
some kinds of supplements are more adequately tested, less likely
to cause harm, more truthfully advertised, and less in need of increased
regulation than other types.
called "mostly untested and unregulated products"
authors of the article consistently characterize supplements as
"mostly untested and unregulated products" -- which is untrue on
supplements are regulated under the provisions of the 1994 Dietary
Supplement Health and Education Act (DSHEA), which established supplements
as a category distinct from drugs (both prescription and over-the-counter),
and also distinct from both foods and food additives.
was passed to protect supplements from the opinion of the federal
Food and Drug Administration (FDA) that supplements are essentially
drugs and should undergo extensive pre-market testing to prove their
absolute safety and absolute efficacy before they can be marketed.
Prior to the passage of DSHEA, the FDA was raiding the offices of
doctors and supplement manufacturers and seizing supplements that
they claimed to be "mislabeled drugs."
effect of such action, especially in the face of sensational media
coverage, was to create a high level of anxiety among users of supplements.
Members of the medical community were subjected to federal charges
and otherwise held up to ridicule. Notably, dietary supplements
and those who make and use them were cast in a negative light. In
the midst of much pontificating of the "doctor knows best" variety,
the age-old use of supplements (botanicals) and more recent use
of vitamins and minerals in concentrated form came under sharp attack.
This context informed the thinking that resulted in passage of DSHEA.
contrast to the pre-market regulatory environment used for
drugs and some food additives, DSHEA established a post-market
framework for supplements. Supplement companies are still required
to have evidence that their products are both safe and effective
for the purposes claimed in their marketing claims and literature.
However, they are not required to present this evidence to the FDA
and gain the FDA's permission before placing products on the market.
Instead, the FDA has full authority to remove unsafe products from
the market. The FDA has stated in Congressional hearings that in
fact they have enough authority to do so. Companies are also required
to be truthful in their advertising, and the Federal Trade Commission
(FTC) has full authority to stop companies from making false claims.
a pamphlet for patients about herbs and anesthesia, the American
Society of Anesthesiologists states, "The government regulates herbal
medicines in the same way that it regulates food and nutritional
supplements, but herbal products do not undergo the same strict
research requirements as prescription drugs or over-the-counter
medications (like aspirin)."
authors of the article may believe that supplements are inadequately
regulated (which is a valid political opinion), but stating
that supplements are unregulated gives inaccurate and misleading
information to their colleagues and the public.
testing, according to the PubMed database maintained by the National
Library of Medicine, more than one million research studies have
been performed concerning the sixty-nine most popular supplements
(see sidebar tabulating citations
of published research). Almost half of this research was conducted
on minerals; another third was conducted on vitamins and amino acids.
Even though botanicals and substances like shark cartilage are much
less extensively researched than these nutrients, the ten botanicals
listed in a sidebar accompanying this article appear in 9,707 articles
in the PubMed database. Even many controversial supplements have
received substantial scientific attention: creatine showed 29,355
studies; glucosamine 13,281; melatonin 8,803; DHEA 6,850.
authors of the article may believe that these studies are flawed
in various ways or that more research is needed, but to call these
supplements untested again gives inaccurate and misleading
information to medical practitioners and their patients. There are
vastly more research studies on most supplements than the number
typically conducted to get FDA approval of a new drug!
called for simplistic answers
on the survey asked for simplistic yes/no answers -- which does
not allow for more complex and realistic responses.
the question "Are supplements good for health and well-being?",
173 people volunteered "some are, some are not," which was not among
the official answers but happens to be the most accurate one. On
this question, 52% of all respondents said "yes", 17% said "no",
another 17% said "some are, some are not", and 14% said they don't
more questions where the most accurate answer is "some are, some
are not" were: "Are supplements adequately tested?" and "Advertising
for supplements is not true, or generally true?"
to "How often are people harmed by supplements?" were grouped into
"often or sometimes", "rarely or never", and "do not know". The
terms often and sometimes are both vague terms that
mean quite different things to different people. Even the term rarely
can reflect a range of meanings. More to the point, the survey might
have asked respondents to estimate or guess how many people are
harmed by supplements in the course of a year.
users said to ignore scientific evidence
result that clearly surprised (and possibly alarmed) the authors
of the article is that two-thirds (66%) of supplement users would
continue to use "the supplement they use most often" even if the
FDA said the supplement is not effective. The researchers characterize
this result as "regular users believe strongly in the usefulness
of various dietary supplements irrespective of the scientific evidence."
used the term belief as if there were no research
demonstrating the benefits of supplements. In fact, there
are hundreds, sometimes thousands of studies . . ."
-- Dr. Shari Lieberman (see full
text of her comments)
we note the obvious in pointing out that "scientific evidence"
and "FDA says" are not the same thing. The FDA takes a very
conservative view of how much scientific evidence is enough to prove
efficacy; supplement users are more likely than nonusers to be aware
of this conservative stance. While this conservatism is appropriate
for a government agency (we wouldn't want the FDA to jump on and
off the bandwagon with every "bleeding edge" research report), individual
citizens are freer to draw their own conclusions.
we would suggest a different interpretation -- that people who experience
positive results from supplement use are most interested in a particular
kind of scientific evidence: the case study. They have asked, "Does
this supplement work for me (and/or family members and close
personal friends)?" And for the supplement(s) they use most often,
these people appear to have gotten an answer that satisfies them:
Yes, it works for me.
doctors are aware that people vary significantly in their body chemistry,
enough to respond differently to various drugs, supplements and
other treatments. Thus, doctors are vitally interested in a similar
case study: "Does the treatment I prescribed work for my patient?"
are unlikely to ask this question about supplements because the
curricula of most American medical and public health schools offer
little or no training in supplementation. Indeed, subsequent to
the Flexner reform of American medical education prior to World
War I, American medical education with a few notable exceptions
has focused exclusively on the allopathic medical model and divorced
itself entirely from non-European, non-Western academic medical
do not challenge the manifold benefits of modern medicine. However,
given the extreme reliance of American medical training on the German
model that Flexner relied upon so closely, future authors of studies
like "Americans' Views on the Use and Regulation of Dietary Supplements,"
might consider including other healing protocols and perspectives
in their surveys.
question of what constitutes rigorous scientific evidence for European,
Eastern and contemporary "alternative, complementary or integrative"
medicine is certainly a valid subject for academic debate. However,
it is unlikely that consumer preferences in supplementation are
driven entirely by false advertising claims of charlatans or "fountain
of youth cure-alls" in popular magazines. Perhaps the authors of
the study in question might more profitably explore the sources
of opinion that led Americans to spend over $16 billion dollars
in 2000 on supplementation.
about government regulation
a section devoted to regulation, the authors of the article say
many Americans are "confused" about the government's regulatory
role in supplements -- and state that 53% are "aware that supplements
are not regulated." However, the authors are themselves among
this 53% who mistakenly believe that supplements are not regulated!
This finding strongly suggests a need for better education of the
public and of medical professionals about existing supplement regulation.
second question in this section -- "Allow new dietary supplements
to be sold only if they have been tested by the FDA" -- says nothing
about undesirable consequences that would result from requiring
supplements to undergo pre-market screening. Consider another survey
question: "How much more money would you be willing to pay for supplements
so they can be tested by the FDA?" with possible answers of nothing
additional, twice as much, four times as much, and more.
One could imagine responses running parallel to the 81% "favor",
14% "oppose", and 5% "do not know" reported for allowing new supplements
to be sold only after gaining FDA approval.
minor aspect of regulation not mentioned in this question is that
the FDA does not do any testing itself. It reviews research performed
by private companies, and licenses private laboratories to conduct
third question -- "Remove dietary supplements from the market if
the FDA shows that they are unsafe" -- is a moot point for any discussion
of regulation. Under DSHEA, the FDA already has this authority.
safety, purity, and truth in advertising
section of the article resembles jokes about "When did you stop
beating your wife?" Of course Americans want to make sure
supplements are not harmful. Of course Americans want to
make sure supplements are pure and doses are consistent. Of course
Americans want to make sure advertising claims are true. A better
question is: Can more stringent regulation achieve these goals --
or is there a better and more cost effective way to achieve these
the consequences of requiring new supplements to go through the
same kind of FDA approval process that is required for drugs before
they can be marketed:
The cost of researching new supplements would skyrocket, innovation
would come to a crashing halt, and prices of new supplements would
are required to show both absolute proof of safety and absolute
proof of efficacy, a process that takes hundreds of millions
of dollars and years of research for each new drug that is approved.
If new supplements are held to these same standards, the cost of
development would be similar to that of a new drug. The FDA has
written extensive guidelines for approving botanical supplements
as drugs (only drugs are allowed to use health claims in their marketing).
These guidelines use essentially the same criteria as those for
pharmaceutical drugs. The authors of the article suggest that new
supplements should also be held to the same criteria as pharmaceuticals.
of pharmaceuticals are able to recoup the costs of drug research
because their products are unique, often synthetic substances that
can be patented. However, most supplements are derived from plants,
animals, and other natural products in ways that cannot be patented,
making it difficult if not impossible to support the expenses of
requiring absolute proof of both safety and efficacy is appropriate
for drugs, where the toxic dose is usually quite close to the therapeutic
dose and where side effects are common and often dangerous. One
wants to be very sure a dangerous substance is effective before
using it to treat patients.
supplements generally have a much wider range of safe dosages, as
well as fewer and less dangerous side effects. For this reason,
the European medical community -- often held up as a regulatory
model for Americans to follow -- also requires botanical products
to have absolute proof of safety, but only relative proof of
efficacy -- a standard that is easier for supplements to meet.
Sending every new supplement through an FDA pre-approval process
would create a logjam of delay.
FDA has stated that it does not have the financial resources to
investigate all questionable supplements, which are a small fraction
of the supplement market. Imagine these same sparse resources applied
to reviewing all new supplement applications. The FDA already receives
criticism for the delays involved in bringing promising new drugs
to market -- an area with vastly greater funding than supplements.
Useful supplements would sit in the pipeline for decades, gathering
current, post-market regulatory process is a more efficient use
of the FDA's funds for supplements. It allows the FDA to focus its
attention where it is needed most: on products whose safety or efficacy
post-market regulatory system does have its drawbacks. It is vulnerable
to fraud and quackery in ways that a pre-market regulatory environment
is not. However, the goals of safety, purity, and truth in advertising
can be achieved in a post-market environment. Two elements are key:
education and quality standards.
is the best prescription"
C. Everett Koop's often-quoted words that "knowledge is the best
prescription" are as applicable to supplements as they are to other
health concerns. When buyer beware is the watchword, consumers
need to know how to choose wisely.
are several simple actions that consumers can take to investigate
product quality and protect themselves from the most blatant varieties
of fraud. However, the biochemical complexities of supplements and
their actions in the body generally mean that recommendations about
supplements are most competently made by health professionals, rather
than by vaguely worded structure/function claims on supplement labels
or advice from health-food-store employees.
Dr. Alan Gaby (MD), a teacher of medical nutrition at Bastyr University,
says, "Patients want to get good advice from their doctors about
supplements. Doctors who practice nutritional therapy will have
patients lined up at the door wanting to see them."
of consumers and health care professionals is one important foundation
of a robust post-market regulatory environment. Having solid quality
standards is equally essential. Several organizations are currently
developing a variety of quality-certification programs; by creating
quality seals and other means of distinguishing product quality,
these initiatives will make it easier to drive most kinds of shoddy
products out of the marketplace.
IMPLICATIONS OF REGULATION
issue of supplement regulation could be debated from the comfort
of plush armchairs except for the context in which it takes place:
Americans pay a disproportionate price for health. Medical professionals,
pharmaceutical producers and health insurance providers assure us
that we are getting top value for our dollars -- but the question
remains: Why are Americans paying such an expensive premium for
the price of computer chips in the open marketplace: Technological
advances combined with plentiful supplies have caused prices to
fall dramatically even as capacity and performance have risen just
as dramatically. The technological complexities of making better
computer chips are no less challenging than those of understanding
the bioactivity of nutritional and botanical supplements. Could
similar advances in improved medical care at reduced cost be achieved
through the same kind of open-market-based regulatory framework?
This is the basic premise of DSHEA: that such advances are possible.
and governmental implications
a deeper question is what kind of society we want to have. American
culture and government are founded on the concept of freedom. Our
most precious documents establish the rights to govern ourselves,
to speak freely, to worship as we choose, and a host of other rights
under the umbrella of life, liberty and the pursuit of happiness.
Are consumers not entitled under this same umbrella to pursue a
variety of options for health and wellness? Is it possible that
a marketplace driven by patient needs would develop better, more
freedoms are restricted only insofar as it is necessary to protect
public safety and establish appropriate rules of interaction. While
the pharmaceutical regulatory structure is touted as necessary to
protect public health, it also forms a gateway that places price
floors under a specific class of commodity: pharmaceutical drugs.
While we all benefit from the fact that these drugs are protected
by patents, we also suffer from the exorbitant prices that result.
Generic alternatives do not become available until a patent expires,
although other patented drugs similar to the original often appear
before the original patent expires, some of them more effective
and at a lower price.
is a fundamental philosophical difference between the provider-centered
view that health is a product that patients buy from a health practitioner
and the patient-centered view that health is a state of being that
people achieve through their own actions. The provider-centered
view is equivalent to saying, "It's okay if you buy a Chevy but
not okay to go down to the auto parts store and get a carburetor
and install it yourself; you might get it wrong and go too fast
and wind up driving off a cliff."
the global medical community has many views of what a "health parts
store" consists of and what kinds of "carburetors" are available
and appropriate. Surely Americans are entitled to benefit from the
best knowledge available through a wide variety of resources, especially
since this country is rapidly becoming a hegemony of minorities
with medical traditions that are thousands of years old. Practitioners
within these traditions are highly experienced in using the very
botanical medicines that are regarded as so controversial by Western
Western medical practitioners used similar botanical medicines well
into the 20th century and many of our "modern" pharmaceutical drugs
are based on botanicals. At the same time, many of these same Western
medical practitioners whisper to their patients, "By the way, you
should probably take a multivitamin every day; I do."
nature of medical care
issue of how we approach supplement regulation goes to the core
concept of what constitutes medical care. Medicine can be viewed
as a nurturing and supportive process between practitioner and patient
-- rather than an isolated collection of technological knowledge
that a practitioner uses to "cure" a patient, much as a mechanic
"fixes" an automobile.
former, interactive view suggests four components to medical practice:
healing trauma, curing infectious disease, alleviating chronic health
conditions, and achieving wellness (as a positive state rather than
an absence of negative states). The Western allopathic approach
has done magnificently well in the first two: healing trauma and
curing infectious disease, but performs poorly on the second two.
Americans are saddled with chronic health problems, from heart disease
and obesity to diabetes and cancer. Millions are plagued with alcoholism
and drug addiction, allergies and asthma, arthritis, back pain,
chronic fatigue syndrome, osteoporosis -- the list goes on and on.
Many of these conditions do not respond well to Western allopathic
approaches, while other traditions are more successful with chronic
problems and basic wellness. This situation suggests why Americans
are turning to alternative medical practices with such enthusiasm.
forum and debate concerning supplement regulation
in health care just as in shoes, one size does not fit all. Patients
respond differently not only to a variety of drugs, but also to
a variety of treatments and basic protocols. Any discussion of supplement
regulation needs to take all these factors into account and address
the most fundamental question of all: What are the most cost-effective
methods of providing effective, appropriate healthcare that meets
the needs of the widely diverse American population?
good answers to this question means that all parties to this debate
need to understand the actual regulatory framework. Fixing a problem
requires that we understand its real nature and causes, not what
we imagine those causes to be. Physicians would unanimously agree
that diagnosis is the first step, and that successful treatment
depends upon making sure the diagnosis is an accurate one. For articles
about supplement regulation just as for heart disease and diabetes
and cancer, there is simply no room for bias and misunderstanding.
Society of Anesthesiologists. "What You Should Know About Herbal
Use and Anesthesia." Park Ridge IL, American Society of Anesthesiologists,
J. Blendon ScD, Catherine M. DesRoches DrPH, John M. Benson MS,
Mollyann Brodie PhD, Drew E. Altman PhD. "Americans' Views on the
Use and Regulation of Dietary Supplements." Archives of Internal
Medicine, Vol 161, No 6, 26 March 2001, pages 805-810.
and Drug Administration. Guidance for Industry: Botanical Drug Products
(Draft Guidance). Center for Drug Evaluation and Research, Food
and Drug Administration, US Department of Health and Human Services,