Part I: Using Science to filter out hype
“The most important thing about science is the scientific method – a way of thinking systematically; a way of assembling evidence and appraising it; a way of conducting experiments so as to predict accurately what will happen under given circumstances; a way of ascertaining and recognizing one’s errors; a way of finding the fallacies in long-held ideas. Science itself is constantly changing, largely as a result of the scientific method.”
Norman Cousins. Human Options. London: W.W. Norton and Co. Ltd. 1982
I. Association is not cause
How often have you seen this: “Joe Athlete takes Megamulti vitamins. He won a silver medal in the Olympics.” This implies that if you use that product, you will also become a champion athlete.
One only has to think this through to see that the association is meaningless. Joe also drank orange juice, and sang in the shower. Were these things also responsible for his success? In fact, it is equally possible that he won the silver medal in spite of the Megamulti, and had he not taken it, would have won a gold medal.
Some associations, of course, do represent a true cause and effect relationship, and this is where the scientific method comes in. The first step in identifying a true cause and effect relationship is to apply the converse test. Let’s say that I tell you that the sun rises every morning because I play my guitar. As proof of this claim, I wake you up at 5:30 AM, play my guitar, and the sun in fact does rise. The healthy skeptic will say, “all right, tomorrow morning do not to play your guitar. Then let’s see whether the sun rises or not.”
The truth of that association will be clearly evident.
The same strategy can be used to examine some nutritional claims. Someone tells you that the natives of South America have a low incidence of heart disease because they eat a certain herb. You can inform them that Greenland Eskimos (who obviously do not consume this herb) have equal or better statistics regarding heart disease.
The dairy industry came up with egg on their face a while back trying to make association look like cause. When it was found that vegetarian women have a lower incidence of osteoporosis than the general population, the dairy people said, “of course, this is because vegetarians rely on milk products for protein.” A number of scientists, however, used the converse rule to point out that association, in this case, is not cause. As it turns out, vegans (vegetarians who do not use dairy products) also have lower osteoporosis risk. Oops.
Advertisers that base their promotion purely on testimonials are relying on the human tendency to assume a causative relationship between the person’s athletic skill, beauty, or fame and the product being advertised. Remember that no matter who is recommending the product, you need more information before you can determine whether the association is coincidental or causative. If the manufacturer won’t give you this information (valid scientific studies) you can assume the promotion is misleading.
Entire populations fall into this trap. Look at the association between a high fat diet and heart attacks. Nearly everyone will tell you that’s a clear cause and effect relationship. Not true. Evidence to the contrary is plentiful. Israelis consume one eighth the fat of Mexicans, and have more than twice the number of heart attacks per capita. Nathan Pritikin was forever touting the low-fat (15% of total calories) diet of the Tarahumara Indians, but they are no better off than the residents of Stromboli, Italy where 40% of calories comes from fat. The all-time best example refuting the simplistic idea that high fat means heart attack are Greenland Eskimos, who have a very low incidence of heart disease, even with a diet where 60% of total calories come from fat.
We now know that the association between diet, lifestyle and heart disease must include:
1. The type of fat, not just the amount. 5. Exercise
2. Intake of antioxidants 6. Blood pressure
3. Emotional and psychological stress factors. 7. Cigarette smoking
4. Blood cholesterol, and HDL 8. Inflammation
Case in point
An internet doctor rants about the dangers of sucralose, a widely used and internationally approved artificial sweetener. His argument is based primarily on patients complaining of a variety of symptoms after they consume a sucralose-containing food. What’s wrong with this argument?
Answer: It’s unscientific.
1. Patients know he is adamantly opposed to the use of artificial sweeteners. They are thus primed to associate their complaint with ingestion of sucralose.
2. It is impossible to determine if the complaint resulted from sucralose or some other ingredient or food in the meal… or sun spots for that matter.
3. In situations like this – where there are strong beliefs and opinions, the ONLY way to discern truth is through a double-blind placebo-controlled study. In this protocol, people would be given either sucralose or a look-alike placebo (eg rice powder). Importantly, neither the subject nor the administrator would know who got which (thus the term “double blind.”). At the end of the trial period, results would be compared. If sucralose produced significantly more side effects than placebo, internet doctor would have evidence that sucralose is harmful to some people. Without such evidence, internet rants against sucralose should be regarded as mere opinions.
2. Incomplete Picture
Good science is always thorough. When experiments are conducted carefully, and results are reported precisely, the result is always a “success”. Even negative results , help to illuminate the question. That is the wonderful thing about objectivity. And to arrive at valid conclusions, you need to know all the facts.
A recent nutrition magazine, for example, announced that “GLA Prevents Cancer” and included references to a study from the Medical University of South Africa. Looking carefully at the article, we find that when GLA (an essential fatty acid) was added to cultures of cancer cells in test tubes, the cancer cells stopped multiplying.
This is very interesting, and points to an important connection between fat metabolism and cancer. But the article’s conclusion that taking GLA supplements will prevent cancer is unscientific and dishonest. After all, adding gasoline to the same culture would stop cell division almost instantly. Does that mean we should drink gasoline?
This brings up an important point regarding in vitro (test-tube) research vs in vivo (livng organism) research. Most breakthroughs start with in vitro testing because it is cheaper, easier and faster. But in vitro results have to be confirmed with in vivo testing, because one simply cannot assume that what happens in a test-tube will also happen in a living organism. Look carefully at supportive evidence and disregard products relying on in vitro testing.
Of course, even in vivo research can fall into the category of Incomplete Picture. If a mouse model was used to test a nutritional product, look to make sure that the material was fed to the mice. After all, you’re not going to inject this product, you’re going to eat and digest it.
The Complete Picture also means that both positive and negative sides are explored. Since advertisers obviously will not do this, the consumer must seek out a balanced point of view. We are often told that we should eat a particular substance because it contains certain nutrients. Is that the whole picture? We are advised, for example, to eat brewers yeast because it is rich in B vitamins, and blackstrap molasses, because it contains iron.
The problem is that brewers yeast is fairly allergenic, and sensitive individuals may find it more harmful than beneficial. Blackstrap molasses is a refined, simple carbohydrate, a by-product of the sugar industry, and many individuals cannot tolerate such concentrated sugars. For them, a little iron is not worth the resulting metabolic stress.
This is not to say that brewers yeast and blackstrap molasses are bad, only that their nutritive contributions must be weighed against the possible disadvantages. We need to get the complete picture. By the way, since sugar cane is not rich in iron, where do you think the iron comes from that is found in blackstrap molasses? Most likely it comes from the machinery used to make the product!
Remember this the next time an enthusiastic salesperson recommends a product based solely on its nutrient content: dirt contains an enormous variety of minerals, some vitamins, and even a few amino acids (made by bacteria). Suggest that they consume two tablespoons of soil each day.
3. Antique Thinking
The common appeal to “the good old days” is more emotional than scientific. Just because a product or practice has been used for a long time does not mean that it has value. In fact, one might well argue that the opposite is true. A careful look at
“grandma’s day” reveals a time of precarious health and a quality of life that few would envy today. There were, no doubt, distinct advantages, not the least being that food was unrefined and unchemicalized. But to obtain the best of both worlds, we need to take the valuable lessons from the past, and add to them the blessings and knowledge of the present.
I’m always amused by the phrase “ancient wisdom.” Not because the ancients were unwise, but because these individuals were operating from an extremely limited knowledge base. With no understanding of circulation, respiration, metabolism or immunity, they simply explained how the body worked in terms that they could understand.
Occasionally one of my students will accuse me of being “Eurocentric,” but the scientific method was not developed by individuals from one cuilture or nation. It was hammered out by systematic thinkers on five continents over many centuries, and that method is still creating a revolution in human understanding. Consider these examples.
In the American Southwest, Indian tribes developed certain treatments for rattlesnake bites. These treatments usually included ritual incantations or chants. Scientists were curious as to how these chants developed. In other words, when a person is bitten by a rattlesnake how did the healer or medicine man know what words to chant? How did that become accepted wisdom?
It turns out that when a rattlesnake bites, it very often does not deliver venom into the wound. Often the bite isn’t deep enough. Sometimes the snake is simply out of venom. The angle of the strike is also a factor. But all of this must have been unknown to the ancients. In their experience, when a rattlesnake bites, the victim is supposed to die. When the person didn’t die, the tribe naturally ascribed the “miracle” to whatever incantation and ritual was performed.
Another excellent example was reported in the Journal of the American Medical Association.[ii] It discusses a common parasitic infection among primitive tribes in Ghana. Western scientists discovered that the parasite (guinea worm) was being spread through contaminated water, but it was nearly impossible to persuade the villagers of this because of their long held belief that the parasite is an innate part of human anatomy. In other words, they had lived for so long with this disease that they had accepted it as an inevitable feature of living. They believed that the worm was “in the blood” and nothing could be done about it. So much for ancient wisdom.
Of course, you don’t have to go back to primitive times. Antique thinking is evident in a great deal of nutrition information available today. It is amazing to me how, in our wistful desire to have perfection (even if only in the past), we confer greatness upon people who, by objective standards, were merely fanatics with vivid imaginations.
Such notables as Sylvester Graham and “Professor” Arnold Ehret, produced outlandish nonsense in this century, and still have acolytes promoting their “therapies”, singing their praises, and selling their books.
The point is, these groups have had nearly a hundred years to demonstrate the validity of such old fashioned wisdom, and in most cases, we’re still waiting. Ehret, for example, was a professor of art, and self-proclaimed medical authority, obsessed with mucus, intestinal waste and “toxins”. He claimed that fever results from the friction of excess waste circulating in the bloodstream, toothaches are caused by eating too much meat, hair turns gray because it is filled with intestinal gas, and mental illness eventually results from this “gas pressure on the brain.”
Sylvester Graham was a religious reformer turned nutrition expert, who ended up mixing the two together by announcing that cholera was caused by chicken pie and “excessive lewdness”. It is astonishing that these books are still sold in health food stores, and are promoted as scientific literature. References to these bizarre theories, as well as arguments for products and practices based upon “old fashioned goodness” should more rightly be called ANTIQUE THINKING.
4. Fantasy Physiology
Often, you will find promotion for products and theories based on physiological effects. Individuals who express these views, however, do not always have a clear understanding of how the human body functions, and in some cases, deliberately distort scientific facts. It is a good idea, therefore, to check with an authoritative source before you buy.
Numerous examples of fantasy physiology can be found in one of America’s all time best selling diet book, Fit For Life. The authors, neither of whom had formal training in anatomy or physiology, promoted an intricate system of “food combining” rules to ensure proper digestion and weight loss. It is not surprising that they trace the roots of their “system” to Sylvester Graham. To the average reader, their arguments seem logical, but to anyone trained in physiology, whether a doctor, biochemist or college student, Fit For Life’s view of human digestion is absurd and indefensible.
Fantasy Physiology is often based on misintrepation of scientific principles. When there is absolutely no reliable evidence for a bizarre health claim, I often hear the presenter ascribe the effect to “quantum physics” or “biomagnetic energy.” While the human nervous system does utilize minute biomagnetic fields, the promotion of nutritional supplements to augment that function is preposterous. One product claiming to enhance the body’s “electric fields” turned out to be nothing more than a mixture of herbs and vitamins. Even an extremely sensitive measuring device failed to detect any magnetic quality to the product, and when I contacted the manufacturer, I was told that this was “too subtle to measure.” Does this remind you of a story called The Emperors New clothes?
5. Unreliable References
As children, we probably all took advantage of this ploy: shifting responsibility for our behavior or opinion to another person who was supposedly well-informed. Unfortunately, the scheme is still commonly used to sell controversial products or points of view. The fact that “Dr. So and So” recommends a vitamin, for example, means nothing unless the Doctor presents valid scientific evidence of the product’s
worth. It means less than nothing when you discover that the “Doctor” has a Ph.D. in music history or economics.
Product claims and points of vew are frequently supported by vague and obscure “authorities”, or studies performed at some unknown location. How often have you heard a statement that references “they?” eg, “When they watered houseplants with microwaved water, the plants died within three days.” Who is “they,” and where was this supposed study performed? In fact, this microwave “experiment” is a hoax; yet it appears on hundreds of websites and is probably believed by millions of health seekers.
Then there are the promotions citing Russian or Chinese athletes or their coaches as authorities. Thankfully, this ploy is being used less today as evidence has surfaced that the “miracle herbs” were often anabolic steroids and other performance-enhancing drugs. [iii]
Demanding reliable references is your first consumer right, yet the vast majority of popular health writers do not bother to support even the most outlandish claims. This is in sharp contrast to medical literature in which all controversial statements are supported by cited references. In scientific literature, the author’s credentials and affiliations are presented at the beginning of the article, and readers are never asked to suspend their critical viewpoint. When I wrote Caffeine Blues in 1998, I realized that the topic (that excessive caffeine is harmful) was controversial. Therefore I cited every arguable point, including 656 references from published biomedical literature. Likewise, The Metabolic Plan (2003) included 300 direct references.
Frequently, promoters of dubious nutritional products or theories become offended when challenged, claiming that they are victims of an orthodox conspiracy. This is absurd. The medical/scientific “establishment” may be conservative, but that does not mean that every whacko idea is therefore valid. There is only one route by which physiological truth is revealed and that is the scientific method; not the development of cults and schools of thought led by pseudo-scientists with persecution complexes.
Good examples of properly supported nutritional products include Coenzyme Q10, EPA/DHA fish oil, anti-inflammatory flavonoids from scutellaria and acacia, DHEA, lycopene, resveratrol, aloe vera, alpha ketoglutarate, green tea, berry and beet concentrates, rhodiola, garlic, panax ginseng and curcumin. These are considered “breakthrough” health products because they have shown significant value in careful research and confirming trials. Risks and benefits have been analyzed objectively with scientific integrity. Compare this to the popular herbal tea manufacturer who, without any scientific evidence, announced in a book that his product cures cancer. When criticized for this outrageous breach of scientific ethics, he claims to be the victim of a plot to suppress genuine cancer therapies. I believe that as consumers become more sophisticated, manufacturers will no longer be able to profit from such questionable tactics.
6. Simplistic Thinking
Probably the most common trap, into which we all fall occasionally, is simplistic thinking; making unreasonable assumptions or taking concepts which are known to be true, and extending them beyond the limitations of reason.
We know for example that vitamin E is necessary for normal fertility and reproduction. But claims are often made that taking vitamin E supplements will increase a man’s virility. At first, this may sound reasonable, but the same argument in a different context will illustrate the error: Spark plugs are necessary for your car’s engine to run (true). Is it therefore true that by putting more spark plugs in your car (perhaps in the glove box), your car will run faster?
Simplistic thinking is so common, in part because the human body is so incredibly complex. We try, in other words, to reduce that complexity down to manageable and understandable pieces, but often find ourselves in the error of simplistic thinking. Because the plaque that blocks an artery contains cholesterol, everyone thought that cholesterol was the cause of heart disease. Turns out to be a series of events that occur long before cholesterol enters the picture. Likewise, osteoporosis has been reduced to a calcium deficiency, when in fact, weight-bearing exercise, hormone balance and vitamin D all appear to be more important.
Simplistic thinking can be expensive. When scientists discovered leptin, the protein messenger that controls appetite, they assumed that obese individuals just didn’t make enough of it. Almost instantly, these scientists were paid $20 million by a drug company for the rights to manufacture a synthetic version of leptin. But less than 90 days later, follow-up research disclosed that obese individuals manufacture plenty of leptin. Their cell receptors are are (for some reason) less receptive to the biochemical signal. (oops!)
Tips on avoiding simplistic thinking.
1. Ask for confirmation. It’s been said before that if something sounds too good to be true, it probably is. That doesn’t mean that the product being pitched to you has to have appeared in the Journal of the American Medical Association. But there should be confirming data somewhere that indicates that what the company is saying is true. Take for example, a common claim made by manufacturers of nutritional supplements that their product is ”95% absorbed.” Since that is a nearly impossible claim (nutrient absorption is influenced by multiple factors that are difficult to control) I always call the company for documentation. To date, I have not received a single scientific study backing such claims.
2. See if the information conforms to nature. The acid test for most health claims is to see if the information fits with what we know about nature. If someone is pitching a weight loss or “energy” product containing caffeine, ask yourself if people are tired or overweight due to a caffeine deficiency. Likewise, promoters of spray vitamins should be asked if spray vitamins are so incredible, why did nature design our bodies to obtain nutrients through the process of digestion?
Machines have been sold for years that produce “alkaline water”, but there is no credible evidence for their value. Aside from the fact that these machines are sold with unreliable references (see # 5) you can ask the seller a very pointed (and relevant) quesiton: “Where on earth does water come out of the ground (or from the sky) with a pH of 9.5?”
7. Magical thinking
When there is absolutely no valid scientific basis for a product, advertisers often turn to the last resort: magical thinking. This usually involves developing an aura of mystery around the product by claiming that it comes from an exotic, far off place, or is used by individuals of legendary health, like the Hunzas. Thousands of products today are promoted with words like legendary, ancient, and secret formula. An herb tea sold in health food stores for decades was claimed to contain a secret “special spice” from Singapore, while another product of the same manufacturer contained a mysterious ingredient from Tibet. Ascribing curative properties to such products, in the absence of real scientific evidence, is called magical thinking.
The tip-off regarding magial Thinking is when the promoter claims that the product is beyond the ability of science to evaluate. They create a veil between what can and cannot be measured, and claim that only special people can see behind this veil. Usually these include long-dead inventors like Nicola Tesla, Reinhold Voll, Ana Aslan (famous for Gerovital) or a cadre of Russian scientists.
No reasonable scientist will automatically dicount claims made for a product or a machine that appears to operate on principles that are poorly understood. But here’s the big truth: the promoters of these devices are making claims that CAN be tested. So in reality, in doesn’t matter if the mechanism of action is mysterious or mundane. Let’s see if the product or device can produce the claimed benefit under blind conditions.
For example, I am the Chief Science Officer for a company with about 15,000 active distributors. So I get pitched by all the manufacturers who want to sell their heath gizmos, most of which measure “subtle energy,” Chi, or a person’s aura. Promotional material is peppered with magical words like meridians, holographic, scalar energy, bioelectric and my favorite, quantum health analyser – of which there are about a dozen for sale on the internet.
I do not purport to know everything. But I am quite confident that if similar results are not obtaned under blind conditions, the device has to be worthless. So I always ask for a demonsration and offer a test subject – one of my colleagues with a serious health disorder. Over the course of three decades, not one of these gizmos has correctly identified the disorder, whether it be kidney disease, a brain tumor or even type 1 diabetes. The promoter always focuses on general or vague issues like adrenal exhaustion, toxins, parasites or yeast overgrowth.
Since there are reliable tests for all of these conditions, I cannot understand the appeal of these gizmos, other than our penchant for magical thinking and the hope that someone can pick up on subtle defects before they become serious illness. And that of course is the out for the testing practitioner. If they can convince you that they have discovered a subtle juju deficiency, and you fix it with whatever nostrum they are selling, and you don’t suffer the serious illness, it appears that their therapy “worked.”
Such is the case with all the devices (usually nothing more than a piece of plastic) that you attach to your phone to protect you from electromagnetic fields (EMF’s). When you don’t get a brain tumor, it obviously was because you were protected by the gizmo. Ha!
8. The Numbers game
You pick up a 1.6 oz Nature Valley Granola Bar, and read the label claim for 110 calories. “Not bad,” you think. But later in the label you see that the package contains two servings. That means your 1.6 oz snack really cost you 220 calories, and you are a victim of the Numbers Game. Check out the serving size on a can of soup or a small bag of chips. No one eats one half of a small can of soup. No one eats 1/2 oz (a small handful) of chips. Manufacturers create artificially small serving sizes to hide the enormous amount of fat, calories and sodium in the product.
Vitamin manufacturers have also learned the numbers game, usually as a way of hiding the actual ingredient quantity. Some manufacturers, for example, list fiber in milligrams instead of grams. After all, 500 mg sounds a lot more impressive than one half of one gram.
Then there’s label padding; including tiny amounts of a substance in a product just to list it on the label. An all-in-one vitamin beverage touts the number of ingredients, but places many of them in “blends.” In the “antioxidant blend,” they list vitamin C and CoQ10. Since vitamin C costs about $6.00 per pound and CoQ10 costs a hundred times that, wouldn’t it be good to know the actual level of CoQ10? According to the manufacturer, that’s a trade secret.
Herbs are often used for label padding. Garlic is an extraordinarily healthful herb. There is abundant evidence that raw garlic is an immune stimulant and can help lower blood pressure. What value, however, could possibly be gained from including 25 mg (or less) of highly processed garlic powder? Often, the amount of an herb is not even given. In cases where the label reads: “In a base containing garlic, alfalfa, kelp, spirulina, golden seal etc.,” you can expect everything in the list to be present in amounts under 10 mg.
How much is 10 mg of an herb? Picture the contents of one tea bag. One third of that is approximately 1 gram. Now divide that third into a hundred tiny portions. One of those portions is 10 mg, a little bigger than a speck.
9. It’s in the PDR!
Lately, it has become fashionable for manufacturers to list their nutritional products in the Physicians’ Desk Reference (PDR). They then advertise this fact, implying that it somehow makes their product “approved” or superior. In reality, PDR is simply a compilation of information written and paid for by pharmaceutical companies. Drug companies, after all, want doctors to have accurate information so they will prescribe their products. But inclusion in the PDR is meaningless as a measure of quality. In fact, the publishers make this quite clear in the Foreword:
“Product descriptions have been prepared by manufacturers…In organizing and presenting the material in PHYSICIANS’ DESK REFERENCE ®, the Publisher is providing all the information made available to PDR® by manufacturers. In making this material available, it should be understood that the Publisher is not advocating the use of any product described.”[iv]
Very often, you will read a brochure or advertisement which presents fairly complex information, but when you sit back and think about it, you really don’t have a clear picture of what you read. Most people automatically blame themselves for being “too dumb,” but more than likely, it’s nutribabble. Manufacturers actually create complicated, impressive-sounding copy to make you feel dumb. It is easier to manipulate people that way.
Example. From the brochure of a nationally distributed line of vitamin supplements.
When I called the company to find out more, I was told that it was “a technical process we don’t like to talk about because it’s not yet patented.” When I explained that I was not interested in trade secrets, but the concept behind this revolutionary process, the representative told me, “It works a little like homeopathy.” Since no one knows how homeopathy works, this is essentially nutribabble.
Putting it all together
I presented these ten principles to my college students, and asked them to find examples in a brochure from a local health foods store. The product advertised was royal jelly, a glandular secretion of honey bees, which apparently determines which egg will become a queen, and which will become a normal worker or drone. The substance is therefore advertised as a “nutritive tonic”, and a host of health benefits are cited. The students labeled this simplistic thinking, noting that its importance to bees has nothing to do with its value to humans.
The brochure cited research conducted at Peking Medical College, but did not give the date or the name of the researchers. The students quickly labeled this unreliable reference. It also stated that royal jelly is a “rich storehouse of B vitamins”, contains all eight essential amino acids, and important minerals. The students called this incomplete picture, observing that no amounts of these substances were given. They also pointed out that since royal jelly is extremely expensive, and the normal serving is only about a teaspoon, it is not likely to be a practical source of B vitamins.
The brochure quoted “one of the world’s foremost authorities on royal jelly”. The students labeled this unreliable reference for two reasons. This person’s credentials are not given, and we have no way of knowing why he is considered an expert. Secondly, this “foremost authority” did not present scientific support for the product, but gave instead testimonials from others. This was labeled Anecdotal Evidence. Furthermore, his claims for royal jelly’s “youth restoring properties” were labeled magical thinking, as simple buzzwords with no clear definition.
Note: While we’re on the subject, I want to point out the essential flaw of the royal jelly story. Promotion of royal jelly depends, for the most part, on the fact that when it’s fed to a normal female larva, that larva grows into a queen bee. Queen bees live 5 years. Drones and workers live only a few weeks. The implication is that if you take royal jelly, you’ll live longer. This is a clear case of Association is not Cause. We now know that an enzyme in the royal jelly acts on a specific coenzyme in the bee larva, leading the to production of a group of proteins that replicate queen bee DNA. Since humans obviously do not possess this coenzyme, royal jelly will have no effect on longevity. Good thing. If this material was biologically active in humans, there’s no telling what effects it might have.
1. Thomas P. FDA moves to limit marketing claims about vitamins, minerals and herbs. The Wall Street Journal, December 15, 1993.[ii] JAMA 1995; 274(18):1412[iii] A number of Chinese swimmers were removed from the 1997 World Championships after authorities found banned substances in their luggage. Previously, their coach had ascribed their remarkable success to herbs such as goji berries and cordyceps mushrooms.[iv] Physicians’ Desk Reference, 43rd edition. Medical Economics Co., Oradell, N.J. 1989. Foreword