Most people learn of medical progress through the media. Whether through short clips on radio or TV or detailed articles in the press, scarcely a day passes without a report of a health development, and how it might affect you. Yet this news is often unhelpful. Exaggerated cures, contradictions, and plainly misleading information can do harm.
While health care policy is a public issue and freedom of the press mandates the free dissemination of health news, misinformation may be expensive, personally harmful, and detract from issues that are more important.
While public education has changed attitudes towards smoking, seat belts, and impaired driving, much reported information is confusing and counterproductive. One day, margarine is safer than butter, but later we learn that it too may harm arteries. Dietary fiber was once thought to prevent colon cancer, but now we are not so sure.
Some reports, but not others suggest saccharin, a boon to diabetics, causes bladder cancer. Such contradictions perplex those unfamiliar with the workings of science. The problem is not the science, but how journalists report it, and how the public interprets it. This essay aims to help readers make sense of health news.
Understand the Scientific Method
One need not have a science background to understand how science progresses. While some medical science is self-evident (the setting of a fracture is one example), most medical facts are established through experiment. In clinical medicine, this means experiments with living persons. In a randomized clinical trial (RCT) of a new drug, individuals with a certain disease are randomly distributed into two groups. Those in one group receive the treatment to be tested, and the others receive a placebo. Neither the subjects nor the researchers know the membership of the groups. If the treatment group is more improved at the end of the trial, the difference is called the therapeutic gain, and if it is sufficiently large, the drug is deemed effective.
However, many things may go wrong with such an experiment. There may be insufficient numbers to make a conclusion; there may be younger patients in the treatment group; patients or doctors may inadvertently find out which patients are on the drug; or there may be a practical or statistical flaw in the trial design. A trial absolutely free of bias is probably impossible. Moreover, the patients selected for the trial may be unrepresentative of the sick population to be treated, side effects may nullify the treatment’s benefit, or similar trials may show contradictory results.
Short-term clinical trials of a treatment are difficult enough, but consider the experimental difficulties inherent in determining whether certain diets, environmental factors, or lifestyles cause or prevent disease. Such studies require the recruitment of dissimilar human beings for observation over years or decades. Usually there is no “treatment,” just an estimation of the degree of exposure of each subject to a diet, toxin, or other environmental factor. This is then related to a health outcome.
Such a study makes several doubtful assumptions: that other determinants of health such as genes, smoking, gender, and occupation are equally distributed among the subjects and will not bias the results; that the estimation of the exposure is accurate and sustained throughout the long study; that no pre-existing risk was overlooked; and that the population studied was such that the results can be generally applied. One report compared colon cancer in Copenhagen and Helsinki, and concluded that the higher incidence in the former was due to the insufficient intake of dietary fiber. This ignored environmental, genetic, and other differences in these two populations, but contemporary press reports encouraged the notion that fiber can prevent colon cancer.
Thousands of medical journals each publish hundreds of scientific papers annually. Only a few of these report a scientific advance, and many of the remainder are flawed, biased, or irrelevant. Very few studies are pivotal, and few valid conclusions rest on a single report. Scientific articles are working documents, subject to criticism, revision, contradictory studies and, for some, eventual confirmation. Each represents a piece of a puzzle which put together over years may reveal an important fact. Smoking as a cause for cancer was not a sudden revelation half a century ago, but rather the building of evidence over many years eventually to inform public health policy. One should interpret media reports accordingly.
Readers should be aware of obvious biases. A cold lasts 5 to 7 days, so any treatment given at the right time will seem to be effective. Those making specific lifestyle or diet changes may simply be the more health conscious among us who also eschew smoking, use seatbelts, and get regular exercise. Bias is almost impossible to eradicate in long-term population studies, so evidence from many sources is required before drawing conclusions.
There are few absolutes in Medicine. Rather, data are interpreted in terms of probability. Suppose a certain diet, if taken over 5 years reduces the chance of death during that period by 2 percent. Note that this is not a large figure – possibly an observation due to chance. Secondly, it suggests that after 5 years, two more of 100 individuals taking the diet will be alive than another 100 not on the diet. Thirdly, many taking the diet will die anyway, and many not on the diet will survive. If the diet is complex, hard to achieve, and expensive over a lifetime, you would want to be certain the data underlying the conclusion was valid before undertaking it. Finally, switching late in life to a diet others have followed for years seems unlikely to benefit you. Risk assessment is difficult and most diet and lifestyle data are works in progress.
Information derived from one population may not apply to another. Genetics, sex, environment, age, nationality, race, and many other factors may have diverse effects on the disease under study. Diet and blood cholesterol levels may have very different impacts upon smokers and diabetics than others.
Science progresses in increments, punctuated by setbacks and only the occasional discovery heralds improved diagnosis or treatment. The “truth” is realized eventually by consensus, based upon data from several sources. Such caveats seldom are declared in media reports, or indeed by the researchers themselves.
Understand the Media
Reporters assigned to the health beat of a broadcaster or newspaper must submit to regular headlines and absolute deadlines. Science seldom obliges with a daily output of exciting news. Pressures to fill space and dramatize the subject are not in the interests of sober reporting. When reading media reports consider the following:
The Nature of the Media – Some news media and broadcasts inspire more trust than others do. Readers should judge the medium’s reputation and independence. Reliability and critical appraisal seems more likely in a national newspaper than in tabloids at the supermarket checkout. Does the outlet have any connection to the news item? For example, does the same issue advertise the reported ‘beneficial’ product?
The Credibility of the Journalist – Does the journalist have a science background or any credential that might assist interpretation of scientific data to the public? A careful reporter will discuss the cons as well as the pros of study results. A good journalist interviews critics as well as the scientists that produce the data. Journalists should try to put research in context and avoid over interpreting results.
The Source of the Information – Readers should insist upon knowing the source of a journalist’s report. For example, pharmaceutical spokespersons are potentially biased sources of drug information. Normally, the researchers producing the data will first present their work at a scientific meeting. This is an important part of the process, but details of the study are sketchy in a 10-minute presentation and peer review is minimal. The real test is when the researchers publish their material in a scientific journal following careful scrutiny by reviewers and editors. The exact source(s) of published data should be included in the media article so the reader may judge their quality, and even look up the original report. Ideally, the journalist should take into account any accompanying editorial and critical letters to the editors in subsequent editions of the scientific journal. A cautious reporter will detail the need for further study to clarify new findings.
Beware of Extravagant Claims – Caveat emptor (Let the buyer beware)! If a news report of medical data seems too good or too outrageous to be true, it probably is. Beware particularly of confusing coincidence with causality. When two phenomena commonly occur together, it does not prove that one causes the other. The human condition is far too complicated to attribute certain diets to certain diseases based on a single observation. Too many other factors are at work. In a proper experiment, researchers try to neutralize or control these so that they do not obscure the results. Of course, associations are important to help generate hypotheses for future studies, but they are seldom worthy of headlines.
A misleading media report may be betrayed by the journalist’s language, or the editor’s headline. Most treatments improve a person’s health, so one should distrust the word “cure.” Despite several instances of dramatic discoveries over the last century (e.g., insulin, penicillin, polio vaccine and a bacterial cause for peptic ulcer), most scientific research is incremental. Contemporary heart treatments, drugs for Crohn’s disease, immunity, and the human genome are taking years to establish. Terms such as “breakthrough,” or “revolutionary” also should be distrusted also. Remember, bloodletting was a common treatment for many diseases for centuries. Its use was due to unwarranted enthusiasm, the placebo effect and the tendency of many diseases to improve. Slow, plodding, and skeptical science is the only sure protection we have against such an outrage.
Beware the Anecdote – Most of us have a friend or have heard of someone who claims to be cured by a treatment that has no scientific basis. Faith-based treatments are not necessarily harmful, but if they have adverse effects, or delay effective treatment, they can be so. Journalism thrives on the anecdote, the human-interest story – the attention-grabbing incident that “proves” the point. Nevertheless, anecdotes are poor science. The human condition is far too complex for us to believe that if a treatment works for someone, it is more likely than by chance to work in ourselves. If there must be anecdotes, they should illustrate all possible outcomes.
If there were no placebo effect, and the natural course of diseases and human lives were predictable, there would be no need for clinical trials. The anecdote would reign supreme in medical decision-making. What happened once, in the same circumstance would happen again. However, we need the careful accumulation of medical evidence with all its detours and false starts if we are to understand what benefits our health and what does not. Even if untrained in science, readers can try to understand the scientific method and learn what to believe from media reports. The evidence-based medicine movement aims to remove chance from medical treatments. A healthy skepticism with rejection of sensational health headlines offers some protection against unscientific and exaggerated claims.
Angell M, Kassirer JP. Clinical research – what should the public believe? New Engl J Med 1994; 331:189-191.
Fitzgerald FT. The tyranny of health. New Engl J Med 1994; 331:196-199. Thompson WG. The Placebo Effect in Health: combining science and compassionate care. New York: Prometheus, 2005.