In our history there has been a rash of junk science in pursuit of a political agenda, such is the case of the science behind the smoking ban. The vast majority of the studies clearly show no statistical significance. The courts will not accept anything less then a relative risk less then 2 as causation for any disease and yet we are passing laws on much smaller results. Also if 1 is included in the CI the results are considered statistically insignificant. Here is a table so that you can judge for yourself. These facts did not stop anti-smoking activists who think that they have collective rights not in the constitution.
No they used a questionable method called Meta Analysis .Much has been written about this questionable method. For one thing there is no set standard as to how much weight to give each study and the other is publication bias.
If meta analysis is to have a future, perhaps it will be due to registries that allow studies to be tracked from their inception and not just on publication. This would make it impossible to publish the results of trials showing benefit while suppressing those that do not. Many medical journals now require that studies be registered at ClinicalTrials.gov before they start in order to be eligible for publication. This may eventually solve the problem of publication bias. It would not capture everything, but it would capture prospectively a (large enough?) cohort of studies. I am less confident of whether it will be able to address the differing quality of the studies or subtle differences in the way outcomes are studied.
At the moment, it is difficult to improve upon the remarks of John C. Bailar, III, taken from his letter to The New England Journal of Medicine, 338 (1998), 62, in response to letters regarding LeLorier et al. (1997), “Discrepancies Between Meta-Analyses and Subsequent Large Randomized, Controlled Trials”, NEJM, 337, 536-542 and his (Bailar’s) accompanying editorial, 559-561:
My objections to meta-analysis are purely pragmatic. It does not work nearly as well as we might want it to work. The problems are so deep and so numerous that the results are simply not reliable. The work of LeLorier et al. adds to the evidence that meta-analysis simply does not work very well in practice.
As it is practiced and as it is reported in our leading journals, meta-analysis is often deeply flawed. Many people cite high-sounding guidelines, and I am sure that all truly want to do a superior analysis, but meta-analysis often fails in ways that seem to be invisible to the analyst.
Bailer was also quoted in the Reference Manual on Scientific Evidence Page 389
[P]roblems have been so frequent and so deep, and overstatements of the strength of conclusions so
extreme, that one might well conclude there is something seriously and fundamentally wrong with the
method. For the present . . . I still prefer the thoughtful, old-fashioned review of the literature by a
knowledgeable expert who explains and defends the judgments that are presented. We have not yet
reached a stage where these judgments can be passed on, even in part, to a formalized process such as
There lies the rub, not only was there publication bias, but deliberate bias. Tobacco control’s studies were based on cherry picked studies the first of which was the 1992 EPA study. This study was not only thrown out in Federal Court but was dismissed by the Congressional Research Service. The EPA has had a long history of producing junk science based on political agenda.
A problem that goes on to this very day.
Of course this applies to the Surgeon Generals report which was done by many of the same activist who did the flawed EPA report.
Jonathan M. Samet, M.D., and the 1992 EPA Report
One might wonder how omissions, distortions, and exaggerations like those pointed out above could occur in a document as important as a Surgeon General’s Report on ETS. To better understand this phenomena one must realize that Samet has dealt with the ETS issue in this manner for many years. In particular, he played a major role in the epidemiologic analysis for the December 1992 report on Health Effects of Passive Smoking: Lung Cancer and Other Disorders: The Report of the United States Environmental Protection Agency . This EPA report classified ETS as a Group A human carcinogen, which causes about 3,000 lung cancer deaths per year in the U.S. The findings from this report were used in the Broin v. Philip Morris litigation described above.
The epidemiologic methodology and conclusions of the EPA report have been severely criticized. One of the harshest critiques is the 92-page Decision issued by Federal Judge William L. Osteen on July 17, 1998, which overturned the report in the U.S. District Court . For instance, in his conclusion Judge Osteen wrote: “In conducting the Assessment, EPA deemed it biologically plausible that ETS was a carcinogen. EPA’s theory was premised on the similarities between MS [mainstream smoke], SS [sidestream smoke], and ETS. In other chapters, the Agency used MS and ETS dissimilarities to justify methodology. Recognizing problems, EPA attempted to confirm the theory with epidemiologic studies. After choosing a portion of the studies, EPA did not find a statistically significant association. EPA then claimed the bioplausibility theory, renominated the a priori hypothesis, justified a more lenient methodology. With a new methodology, EPA demonstrated from the 88 selected studies a very low relative risk for lung cancer based on ETS exposure. Based on its original theory and the weak evidence of association, EPA concluded the evidence showed a causal relationship between cancer and ETS. The administrative record contains glaring deficiencies. . . .”
Even if one were to accept the flawed methodology of meta analysis they still have to overcome the fact that such weak statistical significance has never been accepted as proof of cause of any disease as stated in an award winning article in Science,
“If it’s a 1.5 relative risk, and it’s only one study and even a very good one, you scratch your chin and say maybe.” Some epidemiologists say that an association with an increased risk of tens of percent might be believed if it shows up consistently in many different studies.
That’s the rationale for meta-analysis — a technique for combining many ambiguous studies to see whether they tend in the same direction (Science, 3 August 1990, p. 476).
But when Science asked epidemiologists to identify weak associations that are now considered convincing because they show up repeatedly, opinions were divided — consistently.
Challenge any of the tobacco control activist to name 3 causes of a disease that have been proven with the same weak statistical significance!
Then of course there is the joke of the century, of course that is the claim that there is no safe level of second hand smoke, this flies in the face of the accepted scientific principle that dose makes the poison, now the tobacco control activist will point to the LNT (linear no threshold model) which was based on the theory that there is no safe level of ionizing radiation. There are a few problems with the LNT model. Many critics of the model claim that although radiation is highly toxic they do not measure low enough. That when measured low enough that the dose response curve is not linear. The Surgeon General makes the claim of no safe level and yet has not produced one single study that even implies linearity.
Then of course there are the claims by tobacco control that heart attacks immediately go down as a result of smoking bans. Of course the biggest study done by researchers from the RAND Corporation, Congressional Budget Office, University of Wisconsin, and Stanford University is the first to examine the relationship between smoking bans and heart attack admissions and mortality trends in the entire nation, using national data.
The key conclusions of the study are as follows:1. “In contrast with smaller regional studies, we find that workplace bans are not associated with statistically significant short-term declines in mortality or hospital admissions for myocardial infarction