March 26, 1997 Mervyn Susser MB BCh, Editor American Journal of Public Health American Public Health Association 1015 -- 15th Street NW Washington DC 20005 Re: Hemenway D and Richardson E. "Characteristics of Automatic or Semiautomatic Firearms Ownership in the United States." Am J. Pub Health. February 1997; 87(2):286-288. Dear Dr. Susser, In "Epidemiology Faces its Limits," Taubes' special report in Science,[1] New England Journal of Medicine (NEJM) Assistant Editor Marcia Angell acknowledged the problems that confound epidemiological studies that use "odds ratio" or "relative risk" statistical methodology. She stated, "As a general rule of thumb, we are looking for a relative risk of three or more [before accepting a paper for publication], particularly if it is biologically implausible or if it is a brand new finding." In her book on the silicon breast implant controversy Angell stated,"[Confounding variables] can easily mask or falsely suggest a small risk factor. For this reason, most epidemiologists are very skeptical of a relative risk below 3.0 or 4.0 if the finding is new and unanticipated. A relative risk smaller than this requires a good deal of confirmation.... a relative risk of 2.0 would almost certainly be chalked up to a confounding variable or to some unsuspected bias."[2] In the Science report,[1] a former statistical consultant for NEJM, John Bailar, an epidemiologist at McGill University, expressed skepticism of studies showing odds ratios of less than 10. High rates of non-reporting (27.1%) in the Hemenway & Richardson study[3] and Kellermann's[4] and others'[5] earlier work showing high rates of inaccurate reporting of firearms ownership (12% and 13% respectively) should have alerted the authors to the possibility, even likelihood, that their findings were statistically inaccurate and too insubstantial to draw meaningful conclusions. Even if we ignored how the odds ratios collapse to statistical insignificance in the face of inaccurate and non-reporting of gun ownership (as Kellermann's own 2.7 odds ratio collapses to nothing when his recent gun homicide study[6] is corrected for the reporting inaccuracy), not one of Hemenway & Richardson's "characteristics" of certain gun owners reached Bailar's threshold of credibility and for only one "characteristic," "owns or carries gun for work," did the odds ratio (3.8) exceed Angell's lesser threshold for credibility. So, even if we credulously accepted Hemenway & Richardson's odds ratios at face value, we are presented with a rather uncontroversial and marginally significant observation that police officers have a propensity to own certain firearms. Studying primarily owners of semiautomatic firearms (as the authors admitted), Hemenway & Richardson calculated an odds ratio of 1.89 for "binge drinking" more than once in the 60 days preceding their survey of gun owners. Though the "binge drinking" odds ratio of 1.89 does not even meet the rudimentary threshold of credibility noted above, Hemenway & Richardson spent 7 of 9 "Discussion" paragraphs focusing upon binge drinking and gun ownership, bewailing and speculating about a "disturbing" finding that they have not even convincingly demonstrated. In contrast, their discussion section made no mention at all of their only possible statistically significant finding, that police are more likely to own certain guns than non-police. Cardiovascular researcher Stanton Glantz cited more than a dozen studies in support of his observation that "Approximately half the articles published in medical journals that use statistical methods use them incorrectly. These errors are so widespread that the present system of peer review has not been able to control them."[7] Add the Hemenway & Richardson study to Glantz' evidence of such statistical incompetence. Fifteen studies, fourteen reviewed by Kleck & Gertz[8] and the fifteenth performed by Cook,[9] Kleck's foremost academic critic, demonstrate one to four million or more protective uses of guns annually, far outweighing the combined detriment of criminal, suicidal, and careless gun misuse. The University of Chicago Lott & Mustard study of progressive reform of concealed handgun law allowing good citizens to protect themselves where they are at greatest risk, outside their homes, demonstrated that, rather than blood running in the streets, the net outcome of these reforms has been thousands of lives saved and violent crime prevented.[10] In view of these findings of an overwhelming net benefit of guns in the hands of good citizens, Hemenway & Richardson cannot legitimately seek refuge in a special claim of biological or other plausibility to defend their subthreshold odds ratio. As in this study's technically inappropriate lumping of machine-guns rarely owned by private individuals, "automatics," with non-machine-guns that have been commonly owned by individuals for over a century,[11] "semiautomatics," reviewing Hemenway's prior publications on guns demonstrates his unfamiliarity with technical and safety issues regarding guns, an unfamiliarity that continues to undercut his research design and conclusions. Even if Hemenway's study design had separately inquired of the characteristics of semiautomatic gun owners, that varied group of guns includes an overwhelming majority of semiautomatics having magazine capacities and capabilities approximately the same as old-fashioned cowboy "six shooters," indefensibly characterized by Hemenway as distinctively "rapid fire." Noting Hemenway's published opinions on guns and gun owners begs another question - Does Hemenway want to honestly and scientifically study the characteristics of guns and gun owners or does he merely wish to contribute sound bites, factoids, and demeaning images for use by lobbyists who abhor guns and gun owners? Collegially, Edgar A. Suter MD National Chair Doctors for Integrity in Policy Research Inc. cc: David Hemenway PhD Harvard School of Public Health 677 Huntington Avenue Boston MA 02115 References [1] Taubes G. "Epidemiology Faces its Limits." Science. July 14, 1995. 269:164-169 at 168. [2] Angell M. "Science on Trial: The Clash of Medical Evidence and the Law in the Breast Implant Case." New York: WW Norton & Co. 1996. at 168. [3] Hemenway D and Richardson E. "Characteristics of Automatic or Semiautomatic Firearms Ownership in the United States." Am J. Pub Health. February 1997; 87(2):286-288. [4] Kellermann AL, Rivarra FP, Banton J, Reay D, and Fligner CL. "Validating Survey Responses to Questions about Gun Ownership Among Owners of Reistered Handguns." American Journal of Epidemiology. 1990; 131: 1080-1084. [5] Rafferty A, Thrush JC, Smith PK, and McGee HB. "Validity of a Household Gun Question in a Telephone Survey." Public Health Reports. 1995; 110: 282-288. [6] Kellermann AL, Rivara FP, Rushforth NB et al. "Gun ownership as a risk factor for homicide in the home." N Engl J Med. 1993; 329(15): 1084-91. [7] Glantz SA. "Biostatistics: How to Detect, Correct and Prevent Errors in the Medical Literature." Circulation. January 1980; 61(1):1-7. [8] Kleck G and Gertz M. "Armed Resistance to Crime: The Prevalence and Nature of Self-Defense with a Gun." Journal of Criminal Law & Criminology. 1995; 86(1):150-187. [9] Cook PJ. "You Got Me: How Many Defensive Gun Uses Per Year?" a paper presented to the American Society of Criminology meetings, November 20, 1996. Chicago IL. [10] Lott JR and Mustard DB. "Crime, Deterrence, and Right-To-Carry Concealed Handguns." Journal of Legal Studies. January 1997; 26:1-68. [11] Suter E. "'Assault Weapons' Revisited - An Analysis of the AMA Report." Journal of the Medical Association of Georgia. May1994; 83: 281-89.