Article 67662 of talk.politics.guns: Path: teetot.acusd.edu!network.ucsd.edu!sdd.hp.com!vixen.cso.uiuc.edu!howland.reston.ans.net!cs.utexas.edu!uunet!panix!not-for-mail From: [l--e--y] at [panix.com] (Aldo Tartaglini) Newsgroups: talk.politics.guns Subject: Re: NEJM: Quick and easy way to refute it Date: 11 Nov 1993 12:57:13 -0500 Organization: PANIX Public Access Internet and Unix, NYC Lines: 172 Message-ID: <2btudp$[3 b 2] at [panix.com]> References: <2brkm4$[5 n 6] at [fitz.TC.Cornell.EDU]> <[1993 Nov 11 011002 11839] at [beaver.cs.washington.edu]> <[C G B 1 I 5 H 3] at [usenet.ucs.indiana.edu]> <[1993 Nov 11 073431 25964] at [beaver.cs.washington.edu]> NNTP-Posting-Host: panix.com In <[1993 Nov 11 073431 25964] at [beaver.cs.washington.edu]> [l--p--z] at [cs.washington.edu] (Gus Lopez) writes: >In article <[C G B 1 I 5 H 3] at [usenet.ucs.indiana.edu]> [w--a--f] at [silver.ucs.indiana.edu] (Wayne J. Warf) writes: >>In article <[1993 Nov 11 011002 11839] at [beaver.cs.washington.edu]> [l--p--z] at [cs.washington.edu] writes: >>> >>>This technique being described is called a "case-control" study. You >>>pick out people with the endpoint you're looking at first (for example, >>>people with lung cancer), and then compare them to a carefully matched sample >>>of healthy people (matched for age, education, income, etc.). Then >>>you look at what the people did before they got sick. >> >>"Carefully matched sample..." heres the key. >> >>>So if three times as many lung cancer patients smoked cigarettes >>>as healthy people, I think any normal person would suspect >>>that smoking was connected to lung cancer. If three times as many >>>homicide victims had guns as non-victims, I'd suspect a connection >>>between guns in the home and death by homicide. Of course, this kind >>>of study does not conclusively establish causality, but it's a perfectly >>>valid and commonly used methodology. The case-control (or static group comparison) design is certainly a commonly used method, but what do you mean by "valid?" It is true that the method is considered legitimate by researchers, but this does not in any way speak to the validity of the results obtained from its use (I'm sure you already know this, but I just want to make sure I understand your meaning here). In fact, the case-controlled comparison is considered among the least desireable of the various quasi-experimental designs in terms of its ability to safeguard against confounds and other threats to internal, external, and construct validity. This is to say that, if one really wants to establish a causal relationship among variables, it is best to avoid quasi-experimental methodology entirely in favor of truly experimental designs, such as the randomized control group design. Of course this isn't always an option when using human subjects, which might explain the paucity of well-established causal relationships/mechanisms (much less the discovery of "natural laws") in the "soft" sciences (sociology, psychology, education, etc.). Nontheless, the trend is definitely in the direction of using true experimental designs wherever possible, as these minimize the numerous confounds/threats to validity which plague the use of quasi-experimental methodology. If a researcher wanted to study whether and to what extent possession of handguns in the home was causally related to the likelihood of domestic homicide, the most powerful (capable of producing reliable and valid results) method would entail taking a random sample of households, randomly assigning subjects to gun-in-the-home (maybe define two groups here, e.g., a handgun-only and a rifle/shotgun only group) versus no-gun-in-the-home groups, obtaining pretest measures of subjects on a range of demographic and other variables (e.g., likely confounds) to ensure that the groups are comparable, then obtaining periodic assessments of the extent of domestic homicide among the groups as well as repeated measures of other theoretically relevant variables (possible confounds/mediating variables) over a period of, say, several years. [To maximize the data yield, I might also obtain repeated assessments of the use of guns for self-defense purposes in the with-gun groups, as well as assessments of the overall rate of criminal victimization among all groups.] When the latest NEJM study came out, its authors (aided and abetted by the media and the anti-gun lobby) made it sound as if they had just uncovered some astonishing-but-true force of nature: if you keep a gun in the house, you are 3 times more likely to be a homicide victim... This assertion, while perhaps not deliberately fraudulent, nonetheless represents scientific irresponsibility bordering on misconduct, for the researchers hadn't used a method which permits such extravagant claims. Even true experiments of the sort I described above have a tough time establishing replicable findings, much less making a case for the existence of a causal relationships among variables. The case-controlled comparison is simply not powerful enough to establish such relationships; the best it can do is suggest stable statistical associations, which may or may not be useful for eventually understanding the etiology of a given phenomenon. In the case of the NEJM article, the investigators didn't even seem to look too closely at their own data, limited as it was, which indicated that illicit drug use and domestic violence were far more important risk factors (for homicide) than possession of guns in the home. Alas, given the structural weaknesses of this study, even this finding, which the investigators and media studiously ignored, is tentative at best. >>> >> >>OK, say I want to see how mental patients differ from non mental patients. >>Now, I should match them carefully for age, marital status, income >>level, race or any other variable that may confound my variable of >>interest, namely abortion. I want to see the variance contributed by >>this variable when other theoretical possibilities are excluded as >>causing mental illness. Now, suppose that instead of doing it this >>way, I compared a group of poor, single, young mental patients >>and compared them to a comfortable middle class married sample as a control. >>I then collect information on how many of each sample had abortions. >>I find that 3 times as many mental patients had abortions as my did >>my controls. I then go on every talk show and issuing press releases >>that women would had abortions are 3 times more likely to be mental >>patients ignoring that I failed to control for wealth, marital status >>or age that are theoretically related confounds. What did I establish >>except that I can show a correlation between variables if I don't do >>my controls right? Would you call my study valid? >Of course it would be hard to claim your study is valid because you let >some important variables slip through the cracks. My point is that the >case control methodology is valid and commonly accepted by researchers. >The post to which I was responding claimed otherwise. The authors of the NEJM study are guilty of the same errors that you ascribe to Wayne J. Warf's hypothetical example. >My hunch is that people are looking at this study at the wrong level by >immediately attacking the methods used instead of questioning the >ramifications of the results of the finding. Since the results are very likely noise, I don't think it is very productive to focus on their possible "ramifications." We might just as well speculate on the possible ramifications of various astrological or psychic "findings" (that Hillary does this doesn't make it right!). >Even if you concede the >idea that guns -> increased homicide, I don't think there are any >obvious government policy implications. There is, of course, no evidence of this causal relationship, but the anti-gun lobby acts as if there is and assiduously pushes prohibitionstic gun laws/bans (which egregriously infringe on the rights of peaceable citizens) accordingly. >For instance, if I conducted >a study where I looked at AIDS victims and controlled for wealth, race, and >other variables and found that gay men were 3x as likely to die of AIDS than >straight men, does this mean that gay men should be locked up as way to >solve the AIDS crisis or that gay men should be monitored by the >government? Of course not. You are thinking logically; it is a mistake to believe the chief proponents of the anti-gun/victim disarmament movement are capable and/or willing to do the same. >In other words, accepting that there's a >link between guns and homicide (not such a totally unbelievable thing), Actually, it is utterly impossible to believe. Now, the notion of a link between PEOPLE (sometimes in possession of guns) and homicide is much more plausible. >does not mean you agree with a particular gun-control policy which attempts >to address the homicide variable. Oh, I see what you mean. Once we all agree that guns, not criminals, are responsible for violent crime, we don't need to endorse particular gun control policies. Like, you might suggest enacting more laws which ban guns by type, while someone else might suggest a total ban on all guns over a period of years, while yet another person might suggest a Constitutional ammendment to ban all guns, followed by house-to-house searches by the military, etc. Just so long as we know who the culprit is, I guess it doesn't matter how we go about apprehending him and bringing him to justice, right? >Gus >-- > ___Gus Lopez_______________________ > / University of Washington / > /[lopez] at [cs.washington.edu___]/ A.T. ====================================================================== "And we should -- then every community in the country could then start doing major weapon sweeps and then destroying the weapons, not selling them." - William J. Clinton, President, sworn defender of the U.S. Constitution ======================================================================