no, you can’t do psychology by committee

Virtually every culture, sub-culture, and profession has it’s own method of posturing about how big and oh so very, very important you and your work are, basically, what’s known as a penis waiving contest. Just like male primates are thought to try to intimidate each other with their erections and do so to win mating rights, and as it would seem, female preferences places size limits on those erections, humans use their job titles, cars, and media mentions to boast about how successful they are to win access to more resources or positions of authority in their social group to the extent that we let them before calling them boastful or obnoxious. Pundits like to brag about how many shows they do on a weekly, if not daily basis. Lawyers proudly mention any high profile case on which they worked. Scientists have the impact factor of the journals and studies they publish and the more citations they have, the logic has been going, the better and more prestigious their work. So if a study finding a correlation between violent video games and violence is frequently brought up, well, it must be better than a more obscure one that did not and more valuable in a court case. Right? Actually, no. It isn’t.

One would think that trained scientists, who know that the true value of work is in the data, wouldn’t pay nearly as much attention to it as university quants who use impact factors to judge the worth of a scientific effort. I’m sure you’ll remember how many scientists complained about complicated experiments being reduced to just a couple of meaningless numbers during the tenure decision process, and how many physicists wrote brief rants about impact factors not being what they once used to be on my posts dissecting arXiv studies. So why would a pair of psychologists try to argue that because a legal brief with more popularly cited studies on the alleged link between violent video games and aggression is better than one with less known ones? It’s essentially an argumentum ad populum in academise. Could it be that the psychologists in question, Craig Anderson, Brad Bushman, and Deana Pollard Sacks, who always tend to find that video games and porn are evil and make people more aggressive, are trying to raise their profiles a bit with a few media mentions? The media loves to jump on a controversial topic and they certainly study a few of them. No problem there, but that doesn’t mean they can wave around a paper-thin conclusion as if they had some sort of scientific evidence of relevance to the case, which is exactly what they seem to be trying to do. Their argument boils down to this: in a legal brief arguing that violent video games cause aggression there are more people who published some sort of study in a peer reviewed journal than in the counter-brief, therefore, it’s a better argument.

Do I really even need to point out why this argument is flawed? Shouldn’t I just leave it at that and let this huge and glaring fallacy remain self-evident? We’re talking about 69 people who did a study on the video game and aggression link signing a document saying that video games can make people aggressive and managed to publish it in a top tier journal at some point in time. So because those 69 people were rounded up by a lawyer and signed his brief, that means there’s now a strong link between video games and violent behavior? Say, I seem to remember psychologists who published in respected journals signing on to the Satanic ritual abuse cases concocted by hoaxers to make money from Christian fundamentalists. And there were more than a paltry 69 of them signing just one legal brief. Does this fact mean that we were wrong and there’s now solid, empirical evidence of Satanic ritual abuse and its harm based on how many times papers on it were cited by others, even if it was to show the conclusions as erroneous? Yes, having your paper cited and then dissected actually ads to your impact factor the same way as having it cited as evidence for follow-up or similar work. An impact factor only tells us how much of a splash a paper made, and in the case our trio of psychologists has made, this only extends to the journals. We don’t even know how widely the papers themselves were cited in other scientific literature or in what context. We just know they published in a good journal.

We also can’t rule out the influence of lawyers themselves. One would think that a lawyer on the hunt for a few self-appointed experts in a controversial field is probably not going to solicit perfectly objective advice. He will find those who are willing to agree and ask their friends as well. Plus, a brief worded with enough conditions and qualifiers is easy to sign. Yes, sure, depending on the person’s mood, content of the game, lifestyle and upbringing, and how much Red Bull a player had, some aggression may persist after playing a violent game, fine. Does this now mean that video games make you aggressive and there’s now evidence that it’s harmful to all players? No. Studies on several dozen freshmen at a university and the signatures of 69 people who at some point studied the topic an iron-clad case does not make. And so what if people stay a little aggressive for a few hours if they play Grand Theft Auto or Halo? Rates of violent crime are down across the board, both adult and juvenile. So whatever aggressive feelings a few people might have in a lab or at home don’t exactly seem to be spilling out into the street. If the premise here was true, we’d expect to see spikes in violent crime with new editions of violent blockbusters. We don’t. So why do we insist on making a big deal out of it? To me it seems like just another case of old fogeism in action, coupled with a plea for media attention…