Current mood:	Unreliable?
Current music:	Matchbox Twenty - "How Far We've Come"
Entry tags:	science humor

Someone at CSU Bakersfield thinks I'm educational
but not in a good way. She used an AIR Online article I co-wrote as an example of a website that "is unreliable, bias[sic] and has no authenticity." She had a number of issues with the paper:

"We do not know who the authors are"
The paper claims they are Eric Schulman and Daniel Debowy. Eric Schulman's page has a link entitled Who Am I?. Clicking it reveals that Eric Schulman is a Ph.D. astronomer, author, and member of the editorial board of The Annals of Improbable Research. Daniel Debowy's affiliation is listed as "Massachusetts General Hospital / McLean, Belmont, Massachusetts." There's no direct link, but it only takes a few clicks to find this page. Dr. Debowy is a Clinical Fellow in Psychiatry at Massachusetts General Hospital. Now, one could argue that perhaps an astronomer and a psychiatrist are not the most qualified people in the world to write a paper entitled "Who Will Win the 2008 U.S. Presidential Election?" That's a fair point, but not one that was made.

"[We do not know] how reliable the information, that they presented, is."
The information in the tables is mostly the number of years each presidential candidate served in various offices. But how can the reader trust that we got it right? If only we had included links to our sources. Oh, wait, we did. Click on any candidate name and you will be taken to their Wikipedia page. You could argue that Wikipedia is not the most reliable source in the world. That's a fair point, but not the one that was made.

"We do not know how they came up with the equation that was used to predict the electability."
If only we had included a link to the paper in which we originally described the algorithm. Oh, wait, we did. The first paragraph in the Methods and Results section of that paper reads as follows: "We analyzed the experience of the major party candidates for President and Vice President in each of the U. S. Presidential elections since 1804, and the experience of the two top candidates for President in 1789, 1792, 1796, and 1800 (when the candidate who received the most electoral votes became President, and the candidate who received the next largest share of electoral votes became Vice President). We discovered the following empirical formula after an extensive phase space search." You could argue that trying an unspecified number of weighting functions until finding one that works is not the best way in the world of developing such an algorithm. That's a fair point, but not the one that was made.

"For an example this chart predicted that either Wesley K. Clark, Albert A. Gore Jr., William B. Richardson, or Christopher J. Dodd would be nominee's[sic] for the Democratic Presidential candidate[sic]."
The chart (Table 7) says no such thing. It says that the Democratic presidential candidates with the highest electability (according to our algorithm) are Wesley K. Clark, Albert A. Gore Jr., William B. Richardson, and Christopher J. Dodd. The discussion section (right below Table 7) points out that none of these candidates were highly rated by users of Intrade.Com in March of 2007. The third paragraph points out that the top Democratic presidential candidates at Intrade were Hillary D. R. Clinton and Barack H. Obama (the same two who are still vying for the nomination today).

So, to sum up, let's look at her overall conclusions.

The paper is "unreliable"
How? We did our very best to verify the information used in the paper. Wikipedia may not be the most reliable web site for some topics, but we think it correctly states how long Freddie D. Thompson was a U.S. Senator. In some cases we double-checked with the Biographical Directory of the United States Congress and we never found a discrepancy.

The paper is "bias"
I have written many things, but as far as I know I've never written the embodiment of bias. The paper is not "a line going diagonally across the grain of fabric." It is not "the fixed voltage applied to an electrode." And it is not "a preference or an inclination, especially one that inhibits impartial judgment." Neither is it biased. The paper does not argue that Republicans are better than Democrats. It does not argue that Democrats are better than Republicans. It's true that the algorithm predicts the remaining Democratic presidential candidates are more electable than the Republican presidential candidate. But in 2004, the algorithm predicted (correctly!) that the Republican presidential ticket would defeat the Democratic presidential ticket. So where's the bias?

The paper "has no authenticity"
Does the paper have no "quality or condition of being authentic, trustworthy, or genuine?"
Authentic means "conforming to fact" (as discussed above, the paper does) or "having a claimed and verifiable origin or authorship." That's an interesting question. Perhaps Eric Schulman and Daniel Debowy didn't write the paper. How could one tell? Probably the best way would be to e-mail the editor of The Annals of Improbable Research and ask him for verification. Another would be to do searches for particularly memorable phrases and see if they turned up elsewhere on the web. Let's search for "an electability of 264." We get ten pages. The first is the page she is complaining about. That's a good sign that the paper is authentic. The next nine pages either link to the first page or cite Eric Schulman and Daniel Debowy as the authors. But perhaps none of these websites are real. Perhaps Eric Schulman set up all these websites to convince people that he wrote the paper. If only we had some hard evidence that Eric Schulman is a contributor to AIR and a member of the AIR editorial board. How about a book? Books are pretty hard. Try a Google Books search for "editorial board" "annals of improbable research". The first hit is this. Looks pretty authentic to me. Which leaves us with genuine. Is it a genuine research paper? Is it "free from hypocrisy or dishonesty?" Yes! But is it "sincere"? A fair question, and one that in recent years has been much on my mind. Do I believe with 95 percent confidence that the algorithm will correctly predict the next ten U.S. Presidential elections? No, I do not. Do I believe with 80 percent confidence that the algorithm will correctly predict at least seven of the next ten U.S. Presidential elections? I think I do. There is some truth to our algorithm. We were not the first to realize that, on average, governors do better than senators in U.S. presidential elections. But, as far as I know, we were the first to conclude that being a senator adds nothing to presidential electability. It's not that I believe that being a senator doesn't provide candidates with useful experience, it's that I believe the negatives associated with being senator (e.g., voting records that can be mined for negative ads) might equal the positives. I'm not trying to argue that our tongues were nowhere near our cheeks when writing the paper. All I'm saying is that there is something to what we found. Otherwise it wouldn't be so amusing. Is this paper the most authentic thing in the world? No, of course not. But saying the paper "has no authenticity" is demonstrably false.

In conclusion, my advice to educators using the web is to be careful in evaluating the information you find. Some pages that look reliable are not. And others that look strange may end up, upon closer look, being reliable, unbiased, and authentic.