I read the book a few weeks ago and from what I recall you're both (a) right that Groseclose uses frequency of citations to various liberal/conservative think tanks as part of his analysis; and (b) not even close in suggesting that summarizes his entire methodology.

Thank you, Prof. Kerr. If that is indeed the methodology, it strikes me as strangely circuitous. However, I'd like to read Prof. Groseclose's response before passing judgment.

Does the book describe the methodology underpinning it?

Federal Dog,

I assume it does, but Groseclose is trying to persuade us that the book is worthwhile and thefore that we should buy it: I think he needs to tell us the methodology to know if the book is worthwhile.

Of course, perhaps the strategy is to sell books by refusing to disclose the methodology -- and to promise that the book has it, and that it's awesome once you buy the book. If so, I think it's time for me to disclose that the secret to human happiness has been scientifically proven and can be found in this book. On sale now!

Orin,

"Only 5 left in stock." I guess most of us can just forget about happiness, then. :-(

Kudos for trying to redeem the Groseclose blogging, but I doubt that either Groseclose or the commentariat will let that happen. Titling his book "Left Turn: How Liberal Media Bias Distorts the American Mind" suggests an affinity for button-pushing...

Kirk,

Don't worry, I have more in my office that I can sell you at a very reasonable price. Anything to make sure that you have the secrets to everything -- purely scientific, mind you.

Noblesse,

You're totally wrong -- and this book explains why.

That's pretty much the methodology.

Orin,

I've already got two copies and I haven't found the secret of happiness in either of them. I'll check them again for a refutation of my contention, but I'm starting to doubt your trustworthiness.

Orin Kerr writes:

"Groseclose is trying to persuade us that the book is worthwhile and thefore that we should buy it: I think he needs to tell us the methodology to know if the book is worthwhile.

Of course, perhaps the strategy is to sell books by refusing to disclose the methodology — and to promise that the book has it, and that it’s awesome once you buy the book."

FWIW, a significant portion of the methodology has been disclosed in the Groseclose and Milyo's QJE paper. You can even get it for free and make your own determination:

http://www.sscnet.ucla.edu/...

If this is indeed the methodology, it strikes me as being entirely relative. Are we to believe that the output of all think tanks are of equal value?

For instance, if think tank A says that the world is flat while think tank B says that the world is round, is it "liberal" or "conservative" to cite B instead of A? I would hope that the journalists would cite B, regardless of what the politicians are saying.

We know that there are think tanks (on both sides) who exist mostly to produce propaganda, rather than facts. If the politicians on one side choose to disproportionally cite propaganda outlets, that would skew the results.

Or in other words, not all "expert" opinion is created equal. To assume so might be considered "worshiping the god of Equality".

Orin&#32Kerr: Noblesse,

You’re totally wrong — and this book explains why.

You know, if you'd just have named your book "How idiotic liberals completely screwed up computer crime law" or something, you wouldn't have to resort to these tactics.

Orin,

I think that's about right. Political scientists have a widely accepted methodology for determining how liberal or conservative a politician is, based on how they vote on a number of key votes selected by interest groups. Groseclose's insight was to use think tank citations as a way to map legislators' ideological ratings onto media outlets.

Orin,

As a liberal, I confess I often wonder how thoughtful conservatives like you (and Adler, Anderson, and Elwood) feel when various guest bloggers (and sometimes Bernstein and Zywicki) turn this blog into RedState.

The methodology, presumably, is described in some detail in Groseclose & Milyo's paper, available as a PDF at http://dev.wcfia.harvard.ed... . I haven't read it yet, so can't comment.

Noblesse:

I’ve already got two copies and I haven’t found the secret of happiness in either of them. I’ll check them again for a refutation of my contention, but I’m starting to doubt your trustworthiness.

Obviously you haven't bought the 2011 Supplement yet. (Sheesh, what's with the fools who question me about not getting the secret of happiness but haven't even read the 2011 Supplement?!?!)

Enough about methodology--bring on the discussion about a committee resignation!

136 dollars. Maybe I'll just buy Left Turn instead.

I know about the concept of thread-winners, but is there such a thing as a blog-winner?

This negative review on Amazon, if accurate, looks pretty devastating. According to this reviewer, Groseclose's methodology results in Handgun Control, Inc. being a liberal organization (77.3) while the NRA is a 45.6 (slightly conservative). The result is that a media report quoting an HGI representative favoring a gun control bill and an NRA representative opposing it would be coded as biased to the left. The methodology also results in coding the ACLU, which is a solidly left-wing organization, as 48.9, slightly right of center.

That seems to be the methodology, and as noted, there seems to be no filter regarding the value or accuracy of the information from the think tank or whether the citations are positive or negative. Plus liberals are vicious.

According to this page on Groseclose's website, the "think tank" metric is only one of three methodologies he uses to determine media bias.

The second involves media use of "loaded political phrases" (his words), which is then somehow tied to the PQs of members of Congress. As others have pointed out, though, the two representative "loaded" phrases provided by Groseclose are "death tax" and "estate tax," which are quite obviously not both politically loaded terms (or, at least, not nearly equally loaded).

The third methodology seems to have focused on a particular legislative detail. Groseclose, it seems, studied how often media outlets reported that the Bush tax cuts "made the tax system more progressive" (again, his words), alongside the more widely reported point that the cuts favored the rich. Media sources that failed to report as much on the progressivity of the tax program were classified as liberal. To me, this metric is the strangest of the three.

I find his methodology puzzling as well, though I will say it's fun to run through a list of Senate bills, say how you would have voted on them, then find out which senator most closely matches your voting pattern.

Of course, Professor Groseclose should respond directly, but I am halfway through the book and the professor has written about multiple methods of determining political slant. The first is the citations of think tanks. Another was based on a challenge from two other academics in separate conversations: Apparently, the following two things were both simultaneously true with regard to George Bush's proposed tax cuts:
1) The wealthiest taxpayers got more money back than other groups.
2) The tax structure became more progressive.
As liberals were against the tax cuts and conservatives were for them, the first statement was construed to be a fact which might affect a voter to favor the liberal position, while the second statement was construed to be a fact which might affect a voter to favor the conservative position. So Groseclose and Milyo measured how often each of those two equally true statements were mentioned in newspapers.
Another one--possibly not as rigorously measured--was the frequency of calling partial-birth abortion 'partial-birth abortion' or 'what opponents call partial-birth abortion'. There were others as well, but I don't recall them and don't have the book at-hand.

People here seem to underestimate just how many different ways Groseclose and Milyo measured bias. It's not a short book and it's full of measurements. The many anecdotes of bias mentioned in the book are also very eye-opening.

I really would not comment on any book that I haven't read. This is true even if the methodological summary demanded (in lieu of reading the book) were forthcoming.

Maybe that's just me.

yankee:
This negative review on Amazon, if accurate, looks pretty devastating.

What the professor refers to as "think tanks" are the usual activist organizations - the NRA, the ACLU, etc.

Without getting into the merits of these organizations, I think applying the term "think tank" is misleading.

Cornellian: I find his methodology puzzling as well, though I will say it’s fun to run through a list of Senate bills, say how you would have voted on them, then find out which senator most closely matches your voting pattern.  (Quote)

Doesn't really tell you much, since there's a lot of positioning voting and party stuff.

Like, say -- a given Senate bill, the real deciding factor might have been the cloture vote, not the bill vote itself. (I've seen bills pass cloture 61-39 and then pass the Senate 97-0 or something like that).

In the House, votes are weighed and measured and vulnerable incumbents are released to make votes entirely for political reasons (campaign fodder and the like). So bills they would have voted for, if it came down to just their choice, they'd vote against -- because it played well in Peoria, so to speak, and it was already a done deal.

You'd think politicans voting records would mean something, but not for most of them. Senate's worse, with cloture, with strategic votes (like the Majority Leader often votes against a doomed bill, or one just shy of cloture, so he can bring it up again later. If he votes for it, it's more of a pain for some reason) and the like.

Orin,

I think I can help you understand his methodology: Sift through years of news coverage and select the parts that support his argument. Then assume that, rightly, that nobody else will really care enough to try to wade through all of the data. This is just a guess on my part, but the histrionic title leads me to believe that it has more in common with Ann Coulter than serious scholarship.

Orin, what you've summarized pretty accurately is the methodology of the Groseclose-Milyo paper from 2005. In the book, he supplements that method with some others that he has not previously published. One of them, the analysis of media coverage of the Bush tax cuts of 2001 and 2003, I discussed in a comment on Groseclose's post from earlier today. Briefly, I found that approach flawed because Groseclose tried to equate two statements about the tax cuts that were not really equivalent. The claim that Bush's tax code made the tax system more progressive isn't an unquestioned fact; I cited research by leading economists (William Gale, Thomas Piketty and Emmanuel Saez) which reached the opposite conclusion.

DCGroseclose, it seems, studied how often media outlets reported that the Bush tax cuts “made the tax system more progressive” (again, his words), alongside the more widely reported point that the cuts favored the rich.

Is it right that he looked at that dichotomy only? That's not just a representative example?

Orin&#32Kerr:
If so, I think it’s time for me to disclose that the secret to human happiness has been scientifically proven and can be found in this book.On sale now!  

That seems a little pricey for the secret to happiness. Can you throw in some steak knives?

Also, I think you should change the title to "Computer Crimes: How Leftists Are Using Porn to Steal Your Children."

Put another way, the media is biased because when it looks for experts to quote as authority, it tends to favor liberal experts over conservative experts more than does a centrist politician.

I think what he's saying (using this method--he uses several methods) is that the media is biased because it tends to favor experts who liberals like over experts who conservatives like, not because it favors experts who are liberal over experts who are conservative.

Orin&#32Kerr: I assume it does, but Groseclose is trying to persuade us that the book is worthwhile and thefore that we should buy it: I think he needs to tell us the methodology to know if the book is worthwhile.

Doesn't his peer reviewed paper discuss the methodology? Is it available online?

I do question the PQ ratings, any rating that has Richard Nixon, the President that imposed wage price controls, and the EPA on us, as 12.3, seems like it is measuring partisanship rather than ideology. Nixon certainly was plenty partisan, and anti-communist certainly, but not nearly as conservative as say Goldwater.

It seems like there is an additional methodological component in the book that is not in the paper? Groseclose mentioned that one of his conclusions was that the media bias identified skewed the perspective of the typical voter more liberal than they might otherwise have been.

If anyone who has the book, or Prof. Groseclose, might be able to elaborate on the approach there I would appreciate it.

Jestak—also I'm not sure how you would determine that two statements about some proposed tax cut are equal and opposite. What if the benefits would flow overwhelmingly to the rich, and the increased progressivity is only minimal? What if the Democrats are blathering endlessly about benefits to the rich, but the GOP rarely discusses the increased progressivity and instead makes their case based on how the tax cuts will supposedly create jobs?

Considering that there are an infinite number of true statements about the any given policy, I'm not sure how you could possibly determine that two were equal and opposite in a rigorous and scientific way.

noblesse:
Is it right that he looked at that dichotomy only?That’s not just a representative example?  

That is only one of his three methods, but yes, that method appears to look only at that one dichotomy.

GeoffB:
It seems like there is an additional methodological component in the book that is not in the paper?Groseclose mentioned that one of his conclusions was that the media bias identified skewed the perspective of the typical voter more liberal than they might otherwise have been.

You're quite right, the final chapters of the book where Groseclose goes into estimating the media's impact on elections is also not based on any previously published research.

I think that so far, almost all the discussion I've seen of Groseclose's book has centered on his analysis of media bias. Someone who isn't fully persuaded of the validity of Groseclose's conclusions on that issue is certainly not going to accept what he says about the media's impact on elections, as the latter set of conclusions are completely dependent on the former.

Presumably the people making snarky comments in this thread haven't read Groseclose's explanation of why his methodology is objective:

Is this book biased? On one level, it matters not a whit where I was born or what my political views are. The methods that I use to measure media bias are completely objective—indeed, a computer executes them.

Steve&#32P:
Orin,
I think that’s about right. Political scientists have a widely accepted methodology for determining how liberal or conservative a politician is, based on how they vote on a number of key votes selected by interest groups. Groseclose’s insight was to use think tank citations as a way to map legislators’ ideological ratings onto media outlets.  

This is generally not the case any more. Political science has moved away from interest group ratings as an indicator of ideology for scaling techniques, such as the NOMINATE scores produced by Poole and his co-authors. For more information on that see their website. http://www.voteview.com/

Kazinski:
Doesn’t his peer reviewed paper discuss the methodology?Is it available online?
I do question the PQ ratings, any rating that has Richard Nixon, the President that imposed wage price controls, and the EPA on us, as 12.3, seems like it is measuring partisanship rather than ideology.Nixon certainly was plenty partisan, and anti-communist certainly, but not nearly as conservative as say Goldwater.  

The PQ he reports for Nixon is based solely on Tricky's time in Congress, 1947-52, as PQ's for politicians are simply ADA ratings, adjusted in some way to make the ratings from different periods comparable.

Upthread, Josh Wright provided a link to the Groseclose-Milyo paper.

This is the Gasper paper often referenced as calling seriously into question the methods used by professor Groseclose, in case anyone is interested.

yankee: This negative review on Amazon, if accurate, looks pretty devastating. According to this reviewer, Groseclose’s methodology results in Handgun Control, Inc. being a liberal organization (77.3) while the NRA is a 45.6 (slightly conservative). The result is that a media report quoting an HGI representative favoring a gun control bill and an NRA representative opposing it would be coded as biased to the left. The methodology also results in coding the ACLU, which is a solidly left-wing organization, as 48.9, slightly right of center.  (Quote)

The NRA endorses both Republicans and Democrats, based upon their positions on gun rights. HCI primarily only endorses liberal Democrats. This is the result of the fact that there is far more bipartisan support for our 2nd Amendment rights than there is for banning guns. Hence, HCI is viewed as far more liberal than the NRA is under this methodology.

The ACLU, on the other hand, tends to be a mixed bag. While their stance on the Establishment Clause puts them squarely on the left, they do take the rest of the 1st Amendment seriously, especially regarding hate speech legislation, where their position is more to the right.

He used a computer? Then never mind. Liberals really must be vicious.

Mark&#32N&#46:
Presumably the people making snarky comments in this thread haven’t read Groseclose’s explanation of why his methodology is objective:
  

What more can you ask for in a study than unbiased addition and subtraction?

Mark&#32N&#46: The methods that I use to measure media bias are completely objective—indeed, a computer executes them.

I'm beginning to get the sense that all of this is elaborate satire. Betcha that when Dilan Esper finally gets to the bottom of the funding question, he'll find none other than Steven Colbert. (The Groseclose / Colbert tiff being a misdirection move, obviously.)

Now I'm going to return to my research: developing software that provides "Tim Groseclose" as the answer to the question, "who's the most biased researcher out there?" Hooray for unbiased computers!

A Berman,

Apparently, the following two things were both simultaneously true with regard to George Bush’s proposed tax cuts:

1) The wealthiest taxpayers got more money back than other groups.
2) The tax structure became more progressive.

As liberals were against the tax cuts and conservatives were for them, the first statement was construed to be a fact which might affect a voter to favor the liberal position, while the second statement was construed to be a fact which might affect a voter to favor the conservative position. So Groseclose and Milyo measured how often each of those two equally true statements were mentioned in newspapers.

This illustrates one problem. The second point is open to considerable question. The first is not. Treating them as somehow equal, and balancing each other ideologically is tilting the scales.

1) Whether or not the tax cuts were progressive is an unquestioned fact is not as important as whether or not it was an unquestioned at the time the proposal, debate, and vote. The only Gale, Piketty, and/or Saez studies of the Bush tax cuts that I could find were from years after the debate. A Picketty Saez study from 2007 couldn't possibly be used by the news media in 2001 to rationalize not mentioning the progressivity of the tax cuts if it was the common belief at the time that they were progressive.

2) This Picketty Saez study I found:
http://www.cbpp.org/cms/?fa..., has a very different definition of what is regressive or progressive--total amount of money aggregated over population--than the way people often think about these things--tax rate paid by individuals with a given income.

Jestak:
Orin, what you’ve summarized pretty accurately is the methodology of the Groseclose-Milyo paper from 2005.In the book, he supplements that method with some others that he has not previously published.One of them, the analysis of media coverage of the Bush tax cuts of 2001 and 2003, I discussed in a comment on Groseclose’s post from earlier today.Briefly, I found that approach flawed because Groseclose tried to equate two statements about the tax cuts that were not really equivalent.The claim that Bush’s tax code made the tax system more progressive isn’t an unquestioned fact; I cited research by leading economists (William Gale, Thomas Piketty and Emmanuel Saez) which reached the opposite conclusion.  

Crunchy&#32Frog: The NRA endorses both Republicans and Democrats, based upon their positions on gun rights. HCI primarily only endorses liberal Democrats. This is the result of the fact that there is far more bipartisan support for our 2nd Amendment rights than there is for banning guns. Hence, HCI is viewed as far more liberal than the NRA is under this methodology.

Perhaps Brady should be viewed as more liberal than the NRA, but a story about a gun control bill where somebody from Brady is quoted in support of it and somebody from the NRA is quoted against is (generally speaking) a balanced article. But under the methodology of this study, it's coded as a case of significant left-wing bias, whereas a story that did not quote the supporter and only quoted only the opponent from the NRA would have been coded as only slightly biased to the right. A methodology that produces results like this seems pretty transparently silly, unless the professor has some justification for it.

Byomtov: This illustrates one problem. The second point is open to considerable question. The first is not. Treating them as somehow equal, and balancing each other ideologically is tilting the scales.

Also, it's weird to use only those facts as the inputs to the model. Relative to the degree of tax savings for the rich, tax structure progressivity is a subtle concept that sadly (or perhaps rationally) some consumers of the mass media aren't familiar with. I suspect that much of the media's "balancing" on the tax issue was accomplished by quoting someone for the proposition that tax cuts are good for the economy, etc. That kind of contention may be more debatable, but it's equally (or more) relevant to evaluating bias.