I started this as a comment in an earlier Gallup thread, and it got so long that I decided to post it as a diary. It's largely a stream-of-consciousness commentary on various polling outfits.
But as an intro, here's a quick take on where I think things stand in various battlegrounds (please feel free to ask if you want me to elaborate on my reasoning or if you disagree; I just don't want this post to be be book-length). This isn't based on strict averages, though averages plays a role; Bush's numbers relative to 50. 2000 State Polls and state trends also play a factor)
OH--Total toss-up. I think this is probably the single state where bad polling is influencing folks; look at the three OH polls that have Bush surges; Gallup LV (shocking), Strategic Vision (again shocking), and Zogby Interactive (which is garbage, even though it leans Kerry). The polls thst I find reputable (considering only those who have done post RNC OH polling) all show a dead heat. Other things to consider: a) jobs is a major, major issue; b) LV polls will skew more R in 2000 red states, due to assumptions about turnout; c) OH is extremely capabable of turning blue when it's truly contested all the way through (see '92, '96, and what should have been in 2000; d) 200,000 newly registered and motivated Dems, and an extensive ground game. I'm telling y'all, Kerry's economic message resonates more in the rust belt than in any other contested region.
Toss-ups: FL, NV, WI--Basically, if FL were any other state, I'd say it leans Kerry. If all the votes had been counted, Gore wins here SOLIDLY (some 50,000 votes); since then, the demographic shifts favor the Dems, Bush has lost support amongst Cubans and the elderly (Jeb's ruling on importing prescription drugs). Even more energized Dem base since 2000; Dems with a substantial lead in new voter registration.
Slightly Lean Kerry: MN, IA, NH, NM PA (PA is borderline; I really think it's more lean K)
Lean Kerry: OR, Maine (only here b/c of lack of data points; ME will go Kerry, but who knows about that one Cong. District)
Safe Kerry: WA, MI, NJ (NJ ia absolutely safe)
Slightly Lean Bush: CO (gotta put it here now, too many polls show it a tossup), WV, AR (until I see evidence to the contrary)
Lean Bush: MO, AZ, VA
Safe Bush: NC, TN, LA
______________
Now, for the long part of this post, my pollster rankings (and I'm fully prepared to get lectured on poll-shopping by at least two folks)and comments:
One thing that I'll do when I have more time (being the huge nerd I am--though I'll have to show y'all a pic sometime, I'm kinda meatheadish) is analyze the volatility of the 2000 polls, i.e. see which jumps around the most. I'm willing to bet that Gallup is in the top 3, and probably number 1. They did, after all, have Bush up 52-39 about two weeks before the election.
This cycle, Time has been awfully volatile, as has the WaPo/ABC News to a certain extent.
Off the top of my head, here are the polls (more or less in order) that I really look to give me at least a reasonable take on where things are.
- Anneberg--this is partially b/c I studied at Annenberg with Kathy Hall Jamison, and she just gets it; she's one of the few folks you'll see on CNN who'll openly discuss how the media goes wrong in shaping perceptions; but I love that Annenberg don't even bother with the horserace question until awfully late in the game, and they use a huge sample size.
- Pew (the guy who used to run the Pew Poll actually was another professor of mine at UPenn, and much of what I know about polls (I have some very limited polling experience from when I was an undergrad) I got from him, so again, some partiality)-another poll that uses a huge sample, and really asks relevant questions. They don't race to get their results out; they're more inclined to get an accurate sample, and they are possibly the only polling outlet that I've seen able to witness methodically explain a major shift back to Kerry without the pollsters' head exploding.
T3. Harris--I can't put these guys higher, only b/c they don't sample often enough for my taste; but they're up their w/Gallup in terms of longevity (originally Dem leaning, Lou Harris was JFK's pollster), and, unlike Gallup, they nailed 2000 almost perfectly, to a greater extent than Zogby did (final poll had it 47-47, only national poll on 11/5 or 11/16 showing a basically exact dead heat).
T3. Zogby Telephone--I have to say, I'm wary of Zogby's overall personal integrity right now; I don't really see how a reputable poll can honestly try to draw real conclusions from that interactive bullshit (and note, just because a poll looks reasonable doesn't mean its legit; anyone can massage their numbers enough to make them reasonable--from a methodological point of view, it's largely impossible to do a random sampling over the internet, unless the participants were first identified randomly by some other means--a true random internet sample would basically require either a list of every known email address for the studied population, or else a way to generate email addresses randomly).
But the telephone survey has an undeniably excellent track record for recent presidential races.
4. CBS News--they actually came closest in their final 2000 poll to predicting the margin (they had Gore, 45-44; Zogby had it Gore 48-46; so Zogby got the vote share more accurately, CBS got the margin more accurately)--another, good, solid poll that asks lots of relavent questions, doesn't inflate job approval or favorability ratings and doesn't use a bizarre LV model; I'm not one to freak over what the polls are showing right now, but the poll that hit me the hardest this whole cycle was when CBS had Bush up 8.
T5) NBC News/WSJ--my sense is that this poll, as a matter of methodology, slighly oversamples Repubs--but that's fine, they're consistent, not only in horserace, but in the re-elect, job approval questions, etc. Note that since I'm 100% convinced, and always have been, that this race is basically always gonna be a statistical dead heat at equilibrium, I'm partial to polls that don't often show wild fluctations in either direction.
T5) CSM/IBD--nothing they've ever released has led me to believe that they're not a fairly reliable outfit (though they do use a bizarre method of measuring job approval, which, incidentally, isn't favorable to Bush at all).
7. Fox--note that I'm being very kind; I'm not putting acknowledged partisan pollsters on this list (hence, not ranking D Corp) and I'll be generous and call Fox "not openly partisan." What I have noticed though, both in 2000, and this cycle, is that Fox/OD almost always haz eminently reasonable horserace numbers--when I say reasonable, I basically mean compared to the average of other polls released during a similar time frame--and I appreciate that they don't push leaners very far. Final 2000 poll: Bush 43, Gore 43, on 11/2. Of all of the final 2000 polls taken in November, Fox joined with only Harris to predict a tie. I do think that the Fox internals have a tendency to skew Bush a little bit. Basically, Fox released one terrible poll this cycle, that actually had Kerry winning in the battlegrounds and Bush leading by 1 in 2000 Blue States; if it hadn't been for that blatant garbage, they might be higher on this list, as fas as national top-lines.
T8. AP/Ipsos; first of all, I don't fully trust any poll that never publishes its internals; all you get from these guys is the Fournier article; secondly, and this really isn't meant to be derogatory, but it's my opinion that United States political polls should largely be conducted by firms that are "expert" United States pollsters; I mean, Ispos couldn't even get close in Canada, so I'm not sure why I'd trust them for the U.S. Presidential race.
T8. ABC News/WaPo poll; I don't like this poll, no matter what it shows. It has consistently inflated Bush job approval ratings (i.e. around Abu Ghraib, when at least 4 pollsters had Bush at 42, these guys still had him around 50); they are prone to both massive outliers in both directions, as well as volatile swings. They also, IMHO, have a tendency to really oversample White Southerners.
- Newsweek--Let me say first that I don't think Newsweek in general runs a terrible poll. But I just can't forgive the irresponsibility they displayed in racing their post R convention poll out there, knowing that they had grossly oversampled repubs, and making no attempt to account for it; obviously, they rushed their post DNC poll out, also, but the partisan breakdown wasn't nearly as skewed. Honestly--and I don't care if this sounds like poll shopping--but any poll that has more R's than D's by more than 2-3% just isn't representative; pollsters should either note that at the top of their stories, or weigh their samples, I don't care which. But besides their convention polls, I can generally accept Newsweek results.
- LA Times--Again, not a conspiracy guy. But I always wonder about a poll that has only four data points; of those four, Kerry had a significant lead in one, after which Dowd ripped the organization; and the poll has basically been all roses for Bush since, both pre and post Dem Convention. One of the very few pollsters to never show Bush approval below 50, even when Kerry led by a bunch (LA Times and Time are, I believe, the only polls that never have had Bush approval underm 50)
- Time--another poll that shows hugely volatile swings, constantly inflates Bush job approval compared to the average; their most recent poll gave Bush a 56% approval rating; even this beyond reasonable Gallup Poll out tonight only has Bush at 52%. I mean, they must be kidding. They've had polls that had Kerry up 7 with Bush job approval rating at 52%; sometimes there's an approval gap, but not usually one so large.
- Gallup--in all honesty, Gallup is probably a superior polling organization to the idiots whon run the poll for Time; they probably at least are aware of what proper methodology and reporting shound entail, at least they ought to be. But they have to be last, because they know better. They know their likely voter model skews R (they admit it in their book, "Polling Matters" by Gallup Director Frank Newport; the book also outlines Gallup's absurb LV screen); they know their LV model hasn't nailed a presidential race since 1984 (thanks to Ben P for pointing out the NCPP.com; see this site for a great history of presidential polling); they know their national and state polling don't match up at all; they damn well knew that their post DNC poll was BS--but they report the numbers anyway, and NEVER will you see any kind of acknowledgement that maybe their sample is an outlier. I sound like an idealist all of a sudden, but of all these outfits, Gallup damn well knows better.
What's ironic is that I have almost no doubt that in the final week of the election, Gallup will at least be in the ballpark. As a hypothetical, suppose the race basically doesn't move again (and it will, so don't worry)--that, to me, would suggest a Bush lead of about 2-3 points; I can assure you, Gallup won't be showing any +14 bullshit the last week, because the final number gets remembered, gets analyzed by the National Council on Public Polling, etc. Somehow, their LV model won't magically add 5-8 points to Bush's total. That doesn't mean it won't be wrong; they've been wrong the last few cycles; but it'll at least look reasonable.
So Gallup gets rated last; somewhat for their results, but mostly for the way they conduct themselves. Gallup is by far the most likely outfit to call a Bush margin of 5 a "widening lead" while calling a Kerry margin of 5 a "dead heat, with Mr. Kerry unable to take advantage of such and such." They're the only polling outfit that will ADMIT that they have no experience in state polling but then parrot their results like gospel.
Fuck Gallup. Just to throw out some data: of all the WI polls this cycle, who shows the largest margin? Gallup, with B +8 in LV. How about Ohio? Discounting ZI, it's the Gallup RV poll that showed Kerry +10 (and +3 amongst LV, of course); and MO? Why, shocker, Gallup, with its recent Bush +14.
_
_______________
I'm not a conspiracy guy; I don't think Fox News is a conspiracy, it's too overt to be one. The CNN/Gallup collusion actually makes me wonder more than most things do. I pride myself as a fairly objective news viewer; unlike a few on here, I don't think that any piece that makes some kind of not-entirely-negative Bush reference is part of a diabolical corporate media plot; I think the media bias (which does exist) is pretty easily explainable: it's a combo of de-regulation, the "balanced doctrine," the fact that the Right is much better at media manipulation, and it's the fact that these guys spent 3.5 years telling folks how popular Bush is, and they don't want to look like assholes now.
But the hostility on CNN towards Kerry really makes me wonder. It pushes me to the brink of believing that CNN's ratings are so much in the tank that they're desperate to cut into the Fox viewership to improve their market share.
Looking objectively at certain idiot media commentators; let's compare Chris Matthews and Wolf Blitzer. They're both idiots, don 't get me wrong, but in such different ways.
Now, Matthews will almost invariably miss the key point of whatever discussion he's having; he'll divine some weird cultural notion (the guy reminds me way too much of Reagan, mixing his movies and politics--see his book, "American" which is one of the most pathetic literary attempts I've ever come across) and not be able to shut up about it for days. He no doubt smeared Gore in 2000; but yet--he doesn't hesitate to after wingnuts (see Malkin, Dowd, Zell Miller); he was always against the Iraq war. Here's the big difference for me--even when Matthews repeats verabtim the GOP talking points, he's not smug about it; he'll usually do it in the context of trying to get a Dem to forcefully respond. I mean, if you see Matthews reading a poll that shows Bush up 14, I don't get the definite sense that he's happy about it. I think that, like most moderate adults, Matthews thinks Bush is a disgrace, but he balances it with the reflexive anti-Dem that many Dems leaners (including MANY on this board) are prone to. My basic point is that I don't see any overt bias from Matthews; he chooses the wrong stories, misses the point, and overemphasizes pretty much everything, but it's not malevolent.
Then you've got Blizter. Background as a major right-wing Likudnik (and my apologies if there are single issue Israeal voters reading this, I'm a Jew myself, and consider Israel a crucial issue--and the Likuds are sill nut jobs) Not only is this guy a total moron, but he takes pleasure in his parroting of the GOP points; he can barely hide the smile when Gallup shows up again with a new artifically created LV sample. Same with Woodruff.
That's why I think that, generally, CNN is a bigger disgrace than even Fox. The people who do "news" on Fox are no doubt partisan--but even from them, I don't usually detect the glee over supposed good news for Bush. Call me crazy, but I'd actually rather see Brit Hume analyze a poll than Blitzer. Same definitely goes for Van Susteren. O'Reilly is just an idiot and a truly, insecure malevolent asshole; and if I saw Hannity in the street, I'd beat the shit out of him; but you expect it from those guys. As far as I'm concerned, they're openly partisan, they can say what they want, with the laws how they are. Dems need to deal with that. And fact is, Fox actually spends considerable more time at least attempting to cover real news (not always of course, you've got lots of Laci on Fox) than CNN does. But when you get something like CNN, that many fools actually believe is non-partisan, and they are so clearly rooting for Bush, they've got a problem with me. Note that Aaron Brown is an exception (and Amanpour is a fantastic correspondent).
Man, this is getting long, I'm gonna have to post this as a seperate diary--but my very brief thoughts on other polls I didn't comment on above.
*Economist/YouGov--I like the large sample, but my comments about Ispos above hold, and I need to know more about the methodology. It's a nice large sample, but they claim that the sample has been "recruited." Now, if it was recruited randomly, it's totally legit; if not, then it's not. The poll is generally favorable for Kerry, and I don't see any incentive for the Economist to really want to deliberately boost Kerry, so I'm willing to at least take their numbers in good faith.
Quinnipiac--My guess is that they're very, very solid, but they don't do enough national sampling for me to really form an opinion. Same goes for Marist.
Zogby Interactive--Garbage. I tried justifying this for awhile, bascially thinking that Zogby wouldn't attach his name to something so fundamentally flawed. I wish it weren't so, but there's just no theoretical way to achieve a random sample with the method they use.
Dem Corps--I think they're outstanding, and the message testing is invaluable. I don't see a ton of evidence that they really inflate Kerry numbers, and they don't deflate Bush job approval or direction of the country, but they're still partisan. What I think that means, in this case, is that they'll always be sure to have a reasonable % of Dems in the sample.
Rasmussen--very uncertain. First of all, I will sometimes comment on day to day movement, and I really shouldn't, because they basically never have true statistical day-to-day movement in their national sampling; really only the trends ought to be considered, though this is substantially less the case since the doubling of the sample size. I find the Ras national and state numbers very reasonable; but they're no doubt applying weights, and I'm not sure how I feel about that on a permanent basis. I think polls need to be weighed in the couple weeks following a convention (or else the polls should come with a major disclaimer, or, even better, don't poll right after the convention at all), but beyond that, I'd probably rather see the real data. In any case, a polling outfit has ZERO excuse for not diclosing the sample breakdown, unless they know they got it wrong, and are hiding something.
Now, I basically think Ras in conflicted--he COMPLETELY blew 2000, more than any other pollster, probably 'cause he never weighed. He's an admitted partisan. And yet his analysis is weird--he is extremely objective when describing the horserace and has never deviated from saying that the race is basically a dead heat. HOWEVER--when it comes to Bush approval, handling of certain issues, the headlines and analysis on the site no doubt blatantly spin from Bush's perspective. You can see the conflict at work with the Time Poll--Ras probably deep down wanted Time to be correct, but he knew his numbers showed something different, so he was willing to come out and say Bush didn't have that kind of lead. Hence, I'm inclined to believe that Ras's national sampling, and state polling, is done honestly, which doesn't mean that there aren't methodological flaws with robo-calling and tracking polls in general.
Ras also needs to show me that he can nail races accurately before you can start to rank him with other major national outlets.
ARG--as noted in Salon, there's a little bit of shadiness here, because they don't reveal who funds them. My basic sense of the ARG polling is that they are pretty universally reasonable; but the state polls do tend to be more favorable to Kerry, and Bush's job approval ratings are actually lower here than on average. That being said, ARG technically joined Gallup in showing a negative Kerry convention bounce in the two way (49-46 from 49-45), so I don't get the sense that they're partisan. They are one of the few pollsters I am aware of who actually consider a state's particular partisan breakdown when sampling. The 20 state polls that they released today basically put everything where I thought it would be (except CO, I really didn't think CO was that competetive, but I'm coming around).
SUSA--they may be very good right before the election, and I know that Kos likes them--but before that final week, way too many outliers. I'll accept SUSA polls, just because they get get really good late, but these need to be looked at with cauation, IMHO.
Finally, some major caveats about all the polls in general:
- 99% of them do not take turn out into effect, and the one that does (Gallup) uses it to lump points onto Bush's total.
- All these LV polls are almost entirely missing any new voters this cycle, including the youth, and newly registered folks.
- There are without a doubt issues with cell phones, that I'm not sure to what extent, or if this 'causes the polls to skew in either direction.
- The polls consistently represent the folks who are really in unfortunate financial situations (i.e., no permanent residence, no phone); sadly, it's likely that not many of these folks vote, but every little bit matters in a tight race.
- Always consider the MOE. A great way of checking a lead in a certain poll is statistically significant, is to go www.arg.com.
- A little off topic--tightened NY and NJ polls are not, in this cycle (in my opinion of course) indicative of national Kerry weakness; if this were 2000, these would really worry me; but being here in NY, I can almost categorically say that the tightening in these states is due to the 9/11 focused convention and then 9/11 itself. It's diffcult, unless you were here in 9/01 to fully observe the complex relationship that Bush had/has with New Yorkers; basically, lots of folks are quite capable of hating everything about Bush right now, but still remembering--and being swayed by--how much some of them loved him and needed him right then. It has nothing to do with Bush, per se, but with the need to rally around the president in a time like that, and then being reminded of those emotions. It won't last long..
My apologies for such a long, rambling post. All comments and critiques more than welcome.
In any case, this comment just turned into a diary--sorry for the ramble and the long post.