Replication
What we reran, and what we did not.
Whole group
We estimated the same vote equation in two files. One is the 2008 CCES, the online volunteer survey. The other is a Berkeley extract of the 2008 ANES, the face-to-face survey. An extract is a set of columns downloaded from the full file. Our script reads the finished CCES weight, column V201. It does not rerun YouGov’s distance match, the step that picks the closest interview for each target person. That function lives in the 2008 study guide, not in our code.
The files
The CCES file has 32,800 rows and 477 columns. Each row is one kept interview. The ANES extract has 2,323 rows and the ten columns we downloaded: weights, race, Latino status, age, the party items, and the post-election vote. It has no gender, education, self-reported income, or home ownership. So the full published covariate list, the traits the article’s model holds constant, can only be used in the CCES.
People missing a vote, a party, or a positive weight are dropped. The shared model uses 23,681 CCES interviews and 1,538 ANES interviews. The paper’s CCES sample is 25,814. Ours is smaller because we drop people who skipped income, said “not sure” on party, or named no candidate. The party slope, how much the vote changes per step on the party scale, still matches.
Beyond the article
These next numbers are ours, not the article’s. Party slopes for races lower on the ballot stay near −0.3: House −0.298, Senate −0.306, governor −0.289. On the same 17,318 respondents, adding ideology and three issue questions cuts the presidential party slope from −0.314 to −0.170.
| Check | Result | N |
|---|---|---|
| Shared party slope, CCES | −0.317 | 23,681 |
| Shared party slope, ANES | −0.317 | 1,538 |
| Full CCES party slope | −0.314 | 22,223 |
| Published CCES / ANES | −0.313 / −0.312 | 25,814 / 1,436 |
| CCES weights off | −0.320 | same people |
| ANES weights off | −0.293 | same people |
| State mean error / RMSE | +1.00 / 3.43 pts | 50 states |
| Article state mean / RMSE | +0.27 / 2.43 pts | 50 states |
Figure 1
Party coefficient in the shared model
N = 23,681 · SE 0.002
N = 1,538 · SE 0.010
N = 25,814
N = 1,436
Each bar is the party coefficient: how much the vote score changes for one step toward Republican on the party scale. The vote score is +1 Obama, −1 McCain, 0 another candidate. The shared model uses only party, age, and race, the traits both surveys contain. Bar length is the size of the negative slope. The scale runs from 0 to −0.40. N is the number of respondents. SE is the standard error, a measure of how precise the estimate is. Source: rerun bars from this replication; published bars from Table 5 of Ansolabehere and Rivers (2013).
Figure 3
CCES state vote minus the official two-party share
| Mean error, this rerun | +1.00 points |
|---|---|
| RMSE, this rerun | 3.43 points |
| Mean error, article | +0.27 points |
| RMSE, article | 2.43 points |
Each state error is the CCES two-party Democratic share minus the certified state count, in percentage points. Positive means the survey is more Democratic than the official result. Mean error is the average across fifty states. RMSE, the root mean square error, is the typical size of a state error, ignoring direction. Source: rerun rows from this replication; article rows from Ansolabehere and Rivers (2013).
Sources
- This group’s replication code and results
- 2008 Cooperative Congressional Election Study (CCES), Harvard Dataverse
- 2008 ANES Time Series, American National Election Studies
- 2008 ANES on Berkeley’s Survey Documentation and Analysis site (dataset nes2008)
- Ansolabehere and Rivers, “Cooperative Survey Research,” Annual Review of Political Science (2013)
- Federal election results, Federal Election Commission