WEEK 13:

PANEL DESIGNS, ECOLOGICAL FALLACY

Panel Designs

Problems with cross-sectional surveys that gather data at only one time point include:
1) Inability to study change. You only have data at one time point.
2) Hard to make recursive causal assumptions. With data at only one time point, if a person is a conservative and a Republican, what were they motivated by? Did they become a Republican because they were a conservative, or did their Republican Party identification convince them to become a conservative?

Panel design definition: the same people, are asked the same questions, at two or more time points. Each time point is called a wave.

Problems with panel designs:

• 1) Expensive cost. Because you are doing more than one poll, you may double or triple the cost of doing a survey at only one time point. This is a big problem.
• 2) Tracking movers. Did they leave any forwarding address? If you can’t track down people included in the first wave of the panel, then your later waves have a smaller sample size. The smaller sizes may cause statistical inference problems, or small N size problems in multivariate tabular analyses.
• 3) Panel mortality, which can cause a biased later sample in your later waves. Some people cannot be recontacted. People die, so your later waves may no longer be representative of the population. If you can’t recontact some people, or they refuse to be re-interviewed, then the later waves may no longer be representative of the population. So you have to check to see how representative of the population your later panel waves are, and make some correction for that problem.
• 4) Measurement conditioning. People may remember some of the questions asked in the first wave, and they may have studied up on the subject; so their answers in the later waves are affected by the first wave administered to them. You can check on this by conducting a new cross-sectional design with the same survey questions at the same time as you conduct the later waves of your panel design. The new cross-sectional design would not have been asked any questions previously, of course. However, this increases the cost of your survey even more.

Examples of panel studies:

1) National election studies panels of 1956-58-60, of 1972-74-76, and of 1992, 1994, and 1996. The second set was able to study the effects of Watergate. Major finding of these panels is that party and issue attitudes affect each other in a reciprocal sense. Also, political efficacy (the sense that you can influence government) affects political participation (such as turnout, campaign acts), and political participation affects external efficacy (your belief that public officials are responsive to the people).
2) 1980, 4 wave U.S. national election study. It examined the effects of campaigns on voters. Major finding was that President Carter lost because of voter dissatisfaction and perception that he was a failed leader, not because of ideological issues.
3) The M. Kent Jennings panel of high school seniors and their parents. Wave 1 was in 1965, wave 2 in 1973, and wave 3 in 1982. The subject of the study was socialization and the persistence of attitudes over time. A major finding was that political attitudes (including partisanship) tend to stabilize around age 30.

We’re running out of time in this course, and I normally don’t talk much about this subject. I’ve only used a panel design once, to study how voters seek to acquire cognitive consistency in their presidential candidate choice. They favor candidates whom they believe agree with them on important issues. In other words, they sometimes see what they want to see, so that their political world is consistent with their pre-existing attitudes. Check out my publication:

Balance Theory and Political Cognitions

https://journals.sagepub.com/doi/abs/10.1177/1532673X8100900303

This cognitive consistency theory may help explain the amazing stability of Trump’s approval ratings. He gets impeached, and his popularity goes up 1 point. The Covid virus nearly shuts down our country, and his popularity drops only 2 points. He gets impeached a second time after appearing to support an Insurrection on January 6 after his reelection loss, and yet polls afterwards showed him virtually tied with President Biden. Trump gets indicted for separate felonies by four different courts, and he today has a slight lead over Biden. This cognitive consistency theory can also explain how some liberals and Democrats really seem to hate him, even if he does something liberal (justice reform, paid family leave for federal workers, no more wars). This theory may also increasingly explain how aging political leaders such as President Trump misperceive reality to be consistent with their pre-existing attitudes. Trump thinks he is such a great President that he should be on Mount Rushmore (Really, he asked the Republican governor of South Dakota what the process was to be added to that monument to former “great” Presidents.). Obviously, “great” Presidents must be rewarded by voters by getting re-elected, like FDR (Franklin Roosevelt) was elected four times. Therefore, Trump must have been re-elected. “The election was stolen!!”

AGGREGATE DATA (ECOLOGICAL FALLACY)

Ecological fallacy is the incorrect assumption that relationships existing at the aggregate level also exist at the individual level.

The first example is from the 1990 census- percent foreign born and percent college degrees aggregate relationship, measured at the state unit of analysis.

STATE.....% FOREIGN BORN.....% COLLEGE DEGREE
Mass...................9%......................20%
N.H....................5%......................18%
Vermont................4%......................19%
N.Y...................14%......................18%
N.J...................10%......................18%
Alab...................1%......................12%
Ark....................1%......................11%
La.....................2%......................14%
Miss...................1%......................12%
Ga.....................2%......................15%
S.C.................2%......................13%

The above table suggests that the foreign born are more likely to have college degrees than are U.S.-born adults. Such a conclusion would be committing the ecological fallacy. In reality, the data are merely indicating that states (not people) with a higher percentage of foreign-born residents are also states that happen to have a population that contains a greater percentage of college educated adults, compared to states with a lower percentage of foreign-born residents. The relationship between foreign born and education is a spurious one (non-causal); states with well-funded education systems tend to be located in the Northeast and Midwest, and those are the same states where many immigrants historically have settled.

A second example is from the 2010 census- it is % black of a state and % Republican presidential vote at the state level unit of analysis.

STATE.....% BLACK.....% REPUBLICAN PRES. VOTE IN 2008
Alabama........26%.............60%
Arkansas.......15%.............59%
Georgia........31%.............52%
Miss...........37%.............56%
Iowa............3%.............45%
Minn............5%.............44%
Penn............11%.............44%
Wash............4%.............41%
Wisc............6%.............42%

The above table suggests that African Americans are more likely to vote Republican for President than are whites. Such a conclusion would be committing the ecological fallacy, since the table provides aggregate data, not individual-level data. The table in reality is merely showing that states having a high percentage of African Americans are also states that just happen to be more likely to vote Republican for President, compared to states having a lower percentage of African Americans. The relationship between race and vote at the state unit of analysis is a spurious, non-causal one. African Americans merely happen to be concentrated in southern states, since such states historically relied on slavery on large plantations, and southern whites tend to be more conservative politically than are whites in the north.

So, there is a big problem in using aggregate data. Therefore, we have relied heavily on individual level data for our studies, such as the Mississippi Poll. But then we started getting low response rates, and some really disillusioned people who supported “outsider” types of candidates started dropping out of the polls. So, I am the expert who writes the Mississippi chapter in books on Southern Politics. How can I analyze the results of the 2016 and 2020 presidential elections in Mississippi without a poll? Well, those of you in my Political Parties class can read my chapter in one of those books. I pooled (combined) the Mississippi polls from 2002 thru 2014 to have a large enough sample size for each of Mississippi’s 82 counties. I therefore got a measure of how people in each county responded to key issues such as party identification, abortion, economic issues (jobs, health care), and racial issues (affirmative action, minority aid). I then put them (plus racial composition of each county) into a multiple regression equation explaining the presidential vote result in each county. So county was the unit of analysis- an aggregate study. I tested to rule out the ecological fallacy by showing that each variable was related to past presidential votes at the individual level in the same ways. As we talk about in my Southern Politics class, race continues to be a major factor in the South. The racial composition of a county was hugely important in affecting the presidential vote in Mississippi. Hillary Clinton won every black majority county, and only two of the white majority counties (one was Oktibbeha). A distant second in importance in the multiple regression equation was abortion (pro-life were more pro-Trump counties), and the other factors had no independent effect. Race and abortion played a similar role in the 2020 presidential election results in Mississippi, with Biden able to win only three white majority counties. I also examined the U.S. House elections in 2016, where race of the county was also the most important factor. Second in importance was incumbency (counties with a Republican congressman voted more Republican than counties with a Democratic incumbent, even after controlling for party identification). Third was again abortion attitudes. The editors loved the analysis, and we had to make up additional tables showing it. My approach has influenced some of the authors of other state chapters in the most recent 2020 presidential elections book, which is perhaps the most interesting, readable, and valuable book of the entire presidential elections in the South book series (it started in 1984).