WEEK 6: QUESTIONNAIRES, TEST REVIEW

Wording of survey questions considerations:

1) Avoid loaded or leading questions. Also, ensure that you have reasonable opposing categories.

2) Avoid double barreled questions, which ask about two separate but related issues, such as do you support higher taxes for teacher pay and a new stadium. Break them up into two questions.

3) Avoid complex questions, often having double negatives. In 1981 Miss. Poll, we asked an agree-disagree item, “do you favor or oppose a constitutional amendment stopping all abortions except those needed to protect the health of the mother.” Many people said I oppose that item, because I oppose abortion. We had to explain, then you would favor an amendment stopping all abortions. Uh, I guess. But it would not stop abortions needed to protect the mother’s health. I’m even lost at this point. We soon switched to a simpler agree-disagree item, “do you agree or disagree that a woman should be able to have an abortion as a matter of personal choice.”

4) Avoid unbalanced alternatives. A national poll in 1981 asked respondents whether they thought that blacks were treated the same or worse by public officials. Our poll asked: “How well do you think blacks are treated by public officials in Mississippi? Are blacks treated worse than whites, better than whites, or about the same?” Interesting results, as the races were split on this issue with some whites actually saying that African Americans were treated better than whites.

5) Acquiescence bias. This is an agreement bias, often found on agree-disagree items. An example is the state spending items in the Mississippi Poll. “Now I'm going to ask you about some issues facing state and local government in Mississippi. As you know, most of the money government spends comes from the taxes you and others pay. For each of the following, please tell me whether you think state and local government in Mississippi should be spending more, less, or about the same as now. How about. . .?” There are about 10 different state programs, such as Public Grade School and High School, Streets and Highways, Health Care and Hospitals. It is very easy for people to just say Spend More on all or most of them. Fortunately, you can still compare across the programs to identify which of the programs is most and least popular, such as public education’s popularity. An advantage of such a long list of items with the same response categories is that you can get through the interview very fast, so that makes up for the other 50 or so items that we ask about.

6) Sensitive personal items- people may refuse to answer them. So for age, don’t ask their age, ask in what year they were born. Don’t ask their income, list eight categories that are in $10,000 units, and ask them to choose the right intervals. I initially thought that asking their race might be a problem, so we asked: are you white, black, or what, which students modified by replacing what with the word “other”; it ended up not being a problem, since people answered the question.

7) Social desirability- people may lie when asked about sensitive social issues, such as their sex practices, and what their views are on race or sex equality issues. We use pretty well-established indicators, so we do get fairly honest responses.

And now we review for the first essay exam! The easiest way is to just take a look at a recent test, since this year’s test is likely to be very similar:

POLITICAL ANALYSIS, 2021, FIRST EXAM (100-point test)

1. (12 points) What are the 6 characteristics of the Behavioral era of political science (according to the lecturer), and briefly explain why each of those characteristics is important by applying it to a specific study?

2. (8 points) What are the 4 characteristics of a “good theory,” according to the lecturer, and briefly explain how each one operates? You can explain how each characteristic operates by taking an existing theory mentioned in class and showing how the characteristic does or does not apply to it.

3. (10 points) What are the 5 hypotheses in the model below? Use the precise wording that is necessary for a hypothesis in each of your 5 sentences.

Sex

(women)

Party Id. 2020 Presidential Vote

Age (Democratic)) (Biden)

(young)

This figure did not print out correctly. It has four variables, with Sex and Age being two outside variables (on the left side of the page); Party Identification is the intervening variable (in the middle of the page); the 2020 Presidential Vote is the dependent variable (on the right side of the page). The five arrows are: from sex to party; from age to party; from sex to vote; from party to vote; from age to vote. Categories that go together are women-young-Democrat-Biden, and vice versa.

4. (10 points) What is the unit of analysis (individual, county, state, nation) for each of the following 5 studies? A one-word answer for each is desired.

A. A study of abortion laws in each nation, based on a worldwide study of 180 different nations determining answers to the following questions: extent of legalized abortion, based on the percentage of women desiring abortions who are able to obtain one; percentage of population that is of the Catholic faith; percentage of population that is of the Islamic faith; strength of the left-wing or socialist parties in the Parliament; percentage of women who are college educated.

B. A study of effectiveness of lockdowns in the American states, based on the following information about each state: percentage of businesses shut down; number of days that businesses were shut down; Coronavirus cases reported per capita; number of coronavirus deaths per capita.

C. A study of voter turnout in Mississippi, based on the following information on each of the 82 counties: average hours per week that circuit clerk’s office was open; percentage of county adult population that is college educated; racial composition of the county; percentage of county that are convicted felons; percentage of adults that voted in the last election.

D. A study of policy preferences on LGBTQ issues in the United States, based on a national survey of 1500 individuals, asking each of them the following questions: should transgender females be able to compete in high school and college women’s sports; should transgender people be permitted to serve in the U.S. military; should private businesses be able to refuse to serve LGBTQ persons because of the owner’s religious beliefs; party identification of respondents; ideological self-identification of respondents; frequency of church attendance.

E. A study of migration in western democracies, based on the following information drawn from each of 50 nations: percentage of population that is foreign born; percentage of foreign-born population that are citizens of that country; percentage of foreign-born population that are displaced persons and living in refugee camps; what continent the country is located on; strength of left-wing parties in the Parliament; percentage of foreign-born persons who are not employed.

5. (10 points) What is the level of measurement (nominal, ordinal, interval) of each of the following indicators: A one-word answer for each is desired.

A. Do you strongly agree, agree, disagree, or strongly disagree with the following question:

“Confederate statues honor people who fought against our country, and they should all be taken down?”

B. Which recent American President do you most admire? Donald Trump, Barack Obama, George Walker Bush; Bill Clinton; George Herbert Walker Bush; Ronald Reagan.

C. How many years have you lived in Mississippi?

D. What state are you a resident of?

E. How would you rate the job performance of President Joe Biden? Excellent, good, fair, or poor?

6. (10 points) A public administration graduate student was hired to test the reliability of an Alabama city government’s indicator of job satisfaction of its municipal workers. 1000 workers of the city were surveyed in October of 2020, and the same people were re-surveyed in December of 2020. She conducts a test-retest of the job satisfaction indicator’s reliability, and obtains the following table of workers’ responses at 2 time points. Is this a reliable indicator? Yes, or no? How could you tell, and be specific and include the definition of reliability in your answer?

JOB SATISFACTION LEVEL IN OCTOBER 2020

Job satisfaction in December of 2020	Dissatisfied	Mixed	Satisfied
Dissatisfied	100	130	100
Mixed	110	140	120
Satisfied	90	130	80

7. (10 points) This is a Construct or Criterion Validity Test.

	Strong Dem	Weak Dem	Indep Dem	Pure Indep	Indep Rep	Weak Rep	Strong Rep
% favoring stricter gun control	82%	68%	67%	46%	31%	40%	24%
% favoring affirmative action in universities	38%	24%	22%	12%	9%	11%	6%
% favoring paid family leave	79%	73%	78%	64%	61%	63%	58%
% favoring transgender bathrooms freedoms	69%	56%	66%	49%	32%	37%	19%
% favor death penalty for murder	51%	64%	53%	78%	80%	83%	87%

It is drawn from the 2016 National Election Study conducted by the University of Michigan (with one minor revision). It relates the questionable 7 category indicator of party identification to five well-established issue items. For each row, each cell entry is the percentage of people having that particular partisan orientation (strong Democratic, weak Democratic, etc.) that agrees with the opinion at the extreme left of that row. For example, in the first row and column, 82% of Strong Democrats favor stricter gun control laws (and 18% therefore oppose it). Also, please note that the five well-established issue items are sometimes liberal policies and sometimes conservative policies, so that the highest percentages will sometimes be in the Strong Democrat category and sometimes in the Strong Republican category, so the percentages will either decrease or increase going across the rows depending on the ideology of the policy.

Remember that party identification is measured at the ordinal level of measurement, so the validity question that we are addressing is whether this party indicator is actually a valid ordinal indicator. For each row, just circle each pair of adjacent categories that exhibits a validity problem (in terms of ordinality of the questionable indicator). No other answer is needed.

8. (8 points) What are the four components of Informed Consent? Briefly explain each.

9. (6 points) What is the difference between Anonymity and Confidentiality?

10. (10 points) What are the problems with in-person surveys, the problems with mail surveys, and the advantages of telephone surveys, according to the lecturer? Short sentences are fine.

11. (6 points) What are 2 major historic problems with polls, and give a specific example of each, and explain why and how there were problems?