The Dataset

Before analyzing these data, it is best to have some information about the dataset. First of all, it is useful to know how the data were collected and what sorts of error might be present in the data. The section on survey research on this website attempts to answer those questions by discussing survey research methods in general and the methodology behind the 2016 ANES survey in particular.

It is also important to understand the codebook which describes the dataset. The section on the codebook describes the information in the codebook entries for each variable and provides some additional information for some variables.

Survey Research Methods

The study of voting behavior generally relies on information from sample surveys. Aggregate election statistics from states or counties, another common type of election data, are useful for examining the patterns of election results, such as differences in the presidential vote among the 50 states, but such data are not suitable for an analysis that focuses on the individual voter. In order to investigate the factors that affect how people vote, we need information on individuals. Such information commonly includes data on voting behavior, attitudes and beliefs, and personal characteristics. As it is impractical to obtain this information for each member of the electorate, the common procedure is to draw a sample of people from the population and interview these individuals. Once collected, survey data are usually processed and stored in a form allowing for computer-assisted data analysis. This data analysis generally focuses on describing and explaining patterns of political opinion and electoral behavior.

The data for this instructional package are drawn from the 2016 American National Election Study (ANES), sponsored by the University of Michigan and Stanford University. Funding for the 2016 ANES came from the National Science Foundation (NSF). The study interviewed more than 4,270 respondents before and after the election, but we have included only the 3,649 respondents who completed both pre-election and post-election interviews. The study designers were interested in how people respond to different kinds of surveys, so they designed the 2016 ANES to be conducted both through face-to-face interviews and through the Internet. Approximately 27 percent of the respondents were interviewed face-to-face, while the other 73 percent participated in a Web-based interview. Only a portion of all the information collected by the study is contained in this dataset, and the selected data have been prepared for instructional purposes.

Efficient data analysis requires that the data be recorded, coded, processed, and stored according to standard procedures. This involves representing all information by numeric codes. For example, the information that John Smith is an evangelical Protestant might be stored by recording a value of “2” (evangelical Protestant) on R12 (religion) for respondent “907” (John Smith). This numerically coded information is placed on a storage medium—such as a memory stick—allowing the data to be analyzed with the aid of a computer.

Many people ask how it is possible to make any generalizations about the American public on the basis of a survey sample of 3,649 individuals. The truth of the matter is this: it is not possible to do so unless some methodical type of sampling scheme is used. If we just stood on a street corner and asked questions of the first 3,649 people who walked by, we could, of course, not draw any conclusions about the attitudes and opinions of the American public. If however, we have some kind of sampling scheme, a similar size sample can yield accurate generalizations about the American public.

A full explanation of the theory of survey sampling is beyond the scope of this instructional package. However, we can introduce some basics. The goal of any social science survey is to reduce the error in making generalizations about the population. Error can have two origins—systematic and random. The goal of proper sampling is to reduce both of these. Systematic error is much more serious than is random error, so we are especially concerned about it.

We can reduce error in a survey through good sampling.

The most basic form of sampling is the simple random sample. This involves drawing a sample from a list of all members of the population in such a way that everybody in the population has the same chance of being selected for the sample.

Simple random samples are not appropriate for many social science applications. Often, we want samples in which we are sure there are a similar number of subgroups (women, southerners, Latinos, etc.) in the sample as there are in the population. A simple random sample will not guarantee this. Stratified probability sampling comes closer to this guarantee.

Simple random samples are impractical in national face-to-face surveys for two main reasons:

  • There is no national list of American adults;
  • The sample would be scattered all over the U.S., making it very expensive to conduct face-to face interviews.

Therefore, a form of stratified cluster probability sampling is used in national face-to-face surveys.

The actual form of sampling depends on whether the interviews will be conducted in person or by telephone or in some other way such as on the Internet.

All errors can be of two different types—random errors and systematic errors. Random errors occur any time we seek to measure anything and often re-measuring the same thing will get a slightly different result—which is why carpenters say “measure twice, cut once.” Systematic error is when the device we are using to do our measuring is badly calibrated—like a ruler with the first half-inch missing. Surveys can contain both types of errors.

We can typically deal effectively with random error by phrasing our conclusions in probabilistic terms rather than certainties. So we say, for example, that one position on an issue is favored by more people than the other but that the difference is within the margin of error for a sample of the size taken for this survey.

We also attempt to minimize systematic error by making sure that our questions are well worded, that our interviewers are well trained, and by adhering to proper norms of developing and conducting surveys, such as those developed by the American Association for Public Opinion Research (AAPOR).

Potential sources of error in national surveys include:

  • The sampling procedure itself. Since surveys are based on samples and not the entire population, there is a certain amount of random sampling error in any survey. For a properly constructed national sample with about 2000 respondents, the margin of error is around +/-2 percentage points (meaning that 95 percent of the time the figure that would be obtained from interviewing everyone in the population would be within 2 percentage points of the sample figure).
  • For the large sample of 4270 in the 2016 ANES, the margin of error should be about +/-1.5 percentage points, but the complex nature of the sample makes it difficult to use simple margin of error calculations;
  • Certain unavoidable systematic errors in the sample. For example, the ANES does not conduct face-to-face interviews in Alaska and Hawaii. Also, homeless people and those in penal or mental institutions are not sampled.
  • Survey nonresponse. This is the result of not being able to contact a potential respondent. If non-respondents differ from respondents, this can be a big problem;
  • Refusals to cooperate with the survey by potential respondents. As the number of surveys and polls has increased in recent years, respondents have displayed survey fatigue, and refusals have increased over time;
  • Lack of candor by respondents. This involves questions that have socially acceptable answers (Did you vote in the election?) or socially unacceptable answers (Did you cheat on your income tax last year?);
  • Inability of respondents to remember past behaviors. For many students of politics, it is difficult to believe that some people just don’t remember who they voted for, or they remember it wrongly. ANES has found, for example, that after an election more people remember voting for the winner than actually voted for him or her. Other political behaviors are harder for people to recall (did you contact a public official about some matter in the past year?);
  • Respondents misconstruing survey questions as exams. This can result in them providing answers to questions that they really have not thought much about (what has been termed non-attitudes);
  • Badly trained interviewers. They may give respondents cues as to how to answer questions, or mis-record respondents’ answers, or falsify data;
  • Errors in the preparation, coding, and processing. These can occur when entering the survey into a computer data file.

It is important to be aware of the potential sources of error when working with survey data. Small differences in percentages may be the result of error and so be virtually meaningless.

The 2016 American National Election Study was conducted both face-to-face and via the Internet. For the face-to-face interviews, the interviewer read the questions from a laptop screen to the respondent and recorded his or her answers. For certain sensitive questions—such as household income—the interviewer handed the laptop to the respondent, who recorded his or her own answers, a process known as computer assisted self-interviewing. Great care was taken in identifying the sample, training interviewers, and in the conducting of the interviews. The care that ANES takes in conducting surveys results in data of a very high quality, but it also is expensive. Face-to-face surveys are more expensive to conduct than are telephone or Internet surveys because face-to-face surveys require interviewers to be sent out into the field. To ensure that face-to-face interviews are high quality, the field interviewers must be very highly trained, as there is little supervision when they are out of the office. Many researchers feel that face-to-face interviews yield “richer” data; the interviewer can ask a variety of follow-up questions, can spend adequate time making the respondent feel comfortable in answering sensitive questions, and can note any special circumstances about the interview that might have affected the answers given. Face-to-face interviews were conducted by Westat, with one interview conducted before the November election and one after the election. Some 1,181 respondents were interviewed face to face. Interviews were conducted in both English and Spanish.

Respondents for the Internet survey logged into a website where the questions were displayed and the respondent could answer them online. Some 3,090 respondents participated in the Internet survey. They also completed interviews before and after Election Day.

The response rate for the face-to-face 2016 ANES is approximately 50 percent while the response rate for the Internet survey was 44 percent. The dataset for this instructional package includes only respondents who were interviewed both before and after the election.

The data for this instructional module are weighted. Weighting a dataset is a technical procedure to correct the data for several basic factors. The goal of weighting is to produce a dataset that is more demographically representative of the population. When the data are weighted, some respondents count for more than one person (they have a sample weight greater than 1.0) and some count for less than one person (they have a sample weight less than 1.0). You need not be overly concerned about weighting, as the dataset is designed to be automatically weighted when you open it, and you will only sometimes notice that are you working with weighted data. Some small anomalies that you may occasionally notice will be the result of working with weighted data. More about Weighting.

In order to use a dataset, a codebook is needed. The codebook describes the dataset by providing a list of all variables, an explanation of each variable, and a description of the possible values for each variable. The codebook also indicates how the data are stored and organized for use by the computer. A codebook can be thought of as a combination of a map and an index to the dataset.

The Codebook

The 2016 SETUPS dataset consists of 202 variables drawn from the American National Election Study described in Survey Research Methods section above. These 202 variables contain information about the attitudes, behavior, and characteristics of the 3,649 respondents who were part of both the pre- and post-election surveys. In addition to the 202 substantive variables, the dataset also includes respondent identification numbers and weights for each case, but these are not variables that users will analyze and so are not included in this codebook.

The 202 substantive variables are categorical variables. To make the analysis simpler, we have recoded many of the variables in order to reduce the number of possible categories. Only a few variables have more than seven valid categories.

The 202 substantive variables are grouped into the following categories:

  1. Voting Behavior and Related Items
  2. Political Involvement and Participation Items
  3. Media Exposure and Consumption
  4. Candidate Image Items
  5. Presidential and Congressional Performance Items
  6. Economic Conditions
  7. Ideology
  8. Health Care Policy Issue Items
  9. Economic, Social Welfare, and Spending Issue Items
  10. Social and Moral Issue Items
  11. Women’s and Gender Issue Items
  12. Civil Rights and Race-Related Issue Items
  13. Foreign Policy and National Security
  14. General Political Attitudes and Orientations
  15. Demographic and Social Characteristics

The codebook contains information for each of these variables. To view the list of variables and the details for each one, select the “Browse codebook in this window” option on the main page of SDA. When the list of variables appears, you can click on any variable name, which will produce the codebook information. It does not matter whether you select the sequential or alphabetical variable list, as they contain the same information, just in different orders.

Most of the variables are relatively straightforward and need little additional explanation beyond what is in the codebook, but some variables require a more thorough explanation.

Also, please note that at the time of the preparation of this dataset and website, there were some variables that had not yet been completely processed by ANES. These are noted in the codebook and will be added as they become ready for use.

  1. Variable name. Each variable in the dataset has been assigned a unique name, which is preceded by a unique letter running from “A” to “R” (omitting “I,” “O” and “Q” because of their potential confusion with numerals). Note that each variable name always has two digits (e.g., A02, not A2). You should use these variable names when you specify variables in the SDA dialog boxes.
  2. Variable label. Each variable has been given a unique label. These labels provide a brief description of what the variable refers to (e.g., the label for A02 is “Presidential vote”). Because there are maximum allowable lengths for these variable labels, they sometimes have an abbreviated form.
  3. Text of question or description of variable. An explanation of the meaning of each variable is provided by an approximate description of the question asked or a general description of the variable. For example, the question text for A02 is “Whom did you vote for in the presidential election?”
  4. Value codes and value labels. The possible values for each variable are given in the codebook. Both the numeric codes and a brief explanation of what the codes refer to are provided. Because there are maximum allowable lengths for these labels, they often have an abbreviated form. For example, A02 has three valid codes. A code of “1” indicates a vote for Clinton, a code of “2” indicates a vote for Trump, and a code of “3” indicates a vote for another candidate, such as Gary Johnson. Additionally, a code of “9” is used for respondents who do not fit into any of these categories. For this last group of respondents we have only “missing data.” Missing data occurs because:
    1. The question does not apply to the respondent: e.g., people who did not vote were not asked which presidential candidate they voted for;
    2. The respondent refused to give a response or had no opinion;
    3. The interviewer failed to obtain or record the information for some other reason; or
    4. The respondent did not complete the interview (for Internet respondents). The label “NA” is attached to this category to indicate that the item is “not applicable” or that the information was “not ascertained.”
  5. Frequencies. To the left of the value descriptions and codes is a set of numbers called the frequencies (or marginals or marginal frequencies). The frequencies or marginals indicate the total number of respondents who fall into each category of the variable. These frequencies are based on the unweighted data (see the discussion on weighting). Although the codebook frequencies in SDA are based on the unweighted data, any analysis should use the weighted data. For this reason, there may be some discrepancies between the codebook information and the tables generated in your analysis. In order to view the frequencies based on the weighted data, select the “Run frequencies…” option in SDA and enter the name of the variable for which you desire weighted frequencies. Even though the differences between the unweighted and weighted frequencies can be large, you can obtain a good general idea of the distribution of responses for any variable by viewing the codebook information. You can see a full set of weighted marginals in the PDF codebook (pdf).
  6. Marginal percentages. Besides providing the number of respondents who fall into each category for every variable, the codebook also provides the percentage breakdown for these marginal frequencies. There are two columns of percentages. The first column is based only on the valid responses for the variable (i.e., missing data are excluded). The second column is based on all responses, including the missing data category. (In the PDF file, only the weighted raw marginal frequencies are provided).

Many of the variables are relatively straightforward, but some types of variables require more thorough explanation. This is provided below.

  • There are several feeling thermometer items (such as E07 Feeling thermometer toward Congress), which asked the respondent to indicate his or her feeling toward a specific candidate or party by placing that person or party on a feeling thermometer that ranges from 0 to 100 degrees, where 50 degrees represents a neutral feeling, higher temperatures represent warmer or more positive feelings, and lower temperatures represent cooler or more negative feelings. Placements on the feeling thermometers have been collapsed into five categories for ease of analysis.
  • There are a number of issue-position scales, each of which has a seven-point scale that represents possible positions people might take on a specific issue. For example, there is an issue-position scale on government services and spending (J01). The possible positions on the scale run from “provide many fewer services to reduce spending a lot” to “provide many more services, even if it means increased spending.” Respondents were asked to place themselves on this scale according to their feelings on the issue. Only the end points of the seven-point scale are defined. Respondents who feel that they fall between the two extremes can place themselves on one of the middle points. All of the issue-position scales have this basic structure.
  • There are candidate-placement scales that indicate how the respondents felt that Clinton or Trump stood on the issues. They are similar in structure to the issue-position scales to which they correspond. For example, respondents were asked where they thought Clinton and Trump stood on the health insurance plan scale (H02 and H03). These two candidate-placement scales have seven possible categories, running from “Government health plan” to “Private health plans,” just like the issue-position scale for health insurance plans (H01). The difference is that H01 measures where the respondent falls on this scale, whereas H02 and H03 measure where the respondent thinks that Clinton and Trump fall on the scale. The usefulness of the candidate placement scales is that by using them in combination with the item that measures the respondent’s position, one can see how closely the respondent felt he or she was to each of the two major candidates on the respective issue.
  • There are several indices that summarize how a respondent answered two or more questions that are related to a single topic. For example, the political efficacy index (P02) is based on how the respondent answered two questions dealing with feelings about the ability of people to influence government. Respondents in the “high” category generally answered the questions in a very “positive” or “efficacious” manner. Respondents in the “low” category generally gave very inefficacious responses, and those in between gave mixed responses. Other indices include P01 (Trust in government index) or K07 (Moral traditionalism index). These indices were constructed because each better measures the underlying concept than does any one of the specific questions that were used to construct the index. Note: it is possible for you to construct additional indices by using the recoding program in SDA.