22 Reading: Surveys
When entering a particular social environment, a researcher must be careful. There are times to remain anonymous and times to be overt. There are times to conduct interviews and times to simply observe. Some participants need to be thoroughly informed; others should not know they are being observed. A researcher wouldn’t stroll into a crime-ridden neighborhood at midnight, calling out, “Any gang members around?” And if a researcher walked into a coffee shop and told the employees they would be observed as part of a study on work efficiency, the self-conscious, intimidated baristas might not behave naturally. This is called the Hawthorne effect—where people change their behavior because they know they are being watched as part of a study. The Hawthorne effect is unavoidable in some research. In many cases, sociologists have to make the purpose of the study known. Subjects must be aware that they are being observed, and a certain amount of artificiality may result (Sonnenfeld 1985).
Making sociologists’ presence invisible is not always realistic for other reasons. That option is not available to a researcher studying prison behaviors, early education, or the Ku Klux Klan. Researchers can’t just stroll into prisons, kindergarten classrooms, or Klan meetings and unobtrusively observe behaviors. In situations like these, other methods are needed. All studies shape the research design, while research design simultaneously shapes the study. Researchers choose methods that best suit their study topics and that fit with their overall approaches to research.
In planning studies’ designs, sociologists generally choose from four widely used methods of social investigation: survey, experiment, field research, and secondary data analysis, or use of existing sources. Every research method comes with plusses and minuses, and the topic of study strongly influences which method or methods are put to use.
Surveys
As a research method, a survey collects data from subjects who respond to a series of questions about behaviors and opinions, often in the form of a questionnaire. The survey is one of the most widely used scientific research methods. The standard survey format allows individuals a level of anonymity in which they can express personal ideas.
At some point, most people in the United States respond to some type of survey. The U.S. Census is an excellent example of a large-scale survey intended to gather sociological data. Not all surveys are considered sociological research, however, and many surveys people commonly encounter focus on identifying marketing needs and strategies rather than testing a hypothesis or contributing to social science knowledge. Questions such as, “How many hot dogs do you eat in a month?” or “Were the staff helpful?” are not usually designed as scientific research. Often, polls on television do not reflect a general population, but are merely answers from a specific show’s audience. Polls conducted by programs such as American Idol or So You Think You Can Dance represent the opinions of fans but are not particularly scientific. A good contrast to these are the Nielsen Ratings, which determine the popularity of television programming through scientific market research.
Sociologists conduct surveys under controlled conditions for specific purposes. Surveys gather different types of information from people. While surveys are not great at capturing the ways people really behave in social situations, they are a great method for discovering how people feel and think—or at least how they say they feel and think. Surveys can track preferences for presidential candidates or reported individual behaviors (such as sleeping, driving, or texting habits) or factual information such as employment status, income, and education levels.
A survey targets a specific population, people who are the focus of a study, such as college athletes, international students, or teenagers living with type 1 (juvenile-onset) diabetes. Most researchers choose to survey a small sector of the population, or a sample: that is, a manageable number of subjects who represent a larger population. The success of a study depends on how well a population is represented by the sample. In a random sample, every person in a population has the same chance of being chosen for the study. According to the laws of probability, random samples represent the population as a whole. For instance, a Gallup Poll, if conducted as a nationwide random sampling, should be able to provide an accurate estimate of public opinion whether it contacts 2,000 or 10,000 people.
After selecting subjects, the researcher develops a specific plan to ask questions and record responses. It is important to inform subjects of the nature and purpose of the study up front. If they agree to participate, researchers thank subjects and offer them a chance to see the results of the study if they are interested. The researcher presents the subjects with an instrument, which is a means of gathering the information. A common instrument is a questionnaire, in which subjects answer a series of questions. For some topics, the researcher might ask yes-or-no or multiple-choice questions, allowing subjects to choose possible responses to each question. This kind of quantitative data—research collected in numerical form that can be counted—are easy to tabulate. Just count up the number of “yes” and “no” responses or correct answers, and chart them into percentages.
Questionnaires can also ask more complex questions with more complex answers—beyond “yes,” “no,” or the option next to a checkbox. In those cases, the answers are subjective and vary from person to person. How do plan to use your college education? Why do you follow Jimmy Buffett around the country and attend every concert? Those types of questions require short essay responses, and participants willing to take the time to write those answers will convey personal information about religious beliefs, political views, and morals. Some topics that reflect internal thought are impossible to observe directly and are difficult to discuss honestly in a public forum. People are more likely to share honest answers if they can respond to questions anonymously. This type of information is qualitative data—results that are subjective and often based on what is seen in a natural setting. Qualitative information is harder to organize and tabulate. The researcher will end up with a wide range of responses, some of which may be surprising. The benefit of written opinions, though, is the wealth of material that they provide.
An interview is a one-on-one conversation between the researcher and the subject, and it is a way of conducting surveys on a topic. Interviews are similar to the short-answer questions on surveys in that the researcher asks subjects a series of questions. However, participants are free to respond as they wish, without being limited by predetermined choices. In the back-and-forth conversation of an interview, a researcher can ask for clarification, spend more time on a subtopic, or ask additional questions. In an interview, a subject will ideally feel free to open up and answer questions that are often complex. There are no right or wrong answers. The subject might not even know how to answer the questions honestly.
Questions such as, “How did society’s view of alcohol consumption influence your decision whether or not to take your first sip of alcohol?” or “Did you feel that the divorce of your parents would put a social stigma on your family?” involve so many factors that the answers are difficult to categorize. A researcher needs to avoid steering or prompting the subject to respond in a specific way; otherwise, the results will prove to be unreliable. And, obviously, a sociological interview is not an interrogation. The researcher will benefit from gaining a subject’s trust, from empathizing or commiserating with a subject, and from listening without judgment.
When writing questions for a survey or questionnaire, a sociologist must be concerned with all questions and possible responses. You do not want to accidentally ask two part questions where a respondent may have one answer for one part and a different answer for another response. These are known as double barreled questions. For example, I want to know whether individuals like baseball. I include the question in the survey, “Do you like the baseball and the New York Yankees?” I offer the responses: yes, no, no opinion. The flaw in my question is that someone may love baseball but despise the New York Yankees, so they answer “No”. I am then not getting a true response to whether they like baseball or not.
Also, a sociologist must make sure that everyone who completes the survey has a true answer to select. For example, I am interest in how different age groups feel about politics. I ask the question, “To what political party do you belong?” I offer the responses as: Democrat, Republican, I do not like politics. Anyone who does not belong to the Democrat or Republican party but does partake in the political process does not have an option to choose. For example, Independents would have no true answer to select.
Finally, a sociologist has to make sure that there is only one response that a participant can select as a response to a question. Using the same example above, I want to know how different age groups feel about politics. I ask the question, “How old are you?” I offer the responses: Under 18, 20-25, 25-30, 31-49, 50 and above. There are two issues with the offered responses. First, no one 19 years old has a true response to choose. Secondly, anyone who is 25 could answer 2 different ways, 20-25 or 25-30. As a sociologist, it is crucial to our studies that we avoid these errors in writing survey questions.
Think It Over
What type of data do surveys gather? For what topics would surveys be the best research method? What drawbacks might you expect to encounter when using a survey? To explore further, ask a research question and write a hypothesis. Then create a survey of about six questions relevant to the topic. Provide a rationale for each question. Now define your population and create a plan for recruiting a random sample and administering the survey.
Practice
1. Why is choosing a random sample an effective way to select participants?
- Participants do not know they are part of a study
- The researcher has no control over who is in the study
- It is larger than an ordinary sample
- Everyone has the same chance of being part of the study
Show Answer
d
Show Glossary
- correlation:
- when a change in one variable coincides with a change in another variable, but does not necessarily indicate causation
- Hawthorne effect:
- when study subjects behave in a certain manner due to their awareness of being observed by a researcher
- interview:
- a one-on-one conversation between the researcher and the subject
- participant observation:
- when a researcher immerses herself in a group or social setting in order to make observations from an “insider” perspective
- population:
- a defined group serving as the subject of a study
- primary data:
- data that are collected directly from firsthand experience
- quantitative data:
- represent research collected in numerical form that can be counted
- qualitative data:
- comprise information that is subjective and often based on what is seen in a natural setting
- random sample:
- a study’s participants being randomly selected to serve as a representation of a larger population
- samples:
- small, manageable number of subjects that represent the population
- secondary data analysis:
- using data collected by others but applying new interpretations
- surveys:
- collect data from subjects who respond to a series of questions about behaviors and opinions, often in the form of a questionnaire