16 Research Designs

We have just been looking at models of the research process and goals of research. The following is a comparison of research methods or techniques used to describe, explain, or evaluate. Each of these designs has strengths and weaknesses and is sometimes used in combination with other designs within a single study.

Observational studies involve watching and recording the actions of participants. This may take place in the natural setting, such as observing children at play at a park, or behind a one-way glass while children are at play in a laboratory playroom. The researcher may follow a check list and record the frequency and duration of events (perhaps how many conflicts occur among 2 year olds) or may observe and record as much as possible about an event as a participant (such as attending an Alcoholics Anonymous meeting and recording the slogans on the walls, the structure of the meeting, the expressions commonly used, etc.). The researcher may be a participant or a non-participant. What would be the strengths of being a participant? What would be the weaknesses? Consider the strengths and weaknesses of not participating. In general, observational studies have the strength of allowing the researcher to see how people behave rather than relying on self-report. What people do and what they say they do are often very different. A major weakness of observational studies is that they do not allow the researcher to explain causal relationships. Yet, observational studies are useful and widely used when studying children. Children tend to change their behavior when they know they are being watched (known as the Hawthorne effect) and may not survey well.

Experiments are designed to test hypotheses (or specific statements about the relationship between variables) in a controlled setting in efforts to explain how certain factors or events produce outcomes. A variable is anything that changes in value. Concepts are operationalized or transformed into variables in research which means that the researcher must specify exactly what is going to be measured in the study. For example, if we are interested in studying marital satisfaction, we have to specify what marital satisfaction really means or what we are going to use as an indicator of marital satisfaction. What is something measurable that would indicate some level of marital satisfaction? Would it be the amount of time couples spend together each day? Or eye contact during a discussion about money? Or maybe a subject’s score on a marital satisfaction scale. Each of these is measurable but these may not be equally valid or accurate indicators of marital satisfaction. What do you think? These are the kinds of considerations researchers must make when working through the design.

Three conditions must be met in order to establish cause and effect. Experimental designs are useful in meeting these conditions.

The independent and dependent variables must be related. In other words, when one is altered, the other changes in response. (The independent variable is something altered or introduced by the researcher. The dependent variable is the outcome or the factor affected by the introduction of the independent variable. For example, if we are looking at the impact of exercise on stress levels, the independent variable would be exercise; the dependent variable would be stress.)

The cause must come before the effect. Experiments involve measuring subjects on the dependent variable before exposing them to the independent variable (establishing a baseline). So we would measure the subjects’ level of stress before introducing exercise and then again after the exercise to see if there has been a change in stress levels. (Observational and survey research does not always allow us to look at the timing of these events which makes understanding causality problematic with these designs.)

The cause must be isolated. The researcher must ensure that no outside, perhaps unknown variables are actually causing the effect we see. The experimental design helps make this possible. In an experiment, we would make sure that our subjects’ diets were held constant throughout the exercise program. Otherwise, diet might really be creating the change in stress level rather than exercise.

A basic experimental design involves beginning with a sample (or subset of a population) and randomly assigning subjects to one of two groups: the experimental group or the control group. The experimental group is the group that is going to be exposed to an independent variable or condition the researcher is introducing as a potential cause of an event. The control group is going to be used for comparison and is going to have the same experience as the experimental group but will not be exposed to the independent variable. After exposing the experimental group to the independent variable, the two groups are measured again to see if a change has occurred. If so, we are in a better position to suggest that the independent variable caused the change in the dependent variable. The basic experimental model looks like this:

The major advantage of the experimental design is that of helping to establish cause and effect relationships. A disadvantage of this design is the difficulty of translating much of what concerns us about human behavior into a laboratory setting. I hope this brief description of experimental design helps you appreciate both the difficulty and the rigor of conducting an experiment.

Case studies involve exploring a single case or situation in great detail. Information may be gathered with the use of observation, interviews, testing, or other methods to uncover as much as possible about a person or situation. Case studies are helpful when investigating unusual situations such as brain trauma or children reared in isolation. And they often used by clinicians who conduct case studies as part of their normal practice when gathering information about a client or patient coming in for treatment. Case studies can be used to explore areas about which little is known and can provide rich detail about situations or conditions. However, the findings from case studies cannot be generalized or applied to larger populations; this is because cases are not randomly selected and no control group is used for comparison. (Read “The Man Who Mistook His Wife for a Hat” by Dr. Oliver Sacks as a good example of the case study approach.)

Surveys are familiar to most people because they are so widely used. Surveys enhance accessibility to subjects because they can be conducted in person, over the phone, through the mail, or online. A survey involves asking a standard set of questions to a group of subjects. In a highly structured survey, subjects are forced to choose from a response set such as “strongly disagree, disagree, undecided, agree, strongly agree”; or “0, 1-5, 6-10, etc.” Surveys are commonly used by sociologists, marketing researchers, political scientists, therapists, and others to gather information on many independent and dependent variables in a relatively short period of time. Surveys typically yield surface information on a wide variety of factors, but may not allow for in-depth understanding of human behavior. Of course, surveys can be designed in a number of ways. They may include forced choice questions and semi-structured questions in which the researcher allows the respondent to describe or give details about certain events. One of the most difficult aspects of designing a good survey is wording questions in an unbiased way and asking the right questions so that respondents can give a clear response rather that choosing “undecided” each time. Knowing that 30% of respondents are undecided is of little use! So a lot of time and effort should be placed on the construction of survey items. One of the benefits of having forced choice items is that each response is coded so that the results can be quickly entered and analyzed using statistical software. Analysis takes much longer when respondents give lengthy responses that must be analyzed in a different way. Surveys are useful in examining stated values, attitudes, opinions, and reporting on practices. However, they are based on self-report or what people say they do rather than on observation and this can limit accuracy.

Secondary/Content analysis involves analyzing information that has already been collected or examining documents or media to uncover attitudes, practices or preferences. There are a number of data sets available to those who wish to conduct this type of research. For example, the U. S. Census Data is available and widely used to look at trends and changes taking place in the United States (go to http://www.census.gov/ and check it out). There are a number of other agencies that collect data on family life, sexuality, and many other areas of interest in human development (go to http://www.norc.uchicago.edu/ or http://www.kff.org/ and see what you find.). The researcher conducting secondary analysis does not have to recruit subjects but does need to know the quality of the information collected in the original study.

Content analysis involves looking at media such as old texts, pictures, commercials, lyrics or other materials to explore patterns or themes in culture. An example of content analysis is the classic history of childhood by Aries (1962) called “Centuries of Childhood” or the analysis of television commercials for sexual or violent content. Passages in text or programs that air can be randomly selected for analysis as well. Again, one advantage of analyzing work such as this is that the researcher does not have to go through the time and expense of finding respondents, but the researcher cannot know how accurately the media reflects the actions and sentiments of the population.

Developmental designs are techniques used in life span research (and other areas as well). These techniques try to examine how age, cohort, gender, and social class impact development. Cross-sectional research involves beginning with a sample that represents a cross-section of the population. Respondents who vary in age, gender, ethnicity, and social class might be asked to complete a survey about television program preferences or attitudes toward the use of the Internet. The attitudes of males and females could then be compared as could attitudes based on age. In cross-sectional research, respondents are measured only once. This method is much less expensive than longitudinal research but does not allow the researcher to distinguish between the impact of age and the cohort effect. Different attitudes about the Internet, for example, might not be altered by a person’s biological age as much as their life experiences as members of a cohort.

Longitudinal research involves beginning with a group of people who may be of the same age and background, and measuring them repeatedly over a long period of time. One of the benefits of this type of research is that people can be followed through time and be compared with them when they were younger. A problem with this type of research is that it is very expensive and subjects may drop out over time. (The film 49 Up is a example of following individuals over time. You see how people change physically, emotionally, and socially through time.) What would be the drawbacks of being in a longitudinal study? What about 49 Up? Would you want to be filmed every 7 years? What would be the advantages and disadvantages? Can you imagine why some would continue and others drop out of the project?

Cross-sequential research involves combining aspects of the previous two techniques; beginning with a cross-sectional sample and measuring them through time. This is the perfect model for looking at age, gender, social class, and ethnicity. But it the drawbacks of high costs and attrition are here as well.