3 Discussion 7: Research Design for One-Way ANOVA Whether in a scholarly

3

Discussion 7: Research Design for One-Way ANOVA

Whether in a scholarly or practitioner setting, good research and data analysis should have the benefit of peer feedback. For this Discussion, you will perform an article critique on ANOVA testing. Be sure and remember that the goal is to obtain constructive feedback to improve the research and its interpretation, so please view this as an opportunity to learn from one another.

To prepare for this Discussion:

Review the Learning Resources and the media programs related to ANOVA testing.

Search for and select a quantitative article specific to your discipline and related to ANOVA testing. Help with this task may be found in the Course guide and assignment help linked in this week’s Learning Resources. Also, you can use as guide the Research Design Alignment Table located in this week’s Learning Resources.

By Day 3

Write a 3- to 5-paragraphs critique of the article. In your critique, include responses to the following:

Which is the research design used by the authors?

Why did the authors use ANOVA test?

Do you think it’s the most appropriate choice? Why or why not?

Did the authors display the data?

Do the results stand alone? Why or why not?

Did the authors report effect size? If yes, is this meaningful?

Be sure to support your Main Post and Response Post with reference to the week’s Learning Resources and other scholarly evidence in APA Style.

Frankfort-Nachmias, C., Leon-Guerrero, A., & Davis, G. (2020). Social statistics for a diverse society (9th ed.). Thousand Oaks, CA: Sage Publications.

Chapter 11, “Analysis of Variance” (pp. 373-399)

Wagner, III, W. E. (2020). Using IBM® SPSS® statistics for research methods and social science statistics (7th ed.). Thousand Oaks, CA: Sage Publications.

Chapter 10, “Analysis of Variance”

Chapter 11, “Editing Output” (previously read in Week 2, 3, 4, 5. and 6)

Klugkist, I. (2008). Analysis of variance (ANOVA). In P. J. Lavrakas (Ed.), Encyclopedia of survey research methods (pp. 27-27). Thousand Oaks, CA: SAGE Publications Ltd. doi: 10.4135/9781412963947.n18

Goal & objectives

For this week’s discussion, you will learn how to search for and locate a quantitative article involving analysis of variance (ANOVA) that relates to your discipline by:

choosing an appropriate subject database

conducting a search with relevant keywords

identifying a quantitative research article that uses ANOVA

find and identify a post hoc study article

Choose a database

Start your search by entering a database that fits your subject area with these steps:

1. On the Library Homepage, click the Research by Subject drop-down.
 

2. Click on the subject area that matches your program of study from the list.

3. Once you have clicked on a subject, scroll down until you see the list of databases and click on the title to enter the database.

Note: The databases are organized with the largest collections at the top of the list, so selecting the first database is generally a good way to start.

Build your search

Once you are in a database, you will see a search screen with multiple search boxes. We need to set up our search:

We will use the following keywords to find articles that use this methodology:

ANOVA

“Analysis of variance”

You may also want to search with keywords that describe a topic related to your discipline. You do not have to type in a topic, but if you do, keep the topic broad.

We will build an example search below for articles that use ANOVA with the topic of reading comprehension in the database called Education Source:

1. In the first search box, type:

ANOVA OR “Analysis of variance”

Note: Put quote marks around “Analysis of variance” to glue these words together as an exact phrase.

Note: Type the word OR between these two synonyms to tell the database we will take either the acronym ANOVA OR the phrase “Analysis of variance.”

2. For this example topic, type in the second box:

reading comprehension

Note: Some methodologies are rarely used for certain research topics. You may need to broaden your search topic to find a study that uses this methodology.

The search boxes will look like this:

3. Click Search.

4. Look in the article titles and abstracts to determine if the methods used in the article include ANOVA. You can also look in the Subjects listed for an article:

Refer back to the Evaluate your results box for Week 1 for more help.

Note: Articles may use more than one method in their quantitative analysis of data. You can use an article even if it uses other methods along with ANOVA. For instance, an article my use ANOVA and t-Test to analyze data.

Try it and test yourself:

How did that work for you? Did you get many results? No results?

Learn more about why you may not be finding any results.

ANOVA resources

The following tutorial will show you how to use our database SAGE Research Methods Online to find information about quantitative methodologies, including ANOVA:

Tutorial: SAGE Research Methods Online: A Quick Introduction to Researching Quantitative Methodology

(5 min 42 sec) Closed captioning included in tutorial.

Here are the steps to search inside SAGE Research Methods Online for more information on ANOVA:

1. On the Library Homepage click on the Databases A-Z link.

2. Click on S, scroll down, and then click on Sage Research Methods Online.

3. On the Sage Research Methods Online homepage, type the method into the search box:

ANOVA

4. Click the magnifying glass icon to Search.

You will now see your results with a definition of the ANOVA method above the results list. It will look like this:

This resource from our search gives an overview of the method:

Klugkist, I. (2008). Analysis of variance (ANOVA). In P. J. Lavrakas (Ed.), Encyclopedia of survey research methods (pp. 27-27). Thousand Oaks, CA: SAGE Publications Ltd. doi: 10.4135/9781412963947.n18

Before You Begin

Before reading this Skill Builder, be sure to review the following concepts:

Steps in hypotheses testing

Null and alternative hypothesis

Alpha level

Type I and Type II Errors

t-test for two independent groups

Definitions of categorical vs. continuous variables

Introduction

When researchers conduct a one-way ANOVA, the goal is to examine whether there are mean differences among two or more groups. The one-way ANOVA is a useful statistical analysis for research designs that involve only one independent grouping variable (called a factor in an experimental design) and a single continuous dependent variable. 

Criteria for a One-Way ANOVA

That is, a one-way ANOVA is appropriate in scenarios that meet the following criteria:

bullet

There is one independent variable

bullet

There is one dependent variable

bullet

The independent variable has two or more levels

bullet

The dependent variable can be considered to be continuous

In an ANOVA, just like in a t-test, a “level” is a group or category.

Example: IAT Research Scenario

In our IAT research scenario below, for example, there are four levels: American Indians, inner-city, suburban, and undocumented.

Imagine you want to see if cultural groups have different attitudes toward old people and decide to use the Implicit Association Test (IAT), which is a computerized procedure for measuring attribute discrimination (Greenwald, 2012).  Using latency of responses to different kinds of stimuli, the test has been used to study attitudes towards age, race, skin-tone, religion sexuality, and weight, among other attributes (Lane, Banaji, Nosek, & Greewald, 2007).

You might develop an IAT that measures attitudes toward old people and use it to compare four cultural groups: American Indians living on a reservation, inner-city legal residents, suburban legal residents, and undocumented residents. Your interest in comparing four different groups will lead you to consider the one-way ANOVA. In your one-way ANOVA, you would test the following null hypothesis:

HO  :  μAmerican Indians =  μinner-city = μsuburban = μundocumented

The alternative hypothesis would be:

HA : not  HO

The null states that all four populations have the same mean for the IAT test: that is, that all four populations have the same attitudes, on average, toward old people. The alternative states that one or more of the population mean differs from the others: that is, that the four populations are not equivalent in their attitudes, on average, toward old people.

Note: You can participate in a study using the IAT approach at https://implicit.harvard.edu/implicit/.

bullet

Note that although we can use a one-way ANOVA if we have just two levels, a t-test is probably the simpler approach in this case. One-way ANOVAs are typically used when there are more than two levels.  

bullet

Also note that, although the independent variable in an ANOVA can technically be either categorical or continuous, researchers typically use an ANOVA in cases in which there are not an excessive number of levels. When there are too many levels, there may be very few participants in each level (in each group)  and there will likely not be enough statistical power to discern any differences between the groups. 

Hence, the one-way ANOVA is usually applied when the independent variable has relatively few levels, and most of the time, the independent variable in a one-way ANOVA is categorical.

The F-test

The basic logic of the t-test for two independent groups can be applied to the F-test for the one-way ANOVA. The one-way ANOVA can be thought of as an extension of the t-test. In a t-test, researchers examine mean differences between the two groups. In the one-way ANOVA, researchers use an F-test to examine whether there are mean differences among two or more groups. When conducting a t-test, researchers use a sampling distribution for the t-test to determine whether the t-test is statistically significant. Significance for the F-test is determined by examining a sampling distribution for the F-statistic under the assumption that the null hypothesis is true. 

Figure 1. F with 3 and 96 df

For example, if you were to conduct the study described above involving the four different cultures and the IAT and had 100 total participants, the sampling distribution for the test statistic would have the distribution shown in Figure 1 to the left.

Just as with the t-test, the hypothesis test must account for a type I error (i.e., rejecting the null if the null is true). In this example, the researcher will reject the null hypothesis if the value of F based on the data (the observed value of F ) is greater than the critical value of F (2.69939) that has been determined by alpha along with the degrees of freedom for the F statistic. If alpha has been set at .05, when the observed value of F exceeds the critical value of F, the p-value will be less than .05. With a p-value of less than .05, researchers would reject the null hypothesis. 

In Figures 2 and 3 below, note how the shape of the F distribution changes with different degrees of freedom.  Use the  icons to navigate between the figures.

1

2

Figure 2. F with 1 and 98 df

Figure 3. F with 7 and 92 df

Also, note that alpha is only in the upper tail of the test statistic’s sampling distribution. With the t-test for two independent groups, alpha is divided in half for a two-tailed test, with one half being used in the upper tail of the test statistic’s sampling distribution and the other half in the lower tail. With ANOVA, even though the alternative hypothesis does not specify a direction for the inequalities, only one tail of the F distribution is used to determine whether to reject the null.  

The good news for modern-day researchers is that SPSS and other statistical programs compute p-values for the F-test. Recall that the p-value is the probability of obtaining a value of the test statistic that is more extreme than the observed value if the null hypothesis is true.