Getting Started with Quantitative Research

Topic Overview

Basic Stats: t-tests

The Student’s t-test is one of the simplest statistical tests available. It comes in three types: independent samples t-tests, matched pairs t-tests, and one-sample t-tests.

Independent Samples t-tests

A t-test is used to determine if two sets of data are different enough to conclude that the underlying populations from which the data were drawn are also different. In a controlled experiment, a t-test could be used to determine if control and experimental conditions differed after the application of some treatment, such as evaluating the effectiveness of some learning intervention. In survey research, a t-test could be used to determine if responses drawn from different populations differ, such as evaluating whether domestic and international students view the Georgia Tech OMS program equally favorably.

The following sources explain simple t-tests:

• The T-Test, from the Web Center for Social Research Methods
• T-Test (Independent Samples), from Statwing
• What Is a t-test? And Why Is It Like Telling a Kid to Clean Up that Mess in the Kitchen?, from Patrick Runkel of the Minitab Blog

Matched Pairs t-tests

Simple t-tests will take care of many analyses, but there are slightly more complicated versions for more complex analyses. For example, t-tests assume that the two groups don’t interact. But what if you wanted to test the effectiveness of a learning tool without doing a controlled experiment? What if you simply wanted to evaluate whether the students knew more after using the tool than they knew before? For that, you would use a matched t-test, where you pair up connected values.

The following sources explain matched t-tests:

• Inferences from Matched Pairs, from Elementary Statistics
• Hypothesis Test: Difference Between Paired Means, from Stat Trek
• Dependent T-Test for Paired Samples, from Laerd Statistics
• Paired t-test, from Duke University

One-Sample t-tests

A third kind of t-test, the one-sample t-test, can be used when we know the population mean and want to evaluate whether or not a particular sample matches that population mean. For example, we may want to evaluate whether incoming Georgia Tech OMS students have average GRE scores that match the GRE average: the GRE average is known, and we may take one sample of incoming OMS students and compare their mean GRE scores to the GRE average.

The following sources explain one-sample t-tests:

• One-Sample t-Test, from Emory University
• One Sample t-test, from Kent State University
• Independent One-Sample T-Test, from Explorable

t-test Calculators

Generally, though, you won’t do the math for t-tests by hand. You might use advanced statistical like SPSS or R, but you can also take advantage of simple online calculators like the ones below:

• t-test Calculator, from GraphPad Software
• t-test Calculator for 2 Dependent Means, from SocialScienceStatistics.com
• One sample t-test, from GraphPad Software

Basic Stats ANOVA

t-tests test for differences between groups. However, if you read that information carefully, you may notice something problematic. t-tests usually use a confidence level of 95%, but that means one in every twenty tests could be flawed. What if we want to compare five different groups for differences between any pair of groups? We would need twenty comparisons, raising the odds of getting a false positive.

An Analysis of Variance, or ANOVA, test helps prevent this by giving one test that compares for differences among multiple groups. Below are some sources on ANOVA:

• ANOVA, from Wolfram MathWorld
• Oneway Analysis of Variance, from Rosie Cornish of the Mathematics Learning Support Centre
• Introduction to Analysis of Variance, from Rice University
• ANOVA, from Explorable
• ANOVA: ANalysis Of VAriance between groups, from St. John’s University

Again, you generally won’t do the math for an ANOVA by hand. Instead, use one of these calculators:

• ANOVA Calculator, from St. John’s University
• One-Way Analysis of Variance for Independent or Correlated Samples, from Vassar Stats

Basic Stats: Linear Regression

t-tests and ANOVA both aim to state if two different groups are different. Thus, they rely on data that can be split based on some discrete categories, such as whether the data comes from a pre-test or a post-test, or whether the data comes from a first-year student or a second-year student.

However, what if our explanatory variable is continuous instead of discrete? What if, for example, we wanted to see if class performance varied as a function of number of forum contributions or time spent watching videos? In this case, we would be looking at regression. The simplest form of regression is linear regression, where one variable is assumed to be a linear function of another.

For some information on linear regression, see these sources:

• Simple Linear Regression, from Penn State University
• Linear Regression, from Yale University
• Introduction to Linear Regression, from David Lane of Rice University

As with t-tests and ANOVA, you generally need not perform linear regressions by hand. Instead, here are some calculators or sources on how to carry them out more easily:

• Linear Regression Calculator, from GraphPad
• Simple Linear Regression Using Excel (Video), from Joseph Snider

For more comprehensive information, see:

• Research Methods in Education (Chapter 10)

Advanced Stats

ANOVA, t-tests, and linear regression are three of the simplest statistical tests, and for the scope of this class, they will likely get you what you need. However, it’s possible you might need more complex statistical tests. For example, what if you wanted to analyze the impact of both gender and international status on perception of the OMS program? What about interactions that you suspect will be non-linear, like the interaction between forum interactivity and class performance?

The following are three potentially useful classes of more advanced statistical analyses. To run these, you’ll generally need a tool like SPSS or R, and you should see the section on available online statistics courses for more on how to use those. Note that even more powerful methods exist, but once you start looking at the more powerful methods, you’re getting very close to performing machine learning.

MANOVA

A MANOVA, or Multivariate Analysis of Variance, evaluates whether multiple categories predict variance across some variable. For example, a MANVOA could tell us if there are interactions between gender and international status in predicting students’ perception of the OMS program. Here are some sources on MANOVA:

• Multivariate Analysis of Variance (MANOVA), from Penn State University
• A Primer on Multivariate Analysis of Variance (MANOVA) for Behavioral Scientists, from Russell Warne of Utah Valley University
• MANOVA n R, from Quick-R

Multiple Linear Regression

Linear regression attempts to find a linear interaction between one explanatory variable and one outcome variable. Multiple linear regression allows the same type of analysis, but with multiple explanatory variables. For example, perhaps class performance is a function of both time spent watching class videos and time spent interacting on Piazza: multiple linear regression would allow us to evaluate both of these together. Here are some sources on multiple linear regression:

• Multiple Linear Regression, from Penn State University
• Multiple Linear Regression, from Yale University
• Multiple Linear regression analysis using Microsoft Excel’s data analysis toolpak and ANOVA Concepts (Video), from Knowledge Varsity

Non-Linear Regression

You might speculate that the interaction between study time and class performance is non-linear. After all, is the difference between 100 hours of studying and 101 going to be as significant as the difference between 0 hours and 1? Non-linear regression generalizes linear regression to apply not just to straight lines, but to any function, such as exponential and logarithmic functions. The mathematics are still the same, but the power introduced can be useful.

Here are some sources on non-linear regression:

• Getting started with nonlinear regression, from GraphPad Software
• Data Driven Fitting, from Loren Shure of MathWorks
• itting curves to data using nonlinear regression: a practical and nonmathematical review, from Harvey Motulsky and Lennary Ransnas of the University of California-San Diego

For more comprehensive information, see:

• Research Methods in Education (Chapter 20)