Getting Started with Quantitative Research

Topic Overview

Basic Stats: t-tests

The Student’s t-test is one of the simplest statistical tests available. It comes in three types: independent samples t-tests, matched pairs t-tests, and one-sample t-tests.

Independent Samples t-tests

A t-test is used to determine if two sets of data are different enough to conclude that the underlying populations from which the data were drawn are also different. In a controlled experiment, a t-test could be used to determine if control and experimental conditions differed after the application of some treatment, such as evaluating the effectiveness of some learning intervention. In survey research, a t-test could be used to determine if responses drawn from different populations differ, such as evaluating whether domestic and international students view the Georgia Tech OMS program equally favorably.

The following sources explain simple t-tests:

Matched Pairs t-tests

Simple t-tests will take care of many analyses, but there are slightly more complicated versions for more complex analyses. For example, t-tests assume that the two groups don’t interact. But what if you wanted to test the effectiveness of a learning tool without doing a controlled experiment? What if you simply wanted to evaluate whether the students knew more after using the tool than they knew before? For that, you would use a matched t-test, where you pair up connected values.

The following sources explain matched t-tests:

One-Sample t-tests

A third kind of t-test, the one-sample t-test, can be used when we know the population mean and want to evaluate whether or not a particular sample matches that population mean. For example, we may want to evaluate whether incoming Georgia Tech OMS students have average GRE scores that match the GRE average: the GRE average is known, and we may take one sample of incoming OMS students and compare their mean GRE scores to the GRE average.

The following sources explain one-sample t-tests:

t-test Calculators

Generally, though, you won’t do the math for t-tests by hand. You might use advanced statistical like SPSS or R, but you can also take advantage of simple online calculators like the ones below:

Basic Stats ANOVA

t-tests test for differences between groups. However, if you read that information carefully, you may notice something problematic. t-tests usually use a confidence level of 95%, but that means one in every twenty tests could be flawed. What if we want to compare five different groups for differences between any pair of groups? We would need twenty comparisons, raising the odds of getting a false positive.

An Analysis of Variance, or ANOVA, test helps prevent this by giving one test that compares for differences among multiple groups. Below are some sources on ANOVA:

Again, you generally won’t do the math for an ANOVA by hand. Instead, use one of these calculators:

Basic Stats: Linear Regression

t-tests and ANOVA both aim to state if two different groups are different. Thus, they rely on data that can be split based on some discrete categories, such as whether the data comes from a pre-test or a post-test, or whether the data comes from a first-year student or a second-year student.

However, what if our explanatory variable is continuous instead of discrete? What if, for example, we wanted to see if class performance varied as a function of number of forum contributions or time spent watching videos? In this case, we would be looking at regression. The simplest form of regression is linear regression, where one variable is assumed to be a linear function of another.

For some information on linear regression, see these sources:

As with t-tests and ANOVA, you generally need not perform linear regressions by hand. Instead, here are some calculators or sources on how to carry them out more easily:

For more comprehensive information, see:

Advanced Stats

ANOVA, t-tests, and linear regression are three of the simplest statistical tests, and for the scope of this class, they will likely get you what you need. However, it’s possible you might need more complex statistical tests. For example, what if you wanted to analyze the impact of both gender and international status on perception of the OMS program? What about interactions that you suspect will be non-linear, like the interaction between forum interactivity and class performance?

The following are three potentially useful classes of more advanced statistical analyses. To run these, you’ll generally need a tool like SPSS or R, and you should see the section on available online statistics courses for more on how to use those. Note that even more powerful methods exist, but once you start looking at the more powerful methods, you’re getting very close to performing machine learning.

MANOVA

A MANOVA, or Multivariate Analysis of Variance, evaluates whether multiple categories predict variance across some variable. For example, a MANVOA could tell us if there are interactions between gender and international status in predicting students’ perception of the OMS program. Here are some sources on MANOVA:

Multiple Linear Regression

Linear regression attempts to find a linear interaction between one explanatory variable and one outcome variable. Multiple linear regression allows the same type of analysis, but with multiple explanatory variables. For example, perhaps class performance is a function of both time spent watching class videos and time spent interacting on Piazza: multiple linear regression would allow us to evaluate both of these together. Here are some sources on multiple linear regression:

Non-Linear Regression

You might speculate that the interaction between study time and class performance is non-linear. After all, is the difference between 100 hours of studying and 101 going to be as significant as the difference between 0 hours and 1? Non-linear regression generalizes linear regression to apply not just to straight lines, but to any function, such as exponential and logarithmic functions. The mathematics are still the same, but the power introduced can be useful.

Here are some sources on non-linear regression:

For more comprehensive information, see:

Scholarly Resources

Searching for scholarly readings on quantitative research will generally bring you to statistics as a whole; after all, the entire field of statistics is geared toward conducting quantitative research. Later sections here will cover some statistical methods you might find useful, but the sources below comment on the use of quantitative research as a whole.

For more comprehensive information, see:

Quantitative research and controlled experiments often go together as controlled experiments often generate numeric data. Thus, the list of exemplary controlled experiments gives great demonstrations of quantitative research as well.

General Media

Quantitative research is a massive field. Below are some sources for starting to make sense of this enormous space.

Qualitative vs. Quantitative Research

Choosing between qualitative and quantitative research is not always an easy task. There are some variables and phenomena that seem like they may be measurable numerically, but that really should be described qualitatively before trying to create valid constructs. The sources below should help you choose whether qualitative or quantitative research is right for you. Note that qualitative research is often used in nursing and medicine as well, and you may see sources that talk about it in those domains; generally, the concepts are relatively transferable.

Online Statistics Courses

If you needed to start looking at those advanced statistical methods, however â€” or if you still needed some help with the earlier topics â€” you may need to dive deeper into learning some statistics. Fortunately, statistics is a commonly-taught subject, and there are lots of courses available online, for everyone from beginners to experts.

Here are a few. Note that to access Udacity courses, you’ll need a separate account from your Georgia Tech+Udacity login. You can create such account at Udacity.com

Beginner Courses

Advanced Courses

Beyond Statistics: Data Science and Machine Learning

Data Sets

There are several data sets available on the internet for public analysis. Below are a couple you might be interested in using: