Getting Started with Quantitative Research
Basic Stats: t-tests
The Student’s t-test is one of the simplest statistical tests available. It comes in three types: independent samples t-tests, matched pairs t-tests, and one-sample t-tests.
Independent Samples t-tests
A t-test is used to determine if two sets of data are different enough to conclude that the underlying populations from which the data were drawn are also different. In a controlled experiment, a t-test could be used to determine if control and experimental conditions differed after the application of some treatment, such as evaluating the effectiveness of some learning intervention. In survey research, a t-test could be used to determine if responses drawn from different populations differ, such as evaluating whether domestic and international students view the Georgia Tech OMS program equally favorably.
The following sources explain simple t-tests:
- The T-Test, from the Web Center for Social Research Methods
- T-Test (Independent Samples), from Statwing
- What Is a t-test? And Why Is It Like Telling a Kid to Clean Up that Mess in the Kitchen?, from Patrick Runkel of the Minitab Blog
Matched Pairs t-tests
Simple t-tests will take care of many analyses, but there are slightly more complicated versions for more complex analyses. For example, t-tests assume that the two groups don’t interact. But what if you wanted to test the effectiveness of a learning tool without doing a controlled experiment? What if you simply wanted to evaluate whether the students knew more after using the tool than they knew before? For that, you would use a matched t-test, where you pair up connected values.
The following sources explain matched t-tests:
- Inferences from Matched Pairs, from Elementary Statistics
- Hypothesis Test: Difference Between Paired Means, from Stat Trek
- Dependent T-Test for Paired Samples, from Laerd Statistics
- Paired t-test, from Duke University
A third kind of t-test, the one-sample t-test, can be used when we know the population mean and want to evaluate whether or not a particular sample matches that population mean. For example, we may want to evaluate whether incoming Georgia Tech OMS students have average GRE scores that match the GRE average: the GRE average is known, and we may take one sample of incoming OMS students and compare their mean GRE scores to the GRE average.
The following sources explain one-sample t-tests:
- One-Sample t-Test, from Emory University
- One Sample t-test, from Kent State University
- Independent One-Sample T-Test, from Explorable
Generally, though, you won’t do the math for t-tests by hand. You might use advanced statistical like SPSS or R, but you can also take advantage of simple online calculators like the ones below:
Basic Stats ANOVA
t-tests test for differences between groups. However, if you read that information carefully, you may notice something problematic. t-tests usually use a confidence level of 95%, but that means one in every twenty tests could be flawed. What if we want to compare five different groups for differences between any pair of groups? We would need twenty comparisons, raising the odds of getting a false positive.
An Analysis of Variance, or ANOVA, test helps prevent this by giving one test that compares for differences among multiple groups. Below are some sources on ANOVA:
- ANOVA, from Wolfram MathWorld
- Oneway Analysis of Variance, from Rosie Cornish of the Mathematics Learning Support Centre
- Introduction to Analysis of Variance, from Rice University
- ANOVA, from Explorable
- ANOVA: ANalysis Of VAriance between groups, from St. John’s University
Again, you generally won’t do the math for an ANOVA by hand. Instead, use one of these calculators:
- ANOVA Calculator, from St. John’s University
- One-Way Analysis of Variance for Independent or Correlated Samples, from Vassar Stats
Basic Stats: Linear Regression
t-tests and ANOVA both aim to state if two different groups are different. Thus, they rely on data that can be split based on some discrete categories, such as whether the data comes from a pre-test or a post-test, or whether the data comes from a first-year student or a second-year student.
However, what if our explanatory variable is continuous instead of discrete? What if, for example, we wanted to see if class performance varied as a function of number of forum contributions or time spent watching videos? In this case, we would be looking at regression. The simplest form of regression is linear regression, where one variable is assumed to be a linear function of another.
For some information on linear regression, see these sources:
- Simple Linear Regression, from Penn State University
- Linear Regression, from Yale University
- Introduction to Linear Regression, from David Lane of Rice University
As with t-tests and ANOVA, you generally need not perform linear regressions by hand. Instead, here are some calculators or sources on how to carry them out more easily:
- Linear Regression Calculator, from GraphPad
- Simple Linear Regression Using Excel (Video), from Joseph Snider
For more comprehensive information, see:
- Research Methods in Education (Chapter 10)
ANOVA, t-tests, and linear regression are three of the simplest statistical tests, and for the scope of this class, they will likely get you what you need. However, it’s possible you might need more complex statistical tests. For example, what if you wanted to analyze the impact of both gender and international status on perception of the OMS program? What about interactions that you suspect will be non-linear, like the interaction between forum interactivity and class performance?
The following are three potentially useful classes of more advanced statistical analyses. To run these, you’ll generally need a tool like SPSS or R, and you should see the section on available online statistics courses for more on how to use those. Note that even more powerful methods exist, but once you start looking at the more powerful methods, you’re getting very close to performing machine learning.
A MANOVA, or Multivariate Analysis of Variance, evaluates whether multiple categories predict variance across some variable. For example, a MANVOA could tell us if there are interactions between gender and international status in predicting students’ perception of the OMS program. Here are some sources on MANOVA:
- Multivariate Analysis of Variance (MANOVA), from Penn State University
- A Primer on Multivariate Analysis of Variance (MANOVA) for Behavioral Scientists, from Russell Warne of Utah Valley University
- MANOVA n R, from Quick-R
Multiple Linear Regression
Linear regression attempts to find a linear interaction between one explanatory variable and one outcome variable. Multiple linear regression allows the same type of analysis, but with multiple explanatory variables. For example, perhaps class performance is a function of both time spent watching class videos and time spent interacting on Piazza: multiple linear regression would allow us to evaluate both of these together. Here are some sources on multiple linear regression:
- Multiple Linear Regression, from Penn State University
- Multiple Linear Regression, from Yale University
- Multiple Linear regression analysis using Microsoft Excel’s data analysis toolpak and ANOVA Concepts (Video), from Knowledge Varsity
You might speculate that the interaction between study time and class performance is non-linear. After all, is the difference between 100 hours of studying and 101 going to be as significant as the difference between 0 hours and 1? Non-linear regression generalizes linear regression to apply not just to straight lines, but to any function, such as exponential and logarithmic functions. The mathematics are still the same, but the power introduced can be useful.
Here are some sources on non-linear regression:
- Getting started with nonlinear regression, from GraphPad Software
- Data Driven Fitting, from Loren Shure of MathWorks
- itting curves to data using nonlinear regression: a practical and nonmathematical review, from Harvey Motulsky and Lennary Ransnas of the University of California-San Diego
For more comprehensive information, see:
- Research Methods in Education (Chapter 20)
Searching for scholarly readings on quantitative research will generally bring you to statistics as a whole; after all, the entire field of statistics is geared toward conducting quantitative research. Later sections here will cover some statistical methods you might find useful, but the sources below comment on the use of quantitative research as a whole.
- Doing Quantitative Research in Education, from Daniel Muijs of the University of Southampton
- Toward a New Classification of Nonexperimental Quantitative Research
- Integrating quantitative and qualitative research: how is it done?, from Alan Bryman of the University of Leicester
For more comprehensive information, see:
- Research Methods in Education (Chapters 12, 18, and 20)
Quantitative research and controlled experiments often go together as controlled experiments often generate numeric data. Thus, the list of exemplary controlled experiments gives great demonstrations of quantitative research as well.
Quantitative research is a massive field. Below are some sources for starting to make sense of this enormous space.
- What are the main types of quantitative approaches to research?, from Baltimore County Public Schools
- Quantitative Methods in Education Research, from Ulrike Hohmann of the University of Plymouth
- Quantitative Research Design, from Will Hopkins of SportScience
- Quantitative Research Guidelines, from Teachers of English to Speakers of Other Languages, Inc.
- Quantitative Research Design, from Martyn Shuttleworth of Explorable
- What is quantitative research?, from Marketing Donut
- Quantitative Research, from Dobney.com
- How to structure quantitative research questions, from Laerd Dissertation
- Quantitative Research Definitions, from John Pryor of St. John’s University
Qualitative vs. Quantitative Research
Choosing between qualitative and quantitative research is not always an easy task. There are some variables and phenomena that seem like they may be measurable numerically, but that really should be described qualitatively before trying to create valid constructs. The sources below should help you choose whether qualitative or quantitative research is right for you. Note that qualitative research is often used in nursing and medicine as well, and you may see sources that talk about it in those domains; generally, the concepts are relatively transferable.
- Quantitative and Qualitative Research: A View for Clarity , from Catherine Castellan of Loyola University Maryland
- Quantitative and Qualitative Inquiry in Educational Research: Is There A Paradigmatic Difference Between Them?, from Katrin Niglas of Tallinn Pedagogical University
- Quantitative, Qualitative, and Mixed Research, from Burke Johnson of the University of South Alabama
- Qualitative versus Quantitative Research, from Xavier University
- Nursing Research/Theory â€” Quantitative vs. Qualitative
- Nursing Resources: Qualitative vs. Quantitative, from the University of Wisconsin
- Quantitative and Qualitative Research, from Explorable
- Quantitative Vs. Qualitative Research â€“ When to Use Which, from SurveyGizmo
- Qualitative and Quantitative Research: Comparison of Qualitative and Quantitative Research, from Atlasti
- Qualitative and Quantitative Research, from the British Library
Online Statistics Courses
If you needed to start looking at those advanced statistical methods, however â€” or if you still needed some help with the earlier topics â€” you may need to dive deeper into learning some statistics. Fortunately, statistics is a commonly-taught subject, and there are lots of courses available online, for everyone from beginners to experts.
Here are a few. Note that to access Udacity courses, you’ll need a separate account from your Georgia Tech+Udacity login. You can create such account at Udacity.com
- Intro to Statistics, a Udacity course
- Intro to Descriptive Statistics, a Udacity course
- Intro to Inferential Statistics, a Udacity course
- Descriptive Statistics, a Coursera course from the University of Amsterdam
- Inferential Statistics, a Coursera course from the University of Amsterdam
- Quantitative Research Methods: Multivariate, a course from MIT OpenCourseWare
- Explore Statistics with R, an edX course from Karolinska Institutet
- Introduction to R Programming, an edX course from Karolinska Institutet
Beyond Statistics: Data Science and Machine Learning
- Big Data in Education, a course from Columbia University
- Data Analyst Nanodegree, a program from Udacity
- Machine Learning, a Udacity course from Georgia Tech
- Intro to Machine Learning, a Udacity course
- Intro to Data Science, a Udacity course
- Data Analysis with R, a Udacity course
- Data Science, a Coursera specialization from Johns Hopkins University
- Machine Learning, a Coursera course from Stanford University
- Data Science and Machine Learning Essentials, an edX course from Microsoft