How To: Read an Academic Paper
To start the course, you’re going to be reading a bunch of papers, and so a good place to start is with: How do you read an academic paper? In some ways, that’s putting the cart before the horse because to read a paper, you need to find it — but we’ll assume you’re starting off using the course library, so finding something to get started with should be fine
How Not to Read an Academic Paper
This prompt might sound silly. How do you read an academic paper? Well, you start on page 1, and read to the last page, right?
You could do that, but what you’ll find is that you spend a lot of time on the far right side of the curve of diminishing returns. So, you don’t want to just start on page 1 and read to the last page. What should you do instead? To understand that, it’s important to understand a little bit about how most academic papers are organized.
How Academic Papers are Organized
Most academic papers follow something like this structure:
- Abstract: A high-level summary of the entire paper
- Introduction and Background: What the problem or question is and where it came from
- Related Work: What others have done in this space, likely specifically contextualizing why the authors’ solution is necessary
- Solution: What the authors did to address the problem or answer the question
- Methodology: How the authors evaluated their solution
- Results: The results of evaluating the solution
- Analysis: The authors’ interpretation of the results
- Conclusion: The limitations of the work and what the authors will do next
There is lots of variation there, of course. Pure research papers don’t really have a “Solution” section since they’re often not building something. If a paper was based exclusively on doing some surveys or interviews, for example, it likely would jump straight to methodology. Similarly, there are papers that don’t do much evaluation: they propose a design of some tool or theory, but the contribution is the idea itself.
Generally, though, you can map most papers onto something resembling this structure.
Intention Matters: Getting Started
What you do next depends on why you’re reading a particular paper. What you’re seeking to get out of it matters.
Early in the class, you’re probably just trying to get a high-level view of lots of work going on in a field. For that: start by reading the Abstract. Sometimes, that will be all you need from that paper. That’s ok. You may find that once you settle on an area you want to explore, you might just read a bunch of abstracts to get a view of the field.
When you’re just starting out, though, you probably want to go just a bit deeper. You want to use this paper as an anchor for further exploration. Read the Introduction next. See what problem they’re setting up and how they approach it. Then, jump to the conclusion. See what they found. Oftentimes, this is enough. Then, if you want to find more papers like this one, jump back to the related work section and see what you might want to read next. Your goal here is just to get a feel for what they did and why: once you know what a bunch of people have done and why they’ve done it, you can start to position your own work and take a deeper dive.
For the first couple weeks of this class, that’s about where you’ll stay: you want to get a broad look at what lots of people are working on and start to understand the overall trajectories of the field. You don’t need to get too far into the details of how they did stuff or how they know it worked.
Intention Matters: Zooming In
Once you’re comfortable zooming in on an area, though, your intention shifts a little bit. Now it’s less “Know what others are doing” and more “See what needs to be done”. With that change in intention comes a slight change in how you read. Now, you want to focus a little more on the Future Work and/or Limitations sections of the paper. What do the authors say still needs to be done? That could be more work to expand on the current state of the field, or it could be work to resolve limitations in the existing study. For example, imagine a tool was tested with 15 middle school students and found to be good for learning: does that hold true when tested with 150 students in a less controlled environment? Those are the kinds of things that come next.
At this point, you may also want to finally visit the methodology section, especially if there’s a paper whose conclusions you disagree with. You want to find out how they came to their conclusions, and see if there’s an alternate explanation for their results. Your follow-up work then might be to test that alternate explanation.
In any case, your goal here is to figure out exactly how your new work is going to map to the work that’s already been done: it might fill in some holes, push some boundaries, or even refute existing ideas.
So Why Is It There?
You might be wondering: if I really only need to read the introduction and conclusion, why is the rest of the stuff even there? Here, it’s important to remember that as someone doing research in these areas, you aren’t the original target audience of this publication. The target audience was the academic community in which it was published, and the goal of the paper was to convince that community that the conclusions of the paper are valid and properly scoped. The methodology, raw results, etc. are all in service of that goal: to convince the community that the paper’s conclusions are believable.
You’re welcome to read those areas, too, and come to your own conclusions, and as suggested above, if you’re working on something super-similar to what someone else has done, you probably want to do that. But for the vast majority of your reading, it’s usually sufficient to know, “This paper was selected for publication by this respected venue after a rigorous process of peer review.” That basically says, “If it’s good enough for them, it’s good enough for me.”
You Can Put the Cart Before the Horse
This entire write-up is written from the perspective of someone coming into this world with no prior ideas looking for a problem to solve. However, for many of you, you already have some ideas. You might want to build an intelligent tutoring system for your daughter’s Algebra class, or research whether bring your own technology initiatives improve learning outcomes in disadvantaged areas.
None of the above should suggest that you can’t do that. Rather, your existing ideas just give you a clearer anchor on where to start. If you already know what you’d like to work on, start with that area of the literature, and read with a particular eye toward developing your own idea. It’s a near-certainty that others have done something like what you want to do, but they may have done it in a different domain, a different grade level, with a different technology, etc. If you already know what you want to do, then your goal for this phase of the class is to find out how to put your ideas in the context of the community, as well as to make sure you’re building off whatever lessons have already been learned.
The biggest mistake people make in this class is to assume that their ideas are totally new, and therefore they do not need to look at what others have done. This is never true. Even if your idea is very different from others, there are analogues to others’ work from which you can likely learn. And even if your idea somehow is totally new, you need to be able to explain what makes it new, which requires understanding what others have done.
But I’m getting ahead of myself. We’ll talk about this more when we talk about how to find additional sources later in the week.
Research vs. Tools
You may notice much of this applies to academic research, but many of y’all are looking at developing tools or courseware. The same principles apply, however. You’re looking for what others have done in an area to get a general feel, and then you’re zooming in to your specific competitors or collaborators. So, the skillset we’re describing generally transfers. The big difference in research is that other researchers are usually far more open about reporting their results, and peer review keeps claims a bit more honest. So, developing these skills in academia is a great exercise even if you’re planning on taking a more business-oriented route: the skills are the same, but the business world is more guarded in the data it makes available.
In fact, in many ways, these skills are even more applicable to the business world. If you ever take a class on Entrepreneurship or go through a startup incubator, one of the lessons they’ll drill into your head is that if you think no one has done your idea before, then you haven’t looked hard enough. Nearly every idea has been explored; the question is always: what are you going to do better, or different?
But Don’t Take My Word For It…
Reading academic papers is a well-explored topic. Others have their own takes. If you want more on this, I recommend starting with UBC’s How to Read an Academic Paper video. It’s pretty similar to what I’ve written here, but with a more narrative visual style, so it’s a bit more approachable and digestible.
Then, I’d read Adam Ruben’s “How to read a scientific paper”, in particular his “10 Stages of Reading a Scientific Paper”. It won’t really help you that much, but it’ll reassure you that you’re not alone in finding this somewhat intimidating.
And then I’d read Elisabeth Pain’s “How to (seriously) read a scientific paper”. She asked several scientists how they approach it. I recommend this article because it also gives you a broader diversity of perspectives: maybe something else would work better for you. I’d also recommend William Griswold’s “How to Read an Engineering Research Paper”, especially for more design- and implementation-oriented papers rather than user research and testing papers (thanks to Chu for suggesting this one!).
TL;DR
You’ll rarely read entire papers. Instead, read just enough to get what you need. If you’re just trying to get a high-level feel for the work, read abstracts and conclusions. If you’re trying to understand the field as a whole, read the introduction and related work. If you’re trying to specifically position your work relative to a certain paper, read the methodology, analysis, future work, and limitations sections.
Next time, we’ll talk about how to find papers to read.