How To: Write an Academic Paper

As you near the end of the semester, we’re going to cover the four phases of trying to get an academic paper published:

Selecting a venue
Writing the paper
Submitting the paper
Presenting the work

Of course, two of those phases — 2 and 4, writing and presenting — are also required work for this class. To be clear, since there’s always someone that misunderstands: everyone (or every group) will submit a paper about their project and a presentation of their project. So, to maximize the time y’all have to incorporate these notes into your actual work, we’re going to cover 2 and 4 first, then 1 and 3.

What is an Academic Paper?

The foundation of the scientific community is the peer review process. Under the peer review process, scientists organize conferences and journals around common themes, and researchers who believe they have something worth contributing will write papers and submit to those conferences and journals. Paper submissions are then circulated among the prominent members of the community, who then review the work. These reviews are then sent back to the author, and a decision is made on whether to accept the paper. If accepted, authors typically still get to revise the paper to incorporate the feedback, and then their submission is included in the published journal or conference proceedings. For conference papers, an author of each paper goes to the conference to give a presentation or share a poster about the paper.

So, an academic paper is an attempt to contribute to the field. Most conferences and journals have standard formats to ensure consistent style in the published results. In this class, we’re requiring final papers to be in JDF format for consistency with the rest of the class.

What does it take to contribute?

The typical academic paper will have several parts, all targeted at making sure a paper contributes to the field. The typical outline is:

Introduction. Some background on what it is you’re talking about. Typically, this will include a cursory literature review just to define the problem area. It will culminate in the question the paper is answering or the problem the paper is solving, typically followed by a brief summary of the contribution itself.
Related Work. What is the context of the work? Who else is working on the same sorts of problems or questions? How is this work different? How does this work contribute to the broader field? A strong related work section is the underappreciated foundation of a good paper: it tells the reader exactly where this contribution sits in a broader narrative. This goes back to the first few weeks of this course: the entire reason why we asked you to go so far into the literature was so you could make a case that your work contributes to the field, rather than reinventing the wheel or making the same mistakes as others.
The Solution. If you built something, typically what follows is a description of what you built and why it was designed the way it was.
Methodology. If you built something, you generally need to test it and prove some positive results before it will be accepted for a full publication (it may be accepted for a shorter, lighter contribution based on implementation alone if it’s particularly novel, but for a high-impact contribution there typically needs to be some evaluation). The methodology tells how you tested it. If, on the other hand, you were researching some question, the methodology tells the reader how you set about answering it. The important thing about the methodology is that exists separate and before the results: it tells the reader why they should trust the results.
The Results. What was the result of the investigation or evaluation? This is typically what the entire paper is building toward: some assertion that the solution worked or some answer to the question that was raised.
Limitations. Another underappreciated section of any strong paper: the limitations section clearly articulates exactly how generalizable the conclusions are. For example, if work was done in the context of a middle school, then the results may only be generalizable to middle schools. If there were clear potential lurking variables in the methodology, then those would be disclosed here. The importance of the limitations section is that it clearly and honestly articulates how far the contribution goes. In my experience, any limitation you identify will not be held against your paper, but if the reviewers have limitations you don’t acknowledge, they’ll be far more reluctant to accept the contribution. This is what separates research from advertising.
Conclusion. A summary, basically. Reiterate the context, the problem, the solution, the results, and the limitations.
Future Work. Make a bunch of false promises about what you’re going to do next. (I kid, but generally, I don’t see much value in Future Work sections: they usually raise big questions that the authors have no intent on answering, and I’ve rarely seen a paper receive feedback asking for more future work, even if none was included in the first place.)

What papers get accepted?

So what papers get accepted? In my experience, the two most impactful parts of a paper on its likelihood to be accepted are the Related Work section and the Limitations section.

First, the Related Work section sets the stage for how invested you were in this paper and this community. It’s like a nice web design or a clean animation on a video: it shows the overall level of effort put into the paper, and sets a positive (or negative) expectation in the reader. Plus, a comprehensive Related Work section raises the odds that you actually cited your reviewers, which is never a bad thing.

The Limitations section is where you build trust in your reader. Although we can describe our methodology and analysis in lots of detail, the oversight over whether it actually was executed correctly typically lies within your own university. Falsified results aren’t likely to be caught in the initial peer review process, but rather in follow-up questioning and more comprehensive reporting to funding agencies. For this reason, reviewers are often skeptical of results that seem too good to be true, or work that overclaims its applications. Science is a slow, deliberate, methodical process, and reviewers appreciate when authors proactively emphasize how far their conclusions should be taken.

Of course, the other components are all important, too: a strong methodology, a clear contribution, a nice solution, and strong results (not necessarily positive results, but interesting results). However, those tend to be what most people emphasize anyway. Related Work and Limitations are often saved for the end, and that’s where many papers find themselves rejected: authors are more interested in bragging about their work than putting it in context or delineating its impact.

What about the content track?

All of this applies pretty clearly to the Development and Research tracks, but what about the Content track? In terms of academic publication, the Content track is very similar to the Development track: the solution that you built is content rather than software, but that content still aims to accomplish a goal, and that goal can (in theory) be evaluated.

For a good example of this, consider one of my papers from Learning @ Scale. The paper was about my undergraduate MOOC that we also offer for Georgia Tech credit. The solution is a course — if it was a class project, it would sit squarely within the content track. The paper covers (a) why an online course for this is necessary, (b) what other people have done to teach this topic, (c) why this course design is different, (d) the evaluation methodology for investigating the course, (e) the results of that methodology, and (f) the limitations.

But for this class…

Note that for this class, we don’t expect everyone to have a publication-worthy paper. The hope is that many of you will have that, but that isn’t a requirement of the class. Our goal is to set the stage for some of you to potentially reach that point, but most people won’t.

Specifically, in the context of a college semester, it’s entirely reasonable that you only built something and did not have time to evaluate it. Or, you conducted some research, but the conclusions are too preliminary or the methodology too uncontrolled to make for a strong contribution. That’s perfectly fine.

So for many of you, this may seem like a silly exercise. Most projects in the class aren’t at the point of being publishable, nor should they be after only a few weeks. Maybe it’s a good experience and learning exercise, but is it really worth it?

There’s a reason for this policy, though. This class is very much modeled after the classes that Ellen Do used to teach — they, too, involved an investigation phase, proposal phase, project phase, and delivery phase. She required papers to be conference-ready, too, even if they weren’t going to be submitted for publication. Why?

What we found was that when we require everyone to turn in a publication-ready paper, ~5% of the actual papers would end up getting published! However, if we didn’t require the papers to be conference ready, and instead let that be optional, no one actually wrote conference-ready papers, no one submitted them, and no one was published. The act of requiring everyone to submit conference-ready papers led to more people actually submitting and more people getting accepted, even though the majority still never submitted.

So, for a few extra minutes of thought, we drastically raise the likelihood of some of the papers being published. We’ve seen this in the past: around a dozen projects from this class have been published. So, it’s true: for many people, selecting a conference might not be that useful an exercise. Many of you might have trouble finding a conference that really fits. That’s alright! This is an example of where one size doesn’t fit all — but by having everyone complete these steps, some people will find it fits very well.

And even for those that don’t submit for publication, this exercise still carries value. For one, the formally-formatted papers tend to just look official, even if they aren’t peer reviewed. Taking the time to format the paper well is like putting on your best outfit for a first date: it doesn’t change the underlying content, but it puts it in the best possible light and shows you want it to leave a positive impression. We also encourage you to publish your final papers at SMARTech — that isn’t peer reviewed, but it nonetheless publishes the work in a public repository.