Assignment 1 (Spring 2020)

Journal

First, if you haven’t already, you probably want to peek forward at the Qualifier Question and the Project Proposal assignments. These first several assignments build toward those, so it would be useful to understand where you’re trying to get!

The goal of these first several assignments is to the point of having a project idea in time to receive a Qualifier Question. However, different students will enter the class at different places in that pursuit. Some of you entered the class already knowing exactly what you want to work on. Others of you need some time to explore. The Research Guide is there to help you explore the literature no matter what ideas you have as you enter the class.

For the Journal portion of this assignment, you should document that exploration. This will look different depending on the ideas you came in with: if you already know what you want to work on, you’ll likely document exploring the work others have done in that area. If you came in without many ideas, you’ll likely document your exploration of the literature. Regardless of your starting point, though, we expect everyone to follow the guidelines of the Research Guide, including how to find new sources and read academic papers. Your starting point will just determine how wide a net you cast initially.

Throughout your journal, you should include multiple instances of…

• What papers you read or other sources you perused. We would expect you to look through at least 15 papers per week (but make sure you read How To: Read an Academic Paper — this number isn’t as terrifying as it sounds.) You’ll want to cite these in APA format to start practicing. You might want to peak forward at Assignment 4 as well; any sources you gather now may come in handy then!
• What questions they brought up.
• What you did to further investigate those questions.

Because this is a journal, it is intentionally designed to be personal; you’re welcome to include your personal background, your opinions, your thoughts, and so on. You might want to organize your journal into entries about individual papers, dates of the week, or lines of reasoning to help lend some structure to it. You similarly might not find yourself writing about each paper individually, but rather writing about collections (e.g. “These three papers [1][2][3] provided some background on different approaches to…”), or reading papers without directly referencing them in your text.

The most important things are to (a) document formal progress towards landing on a problem or question, and (b) supply enough information to get feedback from your mentor and classmates.

We would expect a good journal to be around 3-4 pages in JDF. This is neither a minimum nor a maximum, but rather is just a heuristic to understand the level of depth we would expect. We will expect your Journal to show that you’re following the advice prescribed by the Research Guide.

Activity

In a couple weeks, you’re going to be asked to propose your project for the semester. For some of you, that’s going to be building something that solves a problem (Development Track). For others, it’s going to be researching a phenomenon (Research Track), to identify it (“Are cat-owners more likely to succeed in OMSCS than dog-owners?”) and/or to explain it (“Why are cat-owners more likely to succeed in OMSCS than dog-owners?”). For others, you’re going to be teaching some content (Content Track), which in its own way is building something that solves a problem.

No matter what you’re doing, though, there are some commonalities. There will be a need you’re addressing, whether the need is a problem to be solved, a topic to be taught, or a phenomenon to be explained. There will be some audience, either to learn your material, to use your tool, or to participate (actively or passively) in your investigation. There will be some method you use to try to address that need, whether it be implementing some software, conducting some study, or developing some content. There will be a result: you might not reach the point of evaluating your result this semester, but you should be developing something that could be evaluated because that’s simply good design.

Mapping these together, however, can be a difficult task. One of the major things new researchers must learn is how to ensure a method is actually addressing their need and audience, as well as how to ensure an evaluation actually identifies success or failure. These are not trivial concepts.

One of the best ways to learn these things is to take CS6750, but either you’ve already done that or it’s probably too late to plan to do that before you move forward. The next best way, though, is to look at how others have addressed this.

So, for this week’s activity: select five papers. These can be papers you’ve reviewed as part of the first two weeks’ journals, or they can be papers you seek out solely for this activity. Note that not all papers will be compatible with this activity: you need to select papers where the authors were clearly solving a problem or investigating a phenomenon. Don’t worry, this still makes up a large number of papers.

Then, for each paper, introduce it with its full APA citation information and include a link to where the paper can be found. Then, create a logic chart/Guzdial Chart/Blumenfeld Chart. These charts are ways of mapping need, audience, method, and results in order to ensure they connect correctly. For this activity, yours will be a little more general than that link: that link is specifically for research projects, while we’ll include development projects as well.

Your chart should have four “columns”, each with 2-3 sentences answering the following questions. You do not have to actually make them columns, you can do this as subsequent subsections, bullets, or some other format, as long as all these parts are here and answered in sufficient depth:

• What is the need? What problem is trying to be solved or phenomenon trying to be investigated?
• What is method? What did they build to solve the problem? What did they do to investigate the phenomenon?
• Who is the audience? Who are the participants or subjects?
• What were the results? What did they find?

Then, for each paper, critique the alignment between need, audience, method, and results. Are the results justified by the method? Does the method address the need? Is the audience fitting for the need? Overall, how strong is the alignment between need, audience, method, and results?

For example, here would be what you might write about one of my papers, “Toward CS1 at Scale: Building and Testing a MOOC-for-Credit Candidate”:

Paper

Joyner, D. A. (2018). Toward CS1 at Scale: Building and Testing a MOOC-for-Credit Candidate. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale. London, United Kingdom. ACM Press.

Need

There is a need for scalable computer science education that preserves the rigor and reputability of an on-campus program. Modern MOOC platforms make content available, but they are often weaker on assessment, so the credential they generate is not as good as college credit.

Method

The authors developed an online CS1 course that is used as the basis for a for-credit class at Georgia Tech, as well as offered in MOOC form. To investigate its success, the authors had students in the for-credit online section complete a set of surveys and assessments that would allow them to be compared to the for-credit traditional section.

Audience

The audience for the MOOC overall is anyone interested in learning CS, but the audience for this study are undergraduates at Georgia Tech. Some of these undergraduates are in this online section, while some are in a traditional on-campus section.

Results

The authors found that students in the online for-credit section achieved comparable learning outcomes with students in the on-campus for-credit section, and appeared to enjoy the experience more, rating the course more highly. They also reported spending less time per week on the course.

Critique

The study gives pretty compelling evidence that the online for-credit students learn as much as the on-campus for-credit students. There are possible issues with the pre-test and post-test because it is difficult to know how much effort students invest in them, but that should be consistent across both sections, so it doesn’t bias things one way or the other. However, the method and results fall short of answering the core goal. Although this shows that online for-credit students learn as much as on-campus for-credit students, it does not demonstrate that MOOC students learn just as much as well. Without investigating the MOOC students’ actual learning outcomes, the authors can’t claim the MOOC students learn just as much (which they do not actually claim, but simply leave as a possibility).

You may find that some papers are a bit difficult to fit into this structure. For example, the paper used in the example above could be thought of in two ways: as a Development/Content study, where there was a need for scalable CS education and a MOOC was chosen as the method to solve that problem; or as a Research study, where there exists a scalable CS course and there is a need to demonstrate its equivalence with a traditional course through the method of shared assessments. That’s okay: there are no objective answers here. This is an organizing structure to allow you to structure your thinking and investigate your selected papers in depth.

The length of your deliverable will depend in part on your formatting, although we would generally expect 3-4 pages, a bit less than one page to answer these questions for each of the 5 papers.

Submission Instructions

Complete your assignment using JDF, then save your submission as a PDF. Assignments should be submitted to the corresponding assignment submission page in Canvas. You should submit a single PDF for this assignment. This PDF will be ported over to Peer Feedback for peer review by your classmates. If your assignment involves things (like videos, working software prototypes, etc.) that cannot be provided in PDF, you should provide them separately (through OneDrive, Google Drive, Dropbox, etc.) and submit a PDF that links to or otherwise describes how to access that material.

This is an individual assignment. Even if you already plan to work on a team for the project, this assignment should still be completed individually.

Late work is not accepted without advanced agreement except in cases of medical or family emergencies. In the case of such an emergency, please contact the Dean of Students.

Grading Information

As with all assignments in this class, this assignment will be graded on an 11-point scale (0 to 10), in accordance with the grading policy outlined in the syllabus. If your deliverable receives below a 9, you may revise and resubmit it once within one week of receiving a grade. Resubmissions may receive up to a 9. Note that this should not be treated as a de facto free pass to submit sorely lacking work initially; we reserve the right to deny resubmission or grade a resubmission more harshly if we perceive the original submission was lacking in earnest effort.

Peer Review

After submission, your assignment will be ported to Peer Feedback for review by your mentor and classmates. Grading is not the primary function of this peer review process; the primary function is simply to give you the opportunity to read and comment on your classmates’ ideas. All grades will come from the mentors alone.

You will typically be assigned four classmates to review. You receive 1.5 participation points for completing a peer review by the end of the day Thursday; 1.0 for completing a peer review by the end of the day Sunday; and 0.5 for completing it after Sunday but before the end of the semester. For more details, see the participation policy.