Assignment 2 (Fall 2019)
Assignment 2 has two parts. The first, the Journal, is an open-ended opportunity for you to report to your mentor and classmates the progress you’ve made since last week in exploring the literature and refining your idea. Each week until the Qualifier Question, you’ll submit a new journal documenting your research progress since the last week. The second, the Activity, is a more structured opportunity to practice one of the skills you’ll need as you move forward in the class. Each part is worth 50% of this assignment’s grade.
Journal
Your journal for assignment 2 follows the same procedure as that in assignment 1. The journal is a self-reflective account of your exploration of the literature in pursuit of a problem to solve or a question to answer. In this week’s journal, you should continue to document the sources that you encounter, the questions they bring up, and the investigations you perform, in line with the guidance of the Research Guide and with an eye toward the requirements of Assignment 3, the Qualifier Question, and the Proposal. Your journal should be all-new material (as you aren’t repeating the same research you did last week!), but you can include your previous journals as appendices if you think they provide useful context to your peer reviewers.
As a general heuristic, we would expect most people to look at 15-20 new papers this week, but the depth with which you look at each will vary: in some cases you’ll take a deep look, while in others you’ll more briefly just review the abstract and conclusions to get a more general feel for what people are doing. You may want to look ahead to Assignment 4: it will require an annotated bibliography, which will be much easier if you start working on that now!
How far along you are in the process will differ based on the ideas with which you entered the class. For many of you, you’ll still be exploring the literature to try to find what area you want to pursue in more depth. For some of you, though, you might already know what you’re most interested in doing, and so you may be diving deeper into that area. In either case, we would expect your journal to describe your current thought process on where you are and where you’d like to go, and also demonstrate the clear progress you’re making in exploring the literature.
The most important things are to (a) document formal progress towards landing on a problem or question, and (b) supply enough information to get feedback from your mentor and classmates.
We would expect a good journal to be around 3-4 pages in JDF. This is neither a minimum nor a maximum, but rather is just a heuristic to understand the level of depth we would expect. We will expect your Journal to show that you’re following the advice prescribed by the Research Guide.
Activity
In a couple weeks, you’re going to be asked to propose your project for the semester. For some of you, that’s going to be building something that solves a problem (Development Track). For others, it’s going to be researching a phenomenon (Research Track), to identify it (“Are cat-owners more likely to succeed in OMSCS than dog-owners?”) and/or to explain it (“Why are cat-owners more likely to succeed in OMSCS than dog-owners?”). For others, you’re going to be teaching some content (Content Track), which in its own way is building something that solves a problem.
No matter what you’re doing, though, there are some commonalities. There will be a need you’re addressing, whether the need is a problem to be solved, a topic to be taught, or a phenomenon to be explained. There will be some audience, either to learn your material, to use your tool, or to participate (actively or passively) in your investigation. There will be some method you use to try to address that need, whether it be implementing some software, conducting some study, or developing some content. There will be a result: you might not reach the point of evaluating your result this semester, but you should be developing something that could be evaluated because that’s simply good design.
Mapping these together, however, can be a difficult task. One of the major things new researchers must learn is how to ensure a method is actually addressing their need and audience, as well as how to ensure an evaluation actually identifies success or failure. These are not trivial concepts.
One of the best ways to learn these things is to take CS6750, but either you’ve already done that or it’s probably too late to plan to do that before you move forward. The next best way, though, is to look at how others have addressed this.
So, for this week’s activity: select five papers. These can be papers you’ve reviewed as part of the first two weeks’ journals, or they can be papers you seek out solely for this activity. Note that not all papers will be compatible with this activity: you need to select papers where the authors were clearly solving a problem or investigating a phenomenon. Don’t worry, this still makes up a large number of papers.
Then, for each paper, introduce it with its full APA citation information and include a link to where the paper can be found. Then, create a logic chart/Guzdial Chart/Blumenfeld Chart. These charts are ways of mapping need, audience, method, and results in order to ensure they connect correctly. For this activity, yours will be a little more general than that link: that link is specifically for research projects, while we’ll include development projects as well.
Your chart should have four “columns”, each with 2-3 sentences answering the following questions. You do not have to actually make them columns, you can do this as subsequent subsections, bullets, or some other format, as long as all these parts are here and answered in sufficient depth:
- What is the need? What problem is trying to be solved or phenomenon trying to be investigated?
- What is method? What did they build to solve the problem? What did they do to investigate the phenomenon?
- Who is the audience? Who are the participants or subjects?
- What were the results? What did they find?
Then, for each paper, critique the alignment between need, audience, method, and results. Are the results justified by the method? Does the method address the need? Is the audience fitting for the need? Overall, how strong is the alignment between need, audience, method, and results?
For example, here would be what you might write about one of my papers, “Toward CS1 at Scale: Building and Testing a MOOC-for-Credit Candidate”:
Paper
Joyner, D. A. (2018). Toward CS1 at Scale: Building and Testing a MOOC-for-Credit Candidate. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale. London, United Kingdom. ACM Press.
Need
There is a need for scalable computer science education that preserves the rigor and reputability of an on-campus program. Modern MOOC platforms make content available, but they are often weaker on assessment, so the credential they generate is not as good as college credit.
Method
The authors developed an online CS1 course that is used as the basis for a for-credit class at Georgia Tech, as well as offered in MOOC form. To investigate its success, the authors had students in the for-credit online section complete a set of surveys and assessments that would allow them to be compared to the for-credit traditional section.
Audience
The audience for the MOOC overall is anyone interested in learning CS, but the audience for this study are undergraduates at Georgia Tech. Some of these undergraduates are in this online section, while some are in a traditional on-campus section.
Results
The authors found that students in the online for-credit section achieved comparable learning outcomes with students in the on-campus for-credit section, and appeared to enjoy the experience more, rating the course more highly. They also reported spending less time per week on the course.
Critique
The study gives pretty compelling evidence that the online for-credit students learn as much as the on-campus for-credit students. There are possible issues with the pre-test and post-test because it is difficult to know how much effort students invest in them, but that should be consistent across both sections, so it doesn’t bias things one way or the other. However, the method and results fall short of answering the core goal. Although this shows that online for-credit students learn as much as on-campus for-credit students, it does not demonstrate that MOOC students learn just as much as well. Without investigating the MOOC students’ actual learning outcomes, the authors can’t claim the MOOC students learn just as much (which they do not actually claim, but simply leave as a possibility).
You may find that some papers are a bit difficult to fit into this structure. For example, the paper used in the example above could be thought of in two ways: as a Development/Content study, where there was a need for scalable CS education and a MOOC was chosen as the method to solve that problem; or as a Research study, where there exists a scalable CS course and there is a need to demonstrate its equivalence with a traditional course through the method of shared assessments. That’s okay: there are no objective answers here. This is an organizing structure to allow you to structure your thinking and investigate your selected papers in depth.
The length of your deliverable will depend in part on your formatting, although we would generally expect 3-4 pages, a bit less than one page to answer these questions for each of the 5 papers.
Submission Instructions
Complete your assignment using JDF, then save your submission as a PDF. Assignments should be submitted to the corresponding assignment submission page in Canvas. You should submit a single PDF for this assignment. This PDF will be ported over to Peer Feedback for peer review by your classmates. If your assignment involves things (like videos, working software prototypes, etc.) that cannot be provided in PDF, you should provide them separately (through OneDrive, Google Drive, Dropbox, etc.) and submit a PDF that links to or otherwise describes how to access that material.
This is an individual assignment. Even if you already plan to work on a team for the project, this assignment should still be completed individually.
Late work is not accepted without advanced agreement except in cases of medical or family emergencies. In the case of such an emergency, please contact the Dean of Students.
Grading Information
As with all assignments in this class, this assignment will be graded on an 11-point scale (0 to 10), in accordance with the grading policy outlined in the syllabus. If your deliverable receives below a 9, you may revise and resubmit it once within one week of receiving a grade. Resubmissions may receive up to a 9. Note that this should not be treated as a de facto free pass to submit sorely lacking work initially; we reserve the right to deny resubmission or grade a resubmission more harshly if we perceive the original submission was lacking in earnest effort.
Peer Review
After submission, your assignment will be ported to Peer Feedback for review by your mentor and classmates. Grading is not the primary function of this peer review process; the primary function is simply to give you the opportunity to read and comment on your classmates’ ideas. All grades will come from the mentors alone.
You will typically be assigned four classmates to review. You receive 1.5 participation points for completing a peer review by the end of the day Thursday; 1.0 for completing a peer review by the end of the day Sunday; and 0.5 for completing it after Sunday but before the end of the semester. For more details, see the participation policy.