Project Milestone 3: Final Report Draft

For this Milestone, we are requiring you to write a rough draft of your final report (with appropriate blanks for experiments you have not yet conducted). While this may sound strange (How can I write the final report without having results?), writing a draft of the final report early can be a very effective technique for organizing your thoughts and final experiments.

We want to emphasize that you shouldn’t stress out too much about making this submission perfect (we will be fairly lenient with grading here) but we highly recommend taking this opportunity to write a solid first draft. Your future selves will be thankful!

To be specific, for this Milestone, you should:

• Finish all baseline experiments and get as many preliminary results from your main experiments as possible. You should show significant progress towards your main experimental results.
• Create your final report using one of the provided conference paper templates: ACL or AAAI (Please ask us directly if you would like to use a template for a different conference. Also, if you are not familiar with LaTeX, let us know.)
• Fill in as much of the final report as possible given your current results. Your report contain the sections outlined below.

Project Final Report

Your final report is a document that describes what you have accomplished in your project this semester. You are welcome and encouraged to draw content from the previous reports you have submitted so far (MS1 Report, MS2 Report, Lit Review). At this point in the class, all of you have read and presented on academic papers. You should use those journal and conference papers as inspiration for what a good writeup looks like. For additional inspiration, here’s an example of a final report from 2020.

Report Sections

The report should have the following sections. A lot of these can be reused and “fleshed out” from the previous Milestone.

Title & Author Names: What are you calling your project? These are not graded but still important! (No section header for these bits)

1. Project Description: This is the “what” and the “why” of your project.
   • The “what”: Explain the main idea of your project and what you are trying to accomplish. It’s okay if these have changed since the project proposal.
   • The “why”: If it’s a research project, what’s your research question (i.e., what are you trying to prove with this work)? If it’s an artifact that you’re creating, why are you creating it?
2. Related work: What previous research will you be using or building off of? What work are you inspired by? Tell a short “story” using what you found out from the Lit Review. This can include papers from the course schedule as well as anything else you want to cite.
3. Methods: What are the steps you took to accomplish your project? This section should include enough detail that someone reading it could reasonably attempt to reproduce your work. Include figures if applicable.
4. Data: What datasets, if any, did you use? If you collected or built your own dataset, how did you go about doing this? What are some potential problems/challenges you encountered with the data? If you had to clean your data, what did you use to do that? Explain your pre-processing steps.
5. Evaluation: How are you evaluating your results? How do you know if you’ve answered your research question? If you’re creating an interactive experience, what do you use to determine that it’s working well?
   • These can be things like accuracy, diversity metrics, perplexity, BLEU/ROUGE scores, success rate, human evaluation scores, etc. depending on your project.
6. Results: Describe your main results as well as the results for intermediate steps that build toward your main project goal. (Note for the milestone: If you are still waiting on important results, please talk about them and how they will fit into the final report.) Negative results or plans for experiments to run are great to report too! Please include tables, plots, and example system outputs where applicable.
7. Discussion: This is a meta section to discuss how the project went. What things did you try that didn’t work out? If you were to start over from scratch, what would you do differently? If someone were to continue working on this project, what do you suggest are good next steps/good future work?
8. Attribution: For each of your teammates, write a couple sentences describing their main contributions to the project. If you have any concerns about distribution of work, feel free to email us directly. This will not be in the final write-up, but we want to make sure you all still have a final plan for who will be doing what for the last part of the project.

Grading

This section is our grading rubric for the final report. This Milestone will not be graded too harshly but please keep these guidelines in mind. The more effort you put into this draft, the better our suggestions can be so that you can have a great final draft for the next Milestone.

1. Formatting: All of the sections described in the previous section are present and clearly marked. The report includes a title of the project and the name of the authors at the top. Images, tables, and other figures have captions and are referenced in the text. The paper makes use of citations that follow the standard convention of whatever conference the template is from.
2. Communication: Your report should be targeted at an audience who is familiar with NLP but has not taken the class and knows nothing about what you have done so far. You can imagine your audience as either the readers of a technical blog post (if you took the interactive experience option) or as reviwers of a paper submission to an NLP conference such as ACL (if you took the research question option). You should aim to make your methods section clear enough that others could attempt to reproduce your work and your results section clear enough that readers understand exactly what experiments you ran.
3. Style: You should write in paragraphs and complete sentences unless there is a very good reason not to.
4. Content: The report answers all relevant questions posed in the previous section. Images, tables, plots, of other figures are used to support the report contents.
5. Length: There is no strict word length requirement, but if your paper has fewer than 1,500 words (not including referneces) then it is very likely you have not provided enough detail.

What to Submit

Submit the following to Gradescope:

• milestone3.pdf which contains your milestone 3 submission. To make grading easier, your report should include section headers corresponding to each of the bulleted points. As stated earlier, the use of LaTeX and a conference template is required. You may choose to use the template from ACL or the template from AAAI. Please message us directly if you’d like to use a LaTeX template from a different conference.