Project Milestone 2: Project Progress

By now, you should have made some progress on your project. This milestone will flesh out some of the components of your project from the proposal. Be sure to read the appropriate section for your project type below as the requirements start to diverge.

Demo: Build a novel NLP system

For this milestone, you will describe your demo in more detail. The milestone after this one will have a shorter report needed but will require a somewhat-working demo.

Write a report that includes the following sections. The questions here are meant to guide your writing process.

1. Project Description: What novel interactive experience do you propose to design? (This will collapse the Project Description and Related Work sections from your proposal into one section.)
   • Do similar systems exist to the one you propose to create? Give pointers to them and explain how you think they relate to your project idea. What were your inspirations?
   • What niche do you expect this program to fill? (Why are you making it? What does it do? Who is it for?)
   • Give an example mockup of what the user experience will look like. (Also consider: will there be any graphics or AI-generated images?)
   • Describe how you expect users to interact with the system. What is the interface?
   • Include a repo link to your code (as it is so far) in this section. You can link to it in a footnote or put it in the main text.
2. Back End Architecture: What does the overall architecture of your system look like?
   • Describe how your architecture will work.
   • What will each component of your pipeline be? Describe any datasets, models, tools, and tricks you’re using or plan to use. Be specific about the details of why you’re using them and what they’re doing in the overall system.
   • Give an example diagram of the backend system (the pipeline of components).
3. Evaluation: In order to tell if your system is performing well, you need an evaluation. How do you plan on evaluating your system?
   • What baselines, benchmarks, or other systems do you plan on comparing your system to? They can be pre-existing applications. You don’t necessarily need to reimplement them—for example, I’m not expecting you to remake Duolingo—but you need a way of comparing to them.
   • What metrics do you plan to use to compare your system to the baselines/benchmarks/existing applications? You can come up with ways of evaluating the components of your system separately and/or you can evaluate the entire system by doing a little user study where you will ask your friends to evaluate.
4. Team Members: Give a list of the students who are participating in this project, and what contributions you have made and expect to make for the project.
   • In this section also mention what you have completed so far and what is still left to do. You won’t be graded on how far you’ve gotten in the project, but obviously the further you get, the less you’ll have to scramble last-minute.
5. LLM Use Statement: Describe exactly how you used LLMs to generate parts of your proposal document (refer to the syllabus for guidance). If you did not use any generative text, please state so in this section.

Grading

Paper: Investigate an NLP research question

For this milestone, you will take the components of your proposal and begin to move them into more “paper-like” sections. You will also be adding in some more details.

While not required at this stage, we highly recommend that you use LaTeX and a conference template since you will have to for the next milestone. If you do not have a conference/workshop in mind, I recommend that you use the template from the Association for Computational Linguistics.

Your paper should include the following sections. The questions here are meant to guide your writing process.

1. Introduction: What research question(s) or problem are you trying to solve? (This is where your problem definition from the Project Description of your proposal goes.)
   • Why this it worth researching?
   • What contributions does this work make? (i.e., What are you doing that’s novel?)
   • If you are writing any code, either share the link to your repository in a footnote or mention it in the main text.
2. Related Work: What previous research has been done on this research question?
   • Turn your previous list of citations into a little narrative of what has been done previously in the field. Take a look at some previously-published papers to see how they write their related works section.
3. Methods: What are you doing or making to address your research question(s)?
   • If you are using any datasets, does the needed data already exist? If so, how much data is available? What does the data look like? If you are just using a few examples for few-shot prompting, how did you select those?
   • Describe your solution to the problem/research question. What model(s) are you using? If you are using few- or zero-shot prompting, what do your prompts look like?
4. Experiments: What experiments will you be running?
   • What are you measuring and why? What evaluation metrics do you plan to use to answer your research question(s)?
   • What are your proposed baselines? Or are you using a pre-defined benchmark? (That is, what are you comparing your method against?)
   • How do these experiments answer your research questions?
5. Team Members: Give a list of the students who are participating in this project, and what contributions you have made and expect to make for the project.
   • In this section also mention what you have completed so far and what is still left to do. You won’t be graded on how far you’ve gotten in the project, but obviously the further you get, the less you’ll have to scramble last-minute.
6. LLM Use Statement: Describe exactly how you used LLMs to generate parts of your proposal document (refer to the syllabus for guidance). If you did not use any generative text, please state so in this section.

Grading

What to Submit

Submit the following to Blackboard:

• team#-milestone2.pdf which contains your milestone 2 submission. To make grading easier, your proposal should include section headers corresponding to each of the bulleted points.