<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>CS678 Advanced Natural Language Processing | George Mason NLP</title>
    <link>https://nlp.cs.gmu.edu/course/cs678-fall25/</link>
      <atom:link href="https://nlp.cs.gmu.edu/course/cs678-fall25/index.xml" rel="self" type="application/rss+xml" />
    <description>CS678 Advanced Natural Language Processing</description>
    <generator>Source Themes Academic (https://sourcethemes.com/academic/)</generator><language>en-us</language><copyright>Copyright © 2023 George Mason University, Natural Language Processing at George Mason</copyright><lastBuildDate>Fri, 01 Aug 2025 00:00:00 +0000</lastBuildDate>
    <image>
      <url>https://nlp.cs.gmu.edu/images/icon_hu_a92b877b56fb75ef.png</url>
      <title>CS678 Advanced Natural Language Processing</title>
      <link>https://nlp.cs.gmu.edu/course/cs678-fall25/</link>
    </image>
    
    <item>
      <title>Syllabus</title>
      <link>https://nlp.cs.gmu.edu/course/cs678-fall25/syllabus/</link>
      <pubDate>Fri, 01 Aug 2025 00:00:00 +0000</pubDate>
      <guid>https://nlp.cs.gmu.edu/course/cs678-fall25/syllabus/</guid>
      <description>&lt;h1&gt;CS 678&lt;/h1&gt; 
&lt;h2 style=&#34;color: #f2c164;&#34;&gt;Advanced Natural Language Processing&lt;/h2&gt;
&lt;h3 style=&#34;padding-top: 0.2em;&#34;&gt;Instructors&lt;/h3&gt;
&lt;a href=&#34;https://ziyuyao.org/&#34; target=&#34;_blank&#34;&gt;Ziyu Yao&lt;/a&gt; (ziyuyao [at] gmu [dot] edu)&lt;br&gt;Office Hours: Email for appointments.
&lt;p&gt;&lt;h3 style=&#34;padding-top: 0.2em;&#34;&gt;TA&lt;/h3&gt;
Primary TA: Saurabh Srivastava (ssrivas6 [at] gmu [dot] edu)&lt;br&gt;
Office Hours: TBD&lt;br&gt;
Secondary TA: Hao Yan (hyan5 [at] gmu [dot] edu)&lt;br&gt;
Office Hours: TBD
&lt;h3&gt;Meets&lt;/h3&gt;
Tuesday, 4:30 pm - 7:10 pm, Horizon Hall 1014.
&lt;br&gt;
&lt;!-- &lt;strong&gt;Safe Return to Campus:&lt;/strong&gt; Students are expected to follow the university&#39;s &lt;a href=&#34;https://www2.gmu.edu/safe-return-campus&#34;&gt;Safe-Return-to-Campus Policy&lt;/a&gt; (including mask wearing, daily health check, etc.) for attending any classes.
Please check out the policy before coming to the campus and the classroom. Note that students who choose not to abide by these expectations will be referred to the Office of Student Conduct for failure to comply. --&gt;
&lt;h3&gt;Course Web Page&lt;/h3&gt; 
&lt;a href=&#34;https://nlp.cs.gmu.edu/course/cs678-fall25/&#34;&gt;https://nlp.cs.gmu.edu/course/cs678-fall25/&lt;/a&gt;. 
&lt;br&gt;We will use &lt;strong&gt;Canvas&lt;/strong&gt; and &lt;strong&gt;Gradescope&lt;/strong&gt; for course materials/assignments/grading, and &lt;strong&gt;Piazza&lt;/strong&gt; for Q&amp;A (sign up link: &lt;a href=https://piazza.com/gmu/fall2025/cs678&gt;https://piazza.com/gmu/fall2025/cs678&lt;/a&gt;).
&lt;p&gt;&lt;h3&gt;Course Description&lt;/h3&gt; 
Massive amounts of information in our daily life are expressed in natural language. 
In this class, we will study building computing systems that can process, understand, and communicate in natural language. 
The class will start with an introduction to the foundations of natural language processing (NLP), and then focus on cutting-edge research problems in NLP. 
Each section will introduce a particular problem or phenomenon in natural language, describe why it is difficult to model, and demonstrate recent models that were designed to tackle this problem. In the process of doing so, the class will cover different techniques that are useful in creating and applying neural network models.
The class will include assignments, short quizzes, and a final project.
&lt;p&gt;&lt;h3&gt;Learning Outcomes&lt;/h3&gt; 
&lt;ul&gt;
	&lt;li&gt;Be familiar fundamental NLP problems and the methods powering the current state-of-the-art in language technologies, such as large pre-trained neural language models;&lt;/li&gt;
	&lt;li&gt;Understand the limitations of current technologies and be able to make informed decisions about addressing them using advanced machine learning techniques;&lt;/li&gt;
	&lt;li&gt;Design, implement, and evaluate a computing-based solution to address a language-related problem using state-of-the-art tools.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;h3&gt;Prerequisites&lt;/h3&gt; CS 580 (Intro to AI) or CS 584 (Data Mining). You should be proficient in (a) Algorithms and Data Structures and (b) Probability and Statistics (STAT 344) or equivalent. Students should be experienced with writing substantial programs in Python. Please contact the instructor if you have questions about the necessary background.
&lt;p&gt;&lt;h3&gt;Class Format&lt;/h3&gt; The class will be in-person. As the class aims to provide skills necessary to familiarize the students with, and to do cutting-edge NLP research, the classes and assignments will be at least partially implementation-focused. In general, each class will take the following format:
	&lt;ul&gt;
		&lt;li&gt;&lt;i&gt;Reading:&lt;/i&gt; Before some lectures, you will be pointed to some reading materials &lt;i&gt;(see &#34;Reading Materials&#34; in course schedule)&lt;/i&gt; that you should read before coming to class that day.&lt;/li&gt;
		&lt;li&gt;&lt;i&gt;Summary/Elaboration/Questions:&lt;/i&gt; The instructor will summarize the important points of the reading material, elaborate on details that were not included in the reading while fielding any questions. Finally, new material on cutting-edge methods, or a deep look into one salient method will be covered.&lt;/li&gt;
		&lt;li&gt;&lt;i&gt;Demo/Code Walk:&lt;/i&gt; In some classes we will walk through some demonstration code that implements a simple version of the main concepts presented in the reading material.&lt;/li&gt;
	&lt;/ul&gt;
&lt;p&gt;&lt;h3&gt;Grading&lt;/h3&gt; There will be no midterm or final exam. Your final grade will be dependent on:
&lt;p&gt; &lt;strong&gt;Class participation (20%):&lt;/strong&gt; (1) There will be 4 in-class quizzes throughout the semester, comprising 15% of the total grade. Each quiz will ask you 3-4 short-answer questions, designed to evaluate if you understand the lecture contents. (2) You will review in-class presentations from your classmates for project checkpoint 1 (5%). We will assign you to peer-review 3 random groups. You are expected to provide construct feedback to the assigned groups.
&lt;p&gt; &lt;strong&gt;Homework assignments (50%):&lt;/strong&gt; 
	&lt;ul&gt;
		&lt;li&gt;HW1: Preliminaries and n-gram language modeling (10%)&lt;/li&gt;
		&lt;li&gt;HW2: Post-training an open-weight language model (20%)&lt;/li&gt;
		&lt;li&gt;HW3: Implementing a tool-augmented LLM agent (20%)&lt;/li&gt;
	&lt;/ul&gt; 
&lt;/p&gt;
&lt;p&gt; &lt;strong&gt;Project (30%):&lt;/strong&gt; The bulk of your grade will be based on a group research project related to the topics we will discuss in class. &lt;strong&gt;The groups will be of 2-3 people. If you intend to complete the project independently or with more than 3 people in your group, you MUST ask for permission from the instructor.&lt;/strong&gt; 
&lt;br&gt;Briefly, the project will consist of the following milestones (&lt;strong&gt;refer to &lt;a href=&#34;../project&#34;&gt;Project Instruction&lt;/a&gt; for requirements on the checkpoint implementation, report, and in-class presentation&lt;/strong&gt;):
	&lt;ul&gt;
		&lt;li&gt;&lt;strong&gt;Checkpoint 1: Project Kickstart (10%):&lt;/strong&gt; By the first checkpoint, you will make a team, decide a topic, perform literature study, collect datasets, and outline a feasible plan for the project. This checkpoint also involves an in-class presentation, which is &lt;strong&gt;mandatory&lt;/strong&gt; for all team members.  
		&lt;/li&gt;
		&lt;li&gt;&lt;strong&gt;Checkpoint 2: Final Project Submission (20%):&lt;/strong&gt; You are expected to finish the project development, evaluation, and report. This checkpoint also involves an in-class presentation, which is &lt;strong&gt;mandatory&lt;/strong&gt; for all team members.
		&lt;/li&gt;
	&lt;/ul&gt;
	&lt;table style=&#34;cellspacing:0; cellpadding:5; border:0;&#34;&gt;
		&lt;tr&gt;
			&lt;th&gt; Letter Grade&lt;/th&gt;
			&lt;th&gt; Points (out of 100) &lt;/th&gt;
		&lt;/tr&gt;
		&lt;tr&gt;&lt;td&gt; A &lt;/td&gt; &lt;td&gt; 94-100 &lt;/td&gt;&lt;/tr&gt;
		&lt;tr&gt;&lt;td&gt; A- &lt;/td&gt; &lt;td&gt; 90-93 &lt;/td&gt;&lt;/tr&gt;
		&lt;tr&gt;&lt;td&gt; B+ &lt;/td&gt; &lt;td&gt; 86-89 &lt;/td&gt;&lt;/tr&gt;
		&lt;tr&gt;&lt;td&gt; B &lt;/td&gt; &lt;td&gt; 83-85 &lt;/td&gt;&lt;/tr&gt;
		&lt;tr&gt;&lt;td&gt; B- &lt;/td&gt; &lt;td&gt; 80-82 &lt;/td&gt;&lt;/tr&gt;
		&lt;tr&gt;&lt;td&gt; C+ &lt;/td&gt; &lt;td&gt; 76-79 &lt;/td&gt;&lt;/tr&gt;
		&lt;tr&gt;&lt;td&gt; C &lt;/td&gt; &lt;td&gt; 73-75 &lt;/td&gt;&lt;/tr&gt;
		&lt;tr&gt;&lt;td&gt; C- &lt;/td&gt; &lt;td&gt; 70-72 &lt;/td&gt;&lt;/tr&gt;
		&lt;tr&gt;&lt;td&gt; D &lt;/td&gt; &lt;td&gt; 60-69 &lt;/td&gt;&lt;/tr&gt;
		&lt;tr&gt;&lt;td&gt; F &lt;/td&gt; &lt;td&gt; 0-59 &lt;/td&gt;&lt;/tr&gt;
	&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;Late Day Policy:&lt;/strong&gt;&lt;br&gt;
In case there are unforeseen circumstances that don&amp;rsquo;t let you turn in your work on time, 5 late days &lt;i&gt;total&lt;/i&gt; over the assignments and project checkpoint 1 will be allowed without penalty. If you are late beyond the allowed late days, your assignment or project checkpoint 1 will be graded down one half-grade per day late (e.g., Assume your assignment receives a score of 95/100 before penalty. If you have used out your allowed late days and are now 1 more day&amp;rsquo;s late, then your final score becomes 90/100.)&lt;/p&gt;
&lt;p&gt;For project checkpoint 2 (final report submission), no late days are allowed. All quizzes and in-class activities (presentations and peer-review) need to be completed in class and there is no lateness allowed. As a result, for project checkpoint 1, even if you can apply the allowable late days to the report submission, you still need to make the in-class presentation on time.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Contact the instructor IMMEDIATELY and BEFORE DUE DAYS if you know that you will very likely to be late beyond the allowable late days due to illness or other incidents. If you have any questions about the late day policy, similarly reach out to the instructor IMMEDIATELY.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;h3&gt;Readings&lt;/h3&gt; 
	For each topic/class the instructor will provide a list of papers as suggested readings. 
	Students should be able to understand the course content just by following the lecture and by doing the readings. 
	We will mainly use the following textbooks.
	&lt;ul&gt;
		&lt;li&gt;Jurafsky and Martin, Speech and Language Processing, 3rd edition &lt;a href=&#34;https://web.stanford.edu/~jurafsky/slp3/&#34;&gt;[online; January 12, 2025 release]&lt;/a&gt; &lt;strong&gt;(Referred to as &#34;JM&#34;)&lt;/strong&gt;;&lt;/li&gt;
		&lt;li&gt;Jacob Eisenstein, Natural Language Processing &lt;a href=&#34;https://github.com/jacobeisenstein/gt-nlp-class/blob/master/notes/eisenstein-nlp-notes.pdf&#34;&gt;[online]&lt;/a&gt; &lt;strong&gt;(Referred to as &#34;Eisenstein&#34;)&lt;/strong&gt;;&lt;/li&gt;
        &lt;li&gt;Yoav Goldberg, Neural Network Methods in Natural Language Processing &lt;a href=&#34;https://www.morganclaypool.com/doi/abs/10.2200/S00762ED1V01Y201703HLT037?journalCode=hlt&#34;&gt;[publisher]&lt;/a&gt; &lt;a href=&#34;https://u.cs.biu.ac.il/~yogo/nnlp.pdf&#34;&gt;[online primer pdf]&lt;/a&gt; &lt;strong&gt;(Referred to as &#34;Goldberg-Publisher/Primer&#34;)&lt;/strong&gt;; 
        Note that the &#34;publisher&#34; version can be downloaded if you use the school VPN.&lt;/li&gt;
	&lt;/ul&gt;
&lt;p&gt;&lt;h3&gt;Tentative Schedule&lt;/h3&gt;
	&lt;table style=&#34;cellspacing:0; cellpadding:5; border:0; width: 100%;&#34;&gt;
		&lt;colgroup&gt;
       		&lt;col span=&#34;1&#34; style=&#34;width: 7%;&#34;&gt;
       		&lt;col span=&#34;1&#34; style=&#34;width: 30%;&#34;&gt;
       		&lt;col span=&#34;1&#34; style=&#34;width: 20%;&#34;&gt;
       		&lt;col span=&#34;1&#34; style=&#34;width: 43%;&#34;&gt;
    	&lt;/colgroup&gt;
		&lt;tr&gt;
			&lt;th&gt; Date  &lt;/th&gt;
			&lt;th&gt; Topic &lt;/th&gt;
			&lt;th&gt; Assignment Details &lt;/u&gt; &lt;/th&gt;
			&lt;th&gt; Reading Materials &lt;/th&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 08/26 &lt;/td&gt;
			&lt;td&gt; Introduction and ML/NN Basics &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
			&lt;td&gt; JM Ch4.7, Ch5 (Logistic Regression); Eisenstein Ch2 &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 09/02 &lt;/td&gt;
			&lt;td&gt; Language Modeling and Tokenization &lt;/td&gt;
			&lt;td&gt; HW1 Out &lt;/td&gt;
			&lt;td&gt; JM Ch3 (Language Modeling), Ch7 (FFNN); Goldberg-Primer Ch6.1-6.3 (FFNN); &lt;a href=&#34;https://www.youtube.com/watch?v=zduSFxRajkE&amp;t=1457s&amp;ab_channel=AndrejKarpathy&#34;&gt;Andrej Karpathy&#39;s tokenization course&lt;/a&gt; &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 09/09 &lt;/td&gt;
			&lt;td&gt; Neural Language Modeling and Word Embedding &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
			&lt;td&gt; JM Ch8 (RNN, LSTMs), Ch6 (Vector Semantics); Eisenstein Ch6; Mikolov et al., &lt;a href=&#34;https://arxiv.org/pdf/1301.3781.pdf&#34;&gt;2013a&lt;/a&gt;&amp;&lt;a href=&#34;https://arxiv.org/pdf/1310.4546.pdf&#34;&gt;b&lt;/a&gt;&lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 09/16 &lt;/td&gt;
			&lt;td&gt; NN Architectures: Seq2Seq, Attention, and Transformers &lt;/td&gt;
			&lt;td&gt; HW1 Due; HW2 Out &lt;/td&gt;
			&lt;td&gt; JM Ch8 (Enc-Dec), Ch9 (Transformers); Eisenstein Ch18.3.1; &lt;a href=&#34;http://colah.github.io/posts/2015-08-Understanding-LSTMs/&#34;&gt;&#34;understand LSTM&#34; blog by Olah&lt;/a&gt;; &lt;a href=&#34;http://neuralnetworksanddeeplearning.com/chap5.html#the_vanishing_gradient_problem&#34;&gt;&#34;gradient vanishing&#34; blog by Nielsen&lt;/a&gt;; &lt;a href=&#34;https://arxiv.org/pdf/1508.04025.pdf&#34;&gt;Attention-based NMT&lt;/a&gt;; &lt;a href=&#34;https://arxiv.org/pdf/1706.03762.pdf&#34;&gt;Transformer paper&lt;/a&gt; and &lt;a href=&#34;https://jalammar.github.io/illustrated-transformer&#34;&gt;Alammar&#39;s blog&lt;/a&gt; &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 09/23 &lt;/td&gt;
			&lt;td&gt; Large Language Models (LLMs) &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
			&lt;td&gt; JM Ch10-11; &lt;a href=&#34;https://arxiv.org/pdf/1810.04805.pdf&#34;&gt;BERT&lt;/a&gt;; &lt;a href=&#34;https://arxiv.org/pdf/1910.10683.pdf&#34;&gt;T5&lt;/a&gt;;
				&lt;a href=&#34;https://arxiv.org/abs/2001.08361&#34;&gt;Scaling Laws for Neural Language Models (Kaplan et al., 2020)&lt;/a&gt;&lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 09/30 &lt;/td&gt;
			&lt;td&gt; LLM Advancement 1: Prompting and Alignment (Post-Training) &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
			&lt;td&gt; JM Ch12; &lt;a href=&#34;https://arxiv.org/pdf/2107.13586.pdf&#34;&gt;Pre-train, Prompt, and Predict... (Liu et al., 2021)&lt;/a&gt;;
				&lt;a href=&#34;https://arxiv.org/pdf/2201.11903.pdf&#34;&gt;Chain-of-Thought Prompting (Wei et al., 2022)&lt;/a&gt; &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 10/07 &lt;/td&gt;
			&lt;td&gt; LLM Advancement 2: LLM Interpretability &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
			&lt;td&gt; &lt;a href=&#34;https://arxiv.org/abs/1602.04938&#34;&gt;LIME&lt;/a&gt;; &lt;a href=&#34;https://github.com/Eric-Wallace/interpretability-tutorial-emnlp2020&#34;&gt;EMNLP20 tutorial&lt;/a&gt;; &lt;a href=&#34;https://arxiv.org/pdf/2407.02646&#34;&gt;Mechanistic Interpretability (Rai et al. 2024)&lt;/a&gt; &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 10/14 &lt;/td&gt;
			&lt;td&gt; LLM Advancement 3: LLM Agents &lt;/td&gt;
			&lt;td&gt; HW2 Due; HW3 Out &lt;/td&gt;
			&lt;td&gt; &lt;a href=&#34;https://lilianweng.github.io/posts/2023-06-23-agent/&#34;&gt;Lilian Weng&#39;s blog&lt;/a&gt;; &lt;a href=&#34;https://arxiv.org/pdf/2304.03442&#34;&gt;Stanford Smallville&lt;/a&gt; &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 10/21 &lt;/td&gt;
			&lt;td&gt; Linguistic Annotation: Syntactic and Semantic Parsing &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
			&lt;td&gt; JM Ch17.1-17.3 (POS tagging), 18.1-18.2 (const par), 19.1 (dep par), Appendix F (sem par) &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 10/28 &lt;/td&gt;
			&lt;td&gt; Project Checkpoint 1 Presentation &lt;/td&gt;
			&lt;td&gt; Project Ckpt 1 Report Due 10/28, Peer-Review Feedback Due 10/31  &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 11/04 &lt;/td&gt;
			&lt;td&gt; No Class (Election Day) &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 11/11 &lt;/td&gt;
			&lt;td&gt; Applied NLP 1: Chatbot and Dialogue Systems &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
			&lt;td&gt; JM Ch15 &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 11/18 &lt;/td&gt;
			&lt;td&gt; Applied NLP 2: Question Answering, Info Retrieval, and RAG &lt;/td&gt;
			&lt;td&gt; HW3 Due &lt;/td&gt;
			&lt;td&gt; JM Ch14; &lt;a href=&#34;https://github.com/danqi/acl2020-openqa-tutorial&#34;&gt;ACL20 tutorial&lt;/a&gt; &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 11/25 &lt;/td&gt;
			&lt;td&gt; Applied NLP 3: AI/LLM Safety &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
			&lt;td&gt; &lt;a href=&#34;https://genai.owasp.org/llm-top-10/&#34;&gt;OWASP&#39;s Top10 LLM Risks&lt;/a&gt; &lt;/td&gt;
		&lt;/tr&gt;
		&lt;tr&gt;
			&lt;td&gt; 12/02 &lt;/td&gt;
			&lt;td&gt; Final Project Presentation &lt;/td&gt;
			&lt;td&gt; Project Checkpoint 2 Due on 12/09 &lt;/td&gt;
			&lt;td&gt;  &lt;/td&gt;
		&lt;/tr&gt;
	&lt;/table&gt;
&lt;P&gt;&lt;h3&gt;Honor Code&lt;/h3&gt; The class enforces the &lt;a href=&#34;https://academicstandards.gmu.edu/&#34;&gt;GMU Honor Code&lt;/a&gt;, and the &lt;a href=&#34;https://cs.gmu.edu/resources/honor-code/&#34;&gt;more specific honor code policy&lt;/a&gt; special to the Department of Computer Science. You will be expected to adhere to this code and policy.
&lt;ul&gt;
	&lt;li&gt;&lt;strong&gt;IMPORTANT NOTE about the use of ChatGPT:&lt;/strong&gt; Unless permission to do so is granted by the instructor, you (or your group, if a group assignment) may not use assistive technology, artificial intelligence, or other tools to complete assignments which can generate, translate, or otherwise create/correct code or answers (many types of assistive technology may be permitted, but you must ask permission).&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;h3&gt;Note to Students&lt;/h3&gt; Take care of yourself! As a student, you may experience a range of challenges that can interfere with learning, such as strained relationships, increased anxiety, substance use, global pandemics, feeling down, difficulty concentrating and/or lack of motivation. All of us benefit from support during times of struggle. There are many helpful resources available on campus and an important part of having a healthy life is learning how to ask for help. Asking for support sooner rather than later is almost always helpful. GMU services are available, and treatment does work. You can learn more about confidential mental health services available on campus at: &lt;a href=&#34;https://caps.gmu.edu/&#34;&gt;https://caps.gmu.edu/&lt;/a&gt;. Support is always available (24/7) from Counseling and Psychological Services: 703-527-4077.
&lt;p&gt;&lt;h3&gt;Disabilities&lt;/h3&gt;If you have a documented learning disability or other condition which may affect academic performance, make sure this documentation is on file with the &lt;a href=&#34;http://www.gmu.edu/student/drc&#34;&gt;Office of Disability Services&lt;/a&gt; and come talk to me about accommodations. I will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the Office of Disability Services, I encourage you to contact them at ods@gmu.edu.
</description>
    </item>
    
    <item>
      <title>CS 678 Course Project</title>
      <link>https://nlp.cs.gmu.edu/course/cs678-fall25/project/</link>
      <pubDate>Tue, 01 Jul 2025 00:00:00 +0000</pubDate>
      <guid>https://nlp.cs.gmu.edu/course/cs678-fall25/project/</guid>
      <description>&lt;h2 style=&#34;color: #f2c164;&#34;&gt;Advanced Natural Language Processing&lt;/h2&gt;
&lt;p&gt;&lt;br&gt;Students will form groups (each with 2-3 people) for the course project. There will be in total two submissions toward completing the project: Project Kickstart and Final Project Submission.&lt;/p&gt;
&lt;!-- &lt;strong&gt;Acknowledgement: &lt;/strong&gt;
Instructions and the template are adapted from &lt;a href=&#34;https://sites.google.com/iesl.cs.umass.edu/compsci-692a-akbc/projects?authuser=0&#34;&gt;UMass COMPSCI692A by Andrew McCallum&lt;/a&gt;, &lt;a href=&#34;http://www.phontron.com/class/anlp2021/assignments.html&#34;&gt;CMU CS711 by Graham Neubig&lt;/a&gt;, and from the &lt;a href=&#34;https://paperswithcode.com/rc2022&#34;&gt;ML Reproducibility Challenge 2022&lt;/a&gt;. --&gt;
&lt;h3&gt;Project Topics&lt;/h3&gt;
We expect student projects to fall into roughly the following two categories:
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;NLP Application:&lt;/strong&gt; Students are expected to develop an &lt;strong&gt;application&lt;/strong&gt; or a &lt;strong&gt;tool&lt;/strong&gt; using techniques learned from this class. Example applications include:
&lt;ul&gt;
&lt;li&gt;Building a mental health therapy chatbot;&lt;/li&gt;
&lt;li&gt;Building a chatbot for K-12 education or help illiterate people learn to read and write;&lt;/li&gt;
&lt;li&gt;Building a system that finds related literature for scientists;&lt;/li&gt;
&lt;li&gt;Building a domain-specific agent that help people find information and perform tasks automatically on the specified domain context;&lt;/li&gt;
&lt;li&gt;&amp;hellip;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;NLP Research:&lt;/strong&gt; Alternatively, students are also encouraged to pick a &lt;strong&gt;research-oriented&lt;/strong&gt; project, identifying a research problem and proposing a novel approach to address the problem. Example research topics include:
&lt;ul&gt;
&lt;li&gt;Interpreting the mechanisms of LMs in reasoning and planning (e.g., math reasoning, web navigation);&lt;/li&gt;
&lt;li&gt;Benchmarking how well an LM can reason or plan in scenarios that were not covered by prior work;&lt;/li&gt;
&lt;li&gt;Developing approaches to improve state-of-the-art LM performance on an established benchmark (e.g., math reasoning, event extraction, etc.);&lt;/li&gt;
&lt;li&gt;Developing approaches to dynamically control the reasoning length of a large reasonin model (LRM);&lt;/li&gt;
&lt;li&gt;&amp;hellip;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In comparison, an &lt;strong&gt;Application&lt;/strong&gt; may involve more novelty in &lt;em&gt;applying existing techniques to an interested scenario&lt;/em&gt;, whereas a &lt;strong&gt;Research&lt;/strong&gt; project involves more novelty in &lt;em&gt;discovering new scientific insights and/or advancing state-of-the-art methodologies&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Sometimes, a project may lie in the intersection of both categories.&lt;/strong&gt; For example, when developing a domain-specific agent to help people with information seeking and task completion (Application), you may need find that existing techniques all fail to enable a reasonable agent performance, so you eventually devise new methodologies (Research). We encourage such projects as they often lead to both scientific merits and practical tools.&lt;/p&gt;
&lt;p&gt;No matter which kind of project your team decides, students need to submit project progress under the following two checkpoints.&lt;/p&gt;
&lt;h3&gt;Checkpoint 1: Project Kickstart&lt;/h3&gt;
&lt;p&gt;By the first checkpoint, you are expected to:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Form a team. Use Piazza if you have an idea and are looking for teammates or for feedback/iterating on your idea.&lt;/li&gt;
&lt;li&gt;Decide your project topic and do the preliminary work towards your implementation. Your preliminary work should be detailed in a PDF report submission.&lt;/li&gt;
&lt;li&gt;Prepare a short presentation of your project in class and submit a PDF of your slides.&lt;/li&gt;
&lt;li&gt;Peer-review other teams&amp;rsquo; project presentations.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Report Specifics&lt;/strong&gt;
Your report should include the following components &lt;strong&gt;using the provided template (TBD)&lt;/strong&gt;:&lt;/p&gt;
&lt;ul&gt;
    &lt;li&gt;&lt;strong&gt;Project Title:&lt;/strong&gt; Something descriptive but also memorable. Make sure to add names and emails of team members under the title.&lt;/li&gt;
    &lt;li&gt;&lt;strong&gt;Project Description and Motivation:&lt;/strong&gt; What application or research problem are you targeting in this project? What&#39;s your motivation (i.e., why does it matter to build this application or work on this research problem)? &lt;/li&gt;
    &lt;li&gt;&lt;strong&gt;Related Work&lt;/strong&gt;: Have people worked on a similar application or a relevant research problem? Include at least 3 papers directly relevant to the proposed project. For each paper, use 3-4 sentences to describe 1) what prior work has done and 2) how your work is different from their. &lt;/li&gt;
    &lt;li&gt;&lt;strong&gt;Project Plan&lt;/strong&gt;, which must contain the following subsections:
        &lt;ul&gt;
            &lt;li&gt;&lt;strong&gt;Dataset(s)&lt;/strong&gt; - Please list which dataset(s) you plan to use and whether or not you have access to them. &lt;/li&gt;
            &lt;li&gt;&lt;strong&gt;Implementation&lt;/strong&gt; - Describe how to plan to implement this application or research exploration. Be specific about any software, libraries, etc., you plan to use.&lt;/li&gt;
            &lt;li&gt;&lt;strong&gt;Resources&lt;/strong&gt; - How much compute will you need? Have you confirmed the resource availability (from ORC Hopper or other resources)? If not, where do you plan to obtain the resources? &lt;/li&gt;
            &lt;li&gt;&lt;strong&gt;Evaluation&lt;/strong&gt; - How do you plan to evaluate your application or research exploration? Please be specific about the dataset(s) you plan to use and the evaluation metrics. Consider both qualitative (e.g., human evaluation, case studies) and quantitative (e.g., evaluation through automatic metrics) evaluations. &lt;/li&gt;
        &lt;/ul&gt;
    &lt;/li&gt;
    &lt;li&gt;&lt;strong&gt;Team Work and Timeline Clarification:&lt;/strong&gt; (1) Describe each team member&#39;s contribution to this checkpoint. (2) Describe how each team member expects to collaborate towards the final project and make a timeline (as concrete as possible). &lt;strong&gt;Note that each team member should contribute roughly the same amount of work to the project.&lt;/strong&gt; &lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Presentation Specifics&lt;/strong&gt;
The slides of your project pitch should include a summary of what you have in the PDF report:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Title slide (Project Name, Team Members Names, Team Members Emails)&lt;/li&gt;
&lt;li&gt;Project description and motivation (1 page)&lt;/li&gt;
&lt;li&gt;Related work (1 page)&lt;/li&gt;
&lt;li&gt;Project plan and timeline (2-3 pages)
&lt;ul&gt;
&lt;li&gt;What data will you use?&lt;/li&gt;
&lt;li&gt;How do you plan to implement the project and what resources will you need?&lt;/li&gt;
&lt;li&gt;How do you plan to evaluate your project?&lt;/li&gt;
&lt;li&gt;In what timeline do you plan to finish the project?&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Checkpoint 1 Grading&lt;/strong&gt;
This component is worth 10% of your overall grade, consisting of 8% for the report or checkpoint content and 2% for presentation. The project peer review is worth another 5% under &amp;ldquo;Class Participation&amp;rdquo;.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The grading of the checkpoint 1 report or content will be based on (1) completeness (e.g., do you have all required elements in the report and presentation?), (2) clarity and quality (e.g., do you describe each element clearly, carefully, and thoughfully?), (3) novelty (e.g., is your application or research topic novel?), and (4) others, such as whether the team has a reasonable timeline, plan for compute resources, and plan for team work.&lt;/li&gt;
&lt;li&gt;The grading of the checkpoint 1 presentation will be based on whether the team presents the project clearly and carefully under the given time limit. &lt;strong&gt;Note that all team members need to present the project in person; failing to do it will result in significant point deduction for the entire Checkpoint 1 (beyond the 2%).&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Checkpoint 1 Peer Review&lt;/strong&gt;
You will be assigned to review three randomly assigned projects from your classmates. For each project, write a peer-review summary covering the following points:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Project description and motivation: Does the team describe their project scope clearly? Is the project well motivated?&lt;/li&gt;
&lt;li&gt;Related work: Does the team show to research for related work carefully? Do they describe the related work and show the distinction of their project clearly? Do they omit any related work or do you suggest further related work to the team?&lt;/li&gt;
&lt;li&gt;Project plan and timeline: Are the project plan and timeline adequate? Does the team describe clearly what dataset(s) and compute they will use? Is their implementation plan clear and reasonably devised? Similarly, do you think they have a proper evaluation plan?&lt;/li&gt;
&lt;li&gt;Write 1-3 things where you think the team does well;&lt;/li&gt;
&lt;li&gt;Write 1-3 things where you think the team should have done better, and provide your suggestion.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We require the peer review to be specific and constructive. Imagine yourself as a critic, examining the project outline with a critical mind and providing helpful suggestions to your classmates. When you do so, you will also receive helpful feedback from your classmates. The peer review is worth 5% of the total grade as part of the class participation.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Submission Guideline&lt;/strong&gt;
TBD&lt;/p&gt;
&lt;h3&gt;Checkpoint 2: Final Report Submission&lt;/h3&gt;
&lt;p&gt;By the second checkpoint, you are expected to complete the following activities:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Finish the entire project;&lt;/li&gt;
&lt;li&gt;Prepare a short in-class presentation of your project outcome in the last class (Dec 2nd);&lt;/li&gt;
&lt;li&gt;Submit a PDF report including a link to your source code on GitHub one week after the last class (Dec 9th).&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Report Specifics&lt;/strong&gt;
The final project report should includ the following components &lt;strong&gt;using the provided template (TBD)&lt;/strong&gt;:&lt;/p&gt;
&lt;ul&gt;
    &lt;li&gt;&lt;strong&gt;Project Title:&lt;/strong&gt; It is okay if your title is slightly different from the title in your checkpoint 1 report. Make sure to add names and emails of team members under the title.&lt;/li&gt;
    &lt;li&gt;&lt;strong&gt;Detailed System/Methodology Description:&lt;/strong&gt; Describe in full detail what you have implemented, how you implemented it, including what datasets and resources you used in the implementation. &lt;strong&gt;Tip:&lt;/strong&gt; A suggestion for system/methodology description is to draw a figure showing your system/methodology workflow; exmaple see &lt;a href=&#34;https://arxiv.org/pdf/2504.07357&#34;&gt;Fig 2 of Srivastava and Yao (2025)&lt;/a&gt;. Sometimes, you also need a figure or a table demonstrating your data processing; example see &lt;a href=&#34;https://arxiv.org/pdf/2504.07357&#34;&gt;Fig 3 of Srivastava and Yao (2025)&lt;/a&gt; where the authors demonstrated how they process an event extraction task as a python code generation problem with a figure, and &lt;a href=&#34;https://arxiv.org/pdf/2308.04030&#34;&gt;Table 1 of Gentopia AI (2023)&lt;/a&gt;, where the authors illustrated the tasks and datasets used in their experiments.&lt;/li&gt;
    &lt;li&gt;&lt;strong&gt;Evaluation:&lt;/strong&gt;
        &lt;ul&gt;
            &lt;li&gt;&lt;strong&gt;Experimental Setup&lt;/strong&gt; - How did you evaluate your application or research outcomes? Be specific about the evaluation procedure and the used metrics. If your project involves proposing a new approach, you may also need to describe the baseline approaches in this section.&lt;/li&gt;
            &lt;li&gt;&lt;strong&gt;Experimental Results&lt;/strong&gt; - What&#39;s your evaluation result? Does your evaluation tell you where your application or research exploration succeeds and/or fails? &lt;strong&gt;Tip:&lt;/strong&gt; You often need a table to summarize the evaluation results; example see &lt;a href=&#34;https://arxiv.org/pdf/2504.07357&#34;&gt;Table 1 of Srivastava and Yao (2025).&lt;/a&gt; &lt;/li&gt;
            &lt;li&gt;&lt;strong&gt;Analysis&lt;/strong&gt; - Complement your evaluation with a further analysis. The goal is to dive deep and discover the reasons why the application or research exploration succeeds and/or fails, such that future researcher (or yourself!) knows how to further improve it. &lt;/li&gt;
        &lt;/ul&gt;
    &lt;/li&gt;
    &lt;li&gt;&lt;strong&gt;Limitations and Future Work:&lt;/strong&gt;Describe any limitations in your current project design, including the system/approach design, the experimental setup, the implementation procedure, problems revealed from the evaluation, etc., and discuss how these limitations can be addressed in the future.&lt;/li&gt;
    &lt;li&gt;&lt;strong&gt;Team Work Clarification:&lt;/strong&gt; Describe each team member&#39;s contribution to this checkpoint. &lt;strong&gt;Note that each team member should contribute roughly the same amount of work to the project.&lt;/strong&gt; &lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Presentation Specifics&lt;/strong&gt;
The slides of your final project presentation should include a summary of what you have in the PDF report:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Title slide (Project Name, Team Members Names, Team Members Emails)&lt;/li&gt;
&lt;li&gt;System/Methodology description (1 page)
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;You must include a figure (better a real-time demo) showing how your system or approach works.&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;Evaluation (2-3 pages)
&lt;ul&gt;
&lt;li&gt;In what setup did you evaluate your project?&lt;/li&gt;
&lt;li&gt;What&amp;rsquo;s your evaluation result?&lt;/li&gt;
&lt;li&gt;What&amp;rsquo;s the key takeaways from the evaluation?&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;Future work: How can future researchers further improve this project? (1 page)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Grading&lt;/strong&gt;
This checkpoint is worth 20% of your overall grade, consisting of 13% for the report, 5% for the source code, and 2% for project presentation.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The grading of the checkpoint 2 report or content will be based on (1) completeness (e.g., do you have all required elements in the report and presentation?), (2) clarity and quality (e.g., do you describe each element clearly, carefully, and thoughfully? do you use figures or tables properly to demonstrate the system/methodology design and evaluation results? are the analysis and future work discussion thoughtful? etc.), (3) novelty (e.g., is your final application or research exploration novel?), and (4) others, such as whether the team shows reasonable team work.&lt;/li&gt;
&lt;li&gt;The grading of the source code will be based on whether the team has a clear documentation for the grader to reproduce the project (see &lt;a href=&#34;https://github.com/Ziyu-Yao-NLP-Lab/PyCode-TextEE&#34;&gt;
&lt;a href=&#34;https://github.com/Ziyu-Yao-NLP-Lab/PyCode-TextEE&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;https://github.com/Ziyu-Yao-NLP-Lab/PyCode-TextEE&lt;/a&gt;&lt;/a&gt; for example; the key is to include all necessary commands for the grader to train/test your system/methodology), and whether the source code is fully executable (including loading and processing any required datasets and running the evaluation). &lt;strong&gt;Incomplete or problematic source code could threat the validity of the project report, and the grader may accordingly apply deductions to the report grade.&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;The grading of the final project presentation will be based on whether the team presents the project clearly and carefully under the given time limit. &lt;strong&gt;Note that all team members need to present the final project in person; failing to do it will result in significant point deduction for the entire Checkpoint 2 (beyond the 2%).&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Submission Guideline&lt;/strong&gt;
TBD&lt;/p&gt;
</description>
    </item>
    
  </channel>
</rss>
