About
I conduct research in programming languages. For the past few years, I have primarily been focused on (1) program synthesis for "low-resource" programming languages--these are languages that are not Python, Java, etc. but are widely used in specialized fields, such as scientific computing--and (2) understanding how computer science education is impacted by large language models. Here are three representative papers on this work:
- MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation
- Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMs
- How Beginning Programmers and Code LLMs (Mis)read Each Other
I teach a variety of undergraduate and graduate courses. I have been the lead instructor for Northeastern's CS1 course several times, designed several new courses, and taught graduate seminars on topics as varied as Code LLM development and program verification.
I also serve as the Area Chair for Software at Northeastern's Khoury College of Computer Sciences.
Current Teaching
Engineering LLM-Integrated Systems
Prospective Students
- PhD: I encourage you to apply to Northeastern. Feel free to email me to determine if our interests align. You can also determine that by reading recent papers from my group.
- Undergraduates at Northeastern: see this document for more information.
- MS students at Northeastern: I am open to the Khoury Research Apprenticeship Program. Read the document above and adapt appropriately.
Recent News
- "I Would Have Written My Code Differently": Beginners Struggle to Understand LLM-Generated Code will appear at HumanAISE 2025 (April 2025)
- Preprint: PhD Knowledge Not Required: A Reasoning Challenge for Large Language Models (February 2025)
- Substance Beats Style: Why Beginning Students Fail to Code with LLMs will appear at NAACL 2025 (January 2025)
- NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals will appear at ICLR 2025 (January 2025)
- Fully Transparent Self-Alignment for Code Generation will appear at NeurIPS 2024 (October 2024)
- Instructor-Written Hints as Automated Test Suite Quality Feedback will appear at SIGCSE 2025 (October 2024)
- DoE Award on AI for High-Performance Computing in the Sciences, with David Bau, Harshita Menon, William Godoy, Abhinav Bhatale, and Tom Goldstein (September 2024)
- Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMs will appear at OOPSLA 2024 (August 2024)
- Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions will appear at the Conference on Language Modelling (July 2024)
- StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Code will appear in ACL Findings 2024 (May 2024)
- Launch of the National Deep Inference Fabric (NDIF), a computing resource that enables detailed and reproducible experiments on large pretrained AI models such as LLMs (May 2024)
Research Group
I am a member of the Programming Research Laboratory at Northeastern. I work with a great group of students:
- Aleks Boruch-Gruszecki (postdoc)
- Zixuan Wu (PhD)
- Francesca Lucchetti (PhD)
- John Gouwar (PhD)
- Yangtian Zi (PhD)
Alumni
- Abhinav Jangda (PhD → Microsoft Research)
- Ming-Ho Yee (PhD → Meta)
- James Perretta (PhD, co-advised with Jonathan Bell → University of Massachusetts Amherst)
- Donald Pinckney (PhD, co-advised with Jonathan Bell → Gitar)
- Luna Phipps-Costin (MS)
- Emily Herbert (MS)
- Samuel Baxter (MS)
- Rian Shambaugh (MS)
- Nimish Gupta (MS → Amazon)
- Claire Schlesinger (undergrad, NSF GRFP honorable mention)
- Federico Cassano (undergrad, CRA finalist → Cursor)
- Anders Freeman (Wellesley College undergrad → PhD student at University of Massachusetts Amherst)
- Sydney Nguyen (Wellesley College undergrad)
- Nicole Andrews (Mount Holyoke College undergrad)
- Jenna Hammond (Mount Holyoke College undergrad, CRA honorable mention)
- Sam Lee (undergrad)
- Christopher Rybicki (undergrad → MS student at Cornell Tech)
- Harsh Poddar (undergrad → Meta)
- Jane Tangen (undergrad → MITRE)
- Rachit Nigam (undergrad → PhD student at Cornell)
- Ariel Weiss (undergrad, NSF GRFP → PhD student at Northeastern University)
- Akshay Gadre (undergrad → PhD student at Carnegie Mellon)