Publications

View on Semantic Scholar / Google Scholar / ACL Anthology / DBLP

Showing (All/None): Conferences / Findings / Journals / Workshops / Technical reports / Theses / Preprints

2025

1. Why Do Some Language Models Fake Alignment While Others Don’t?
   Abhay Sheshadri, John Hughes, Julian Michael, Alex Mallen, Arun Jose, Janus and Fabien Roger
   preprint
   pdf arxiv bib
2. FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
   Christina Q. Knight, Kaustubh Deshpande, Ved Sirdeshmukh, Meher Mankikar, Scale Red Team, SEAL Research Team and Julian Michael
   preprint
   s2 pdf arxiv bib
3. AI Debate Aids Assessment of Controversial Claims
   Salman Rahman, Sheriff Issaka, Ashima Suvarna, Genglin Liu, James Shiffer, Jaeyoung Lee, Md Rizwan Parvez, Hamid Palangi, Shi Feng, Nanyun Peng, Yejin Choi, Julian Michael, Liwei Jiang and Saadia Gabriel
   MAS 2025
   s2 pdf arxiv bib
4. A Red Teaming Roadmap Towards System-Level Safety
   Zifan Wang, Christina Q. Knight, Jeremy Kritz, Willow E. Primack and Julian Michael
   preprint
   s2 pdf arxiv bib
5. International AI Safety Report
   Yoshua Bengio, Sören Mindermann, Daniel Privitera, Tamay Besiroglu, Rishi Bommasani, Stephen Casper, Yejin Choi, Philip Fox, Ben Garfinkel, Danielle Goldfarb, Hoda Heidari, Anson Ho, Sayash Kapoor, Leila Khalatbari, Shayne Longpre, Sam Manning, Vasilios Mavroudis, Mantas Mazeika, Julian Michael, Jessica Newman, Kwan Yee Ng, Chinasa T. Okolo, Deborah Raji, Girish Sastry, Elizabeth Seger, Theodora Skeadas, Tobin South, Emma Strubell, Florian Tramèr, Lucia Velasco, Nicole Wheeler, Daron Acemoglu, Olubayo Adekanmbi, David Dalrymple, Thomas G. Dietterich, Edward W. Felten, Pascale Fung, Pierre-Olivier Gourinchas, Fredrik Heintz, Geoffrey Hinton, Nick Jennings, Andreas Krause, Susan Leavy, Percy Liang, Teresa Ludermir, Vidushi Marda, Helen Margetts, John McDermid, Jane Munga, Arvind Narayanan, Alondra Nelson, Clara Neppel, Alice Oh, Gopal Ramchurn, Stuart Russell, Marietje Schaake, Bernhard Schölkopf, Dawn Song, Alvaro Soto, Lee Tiedrich, Gaël Varoquaux, Andrew Yao, Ya-Qin Zhang, Olubunmi Ajala, Fahad Albalawi, Marwan Alserkal, Guillaume Avrin, Christian Busch, André Carlos Ponce de Leon Ferreira de Carvalho, Bronwyn Fox, Amandeep Singh Gill, Ahmet Halit Hatip, Juha Heikkilä, Chris Johnson, Gill Jolly, Ziv Katzir, Saif M. Khan, Hiroaki Kitano, Antonio Krüger, Kyoung Mu Lee, Dominic Vincent Ligot, José Ramón López Portillo, Oleksii Molchanovskyi, Andrea Monti, Nusu Mwamanzi, Mona Nemer, Nuria Oliver, Raquel Pezoa Rivera, Balaraman Ravindran, Hammam Riza, Crystal Rugege, Ciarán Seoighe, Jerry Sheehan, Haroon Sheikh, Denise Wong and Yi Zeng
   UK DSIT/AISI Report
   website s2 pdf arxiv bib

2024

1. Alignment faking in large language models
   Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, Sören Mindermann, Ethan Perez, Linda Petrini, Jonathan Uesato, Jared Kaplan, Buck Shlegeris, Samuel R. Bowman and Evan Hubinger
   preprint
   website s2 pdf arxiv code video memo bib
2. Rapid Response: Mitigating LLM Jailbreaks with a Few Examples
   Alwin Peng, Julian Michael, Henry Sleight, Ethan Perez and Mrinank Sharma
   preprint
   s2 pdf arxiv code openreview bib
3. Training Language Models to Win Debates with Self-Play Improves Judge Accuracy
   Samuel Arnesen, David Rein and Julian Michael
   preprint
   s2 pdf arxiv code bib
   Media: Quanta Magazine
4. GPQA: A Graduate-Level Google-Proof Q&A Benchmark
   David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael† and Samuel R. Bowman†
   COLM 2024 (Spotlight)
   pdf arxiv data talk twitter reviews bib
   Media: AI Index (2024; Ch. 2), Nature
5. Research Agenda for Sociotechnical Approaches to AI Safety
   Samuel Curtis, Ravi Iyer, Cameron Domenico Kirk-Giannini, Victoria Krakovna, David Krueger, Nathan Lambert, Bruno Marnette, Colleen McKenzie, Julian Michael, Evan Miyazono, Noyuri Mima, Aviv Ovadya, Luke Thorburn and Deger Turan
   preprint
   pdf bib
6. Analyzing the Role of Semantic Representations in the Era of Large Language Models
   Zhijing Jin,* Yuen Chen,* Fernando Gonzalez,* Jiarui Liu, Jiayi Zhang, Julian Michael, Bernhard Schölkopf and Mona Diab
   NAACL 2024
   pdf arxiv code bib
7. Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought
   James Chua,* Edward Rees,* Hunar Batra, Samuel R. Bowman, Julian Michael, Ethan Perez and Miles Turpin
   arXiv preprint
   pdf arxiv code twitter bib
   Media: NYU CDS Blog

2023

1. Debate Helps Supervise Unreliable Experts
   Julian Michael,* Salsabila Mahdi,* David Rein,* Jackson Petty, Julien Dirani, Vishakh Padmakumar and Samuel R. Bowman
   arXiv preprint
   website pdf arxiv code data twitter bib
2. Eliciting Language Model Behaviors using Reverse Language Models
   Jacob Pfau,* Alex Infanger,* Abhay Sheshadri,* Ayush Panda, Curtis Huebner and Julian Michael
   SoLaR 2023 (Spotlight)
   reviews pdf bib
3. Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
   Miles Turpin, Julian Michael, Ethan Perez and Samuel R. Bowman
   NeurIPS 2023
   s2 pdf arxiv code lw twitter bib
   Media: Nature, Neue Zürcher Zeitung
4. The Case for Scalable, Data-Driven Theory: A Paradigm for Scientific Progress in NLP
   Julian Michael
   The Big Picture Workshop (Best Paper)
   pdf arxiv slides twitter bib
5. We’re Afraid Language Models Aren’t Modeling Ambiguity
   Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah A. Smith and Yejin Choi
   EMNLP 2023
   s2 pdf arxiv data code twitter bib
6. Building Blocks for Data-Driven Theories of Language Understanding
   Julian Michael
   PhD Thesis, University of Washington, 2023
   pdf bib
7. What Do NLP Researchers Believe? Results of the NLP Community Metasurvey
   Julian Michael, Ari Holtzman, Alicia Parrish, Aaron Mueller, Alex Wang, Angelica Chen, Divyam Madaan, Nikita Nangia, Richard Yuanzhe Pang, Jason Phang and Samuel R. Bowman
   ACL 2023
   website s2 pdf arxiv acl poster talk twitter bib
   Media: AI Index (2022; Ch. 8), Data Skeptic Podcast, IFLScience, Tages-Anzeiger, New Scientist, The Times, Yahoo, NLP Deep Dive, NYU CDS Blog

2022

1. Overconfidence in the Face of Ambiguity with Adversarial Data
   Margaret Li* and Julian Michael*
   DADC 2022 (Best Paper)
   s2 pdf code reviews bib
2. Nearest Neighbor Zero-Shot Inference
   Weijia Shi, Julian Michael, Suchin Gururangan and Luke Zettlemoyer
   EMNLP 2022
   s2 pdf arxiv twitter bib

2021

1. Asking It All: Generating Contextualized Questions for any Semantic Role
   Valentina Pyatkin, Paul Roit, Julian Michael, Yoav Goldberg, Reut Tsarfaty and Ido Dagan
   EMNLP 2021
   s2 pdf arxiv code twitter bib
2. Inducing Semantic Roles Without Syntax
   Julian Michael and Luke Zettlemoyer
   Findings of ACL 2021
   website s2 pdf code bib
3. Prompting Contrastive Explanations for Commonsense Reasoning Tasks
   Bhargavi Paranjape, Julian Michael, Marjan Ghazvininejad, Hannaneh Hajishirzi and Luke Zettlemoyer
   Findings of ACL 2021
   s2 pdf arxiv code bib

2020

1. Asking without Telling: Exploring Latent Ontologies in Contextual Representations
   Julian Michael, Jan A. Botha and Ian Tenney
   EMNLP 2020
   s2 pdf arxiv twitter bib
2. Controlled Crowdsourcing for High-Quality QA-SRL Annotation
   Paul Roit, Ayal Klein, Daniela Stepanov, Jonathan Mamou, Julian Michael, Gabriel Stanovsky, Luke Zettlemoyer and Ido Dagan
   ACL 2020
   s2 pdf arxiv code data bib
3. AmbigQA: Answering Ambiguous Open-domain Questions
   Sewon Min, Julian Michael, Hannaneh Hajishirzi and Luke Zettlemoyer
   EMNLP 2020
   website s2 pdf arxiv code data twitter bib

2019

1. SuperGLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
   Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy and Samuel R. Bowman
   NeurIPS 2019
   website s2 pdf arxiv code bib
2. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
   Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy and Samuel R. Bowman
   ICLR 2019
   website s2 reviews pdf arxiv code bib

2018

1. Large-Scale QA-SRL Parsing
   Nicholas FitzGerald, Julian Michael, Luheng He and Luke Zettlemoyer
   ACL 2018 (Honorable Mention)
   website s2 pdf arxiv code data twitter bib
2. Crowdsourcing Question-Answer Meaning Representations
   Julian Michael, Gabriel Stanovsky, Luheng He, Ido Dagan and Luke Zettlemoyer
   NAACL 2018
   s2 pdf arxiv poster code data bib
3. Supervised Open Information Extraction
   Gabriel Stanovsky, Julian Michael, Luke Zettlemoyer and Ido Dagan
   NAACL 2018
   s2 pdf poster code bib

2016

1. Human-in-the-Loop Parsing
   Luheng He, Julian Michael, Mike Lewis and Luke Zettlemoyer
   EMNLP 2016
   s2 pdf code slides bib
2. Proving Infinitary Formulas
   Amelia Harrison, Vladimir Lifschitz and Julian Michael
   TPLP Vol. 16, 5–6; Presented at ICLP 2016
   s2 pdf bib

2015

1. The Theory of Correlation Formulas and Their Application to Discourse Coherence
   Julian Michael
   Undergraduate Honors Thesis, UT Austin, 2015
   s2 pdf bib
2. The Winograd Schema Challenge and Reasoning about Correlation
   Daniel Bailey, Amelia Harrison, Yuliya Lierler, Vladimir Lifschitz and Julian Michael
   Commonsense 2015
   s2 pdf bib