publications

2024

  1. Scalable Influence and Fact Tracing for Large Language Model Pretraining
    Tyler A. Chang, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon, and Ian Tenney
    arXiv preprint, 2024
  2. Interactive Prompt Debugging with Sequence Salience
    Ian Tenney, Ryan Mullins, Bin Du, Shree Pandya, Minsuk Kahng, and Lucas Dixon
    arXiv preprint, 2024
  3. Tech Report
    Gemma: Open Models Based on Gemini Research and Technology
    Gemma Team
    Google DeepMind, 2024
  4. Can Generative Multimodal Models Count to Ten?
    Sunayana Rane, Alexander Ku, Jason Michael Baldridge, Ian Tenney, Thomas L. Griffiths, and Been Kim
    In Proceedings of the Annual Meeting of the Cognitive Science Society, 2024
  5. LLM Comparator: Visual Analytics for Side-by-Side Evaluation of Large Language Models
    Minsuk Kahng, Ian Tenney, Mahima Pushkarna, Michael Xieyang Liu, James Wexler, Emily Reif, Krystal Kallarackal, Minsuk Chang, Michael Terry, and Lucas Dixon
    IEEE Transactions on Visualization and Computer Graphics, 2024

2023

  1. Tech Report
    Gemini: A Family of Highly Capable Multimodal Models
    Gemini Team
    Google DeepMind, 2023
  2. DIS
    From Discovery to Adoption: Understanding the ML Practitioners’ Interpretability Journey
    Narges Ashtari, Ryan Mullins, Crystal Qian, James Wexler, Ian Tenney, and Mahima Pushkarna
    In Proceedings of the 2023 ACM Designing Interactive Systems Conference, 2023
  3. Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs
    Kelvin Guu, Albert Webson, Ellie Pavlick, Lucas Dixon, Ian Tenney, and Tolga Bolukbasi
    arXiv preprint, 2023

2022

  1. Findings of EMNLP
    Tracing Knowledge in Language Models Back to the Training Data
    Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, and Kelvin Guu
    In Findings of the Association for Computational Linguistics: EMNLP, 2022
  2. ACL
    Retrieval-guided Counterfactual Generation for QA
    Bhargavi Paranjape, Matthew Lamm, and Ian Tenney
    In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 2022
  3. The MultiBERTs: BERT Reproductions for Robustness Analysis
    Thibault Sellam, Steve Yadlowsky, Ian Tenney, Jason Wei, Naomi Saphra, Alexander D’Amour, Tal Linzen, Jasmijn Bastings, Iulia Turc, Jacob Eisenstein, Dipanjan Das, and Ellie Pavlick
    ICLR (spotlight), 2022

2020

  1. Measuring and reducing gendered correlations in pre-trained models
    Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, and Slav Petrov
    arXiv preprint, 2020
  2. The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models
    Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, and Ann Yuan
    In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2020
  3. Asking without Telling: Exploring Latent Ontologies in Contextual Representations
    Julian Michael, Jan A. Botha, and Ian Tenney
    In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020
  4. Do Language Embeddings capture Scales?
    Xikun Zhang, Deepak Ramachandran, Ian Tenney, Yanai Elazar, and Dan Roth
    In Findings of the Association for Computational Linguistics: EMNLP, 2020
  5. What Happens To BERT Embeddings During Fine-tuning?
    Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick, and Ian Tenney
    In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, 2020
  6. ACL
    jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models
    Yada Pruksachatkun, Phil Yeres, Haokun Liu, Jason Phang, Phu Mon Htut, Alex Wang, Ian Tenney, and Samuel R. Bowman
    In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2020

2019

  1. ACL
    BERT Rediscovers the Classical NLP Pipeline
    Ian Tenney, Dipanjan Das, and Ellie Pavlick
    In Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019
  2. ACL
    Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling
    Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, and others
    In Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019
  3. Probing What Different NLP Tasks Teach Machines about Function Word Comprehension
    Najoung Kim, Roma Patel, Adam Poliak, Patrick Xia, Alex Wang, Tom McCoy, Ian Tenney, Alexis Ross, Tal Linzen, Benjamin Van Durme, and others
    In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM), 2019
  4. What do you learn from context? Probing for sentence structure in contextualized word representations
    Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick
    In International Conference on Learning Representations, 2019

2018

  1. WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discourse
    Manaal Faruqui, Ellie Pavlick, Ian Tenney, and Dipanjan Das
    In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018

Older

  1. Collisional decoherence and rotational quasirevivals in asymmetric-top molecules
    Ian F Tenney, Maxim Artamonov, Tamar Seideman, and Philip H Bucksbaum
    Physical Review A, 2016
  2. Ultrafast isomerization initiated by X-ray core ionization
    CE Liekhus-Schmaltz, I Tenney, T Osipov, A Sanchez-Gonzalez, N Berrah, R Boll, C Bomme, C Bostedt, JD Bozek, S Carron, R Coffee, J Devin, B Erk, KR Ferguson, RW Field, L Foucar, LJ Frasinski, JM Glownia, M Gühr, A Kamalov, J Krzywinski, H Li, JP Marangos, TJ Martinez, BK McFarland, S Miyabe, B Murphy, A Natan, D Rolles, A Rudenko, M Siano, ER Simpson, L Spector, M Swiggers, D Walke, S Wang, T Weber, PH Bucksbaum, and VSS Petrovic
    Nature communications, 2015
  3. Ultrafast X-ray Auger probing of photoexcited molecular dynamics
    BK McFarland, JP Farrell, S Miyabe, F Tarantelli, A Aguilar, N Berrah, C Bostedt, JD Bozek, PH Bucksbaum, JC Castagna, RN Coffee, JP Cryan, L Fang, R Feifel, KJ Gaffney, JM Glownia, TJ Martinez, M Mucke, B Murphy, A Natan, T Osipov, VS Petrovic, S Schorb, Th Schultz, LS Spector, M Swiggers, I Tenney, S Wang, JL White, W White, and M Gühr
    Nature communications, 2014
  4. Semicrystalline dihydroxyacetone copolymers derived from glycerol
    Jeff Simon, Johan V Olsson, Hyunuk Kim, Ian F Tenney, and Robert M Waymouth
    Macromolecules, 2012