Ian Tenney

I am a software engineer on the Language team at Google Research. My research focuses on interpretability and analysis of deep NLP models (a.k.a. “BERTology”). I’m interested in how these models encode linguistic structure, how these structures develop, and what they can tell us about model behavior and robustness. I’m also interested in the practical workflow of interpretability: how does the way we interact with these models inform our own mental models of how they work?

I am one of the tech leads for the Language Interpretability Tool (LIT).

I am based in Seattle, but collaborate with researchers across many sites, including Google’s People + AI Research and the LUNAR group at Brown.

In 2018, I was a Senior Researcher on the Sentence Representation Learning Team as part of the 2018 JSALT workshop at Johns Hopkins University. Among other things, I took this terrible group photo.

From 2016 to 2018, I taught Data Science W266: Natural Language Processing with Deep Learning at UC Berkeley School of Information.

In a past life, I was a physicist, studying ultrafast molecular and optical physics in the lab of Philip H. Bucksbaum at Stanford / SLAC.

Contact: "if" + lastname + "@gmail.com"

projects

The Language Interpretability Tool (LIT)

An open-source platform for visualization and understanding of NLP models.

Probing and BERTology

Understanding linguistic structure in pre-trained language models, such as ELMo and BERT.

selected publications

EMNLP

The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models

Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, and Ann Yuan

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2020

Abstract arXiv EMNLP 2020 Blog Code Poster Website

We present the Language Interpretability Tool (LIT), an open-source platform for visualization and understanding of NLP models. We focus on core questions about model behavior: Why did my model make this prediction? When does it perform poorly? What happens under a controlled change in the input? LIT integrates local explanations, aggregate analysis, and counterfactual generation into a streamlined, browser-based interface to enable rapid exploration and error analysis. We include case studies for a diverse set of workflows, including exploring counterfactuals for sentiment analysis, measuring gender bias in coreference systems, and exploring local behavior in text generation. LIT supports a wide range of models—including classification, seq2seq, and structured prediction—and is highly extensible through a declarative, framework-agnostic API. LIT is under active development, with code and full documentation available at https://github.com/pair-code/lit.
ACL

BERT Rediscovers the Classical NLP Pipeline

Ian Tenney, Dipanjan Das, and Ellie Pavlick

In Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019

Abstract arXiv ACL 2019 Code Poster

Pre-trained text encoders have rapidly advanced the state of the art on many NLP tasks. We focus on one such model, BERT, and aim to quantify where linguistic information is captured within the network. We find that the model represents the steps of the traditional NLP pipeline in an interpretable and localizable way, and that the regions responsible for each step appear in the expected sequence: POS tagging, parsing, NER, semantic roles, then coreference. Qualitative analysis reveals that the model can and often does adjust this pipeline dynamically, revising lower-level decisions on the basis of disambiguating information from higher-level representations.
ICLR

What do you learn from context? Probing for sentence structure in contextualized word representations

Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick

In International Conference on Learning Representations, 2019

Abstract arXiv ICLR 2019 Code Poster

Contextualized representation models such as ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2018) have recently achieved state-of-the-art results on a diverse array of downstream NLP tasks. Building on recent token-level probing work, we introduce a novel edge probing task design and construct a broad suite of sub-sentence tasks derived from the traditional structured NLP pipeline. We probe word-level contextual representations from four recent models and investigate how they encode sentence structure across a range of syntactic, semantic, local, and long-range phenomena. We find that existing models trained on language modeling and translation produce strong representations for syntactic phenomena, but only offer comparably small improvements on semantic tasks over a non-contextual baseline.