UNIVERSITY OF PENNSYLVANIA

human
computer
interaction

Welcome to Penn HCI! Coming to Penn in 2022, our group will study a range of topics in Human-Computer Interaction with the goal of understanding, designing, engineering, and improving technologies to make a positive impact on individuals and communities.

scroll

people

Penn HCI is founded and led by incoming Assistant Professors Andrew Head and Danaë Metaxa. We are currently recruiting postdocs, graduate students, and undergraduate researchers to join us next year at Penn.

Andrew Head

Andrew Head

Asst Professor, University of Pennsylvania CIS

Danaë Metaxa

Danaë Metaxa

Asst Professor, University of Pennsylvania CIS

Tal August

Tal August

Ph.D. Student, University of Washington

Keitany Alison Jelagat

Keitany Alison Jelagat

Sophomore, Stanford University

Ayush Pandit

Ayush Pandit

Senior, Stanford University

Raymond Fok

Raymond Fok

Ph.D. Student, University of Washington

Daniel S Weld

Daniel S Weld

WRT/TJ Cable Professor, University of Washington

Michelle Lam

Michelle Lam

Ph.D. Student, Stanford University

Marti Hearst

Marti Hearst

Professor, UC Berkeley

Colin Kalicki

Colin Kalicki

Senior, Stanford University

meet everyone

publications

View recent publications and filter by topic, author, year, and more.

An Image of Society: Gender and Racial Representation and Impact in Image Search Results for Occupations

Danaë Metaxa, Michelle A. Gan, Su Goh, Jeff Hancock, and James A. Landay
CSCW 2021

Algorithmically-mediated content is both a product and producer of dominant social narratives, and it has the potential to impact users’ beliefs and behaviors. We present two studies on the content and impact of gender and racial representation in image search results for common occupations. In Study 1, we compare 2020 workforce gender and racial composition to that reflected in image search. We find evidence of underrepresentation on both dimensions: women are underrepresented in search at a rate of 42% women for a field with 50% women; people of color are underrepresented with 16% in search compared to an occupation with 22% people of color (the latter being proportional to the U.S. workforce). We also compare our gender representation data with that collected in 2015 by Kay et al., finding little improvement in the last half-decade. In Study 2, we study people’s impressions of occupations and sense of belonging in a given field when shown search results with different proportions of women and people of color. We find that both axes of representation as well as people’s own racial and gender identities impact their experience of image search results. We conclude by emphasizing the need for designers and auditors of algorithms to consider the disparate impacts of algorithmic content on users of marginalized identities.


Document-Level Definition Detection in Scholarly Documents: Existing Models, Error Analyses, and Future Directions

Dongyeop Kang, Andrew Head, Risham Sidhu, Kyle Lo, Daniel S. Weld, and Marti A. Hearst
SDP 2021

The task of definition detection is important for scholarly papers, because papers often make use of technical terminology that may be unfamiliar to readers. Despite prior work on definition detection, current approaches are far from being accurate enough to use in real- world applications. In this paper, we first perform in-depth er- ror analysis of the current best performing definition detection system and discover major causes of errors. Based on this analysis, we develop a new definition detection system, HEDDEx, that utilizes syntactic features, transformer encoders, and heuristic filters, and evaluate it on a standard sentence-level benchmark. Because current benchmarks evaluate randomly sampled sentences, we propose an alternative evaluation that assesses every sentence within a document. This allows for evaluating recall in addition to precision. HEDDEx outperforms the leading system on both the sentence-level and the document-level tasks, by 12.7 F1 points and 14.4 F1 points, respectively. We note that performance on the high-recall document-level task is much lower than in the standard evaluation approach, due to the necessity of incorporation of document structure as features. We discuss remaining challenges in document-level definition detection, ideas for improvements, and potential is- sues for the development of reading aid applications.


Augmenting Scientific Papers with Just-in-Time, Position-Sensitive Definitions of Terms and Symbols

Augmenting Scientific Papers with Just-in-Time, Position-Sensitive Definitions of Terms and Symbols

Andrew Head, Kyle Lo, Dongyeop Kang, Raymond Fok, Sam Skjonsberg, Daniel S. Weld, and Marti A. Hearst
CHI 2021

Despite the central importance of research papers to scientific progress, they can be difficult to read. Comprehension is often stymied when the information needed to understand a passage resides somewhere else—in another section, or in another paper. In this work, we envision how interfaces can bring definitions of technical terms and symbols to readers when and where they need them most. We introduce ScholarPhi, an augmented reading interface with four novel features: (1) tooltips that surface position-sensitive definitions from elsewhere in a paper, (2) a filter over the paper that “declutters” it to reveal how the term or symbol is used across the paper, (3) automatic equation diagrams that expose multiple definitions in parallel, and (4) an automatically generated glossary of important terms and symbols. A usability study showed that the tool helps researchers of all experience levels read papers. Furthermore, researchers were eager to have ScholarPhi’s definitions available to support their everyday reading.


all publications

courses

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud.

CS 2XX

Designing Interactive Programming Tools

This hands-on course explores a selection of techniques from Programming Languages and Human-Computer Interaction that can help us create useful, usable programming languages and programming tools. We will cover strategies for designing programming systems—e.g., need finding, formative studies, user-centered design broadly. We will also cover tools and techniques that help us build user-friendly programming systems—e.g., program synthesis, structure editors, abstraction design, program slicing. For the final project, individuals or teams will develop a usable abstraction, language, or programming tool of their own design. The course will include a mix of formats: lecture; seminar-style discussion; small design projects and programming projects for building familiarity with key techniques; and a final project, which can be small- or medium-scale.

DETAILS:

Spring / Summer
Mondays
Andrew Head


CS 2XX

Designing Media for Thinking the Unthinkable

As pedagogy and communication increasingly take place online, authors have the ability to write about complex ideas in media that provide new visual and interactive affordances. This course reviews the history of interactive media, from the SuperBook to DynamicLand. Students conduct formative studies of how ideas can be presented in cutting-edge formats like executable books, video explainers, AR tutorials, and explorable scientific publications. Students build and evaluate interactive prototypes that envision the future of media for communicating about complex ideas.

DETAILS:

Fall / Winter
Mondays
Danaë Metaxa


CS 2XX

Research in Human-Computer Interaction

This course is a broad introduction to conducting research in Human-Computer Interaction. Students will become familiar with seminal and recent literature; learn to review and critique research papers; re-implement and evaluate important existing systems; and gain experience in conducting research. Topics include input devices, computer-supported cooperative work, crowdsourcing, design tools, evaluation methods, search and mobile interfaces, usable security, help and tutorial systems.

DETAILS:

Fall / Winter
Mondays
Andrew Head


join us

We are actively recruiting students at all levels to join our group, beginning in 2022. Join us!

connect