Dongwook Yoon

In robot-mediated telepresence, the teleoperator's physical embodiment enhances social interaction with on-site users. Current user representation techniques may result in disparities between the robotic embodiment and the teleoperator's representation, resulting in perceptual dissonance in on-site users which can degrade the interaction quality. To address this dissonance felt by on-site users, we introduce AvatARoid, a novel system that emphasizes the teleoperator's embodiment by superimposing a motion-mapped AR overlay of the teleoperator’s avatar on a humanoid robot. To assess the conceptual validity of AvatARoid, we implemented our system with a virtual robot in a realistic AR simulation carefully designed to maximize the sense of co-presence with the virtual robot. Since our evaluation focused on the different visual representations of the teleoperator, we designed the appearance and behavior of the virtual robot in our simulation to appear visually plausible and realistic within the experimental environment when augmented with different representations of the teleoperator. We conducted a mixed-method between-participants study (n=48) where participants engaged in a "get-to-know-you'' conversation followed by a word guessing game with a confederate teleoperator, represented as a virtual humanoid robot with (a) only the voice of the teleoperator (b) voice and video, or (c) AvatARoid. Our findings suggest that the AvatARoid system significantly improved the on-site user's perceived embodiment of the teleoperator compared to other conditions, particularly enhancing the sense that the teleoperator was situated in the same location as the robot and adding confidence that the teleoperator was in control of the robot. Additionally, AvatARoid allowed for more expressive non-verbal gestures, leading to engaging interactions. In contrast, the video and baseline conditions often resulted in on-site users feeling a pronounced disconnect between the teleoperator and the robot. Our study offers new insights into conceptualizing and evaluating novel user representation techniques to promote social interaction in robot-mediated telepresence systems.

View record

Adopting the perspective of another person can enhance one's understanding of that individual, which can be especially valuable in the context of intergenerational communication. However, it can be challenging to achieve this shift in perspective. We explored leveraging the Proteus Effect -- behaviour and perspective modulation in Embodied VR -- with the aim of improving younger adults’ understanding and empathy of their older family members’ life experiences. We asked younger adults to adopt the identity of their older family members at different ages through avatar swapping under several conditions. We collected rich qualitative data from 10 participants (5 dyads with a one-generation gap), and analyzed the data using reflexive thematic analysis. Our work suggests: 1) important design considerations for a VR setup that can facilitate intergenerational communication and lead to greater empathy; 2) the manner in which this VR setup affects both older adults and younger adults individually within the same familial context; and 3) the extent to which this VR setup shapes their mutual understanding and their relationship dynamics. Our work contributes a novel approach to bridging intergenerational communication.

View record

Modern large-scale language models approach the quality of human-level writing. This promises the advent of Artificial Intelligence (AI) writing companions performing AI-led writing under human control, surpassing traditional writing tools limited to revision and ideation supports. However, human-AI co-writing may endanger writers’ control, autonomy, and ownership by overstepping co-creative boundaries. My research team and I first conducted a design workbook study with 7 hobbyists and 13 professional writers and elicited three sets of primary barriers to the adoption of human-AI co-writing. Storywriters desire retaining control over writing rather than letting AI take the lead when they (1) prioritize emotional values in turning ideas into words over the productivity of AI-generated writing; (2) have high self-confidence and distrust AI in challenging sub-tasks (e.g., creating characters and dialogue); and (3) expect the AI control mechanism to mismatch their writing strategies. To expand on the results of the design workbook study, I led the design of two AI-assisted writing tools which support UX scenario writing. I subsequently conducted a summative evaluation of these systems. Participants prioritized productivity and reacted positively to both systems, perceiving them as Creativity Support Tools (CST)’s for achieving prolific writing. The results reveal how the tool design interacted with designers’ value structure: both systems allowed them to maintain their integrity and enhance their enjoyment. Additionally, increased control promoted a greater sense of ownership and satisfaction, while adding additional overhead for the designer. This work lays the groundwork for AI companions that respect storywriters’ personal values and writing methods.

View record

Pull Requests (PRs) are a frequently used method for proposing changes to source code repositories. When discussing proposed changes in a PR discussion, stakeholders often reference a wide variety of information objects for establishing shared awareness and common ground. Previous work has not considered how the referential behavior impacts collaborative software development via PRs. This knowledge gap is the major barrier in evaluating the current support for referencing in PRs and improving them. We conducted an explorative analysis of ~7K references, collected from 450 public PRs on GitHub, and constructed taxonomies of referent types and expressions. Using our annotated dataset, we identified several patterns in the use of references. We found that despite a prevalent use of references in PR discussions, GitHub's interface lacks the support for referencing the majority of information types. We provide qualitative descriptions of how different contextual factors shape the use of references in discussions. We also discovered distinct referencing patterns in merged and closed PRs which signifies a potential ground for future research to establish a relationship between reference use and PR outcomes. These findings suggest that what is and is not referenced within a PR discussion has an important impact on the software development process, and warrants continued platform support and research. We conclude with design implications to support more effective referencing in PR discussion interfaces.

View record

In human-computer interaction (HCI) studies, bias in the gender representation of participants can jeopardize the generalizability of findings, perpetuate bias in data driven practices, and make new technologies dangerous for underrepresented groups. Key to progress towards inclusive and equitable gender practices is diagnosing the current status of bias and identifying where it comes from. In this mixed-methods study, we interviewed 13 HCI researchers to identify the potential bias factors, defined a systematic data collection procedure for meta-analysis of participant gender data, and created a participant gender dataset from 1147 CHI papers. Our analysis provided empirical evidence for the underrepresentation of women, the invisibility of non-binary participants, deteriorating representation of women in MTurk studies, and characteristics of research topics prone to bias. Based on these findings, we make concrete suggestions for promoting inclusive community culture and equitable research practices in HCI.

View record

Throughout the COVID-19 pandemic, older adults have been encouraged to stay indoors and isolated, leading to potential disruptions in their social activities and interpersonal relationships. We conducted an interview study (N=24) to examine older adults' technology adoption and communication practices in light of new circumstances related to the pandemic. Our interviews revealed that the pandemic motivated many older adults to learn new technology and become more tech-savvy in an effort to stay connected with others. However, they also reported challenges related to the pandemic that were major impediments to technology adoption. These were: (1) lack of access to in-person technology support under physical distancing mandates, (2) lack of opportunities for online participation due to negative age stereotypes and assumptions, and (3) increased apprehension to seek help from family members and friends who were suffering from pandemic-related stresses. This study extends technology adoption literature and contributes an up-to-date examination of the "grey digital divide" (the gap between older adults who use technology and those who do not). Our findings demonstrate that despite the rapidly increasing number of tech-savvy seniors, a digital divide not only persists, but has been exacerbated by the transition to virtual-only offerings. We reveal the challenges and coping strategies of older adults who remain separated from technology, and propose actionable solutions to increase digital access during the COVID-19 pandemic and beyond.

View record

With substantial industrial interests, conversational voice user interfaces (VUIs) are becoming ubiquitous through devices that feature voice assistants such as Apple’s Siri and Amazon Alexa. Naturalness is often considered to be central to conversational VUI designs as it is associated with numerous benefits such as reducing cognitive load and increasing accessibility. The literature offers several definitions for naturalness, and existing conversational VUI design guidelines provide different suggestions for delivering a natural experience to users. However, these suggestions are hardly comprehensive and often fragmented. A precise characterization of naturalness is necessary for identifying VUI designers’ needs and supporting their design practices. To this end, we interviewed 20 VUI designers, asking what naturalness means to them, how they incorporate the concept in their design practice, and what challenges they face in doing so. Through inductive and deductive thematic analysis, we identify 12 characteristics describing naturalness in VUIs and classify these characteristics into three groups, which are ‘Fundamental’, ‘Transactional’ and ‘Social’ depending on the purpose each characteristic serves. Then we describe how designers pursue these characteristics under different categories in their practices depending on the contexts of their VUIs (e.g., target users, application purpose). We identify 10 challenges that designers are currently encountering in designing natural VUIs. Our designers reported experiencing the most challenges when creating naturally sounding dialogues, and they required better tools and guidelines. We conclude with implications for developing better tools and guidelines for designing natural VUIs.

View record

Speech shadowing, i.e., listening to some audio and simultaneously vocalizing the words, is a popular language-learning technique that is known to improve listening skills. However, despite strong evidence for its efficacy as a listening exercise, existing software tools do not adequately support listening-focused shadowing practice, especially in self-regulated learning environments with no external feedback. To bridge this gap, we introduce CAST, a shadowing system that makes self-regulation easy and effective through four novel interface features — contextual blurring for inducing self-reflection on misheard portions, in-situ annotations for self-monitoring progress through tracking and visualization, embedded recordings for post-practice self-evaluation, and adjustable pause-handles for self-paced practice. We base CAST on a formative user study (N=15) that provides fresh empirical grounds on the needs and challenges of those who practice shadowing using conventional software tools. We validate our design through a summative evaluation (N=12) that shows learners can successfully self-regulate their shadowing practice with CAST while retaining focus on listening.

View record