Bingsheng Arthur Yao
Associate Research Scientist, Khoury College of Computer Sciences, Northeastern University
My research investigates how people collaborate, and what it takes for AI agents to become effective partners in human work. By studying collaborative processes and designing human–AI systems, I envision a future of collaborative symbiosis, where humans and AI agents complement one another to expand what we can accomplish together.
Previously a postdoctoral researcher with Prof. Dakuo Wang. PhD from Rensselaer Polytechnic Institute, advised by Prof. Jim Hendler.
News
- Five papers (two first-authored) and one workshop were accepted to CHI 2026!
- Two papers (one first-authored) were accepted to CSCW 2026!
- I received a Microsoft Agentic AI Research and Innovation (AARI) award to support my research on human-agent collaboration.
- A Co-first authored survey paper of LLM Role-Playing Agent Evaluation is now publicly available.
Playground
PhD Simulator Five years, give or take. A game about doing a PhD in NLP and HCI, built from real stories. Play All of Playground Research
Domain
Topic
67 papers, newest first
- CSCW 2026 RECOVER: Designing a Large Language Model-based Remote Patient Monitoring System for Postoperative Gastrointestinal Cancer Care
- CSCW 2026 Bridging Knowledge Gaps in Clinical AI: An Activity Theory Perspective on Interdisciplinary Data Work for Telehealth
- CHI 2026 Balancing Efficiency and Empathy: Healthcare Providers' Perspectives on AI-Supported Workflows for Serious Illness Conversations in the Emergency Department
- CHI 2026 Exploring Collaboration Breakdowns Between Provider Teams and Patients in Post-Surgery Care
- CHI 2026 Through the Lens of Human-Human Collaboration: A Configurable Research Platform for Exploring Human-Agent Collaboration
- CHI 2026 Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight
- CHI 2026 LLM-based Embodied Conversational Agent for Reducing Foreign Language Speaking Anxiety in Social VR
- CHI EA 2026 From Human-Human Collaboration to Human-Agent Collaboration: A Vision, Design Philosophy, and an Empirical Framework for Achieving Successful Partnerships Between Humans and LLM Agents
- CHI EA 2026 Agent A/B: Automated and Scalable A/B Testing on Live Websites with Interactive LLM Agents
- ACL 2026 Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluation
- ACL 2026 Can LLM Agents Simulate Multi-Turn Human Behavior? Evidence from Real Online Customer Behavior Data
- ACL 2026 LLMs for Now, Fine-Tuning for Later: An Ensemble Approach to Data Drift in Domain-Specific Tasks
- TOSEM 2026 “Should I Give Up Now?” Investigating LLM Pitfalls in Software Engineering
- TAISAP 2026 The Obvious Invisible Threat: Vulnerabilities of LLM-Powered GUI Agents to Adversarial UI Manipulations in Web Interaction Tasks
- ACM HEALTH 2026 WatchGuardian: Enabling User-Defined Personalized Just-in-Time Intervention on Smartwatch
- npj Digit. Med. 2026 SurgWound-Bench: A Benchmark for Surgical Wound Diagnosis
- PacificVis 2026 Striking a Balance: Evaluating How Aggregations of Multiple Forecasts Impact Judgment Under Uncertainty
- Preprint 2026 Humans' ALMANAC: A Human Collaboration Dataset of Action-Level Mental Model Annotations for Agent Collaboration
- Preprint 2026 CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments
- Preprint 2026 Deco: Extending Personal Physical Objects into Pervasive AI Companion through a Dual-Embodiment Framework
- CSCW 2025 Secret Use of Large Language Model (LLM)
- CSCW 2025 “Mango Mango, How to Let The Lettuce Dry Without A Spinner?”: Exploring User Perceptions of Using An LLM-Based Conversational Assistant Toward Cooking Partner
- CHI 2025 CardioAI: A Multimodal AI-Based System to Support Symptom Monitoring and Risk Prediction of Cancer Treatment-Induced Cardiotoxicity
- CHI 2025 Characterizing LLM-Empowered Personalized Story Reading and Interaction for Children: Insights from Multi-Stakeholder Perspectives
- CHI 2025 Examining Student and Teacher Perspectives on Undisclosed Use of Generative AI in Academic Work
- CHI 2025 Live-Streaming-Based Dual-Teacher Classes for Equitable Education: Insights and Challenges From Local Teachers' Perspective in Disadvantaged Areas
- CHI 2025 Promoting Prosociality via Micro-Acts of Joy: A Large-Scale Well-Being Intervention Study
- IMWUT 2025 Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-in-the-Loop LLM
- IJHCS 2025 User Interaction Patterns and Breakdowns in Conversing with LLM-Powered Voice Assistants
- ACL Findings 2025 Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agents
- EMNLP Findings 2025 Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction
- KDD 2025 SepsisCalc: Integrating Clinical Calculators into Early Sepsis Prediction via Dynamic Temporal Graph Construction
- Preprint 2025 More Modality, More AI: Exploring Design Opportunities of AI-Based Multi-modal Remote Monitoring Technologies for Early Detection of Mental Health Sequelae in Youth Concussion Patients
- Preprint 2025 “It Felt Like I Was Left in the Dark”: Exploring Information Needs and Design Opportunities for Family Caregivers of Older Adult Patients in Critical Care Settings
- Preprint 2025 UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents
- Preprint 2025 LLM Agent Meets Agentic AI: Can LLM Agents Simulate Customers to Evaluate Agentic-AI-based Shopping Assistants?
- Preprint 2025 DPRF: A Generalizable Dynamic Persona Refinement Framework for Optimizing Behavior Alignment Between Personalized LLM Role-Playing Agents and Humans
- CHI 2024 Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis
- CHI 2024 “It's a Fair Game”, or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents
- CHI EA 2024 Human-Centered Privacy Research in the Age of Large Language Models
- CHI EA 2024 Building LLM-Based AI Agents in Social Virtual Reality
- CSCW 2024 Exploring Parent's Needs for Children-Centered AI to Support Preschoolers' Interactive Storytelling and Reading Activities
- CSCW 2024 workshop Challenges and Opportunities of LLM-Based Synthetic Personae and Data in HCI
- IMWUT 2024 Talk2Care: An LLM-Based Voice Assistant for Communication Between Healthcare Providers and Older Adults
- IMWUT 2024 Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data
- EMNLP 2024 StorySparkQA: Expert-Annotated QA Pairs with Real-World Knowledge for Children's Story-Based Learning
- NAACL Findings 2024 More Samples or More Prompts? Exploring Effective Few-Shot In-Context Learning for LLMs with In-Context Sampling
- KDD 2024 SepsisLab: Early Sepsis Prediction with Uncertainty Quantification and Active Sensing
- PhD dissertation 2024 Enhance Machine Reasoning via Active Learning with Human Rationales
- Preprint 2024 Sunnie: An Anthropomorphic LLM-Based Conversational Agent for Mental Well-Being Activity Recommendation
- Preprint 2024 Clinical Challenges and AI Opportunities in Decision-Making for Cancer Treatment-Induced Cardiotoxicity
- Preprint 2024 Human and LLM-Based Voice Assistant Interaction: An Analytical Framework for User Verbal and Nonverbal Behaviors
- Preprint 2024 Designing and Evaluating Sampling Strategies for Multiple-Forecast Visualization (MFV)
- ACL 2023 Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanations
- EMNLP 2023 ‘Don't Get Too Technical with Me’: A Discourse Structure-Based Framework for Automatic Science Journalism
- EMNLP Findings 2023 Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture
- Preprint 2023 Human Still Wins over LLM: An Empirical Study of Active Learning on Domain-Specific Annotation Tasks
- ACL 2022 Fantastic Questions and Where to Find Them: FairytaleQA, an Authentic Dataset for Narrative Comprehension
- ACL 2022 It is AI's Turn to Ask Humans a Question: Question-Answer Pair Generation for Children's Story Books
- CHI 2022 StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement
- EMNLP Demo 2022 GEMv2: Multilingual NLG Benchmarking in a Single Line of Code
- LREC 2022 A Corpus for Commonsense Inference in Story Cloze Test
- Preprint 2022 Efficient Long Sequence Encoding via Synchronization
- Preprint 2022 NECE: Narrative Event Chain Extraction Toolkit
- TACL 2021 Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study
- IUI 2020 Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning Systems
- NUSE @ ACL 2020 Frustratingly Hard Evidence Retrieval for QA Over Books