RESEARCHER & SCIENTIST

Hi, I’m Zhaobo.

Exploring how intelligent systems can better understand and interact with people.

I’m Zhaobo K. Zheng, a Senior Scientist at Honda Research Institute USA. My research brings together multimodal machine learning, human-computer interaction, and human-AI teaming.

Headshot of Zhaobo K. Zheng

01 / RESEARCH

Ideas in action.

How can technology better understand people and support the way we live, work, and move? Explore the questions behind the work.

RESEARCH AREA

Multimodal Machine Learning

Understanding people through multiple signals and perspectives.

01 / GROUP INTERACTION

Diagram showing teammate signals used to model group dynamics and infer an individual's engagement

Human-as-a-Sensor: Inferring Individual Engagement from Teammates Alone via Latent Group State Modeling

In a team, one person’s engagement can show up in the way others respond. Could we learn something about a teammate without observing that person directly? This work models the group’s evolving interaction and uses signals from teammates to estimate an individual’s engagement, exploring a different way to understand social dynamics.

Read the paper

02 / AUDIO + VIDEO

Diagram aligning face tracks and audio-based speaker identities across video and audio streams

Speaker Diarization in Static and Egocentric Videos via Speech-Face Alignment

When several people talk, figuring out who said what can be surprisingly difficult. This work combines audio with faces seen by stationary cameras and wearable, first-person cameras. By aligning voices and visible speakers across views, it aims to make the conversation easier for a machine to follow.

Read the paper

03 / COGNITIVE WORKLOAD

Diagram showing behavioral and physiological features feeding a hierarchical model and Bayesian workload estimate

A Hierarchical Latent Cognitive Model for Workload Estimation with Bayesian Inference

How busy is someone’s mind when a task becomes demanding? Cognitive workload is not directly visible, so this research uses a hierarchical latent cognitive model and Bayesian inference to estimate it. The goal is to better understand human performance in complex task environments.

Read the paper

04 / PROSOCIAL BEHAVIOR

Diagram of physiological model pre-training and multimodal prediction of prosocial behavior intention

Self-Supervised Learning-Based Multimodal Prediction on Prosocial Behavior Intentions

Predicting when someone intends to help is hard, especially when labeled examples are scarce. This work first learns from existing physiological and behavioral data, then adapts the model to a smaller dataset about prosocial intentions. It explores how knowledge learned across tasks can help intelligent systems understand human behavior.

Read the paper

05 / CAREGIVER SUPPORT

Body motion tracking illustration, modeled body joints, and a participant using objects at a table

PreMAC: A Predictive Multimodal Framework to Alert Caregivers of Problem Behaviors for Children with ASD

Early signs of distress can be subtle and difficult to notice. PreMAC investigates whether physiological, motion, and behavioral signals can help identify precursors of escalating behavior, giving caregivers information that could support earlier responses. The work was evaluated in a controlled research setting.

Read the paper
RESEARCH AREA

Human-Computer Interaction

Designing and studying technology around human needs.

01 / HUMAN + AI TEAMS

Example of a reasoning task followed by a discussion among language-model agents

From Process Loss to Assembly Bonus: Human-Grounded Diagnosis of Multi-Agent LLM Collaboration

Does a group make better decisions than its best individual member? This research examines that question for groups of language-model agents using ideas from human group problem-solving. It looks beyond the final answer to explore when discussion helps—and when conformity can cause a good answer to be lost.

Read the paper

02 / FAMILY SUPPORT

Four screens from the Family on Track app asking about services and barriers

Development of an App for Tracking Family Engagement With Early Intervention Services

Finding and navigating early intervention services can be overwhelming for families. Informed by conversations with families and providers, the Family on Track app was developed and piloted to help track engagement, barriers, and experiences. The project shows how listening to users can shape a more useful support tool.

Read the paper

03 / AUGMENTED REALITY

CheerBrush system diagram and a participant using the toothbrushing setup

CheerBrush: An Interactive Augmented Reality Coaching System

Learning to brush your teeth involves a sequence of small, coordinated actions. CheerBrush turns those steps into an interactive augmented reality experience, using a virtual coach and real hand movements to help children with autism practice the skill.

Read the paper

04 / CLINICAL TOOLS

PB.MI interface showing behavior recording controls and a real-time graph

PB.MI: The Problem Behavior Multilevel Interpreter

During a behavioral assessment, clinicians need to see patterns as they emerge. PB.MI displays data in real time and offers a computerized interpretation to support visual analysis. The research evaluates the tool as an aid to clinical interpretation—not a replacement for it.

Read the paper

05 / HAPTICS + VIRTUAL REALITY

Hand holding the instrumented haptic gripper beside a screen showing a virtual task

Design of a Haptic-Gripper Virtual Reality System (Hg) for Analyzing Fine Motor Behaviors in Children with Autism

What if a virtual task could let a child practice both movement and grip? The Hg system combines a sensor-equipped gripper with virtual activities and tactile feedback, providing a way to study and practice fine motor control.

Read the paper
RESEARCH AREA

Intelligent Vehicle Research

Making automated mobility more responsive to people.

01 / PERSONALIZED AUTOMATION

Participant seated at a driving simulator with three displays

Adaptive Driving Style for SAE Level-2 Driving Automation: Minimizing Preference Mismatch

One driver’s comfortable maneuver may feel too cautious—or too assertive—to another. This research investigates an adaptive driving-style framework that predicts preference changes and chooses among bounded driving styles. In a simulator study, the aim was to reduce mismatches and support trust in driving automation.

Read the paper

02 / TRUST IN AUTOMATION

Simulated driving scene showing two pedestrians

Beyond Empirical Windowing: An Attention-Based Approach for Trust Prediction in Autonomous Vehicles

Trust can change over the course of a drive, but not every moment is equally informative. Instead of relying on a single hand-chosen time window, this work uses an attention-based model to identify relevant intervals in multimodal signals and study changes in trust.

Read the paper

03 / DRIVING PREFERENCES

Diagram of two signal inputs processed by a Siamese neural network

Learn-able Evolution Convolutional Siamese Neural Network for Adaptive Driving Style Preference Prediction

People rarely need to explain every driving preference out loud. This project explores how multimodal signals and a Siamese neural network can help identify preferred driving styles, with the longer-term aim of making automated driving more responsive to individual users.

Read the paper

04 / FUTURE MOBILITY

Participants using car and sidewalk mobility simulators

The Future of Mobility-as-a-Service: Trust Transfer Across Automated Mobilities, from Road to Sidewalk

If you trust one automated vehicle, does that experience affect how you feel about another? This study explores how trust develops and transfers between automated car and sidewalk mobility experiences, offering insight into a future with multiple connected mobility services.

Read the paper

02 / PUBLICATIONS

Research in print.

Full Google Scholar profile
  1. CheerBrush: a novel interactive augmented reality coaching system for toothbrushing skills in children with autism spectrum disorder

    Z. K. Zheng, N. Sarkar, A. Swanson, A. Weitlauf, Z. Warren, and N. Sarkar · ACM Transactions on Accessible Computing · 2021

  2. A predictive multimodal framework to alert caregivers of problem behaviors for children with ASD (PreMAC)

    Z. K. Zheng, J. E. Staubitz, A. S. Weitlauf, and collaborators · Sensors · 2021

  3. Design of a haptic-gripper virtual reality system (Hg) for analyzing fine motor behaviors in children with autism

    H. Zhao, Z. Zheng, A. Swanson, A. Weitlauf, Z. Warren, and N. Sarkar · ACM Transactions on Accessible Computing · 2018

  4. Driving aggressively or conservatively? Investigating the effects of automated vehicle interaction type and road event on drivers’ trust and preferred driving style

    Y. Lee, M. Dong, V. Krishnamoorthy, K. Akash, T. Misu, Z. Zheng, and G. Huang · Human Factors · 2024

  5. Identification of adaptive driving style preference through implicit inputs in SAE L2 vehicles

    Z. Zheng, K. Akash, T. Misu, V. Krishnamoorthy, M. Dong, Y. Lee, and G. Huang · International Conference on Multimodal Interaction · 2022

  6. The future of mobility-as-a-service: trust transfer across automated mobilities, from road to sidewalk

    J. G. Hunter and collaborators · Frontiers in Psychology · 2023

  7. Trust in shared automated vehicles: Study on two mobility platforms

    S. Mehrotra, J. G. Hunter, M. Konishi, K. Akash, Z. Zheng, and collaborators · arXiv preprint · 2023

  8. Investigating users’ preferences in adaptive driving styles for level 2 driving automation

    Z. Sajedinia, K. Akash, Z. Zheng, and collaborators · International Conference on Automotive User Interfaces and Interactive Vehicular Applications · 2022

  9. Detection of perceived discomfort in SAE L2 automated vehicles through driver takeovers and physiological spikes

    Z. K. Zheng, K. Akash, and T. Misu · IEEE International Conference on Intelligent Transportation Systems · 2022

  10. Pilot study of a real-time early agitation capture technology (REACT) for children with intellectual and developmental disabilities

    N. Khan, A. Plunk, Z. Zheng, and collaborators · Digital Health · 2024

  11. Collective States in Multimodal Interaction

    T. Misu, G. Varni, K. Inoue, Z. Zheng, and C. Maeda · International Conference on Multimodal Interaction · 2026

  12. Human-as-a-Sensor: Inferring Individual Engagement from Teammates Alone via Latent Group State Modeling

    Z. Zheng, N. Salami Pargoo, K. Akash, and T. Misu · International Conference on Multimodal Interaction · 2026

  13. Speaker Diarization in Static and Egocentric Videos via Speech-Face Alignment

    Z. Zhang, X. Yin, Z. Zheng, K. Akash, T. Misu, and C. Busso · International Conference on Multimodal Interaction · 2026

  14. From Process Loss to Assembly Bonus: Human-Grounded Diagnosis of Multi-Agent LLM Collaboration

    A. N. Tak, T. Misu, K. Akash, Z. Zheng, K. Joo, and J. Gratch · Findings of EMNLP · 2026

  15. A Hierarchical Latent Cognitive Model for Workload Estimation with Bayesian Inference

    Z. Zahedi, Z. Zheng, S. Mehrotra, and K. Akash · IEEE Transactions on Affective Computing · 2026

03 / ABOUT ME

A little about me.

I’m a Senior Scientist at Honda Research Institute, working on multimodal machine learning, social signal processing, and human-AI teaming. My research explores how we can detect human and collective states in human-AI interactions, such as in human teams, human-robot interaction, and automated driving.

I received my PhD in Mechanical Engineering from Vanderbilt University, and my Bachelor’s degree in Mechanical Engineering from Xi’an Jiaotong University. I was advised by Prof. Nilanjan Sarkar, Prof. Zachary Warren, and Prof. Amy Weitlauf. I work in inter-discipline areas spanning machine learning, human computer interaction, robotics and mechatronics, and human cognition. This combination motivates me to build technologies that can facilitate smooth interactions between humans and AI systems.

Find me on Google Scholar