Add to calendar

Electrical Engineering and Computer Science (EECS)

Ph.D. Dissertation Defense


“Lip Reading as an Active Mode of Interaction with Computer Systems”

Laxmi Pandey

Electrical Engineering and Computer Science

University of California, Merced




Silent speech that converts lip movements into text can mitigate many challenges of speech and traditional input methods. Yet, existing silent speech recognition models are slow and error-prone, or use impractical and invasive setups, impeding their scalability in real-world scenarios. In this talk, I will present the findings of studies involving silent speech input.

First, a social study exploring the social acceptance of silent speech input in different social contexts. Second, two empirical studies investigating user tolerance of recognition errors in silent speech input and identifying suitable feedback mechanism for real-time silent speech input on mobile devices. Third, a new end-to-end deep neural network that can automatically segment lip sequence videos and classify them into text. Fourth, an independent repair model which is a multi-stage pipeline that processes video input for poor lighting conditions, when applicable, and corrects potential errors in output for increased accuracy. Fifth, two studies exploring how speaking rate affects silent speech recognition and whether these effects are different than those of speech recognition. Finally, design and evaluation of silent speech as a hands-free selection method in eye-gaze pointing.


Speaker Bio:

Laxmi Pandey is a PhD student of Electrical Engineering and Computer Science at the University of California, Merced, where she works at the Inclusive Interaction Lab. Her research focuses on multiple areas, including natural language processing, audio-visual speech recognition, and applied artificial intelligence. Her aim is to design and develop effective and accessible communication technologies for mobile devices. She has received many awards for her research activities, including the Fred and Mitzie Ruiz Fellowship, the Hatano Cognitive Development Research Fellowship, ACM-W Scholarship, and the SIGIR Travel Grants. Additionally, she has been an intern at Meta @Speech AI and will now join the team as a full-time researcher after receiving her PhD. She received her Master’s in Electrical Engineering from IIT Kanpur, India, and Bachelor’s in Electronics and Communication Engineering from Uttar Pradesh Technical University, India. More details about her are available at

Event Details

0 people are interested in this event

User Activity

No recent activity

University of California Merced Events Calendar Powered by the Localist Community Event Platform © All rights reserved