Research

Human Centred Design and Evaluation of AI Assisted Interview Systems

Investigating how artificial intelligence can be intentionally designed around human psychological needs rather than rigid automation. Grounded in Self-Determination Theory (SDT) and participatory co-design methodologies, this research explores how job seekers experience, perceive, and learn from AI-driven interview preparation platforms. By centering affected users throughout the design lifecycle, these works uncover critical experiential tensions—such as the risk of producing homogenized "robotic answers"—and demonstrate how structured practice, video playback, and autonomy-supportive conversational feedback can genuinely empower candidates.

Robotic Answers: Participatory Identification of Risks in AI-Assisted Interview Training by Job Seekers

Pooja S. B. Rao, Laxminarayen Nagarajan Venkatesan, Dinesh Babu Jayagopi, and Mauro Cherubini

ACM FAccT 2026 Participatory Co-Design Psychological Risks
Participatory Identification of Psychological Risks in AI-Assisted Interview Training
Participatory Risk Framework: Centering job seekers' perspectives to uncover psychological harms in AI interview training.
Abstract
AI-assisted interview hiring and training systems are increasingly positioned as tools to support job seekers; however, their deployment may introduce risks that are not sufficiently understood. Existing research on fairness and accountability in AI systems has largely emphasized model-centric harms, often overlooking how affected users themselves perceive and articulate risks. We present a participatory study that centers job seekers' perspectives on the potential harms of AI-assisted interview training systems. Through a workshop with N=20 job seekers, participants engaged in collective reflection and peer critique of proposed AI-driven feedback designs. An inductive thematic analysis reveals four psychological risks: homogenization of communication ("robotic answers"), overreliance on AI, illusion of human interaction, and reinforcement of stress and anxiety. By foregrounding job seekers as active contributors to risk identification, this work provides a user-centered perspective on AI harms in hiring technologies.

On the Potential of Supporting Autonomy in Online Video Interview Training Platforms

Pooja S. B. Rao, Laetitia Renier, Marc-Olivier Boldi, Marianne S. Mast, Dinesh B. Jayagopi, and Mauro Cherubini

IJHCS 2024 Self-Determination Theory User Evaluation
InterviewApp Autonomy Support Architecture and Outcomes
InterviewApp Framework: Supporting candidate autonomy, competence, and self-efficacy through mock recording, playback, and structured feedback.
Abstract
Asynchronous video interviews (AVIs) have become standard in recruitment, yet preparing for them often induces high anxiety and feelings of depersonalization. This study introduces InterviewApp, an online interview training platform built upon Self-Determination Theory (SDT). Across experimental interventions with job seekers, we investigate how structured mock interview recordings, video playback self-reflection, and autonomy-supportive expert feedback influence interview performance and motivation. Our findings demonstrate that perceived autonomy plays a decisive mediating role: empowering candidates with control over their training pace and non-evaluative self-reflection significantly enhances interview performance and self-efficacy while mitigating anxiety, pointing to key design guidelines for autonomy-supportive career preparation systems.

ResumeGenAI: Supporting Job Seekers with LLM-Driven Resume Feedback

Dhruv Awasthi, Pooja S. B. Rao, and Dinesh Babu Jayagopi

ACM CUI 2025 Conversational AI LLM Feedback
ResumeGenAI Conversational Feedback Workflow
ResumeGenAI Workflow: Conversational, multi-turn LLM guidance supporting candidate agency and structured resume improvements.
Abstract
Resume screening is frequently candidates' first gate in automated hiring pipelines, yet feedback on rejected resumes remains virtually non-existent. We present ResumeGenAI, a conversational AI system designed to provide job seekers with transparent, personalized, and actionable resume feedback. By combining role-targeted criteria analysis with multi-turn conversational interaction, ResumeGenAI enables candidates to discuss critique points, test rewording suggestions, and maintain authentic ownership of their qualifications, bridging the gap between automated screening standards and job seeker empowerment.

Responsible Risk Identification and Reporting

Interrogating how artificial intelligence risks, socio-technical hazards, and algorithmic biases are communicated, audited, and mitigated across the AI development lifecycle. This research combines large-scale empirical auditing of over 460,000 open-source model cards, retrieval-augmented generation tools (RiskRAG) for actionable risk reporting, and experimental fairness auditing of Large Language Models in recruitment. By identifying severe blind spots between what developers report and how AI causes real-world harm, this work contributes practical solutions for transparent, culturally sensitive, and accountable AI systems.

RiskRAG: A Data-Driven Solution for Improved AI Model Risk Reporting

Pooja S. B. Rao, Sanja Šćepanović, Ke Zhou, Edyta Paulina Bogucka, and Daniele Quercia

ACM CHI 2025 Talk Available Retrieval-Augmented Generation
RiskRAG Presentation Talk Thumbnail
Watch ACM CHI 2025 Presentation
Conference presentation of RiskRAG at ACM CHI 2025: Data-driven AI model risk reporting.
Abstract
Risk reporting is essential for documenting AI models, yet only 14% of model cards mention risks, out of which 96% copy content from a small set of cards, leading to a critical lack of actionable insights. To address this, we introduce RiskRAG, a Retrieval-Augmented Generation (RAG) based risk reporting solution guided by five design requirements co-designed with 16 developers: identifying diverse model-specific risks, clearly presenting and prioritizing them, contextualizing for real-world uses, and offering concrete mitigation strategies. Drawing from 450K model cards and 600 real-world incidents, RiskRAG pre-populates contextualized risk reports. Evaluated with 115 practitioners across development, design, and media, RiskRAG significantly improved AI model selection, fostering more deliberative and safety-conscious deployment choices.

The AI Model Risk Catalog: What Developers and Researchers Miss About Real-World AI Harms

Pooja S. B. Rao, Sanja Šćepanović, Dinesh Babu Jayagopi, Mauro Cherubini, and Daniele Quercia

AAAI/ACM AIES 2025 Risk Taxonomy 460K Model Audit
The AI Model Risk Catalog Tripartite Analysis
The AI Model Risk Catalog: Comparative taxonomy revealing severe underreporting of fraud, manipulation, and human-interaction harms.
Abstract
We analyzed nearly 460,000 AI model cards from Hugging Face to examine how developers report risks in practice. From these, we extracted approximately 3,000 unique risk mentions and synthesized the AI Model Risk Catalog. Comparing developer-reported risks with those documented in academic taxonomies (MIT Risk Repository) and verified historical harms from the AI Incident Database revealed substantial misalignments: developers focus heavily on internal technical defects and basic bias, whereas researchers prioritize broad societal impacts. Alarmingly, both groups systematically overlook risks tied to fraud, deception, and social manipulation—harms that frequently emerge once AI systems interact with human users.

Invisible Filters: Cultural Bias in Hiring Evaluations Using Large Language Models

Pooja S. B. Rao, Laxminarayen Nagarajan Venkatesan, Mauro Cherubini, and Dinesh Babu Jayagopi

AAAI/ACM AIES 2025 Algorithmic Fairness Cross-Cultural Evaluation
Invisible Filters: Cultural Bias in Hiring Evaluations Research Poster
Research Poster: Investigating cultural bias and linguistic scoring disparities in LLM-assisted hiring evaluations.
Abstract
Large language models (LLMs) are increasingly deployed to screen and score interview candidates, raising critical questions of fairness across diverse cultural backgrounds. Using balanced datasets of video interview transcripts from UK and Indian job seekers, we evaluate cross-cultural differences in LLM-generated ratings for hirability, self-promotion, and storytelling. Indian candidates consistently receive lower evaluations, even when transcripts are completely anonymized. Disparities correlate strongly with linguistic style indicators—such as syntactic complexity and lexical diversity—rather than surface identity markers (names, gender, or caste). These findings demonstrate that standard LLMs harbor covert cultural evaluation filters, emphasizing the necessity of culturally grounded audits in AI hiring.

Good Research Practices

Advancing scientific transparency, methodological rigor, and research ethics across empirical computing and Human-Computer Interaction literature. Through large-scale meta-research, rigorous multi-annotator audits, and the development of automated verification tools, this work measures community-wide adherence to open science principles—including open data and code availability, ethical consent disclosures, participant compensation reporting, and experimental reproducibility.

Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022

Kavous S. Niksirat, Lahari Goswami, Pooja S. B. Rao, James Tyler, Alessandro Silacci, Sadiq Aliyu, Annika Aebli, Chat Wacharamanotham, and Mauro Cherubini

🏆 Best Paper Award (Top 1%) ACM CHI 2023 Talk Available Open Science
Changes in Research Ethics CHI 2023 Presentation Thumbnail
Watch ACM SIGCHI Award Presentation
ACM SIGCHI Conference presentation: Meta-evaluating 245 empirical papers across 45 good research practice criteria.
Abstract
In recent years, the Human-Computer Interaction (HCI) community has increasingly embraced Good Research Practices (GRPs) to improve the reliability, replicability, and ethical soundness of empirical findings. To examine the real-world impact of these initiatives, we conducted a meta-analysis evaluating stratified random samples of empirical papers from CHI 2017 (N=118) and CHI 2022 (N=127) across 45 operationalized criteria covering research ethics, openness, and transparency. The CHI 2022 sample demonstrated statistically significant improvements in 18 out of 45 criteria, with the strongest progress observed in research transparency (10 of 17 criteria). To support authors and reviewers in upholding these standards, we developed a proof-of-concept automated screening tool that accurately flags adherence for key criteria, providing a foundation for scalable meta-science in computing.

Automated Interview Assessment

Pioneering multimodal machine learning and computational behavioral analysis to assess non-verbal communication, speaking fluency, and interaction dynamics in video interviews. Combining acoustic signal processing (pitch, rhythm, pause structures), visual non-verbal tracking (head pose, gaze, facial action units), and natural language processing, this body of work established foundational empirical baselines comparing asynchronous video interviews against traditional face-to-face formats, and introduced dynamic follow-up question generation to make automated interviews interactive and conversational.

Asynchronous Video Interviews vs. Face-to-Face Interviews for Communication Skill Measurement: A Systematic Study

Sowmya Rasipuram, Pooja S. B. Rao, and Dinesh B. Jayagopi

🏆 Best Student Paper Nominee ACM ICMI 2016 Multimodal Signal Processing
Multimodal Communication Skill Assessment Pipeline
Multimodal Assessment Architecture: Feature extraction across acoustic and visual modalities for automated communication skill scoring.
Abstract
Asynchronous video interviews (AVIs) have increasingly replaced traditional in-person interviews for preliminary hiring screening. However, how candidate behaviors and perceived communication competence translate across these distinct interaction modes remains poorly understood. In this systematic comparative study, we collect synchronized audio-visual recordings of candidates participating in both AVIs and conventional face-to-face interviews. We extract multimodal non-verbal behavioral cues—including prosodic features, facial expressions, and head movements—and train regression models to automatically score communication skills. Our comparative analysis reveals that while acoustic features remain consistently predictive across both formats, visual engagement and expressive cues exhibit significant modality-specific variations that automated evaluation algorithms must account for.

Improving Asynchronous Interview Interaction with Follow-up Question Generation

Pooja S. B. Rao, Manish Agnihotri, and Dinesh B. Jayagopi

IJIMAI 2021 Natural Language Processing Conversational Interaction
Dynamic Follow-Up Question Generation System
Adaptive Follow-Up Generation: Parsing candidate speech to identify information gaps and generate relevant follow-up questions in real time.
Abstract
A critical drawback of asynchronous video interviews is their rigid, one-sided nature: candidates respond to pre-recorded prompts without the opportunity for clarifying dialogue or deeper probing. We propose an automated natural language processing pipeline to incorporate dynamic follow-up question generation into asynchronous interviews. By transcribing candidate utterances in real time, extracting key named entities, and detecting incomplete thematic segments, the system formulates contextually grounded follow-up questions. User evaluations demonstrate that automated follow-up questions significantly increase interview realism, candidate engagement, and the depth of verifiable information obtained.

Automatic Prediction of Fluency in Interface-Based Interviews

Sowmya Rasipuram, Pooja S. B. Rao, and Dinesh B. Jayagopi

IEEE INDICON 2016 Speech Analysis Fluency Modeling
Abstract
Oral fluency is a crucial determinant of communication ratings in job interviews. In interface-based asynchronous interviews, speech is often affected by the unnatural interaction setting. We present an automated approach for predicting speaking fluency ratings using acoustic and temporal prosodic features extracted from interview audio, including syllable articulation rates, phonation-time ratios, pause length distributions, and pitch contours. Evaluated against expert human ratings, our machine learning models demonstrate high correlation with human assessors, validating speech prosody modeling as an effective component for scalable interview evaluation.