中文摘要
当组织采用人工智能(AI)来识别个体的负面情绪并据此配置有限资源时,策略性用户便有动机通过虚报自身情绪来操纵系统。AI 在自动化此类情绪驱动资源配置方面的价值,可能因操纵行为、情绪检测中的算法噪声以及负面情绪的溢出效应而被削弱。我们建立博弈论模型以理解情感 AI 的采用(尤其是在客户关怀领域的采用),并分析相关配置政策的设计。我们发现,如果负面情绪的溢出效应相较于资源错配损失可以忽略不计,那么无论算法噪声和操纵行为如何,采用情感 AI 都是有价值的。我们还量化了情感 AI 对用户、组织和社会的福利影响。值得注意的是,更强的 AI 并非总是符合社会期望,因而需要对情绪驱动的资源配置进行监管。最后,我们刻画了在情绪驱动配置中利用 AI 系统优于雇用人类员工的条件。我们还探讨了情感 AI 的另一种应用,即利用其监督策略性员工,并将其与雇用人类管理者进行监督的方式进行比较。耐人寻味的是,算法噪声可能提高 AI 监督的利润。我们的研究为情感 AI 的设计、采用与监管提供了启示。
Abstract
When organizations adopt artificial intelligence (AI) to recognize individuals’ negative emotions and accordingly allocate limited resources, strategic users are incentivized to game the system by misrepresenting their emotions. The value of AI in automating such emotion-driven allocation may be undermined by gaming behavior, algorithmic noise in emotion detection, and the spillover effect of negative emotions. We develop a game-theoretical model to understand emotion AI adoption, particularly in customer care, and analyze the design of the associated allocation policies. We find that adopting emotion AI is valuable if the spillover effect of negative emotions is negligible compared with resource misallocation loss, regardless of algorithmic noise and gaming behavior. We also quantify the welfare impacts of emotion AI on the users, organization, and society. Notably, a stronger AI is not always socially desirable and regulation on emotion-driven allocation is needed. Finally, we characterize conditions under which leveraging the AI system is preferred to hiring human employees in emotion-driven allocation. We also explore the alternative application of using emotion AI to monitor strategic employees and compare it with hiring a human manager for monitoring. Intriguingly, algorithmic noise may increase the profit of AI monitoring. Our work provides implications for designing, adopting, and regulating emotion AI.