Unlocking the Power of Generative AI in Customer Service: Surprising Insights from Alibaba

Introduction

Tuck professor Lauren Xiaoyuan Lu has analyzed a groundbreaking generative AI experiment at Alibaba, revealing its potential as a powerful tool with unexpected outcomes.

The Challenge of Customer Service

Every day, over 380 million users visit Taobao, Alibaba's e-commerce platform. When customers encounter issues, they first interact with a chatbot before being connected to human agents. Taobao processes an astonishing 400,000 customer service requests daily, making effective service crucial for business success.

The Experiment

In early 2024, Taobao conducted a large-scale randomized field experiment, providing some after-sales agents access to a gen AI assistant while others remained in a control group. The AI assistant analyzed customer data and past interactions in real-time, crafting messages for agents that included a problem synopsis and proposed solutions. Agents could use, modify, or ignore these AI-generated messages.

Key Findings

• Improved Speed and Quality: The deployment of the gen AI assistant significantly enhanced both the speed and quality of customer service. Agents spent less time on chats, leading to increased customer satisfaction.
• Augmentation, Not Replacement: Surprisingly, the AI assistant augmented agents' performance rather than replacing them. Agents’ engagement levels increased, as indicated by a rise in message volume, despite no significant decrease in typing time.
• Bridging Performance Gaps: The AI assistant helped improve the performance of lower-performing agents but slightly decreased the efficiency of high performers. This dual effect poses a challenge for management in maintaining high standards among top agents.

Considerations for the Future

Despite the promising results for e-commerce firms, Lu expresses concerns about the potential displacement of workers due to advancements in AI. As companies seek to cut costs, the temptation to reduce human workforce may increase, urging firms to consider re-deployment and upskilling strategies for their employees.

“I do have a concern about the displacement of workers. Firms should ponder how they can re-deploy these workers and upskill them for other value-added services.”
— Lauren Xiaoyuan Lu, Professor of Business Administration