This text is a review of the article
Brynjolfsson, E., Li, D., & Raymond, L. R. (2023). Generative AI at Work
It is clear that new AI tools have the potential to change the way employees work and train. However, little is known about their impact on work activities. One of the recent works in this area is a study by Brynjolfsson, E., Li, D., & Raymond, L. R. (2023). In this article, the authors study the phased implementation of generative AI based on a conversational assistant. Specifically, the authors study the impact of generative AI on the productivity and experience of employees in the customer service sector, an industry with one of the highest rates of AI adoption at the moment. The authors explore the phased implementation of a chat assistant using data from 5,000 agents working for a Fortune 500 software company (a company that provides business process software). The AI technology under study is based on the latest version of the Generative Pre-trained Transformer (GPT) family of large language models developed by OpenAI. It monitors customer chats and provides employees (agents) with real-time advice on how best to respond to the customer in the chat.
Overall, the emergence of generative artificial intelligence has attracted considerable attention, but few studies have examined its economic impact. While various generative AI tools have proven themselves well in the laboratory, excitement about their potential has been tempered by concerns that these tools may be less effective in real-world settings, where they may encounter unfamiliar problems, resistance from organizations, or provide misleading information.
However, it should be noted that machine learning (ML) algorithms operate differently from traditional computer programs: instead of requiring explicit instructions to function, these systems derive instructions on their own from the examples that were given to them for training. For example, with a training set of images, ML systems can learn to recognize specific individuals, even if it is not possible to fully explain what physical features characterize the identity of that person. This ability underscores a key, distinctive aspect of ML systems: they can learn to perform tasks even in the absence of instructions, including tasks that require implicit knowledge that was previously only acquired through life experience.
When it comes to the customer communication sphere, ML systems developed as assistants for this field are often trained on data obtained from human workers, who naturally vary in their abilities. By observing numerous examples of how tasks—such as preparing advertising materials, driving a truck, or diagnosing a patient—are performed well or poorly, these models can indirectly learn which specific behavioral patterns and characteristics distinguish highly effective workers from their less effective colleagues. In other words, generative AI models are not only capable of performing complex tasks, but they may also be able to acquire the skills that differentiate top-performing workers. Thus, the use of ML tools can introduce lower-skilled workers to new skills and lead to significant changes in labor productivity. This hypothesis is supported by the authors of the analyzed work, who draw several conclusions at once.
In the work by Brynjolfsson, E., Li, D., & Raymond, L. R. (2023), it was concluded that the support of artificial intelligence increases the productivity of employees in the customer service sector (chatting with customers), measured by the number of solved problems per hour, by an average of 14%. This increase reflects changes in three performance components: a reduction in the time spent by the agent processing an individual chat, an increase in the number of chats processed by the agent per hour (agents can process multiple chats simultaneously), and a slight increase in the proportion of successfully resolved chats.
However, it is important to note that the impact of AI assistance on productivity is quite uneven. It has been found that less skilled and experienced workers show higher results across all productivity metrics considered, including a 34% increase in the number of tasks they can handle per hour. Access to artificial intelligence tools also helps new agents advance in their careers more quickly: agents who use the chat assistant and have two months of work experience perform as well as agents without it and with over six months of work experience. On the other hand, the authors found minimal impact on the productivity of more experienced or qualified workers. Conversely, the authors find evidence that AI assistance for highly skilled and experienced agents may even reduce the quality of their conversations.
In other words, the authors of the study provide preliminary evidence that the AI model helps beginners to increase productivity to a greater extent, as it spreads the best practices of more experienced workers, but generative AI has little or even negative impact on the productivity of experienced workers.
The second important conclusion made by the authors of the study lies in the learning ability of employees using AI. It turned out that agents who follow the AI recommendations more closely see a greater increase in productivity and, as a result, try to learn from these recommendations. So, using data on AI downtime — periods during which the AI software stopped providing any suggestions to employees —workers exhibited higher levels of productivity compared to their baseline levels before AI, even when AI recommendations were not available to them. In addition, the authors of the article analyze the text of agent chats and provide convincing evidence that access to AI contributes to the convergence of communication models between beginners and experienced workers: agents with low qualifications begin to communicate close to how highly qualified agents communicate.
Thus, we conclude that access to generative AI can enhance productivity in the customer service/communication sphere. However, the authors of the article emphasize that the conclusions of the study do not extend to the overall impact of generative AI tools on employment or wages in all sectors. The research focused on a specific generative AI in a specific field.
