A hot potato: Most CEOs love generative AI, lauding its ability to streamline companies and make them more efficient – a process that usually involves laying off workers. But wouldn't it be ironic if the chief executives found themselves out of a job, replaced by a machine? A new study suggests AI can perform a CEO role better than a human, but there are scenarios where it struggles, meaning it's more likely to get fired more quickly.
An experiment carried out this year by University of Cambridge researchers included college students, senior executives at a South Asian bank, and GPT-4o, OpenAI's advanced LLM.
Participants took part in a game, designed by the Cambridge researchers' ed-tech startup, Strategize.inc, that put them in the role of CEO at a car company. It was designed to replicate the kinds of decision-making challenges CEOs face, with various metrics tracking the quality of their choices.
The ultimate goal of the game was to survive as long as possible without being fired by a virtual board while maximizing the company's market cap.
Hamza Mudassir, one of the researchers behind the experiment, told Business Insider that the AI outperformed the human participants on the majority of metrics, including profitability, product design, managing inventory, and optimizing prices.
Mudassir gave the example of designing a car as an area where the LLM was significantly better than humans. Using factors such as available parts, price, consumer preferences, and demand, the AI was able to put together a combination that offered the most optimal value for what the customer wanted, whereas humans let personal bias and taste influence areas like the shape of the car.
Before CEOs start getting worried, there was a major caveat with the AI's performance: it struggled with black swan events. In the business world, these are defined as an extremely negative event or occurrence that is impossibly difficult to predict, such as the market collapses seen during the pandemic.
While the top-performing human participants were able to account for unpredictable changes in customer demand, collapsing pricing levels, and strained supply chains, the AI was unable to react as quickly or effectively.
"How do you react to COVID if you're dealing with it for the first time? A lot of people, and a lot of CEOs, have different strategies," Mudassir said. "In this case, it [the AI] did not have enough information on how to react in time to prevent itself from getting fired," he said.
Interestingly, the top executives who took part in the experiment lasted longer than GPT-4o but were fired by the virtual board faster than the students. Like the LLM, the execs had overconfidence in a system that rewards flexibility and long-term thinking as much as aggressive ambition, states the report.
The conclusion is that while generative AI might not replace CEOs, at least not yet, those who fail to utilize its benefits could be left behind.
Experiment shows AI can perform a CEO's job better than humans, but it struggles in a crisis
