US Open to feature AI sports commentary after other experiments

Representatives of Eurovision Sports and IBM who spoke to NBC News said AI hasn’t replaced real commentators, but that it’s doing the tedious parts of the job, such as summarizing events and play-by-play announcing on less notable matches. AI technology generates audio commentary quickly and can cover events that didn’t have commentary before, according to Syken. 

The AI commentary at the Masters and Wimbledon was powered by IBM’s generative AI platform watsonx. According to IBM, generative AI provided golf narration for more than 20,000 video clips on the Masters app. Achieving the same feat with real commentators would be difficult, according to Syken. 

While AI is alleviating some of the burdens of sports broadcasting, some people have speculated that AI commentators could replace human ones in the future.  

But Syken said AI is here to assist rather than replace real commentators, despite advancing technology making AI voices sound increasingly realistic. 

“The same way we’re not trying to replace humans, we’re not trying to overly humanize the voices that we use,” he said. 

Several synthetic voices were used for AI commentary at Wimbledon and one synthetic voice was used at the Masters, according to Syken. He said patrons gave positive feedback on the AI commentary at both the events. 

The AI commentators at Wimbledon and the Masters received some criticism from sports audiences on social media for their botlike delivery. 

A Twitter user referred to the golf AI commentary as “monotonous with no emotions or voice modulations,” in a thread about a video.

“Wimbledons use of AI in highlight reels is completely emotionless, lacks substance or any sense of timing in commentary. So blatantly clear that it’s AI,” another Twitter user wrote. 

Kevin Sylvester, a sports broadcaster of more than 25 years, whose work includes PGA Tour golf coverage and hosting for WGRZ, an NBC affiliate, said the Masters’ AI lacked commentary techniques such as the inflection of the voice to keep audiences engaged with the game.

“It’s an art form, really,” he said. “There’s an expertise that goes into it through experience, and being there and to convey that to the viewer or to the listener. And I don’t think any computer can replace that.”

In the European Athletics Team Championships audio commentary, the AI voice was cloned from commentator Hannah England, who was chosen for her public profile as a former competitive athlete and because she has a recognizable voice to listeners, according to Christophe Pasquier, Eurovision Sport’s head of audio and innovation. 

Thanks to voice cloning, the voices of ‘AI Hannah’ and the real Hannah England are difficult to distinguish, prompting the broadcast to add disclaimers before the AI speaks to inform listeners that what they’re hearing is not a real human. 

“For us, it’s crucial because we want to respect the listeners, we don’t want to cheat them. And we want to educate them. We want to offer them an opportunity to fully apprehend the pros and cons, the benefit and the danger, of AI,” Pasquier said. 

He said England was not compensated for providing her likeness and he declined to disclose financial details related to the AI voice project. He said AI’s added value was not convincing yet. 

“Economically speaking, it’s very time-consuming, and it costs a lot. So it was just an experiment,” Pasquier said. “We’re going to debrief within our community in the coming month and then we’ll see if we’re fit to continue or not.”

An IBM spokesperson would not comment on details of their financial arrangements with their partners.

Pasquier said AI helped reduce the demanding workload on commentators. The format of major tournaments requires professionals to deliver live commentary for nine to 10 hours per day over multiple consecutive days. 

“In the past, it was very painful for the commentator after such a long day, to ask them to stay in the commentary position for let’s say, one extra hour, in order to record all the content we actually generated with AI for highlights post-production,” he said. 

Syken said that if a tennis tournament had 14 matches, “that would require 14 production crews, 14 sets of commentators, to all do original commentary. And that’s not really how media companies choose to produce events these days.” 

Sports broadcasting has been impacted by declining viewership in traditional media and the rise of streaming. According to the Bureau of Labor Statistics, employment for broadcast announcers and DJs is projected to decline by 11% between 2021 and 2031. In June, the sports channel ESPN laid off around 20 sports commentators to manage costs.

As cost-cutting has continued in media companies, AI models are improving.

Syken said its AI model is being trained in the nuances of commentary, to adapt to the vocabulary of a tournament’s country and sport. 

According to IBM, its team sourced data from almost 130 million documents to train the large language model for Wimbledon commentary. 

Pasquier said that from an ethical standpoint and to ensure accuracy, Eurovision Sports used text-to-speech capabilities: a human producer wrote the script read by the AI commentator. 

AI is yet to be used in live matches since processing time could take a few seconds or minutes, depending on the length of the footage, Syken said. “The live nature of the capability we haven’t brought to bear yet, but that’s on the road map.”

Some people said the AI commentary at Wimbledon mispronounced some players’ names. 

“3 minutes highlights with emotionless AI commentary on Wimbledon website was all I found and Wawrinka pronounced War-Rinka,” one Twitter user wrote. Another user said the Wimbledon commentary “sounds artificial and robotic, can’t pronounce half the players’ names correctly, and is profoundly distracting.”

An author who listened to the AI commentary during a Wimbledon match between Ons Jabeur and Magdalena Fręch, said in an article for The Atlantic, “Fręch is mispronounced, as is Tunisia.” 

AI performs better in English than in other languages, Pasquier said, which presents a challenge in producing AI commentary for non-English speaking markets. 

Zohaib Ahmed, founder of Resemble AI, an AI voice-generating company that also does dubbing with AI for entertainment and gaming companies, said that AI’s language capabilities have been progressing: “I think we’ve come a long way in a very short period of time.”

Syken said IBM is exploring dialects and languages to add to its AI commentary moving forward, in consideration of international players and audiences. 

“Sport is an incredibly international activity, whether it’s Wimbledon or what’s coming up at the U.S. Open, to our players from around the world who participate in these events,” he said.