AI-powered broadcast monitoring at the Tokyo Olympics

The situation

A major European sports broadcaster found itself sending commentary in the wrong language out to its viewers. Finns could be subjected to the Tour de France in Bulgarian. Each video feed, with ambient sound from the event, could be merged with the sound feeds from commentators in literally dozens of languages and then had to be routed out to the right broadcast endpoints for each local market. The scope for getting it wrong or not sending the commentary out was significant and each error was a source of enormous customer dissatisfaction.

In the summer of 2021 global society, and sport along with it, was struggling to recover from COVID. A bonanza of sport, and new subscribers with it, was arriving in the summer. The French Open tennis at Roland Garros was resuming its normal May schedule (after being pushed to September the year before), followed 9 days later by the first Wimbledon in two years (having been cancelled in 2020) and then 11 days after that the postponed Tokyo Olympics would commence. The number of feeds being edited together to cover the Olympics would spike and with it the chances of making a bad impression on new subscribers.

At the beginning of June we got a call. The solution had to be deployed for test before the French Open finished on the 13th, it was to be integrated into the customer's processes during Wimbledon on the 21st and part of production support workflows for the start of the Olympics. In the inimitable deadpan tone of the lead BA: "exciting times".

The solution

Thankfully our client, a hyperscale cloud provider, was a couple of weeks into the work already. Their go-to engineer for these rapid-response situations had already produced the core pieces. We have a track record for productionising and deploying solutions via infrastructure-as-code in a way that passed security audits, hence the call.

The system monitored every outbound video stream in real time, checking for two conditions: silence on the audio channel (detected by comparing frequency spectrograms against a silence baseline at regular intervals) and language mismatches (detected by sending audio samples to an AI classification model behind a REST API).

The biggest sticking point was that the cloud platform provider's own language detection service did not cover the full set of broadcast languages. A third-party provider had been contacted and could supply a suitable model, but their infrastructure could not support the volume of continuous inference the tournament would require. A data scientist and a senior solution architect from our client's internal team containerised the model and deployed it as a serverless function behind the API, making the inference layer sufficiently horizontally scalable to support the anticipated load.

The live dashboard displayed each channel's red-amber-green alert state. We performed the SSO and messaging integration with the support team's incident management platform so that engineers on the support rota were notified immediately when a mismatch or silence event was detected.

The outcome

The system went into test and went live to the incredibly aggressive schedule required. A potential crisis, which would have resulted in enormous reputational and financial damage and that was being tracked at board level, had been averted.

It was also a significant moment for us professionally. For the first time in over 20 years of industry we had worked with a genuinely new kind of colleague: a professional AI expert whose tools, processes and needs were, at the time, completely unfamiliar to us. Summer 2021 was the point at which we understood that AI was going to be a part of the regular technology workflow, not just a niche or exotic service to be consumed. As with our early adoption of cloud, a commitment to acquiring a deep understanding of the emerging tools put us in a strong position to take advantage of the revolution that arrived just 16 months later with the launch of ChatGPT.