__MAIN_TEXT__
feature-image

Page 48

TECHNOLOGY

A FACE ODYSSEY Richard Welsh tells Dan Meier about Sundog Media Toolkit’s work with AI

A

rthur C. Clarke’s vision for 2001 may have been slightly premature, but given the rapid development of AI and machine learning tools, it’s surely a matter of time before robots are plowing our fields, writing our movies or managing our football teams. That said, RoboKlopp might still be some way off, as Sundog Media Toolkit CEO Richard Welsh reassures: “AI in the traditional kind of HAL or Terminator sense, we’re a long way from that truthfully. With my futurist hat on, I don’t see artificial general intelligence on the near horizon, so something that’s self-aware or something that can solve arbitrary problems through AI.” However, there are all manner of manual tasks in the film and TV production workflow that are already being automated using AI. “They may not even be very high-value tasks, ie. your operator cost is not the big issue; the problem is the time,” Welsh explains. “So if it’s taking a day or a week to do something when really it’s kind of laborious, non-creative, every production wants that week back if they can get it.” I, ROTOSCOPE To this end, Sundog has developed the DubSafe tool, a Cloud-based rotoscoping AI designed for content versioning security. “We had several customers talk to us about a particular use case where they do rotoscoping and masking of a movie or TV show for the pre-release dubbing copy - so this goes to the voice artists, the sound remixing and foreign language remixing stages, the translators, the subtitlers - and there’s a large team of people that have to touch that content, so to protect the high-security or the particularly sensitive content they do this rotoscoping where they put masks around the faces.” Even at the hands of a skilled operator, this manual process can take up a week of valuable post production time. “There’ll be someone in the post production team who basically sits there for the week and masks out all the faces; they have to go and find all the places where there’s dialogue,” says Welsh. “The other thing is it’s not always actual dialogue, it could be things like breaths, like gutteral noises if it’s a fight.” This means dialogue detection tools are of limited use, hence Sundog’s work in this area. “We built an AI specifically to do the masking automatically; it does do face detection but it does a lot more than that,” notes Welsh. “We found face detection was an interesting challenge because off-the-shelf face detection is really built off large catalogues of still-frame images pretty much off the internet, and these

48 | TVBEUROPE APRIL 2019

are things like people’s holiday snaps and stuff on their phone and selfies and that sort of thing. When actually movies and TV don’t look like that. They’re not framed that way, the lighting is not the way it is in snapshots. So we found that building our own face detection and training it specifically on that type of content is way more successful that just trying to use off-the-shelf face detection.” The other issue with off-the-shelf face detection is that faces might not even look like faces in the traditional sense. “You might get an alien or an animal or a car, or something that’s got a ‘face’, so again being able to train on those different things is really important because in movies that’s pretty common,” says Welsh. “We use different methodologies of object detection, motion tracking and so on, because once we’ve got a face we then need to track it for the whole shot, so the first thing we do is go through and cut up the movie or TV show into shots (we have a scene detect we built ourselves). We then look for a face at any point during each shot.” Sundog’s object detection system can then track a face for the entire shot, even if the character turns

Profile for Future PLC

TVB Europe 63 April 2019  

Get ready, coz here is come - 5G

TVB Europe 63 April 2019  

Get ready, coz here is come - 5G