Ready for your close-up HAL?

AI IS ALREADY EMBEDDED IN THE CREATIVE ARTS. WHAT ARE YOU GOING TO DO ABOUT IT?

As artificial intelligence gets more and more sophisticated to produce longer-form narrative video, including deepfake or CG actors, the dividing lines will increasingly blur. makers explores some of the latest practical examples benefitting production and addresses the concern it might automate some, or even all, creative roles.

The latest season of America’s Got Talent presented Elvis singing live in the final. Meanwhile the second series finale of BBC /Peacock drama The Capture reached its altogether more sinister conclusion. What they have in common, aside from NBC Universal as producer, is the use of deepfakes and video manipulation to twist the truth.

While America’s Got Talent ’s was an entertaining gimmick and The Capture’s was a compelling hook on which to hang conspiracy theories, they both prompt questions about the use of AI as a means to unlock creative opportunities and about the ethics of doing so.

“It’s too late to be worried about AI,” says Sami Arp, Founder and CEO of Largo.AI. “AI is already here and companies should be open to embracing it, looking to redesign roles and to create with it.” Many media companies are indeed adopting AI tools for everything from storyboarding in preproduction to powering search and recommendation engines for service subscribers.

Largo, a Swiss-based company, provides data-assisted intelligence to the film and television industry. Its algorithm will analyse all the main ingredients of a proposed piece of content including the script to identify important patterns related to demographics, actors and other key talent, and estimated box office return.

“The aim is to give producers more information and insights to help them make the best final product,” says Arp. “Our system predicts the potential revenue of your film at each step of the development from script, rough-cut, and fine cut. Our accuracies are

The Capture © BBC/Heyday/Universal International Studios.

AI IS INCREASINGLY USED TO GENERATE CG PERFORMANCES FROM MOTION CAPTURE AND TO AUTOMATE AND ANIMATE FACE REPLACEMENT.

The Andy Warhol Diaries ©Andy Warhol Foundation/Netflix. over 80%. After making any decision during the development process, you can easily see its impact on the resulting revenue of the film.”

Among benefits to indie producers who are using the technology is a reduction in the time taken to go through this process manually and an increase in the chance of getting a project off the ground.

“Producers have much better understanding of the potential of their film – especially small producers which have limited access to market data compared to streamers. It also creates much more confidence among distributors, investors and funding bodies if you provide analysis like this to overcome their scepticism.”

Arrow International Media is extending the integration of AI services into its production workflows to assist in managing the ever increasing amounts of data generated on a shoot. A recent production was shooting upwards of 20 hours per day in the US. Overnight the rushes were uploaded, processed for edit and transcoded to web proxy. This triggered an automatic transcription service, and the content analysis of the footage. Within 24 hours of being shot in the US, the footage was available in the edit with a transcription and was searchable in the cloud.

“Mistakes are to be expected and the results still require human interpretation, but the material to interpret is reduced to a realistic and manageable level,” says Arrow’s post production consultant Dan Carew-Jones. “It is a triage process, not microsurgery.”

Arrow continues to embrace the possibilities of AI and ML. Footage restoration, colourisation, and categorisation, facial recognition, metadata enhancement and integration are all areas that it is studying.

AI is already proficient at tackling time consuming tasks like de-noising, rotoscoping, and motion capture tracking removal. As Pete Divers, Co-Founder and Head of VFX, Fika Entertainment says, “If an AI can knock-off the facial animation of 20 background characters that would leave your team more time to concentrate on the hero characters and the parts of the frame where viewers are drawn to.”

AI can also mimic speech and has been used in documentaries. Roadrunner: A Film About Anthony Bourdain used synthetic audio to have the late chef and traveller say things he didn't, while Andy Warhol was voiced by an AI and used extensively to narrate the Netflix documentary The Andy Warhol Diaries. While Roadrunner producer/director Morgan Neville was criticised for not being up front about the dupe, the makers of Warhol’s diaries ensured they put in a disclaimer up front and won plaudits for its storytelling nous.

AI is increasingly used to generate CG performances from motion capture and to automate and animate face replacement.

A version of this technique was used by documentary Welcome to Chechnya (2021) in which director David France protected the identity of witnesses but retained more of the power of their testimony on screen.

“There’s now this AI tool for filmmakers to tell their stories in ways that haven’t been done before,” says France. “It also provides some additional security for witnesses to tell their stories and do it in a human way. They can have a voice and have their story translated effectively and truthfully.”

Face replacement is now possible in a live broadcast. Metaphysic, the company behind the America’s Got Talent Elvis deepfake, did the same to judges Heidi Klum, Sofia Vergara and Simon Cowell on the show. They are also behind the viral internet sensation DeepTomCruise.

Co-founder Chris Umé describes what they do as creating Synthetic Media. This is the rapidly-expanding world of digital experiences and objects generated with input from artificial intelligence.

It requires training an AI model on images and videos of one person (dead in the case of Elvis), capturing them from multiple angles and in a variety of lighting conditions. They then shoot base videos using a body double, before generating a deepfake by combining the body double footage with video of the other person’s face.

“DALL-E 2 WILL CHANGE THE INDUSTRY RIGHT ACROSS THE BOARD. NEW JOBS WILL BE CREATED SUCH AS PROMPT ENGINEER, WHO WRITES THE PROMPT INTO THE AI TO GENERATE VERY SPECIFIC OUTPUTS.”

ART IS AN ACCIDENT AND MACHINES PRODUCE LOGIC. MACHINES DON’T KNOW HOW TO PRODUCE ACCIDENTS. In an illustration of how this technology is bleeding into the mainstream, the same process was used by state security services in The Capture to fake CCTV footage.

Metaphysic plan to commercialise their technology in the belief that as much as 80% of the time we spend online in future will be spent interacting with each other’s avatars.

“Soon, studios will simply need to rent Brad Pitt’s face value rights for him to appear in the upcoming blockbuster film without having to leave the comfort of his couch,” says Pinar Seyhan Demirdag, co-founder of Seyhan Lee which offers creative AI solutions to the movie industry.

“I predict that letting AI recreate your footage in a style of your choice will be one of the first AI art tools that studios will adopt. Give it a couple of years, and you’ll be able to film a scene of four random people walking down an aisle, and you will be able to turn them into Dorothy and the gang in The Wizard of Oz.”

It’s only a short hop to being able to generate coherent and screen-ready feature length content that are entirely AI generated.

In fact, a step closer to that inevitable day has been provided by Irish computer artist Glenn Marshall. In 2008 he won the prestigious Prix Ars Electronica for a music video he created for Peter Gabriel –unique in that it was created entirely out of programming and algorithms. He also created an AI-generated Daft Punk video. His new short The Crow is a finalist for The Lumen Prize, considered to be two of the most prestigious digital arts awards in the world, and is also eligible for submission to the BAFTA Awards.

Marshall fed video frames of a short live-action dance film called Painted into CLIP, a neural network created by OpenAI. He then prompted the system to generate a video of “a painting of a crow in a desolate landscape.”

Marshall says the AI is trying to make every live action frame look like a painting with a crow in it. “I’m meeting it half way, and the film becomes kind of a battle between the human and the AI – with all the suggestive symbolism.”

He’s exploring CLIP-guided video generation, which can add detailed text-based directions, such as specific camera movements. That could lead to entire feature films produced by text-to-video systems.

Johnny Johnson, who teaches immersive production at the UK’s National Film and TV School’s (NFTS) StoryFutures Academy thinks future versions of AI tools like text to image engine DALL-E 2 will be capable of making entire feature films with AI-generated scripts and AI generated audio performances alongside the images.

“DALL-E 2 will change the industry from production design and concept art right across the board,” he says. “New jobs will be created such as Prompt Engineer, who writes the prompt into the AI to generate very specific outputs.” There are two main concerns to the creative industries. One is that AI will automate the job of any creative. The other is a potential copyright black hole.

Small-scale content creators or artists just starting out are most under threat of an AI stunting their career.

Video game artist RJ Palmer envisions a scenario where using AI a single art director could take the place of five to ten artists. “The unfortunate reality of this industry is that speed is favoured over quality so often that a cleaned up, ‘good enough’ AI-generated image could suffice for a lot of needs,” he told gaming site Kotaku.

This would affect illustrators, photographers, graphic designers, models, pretty much any job that requires visuals – all potentially outsourced to AI.

However, those makers spoke to were not convinced this will happen.

“AI can spawn a huge amount of images at concept stage but creative leads will always want to take over and refine those ideas,” says Divers.

“The industry is driven by creativity and for that reason I don’t think human creators will ever accept AI [dominion],” says Arp. “What’s more if you replace actors with deepfakes and writers with AI you will lose a big part of the audience in the process.”

Marc du Pontavice, CEO of animation studio Xilam agrees. “Art is an accident and machines produce logic,” he says. “Machines don’t know how to produce accidents.

“Maybe I’m old fashioned but I am very sceptical about AI. When it comes to technology I’m interested in whatever will help creators get their imagination to the screen. But consider that it takes as many people if not more to create a Pixar movie today as it took to make The Lion King 28 years ago.”

On the copyright front, AI is already so proficient at copying a particular artist’s work that it won’t be long before filmmakers need to protect themselves from plagiarism.

There’s an argument that right now camera moves, editing choices, colour palettes or lighting schemes, should be copyrighted because there is nothing to prevent an AI from entirely generating a new gangster movie in the style of Scorsese or a sci-fi that looks and feels like it has come from Stanley Kubrick.

There will be some Hollywood execs no doubt calculating that if an AI could perfect a hit movie without having to pay for the risk and fuss that human talent brings it’s a price worth paying.

“IT’S TOO LATE TO BE WORRIED ABOUT AI. AI IS ALREADY HERE.”

The Year of Greece

GREECE HAS BECOME A TOP GLOBAL FILMING DESTINATION IN A VERY SHORT TIME, THANKS TO ITS UNIQUE LOCATIONS, COMPETITIVE INCENTIVES, SKILLED PROFESSIONALS AND NETWORK OF FILM OFFICES.

Crowd scene from Smyrna, by Grigoris Karantinakis. Attica, Greece. © Tanweer Productions.

Kathryn Hahn, Jessica Henwick, Kate Hudson, Leslie Odom Jr, Madelyn Cline (l to r) in Glass Onion: A Knives out Mystery, by Rian Johnson. Spetses, Greece. © Glass Onion: A Knives out Mystery.

Camera view from first day of filming Glass Onion: A Knives out Mystery, by Rian Johnson. Spetses, Greece. © Rian Johnson.

BROUGHT TO YOU BY

Greece is an incomparable natural set, diverse, accessible, and above all friendly. It offers a variety of options that serve your story. Magnificent coastlines, sand dunes, waterfalls, snow capped mountains, tropical forests, ancient sites, historical monuments, industrial buildings, urban, rural, ancient, medieval, classical, modern: one of the richest cultural heritages in the world! All easy to reach within one hour drive from coast to mountain. Add to all the above the well-known traditional Greek hospitality and one of the best financing schemes in Europe administered by EKOME, and you have your perfect location!

During the last two years, the successful management of the pandemic crisis by the Greek government and the ongoing improvements in legislation helped the country soar to one of the top filming destinations not only in Europe but worldwide. Major studios (Universal, Disney, Amazon, Paramount, Warner, Netflix, Apple TV, Millenium Media) with high-end productions now systematically choose Greece to film their projects.

With foreign production booming, Greece is also famous for its highly qualified experienced film professionals and state of the art facilities in all the stages of film production (crew, talentactors/dancers/extras, PSCs, soundstages, VFX, post-production companies, sound design, dubbing/subtitling, equipment rental, etc.).

One of the major selling points of the Greek incentives, concerns productions with over EUR8 million in local spend, providing for subsidies on non-resident labour (e.g., for scriptwriter and director fees), making the country a top destination for high-end projects. Latest data provided by EKOME (running the 40% cash rebate, 30% tax relief incentives and the National Network of Film Offices across Greece show a record EUR500 million in investments, and over EUR80 million returned to productions, in the past four years. As a result, major investors are eyeing Greece for a permanent presence, namely for the development of sound stages across the country.

In fact, David Cronenberg’s Cannes nominated Crimes of the Future, Ruben Östlund’s Palme D’ Or winner Triangle of Sadness, Netflix’s Knives Out sequel Glass Onion: A Knives out Mystery, Disney+’s Rise, Millennium’s Expendables 4, Bricklayer and The Enforcer, Apple TV+’s Tehran series (S1-S2) are amongst the 300 projects that have chosen Greece as their destination for production.

The inflow of foreign “GREECE IS AN capital has a huge INCOMPARABLE positive impact not only NATURAL AND on the domestic sector (production companies, actors, Greek workshop DIVERSE SET: UNIQUE, ACCESSIBLE, AND workers) but also on FRIENDLY. IT OFFERS other sectors of the A VARIETY OF Greek economy (tourism, OPTIONS THAT SERVE accommodation, catering, YOUR STORY.” transport, banking, insurance services). It is noteworthy that European cinema and in particular European co-productions have largely benefited through the Greek audiovisual investment incentives. With such an impressive track record, it is easy to assume that we are looking at 2023 as the year of Greece for all things audiovisual production.

FOR MORE INFORMATION CONTACT: Vasiliki Diagouma, Head Communication & International Relations EKOME vdiagouma@ekome.media

www.ekome.media