From Script To Distribution: AI's Role In The Film World
From R2-D2 of Star Wars to Ultron of The Avengers, the "reel" world has long been fascinated with AI. But in today's cinema, AI isn't just a character; it's a behind-the-scenes maestro.
From conceiving the idea to final distribution, the “AI in entertainment and media” market is projected to reach $132.16 billion by 2032.
Here's how AI is influencing multiple phases of filmmaking.
AI In Pre-Production
AI is applied in pre-production for script analysis, as exemplified by companies like StoryFit. It uses AI to analyze scripts, characters and audience responses, offering studios valuable insights.
For instance, StoryFit's analysis revealed communication patterns between male and female characters in films, providing insights for improving character dynamics and dialogue. They examined zombie-related films like HBO’s Last of Us to understand how socio-political events influence the horror genre, offering insights for filmmakers tackling current societal themes.
AI also supports the scriptwriting process, with tools like Pickaxe aiding writers in generating scenes by prompting plot and character descriptions into a text box. While the output may not be a final draft, it serves as a creative spark, providing inspiration.
Moreover, AI can predict key elements of a film's success, such as box office revenue and audience preferences. Warner Bros., for instance, collaborates with AI platforms like Cinelytic to guide choices on release dates, marketing and distribution. Cinelytic's forecasting proved accurate with Hellboy, closely aligning with the film's actual box office performance.
AI In Production
In cinematography, AI offers assistance in executing precise camera movements, scene optimizations for ideal lighting and composition, and suggesting adjustments for enhanced visuals and shot composition. AXIBO, for example, pioneers AI-powered camera automation, providing noise-free motion and diverse control options.
Another tool, Move.ai, employs AI for markerless 3D motion capture, eliminating the need for actors in traditional motion capture suits. This innovation is particularly significant as it allows remote motion capture.
AI enhances real-time special effects, like Massive software simulating lifelike crowd behaviors in movies such as the Battle of Helm's Deep in The Lord of the Rings. This technology uses AI to create realistic computer-generated armies, aiding cinematic battles.
AI In Post-Production
Post-production tasks, such as editing, special effects, audio editing and soundtracks, are often technical and time-consuming. AI has the potential to streamline these processes by automating repetitive tasks.
AI tools like Blackmagic Design's DaVinci Neural Engine leverage ML algorithms to adjust color tone, remove noise and track objects across footage, considerably reducing the time and effort required for post-production tasks.
Another notable AI tool, Runway ML is used by VFX artists for tasks such as inpainting, motion tracking, greenscreen, and object/background removal, streamlining manual processes. This was used in the award-winning film Everything Everywhere All At Once for background and visual effects.
Similarly, Wonder Dynamics' AI tool, Wonder Studio, automates actor-to-CG character replacement, managing animation, lighting and composition in real time. This innovation democratizes CGI and VFX for small budgets, freeing artists to concentrate on creativity over technical tasks.
De-aging actors is gaining popularity, with companies like Metaphysic providing AI alternatives to traditional VFX and CGI.
In a video showcasing a de-aged Harrison Ford in Cowboys & Aliens, AI tracked and diffused his face every 30th frame using Stable Diffusion, presenting a more efficient and cost-effective approach compared to CGI-heavy methods seen in films like The Irishman.
AI goes beyond visuals—it's handy for audio editing, too. Izotope's Neutron software, featuring Track Assistant and Mix Assistant, analyzes and creates custom presets to enhance audio tracks. This provides a solid starting point for audio mixers, significantly reducing editing time.
Marketing And Distribution
AI recommendation engines improve viewer satisfaction by personalizing content suggestions on platforms like Netflix. The company continually refines its algorithms through user data, studying viewing habits and analyzing platform interactions.
AI is also playing a role in trailer creation. For instance, in 2016, IBM Watson used AI to craft a trailer for Morgan by analyzing key film moments, suggesting soundtracks and refining edits. In 2019, Adobe Sensei analyzed past Sundance Film Festival winners, using AI to create an original short film that captured the festival's essence.
Ethical Considerations
While AI leads to benefits, from cost and time savings to improved outputs and aid in creativity, it is not without its concerns.
A significant concern is the potential bias in AI-generated content. AI algorithms learn from data, and if that data is biased, the content may reinforce stereotypes and marginalize certain groups. To address this, it's crucial to use diverse and representative training data.
Last year’s 118-day-long joint strike by SAG-AFTRA and the WGA brought attention to intellectual property rights and privacy issues. It focused on AI use and safeguarding actors and writers. Eighty-six percent of the union vote was in favor of AI use, but studios must obtain an actor's consent before digitally emulating them; actors will be paid for the days a digital replica is used in their stead.
Another major concern is determining ownership and royalties for AI-generated content since it challenges existing copyright laws. Recently, it was highlighted by the U.S. judgment announcing that AI-generated content cannot be copyrighted.
AI-generated storytelling sparks concerns about diminishing human touch, originality and creativity, prompting a crucial question: Can these narratives retain relatable flaws and uniqueness, resonating with our shared human experience?
What Comes Next?
As AI-generated short films and stories gain popularity and AI becomes more prevalent in mainstream post-production, it's clear that AI's influence in the film industry will keep growing, paving the way for further innovation.
For example, Pika Labs, an AI video generator, raised $55 million in funding, valued between $200 million and $300 million. The platform, initially focused on anime, now caters to a broader audience, generating AI-created videos based on text inputs.
Picture an AI-powered casting assistant, efficiently recommending actors based on project requirements, a video editor that can make most of your required changes by simply following text prompts or a camera that filters the background and adjusts lightning at the time of shooting using AI; the possibilities are endless.
