Facebook owner Meta announced on Friday that it has built a new artificial intelligence model called Movie Gen that can create realistic-looking video and audio clips based on user prompts, claiming it can work with leading media generation such as OpenAI and ElevenLabs tools for startups.
Meta said in a blog post that Movie Gen can also generate background music and sound effects that sync with video content, and use the tool to edit existing videos.
In one such video, Mehta had the tool insert pompoms into the hands of a man running alone in the desert, while in another, it transformed a parking lot where a man was skateboarding from dry ground into A parking lot covered in splashing puddles.
Meta’s Movie Gen AI model lets users edit videos with text prompts
Photo credit: Meta
Meta says Movie Gen can create videos up to 16 seconds long and audio up to 45 seconds long. It shared data showing that blind tests showed the model outperformed products from startups such as Runway, OpenAI, ElevenLabs and Kling.
The news comes as Hollywood has been grappling with how to harness generative AI video technology this year, after Microsoft-backed OpenAI first demonstrated in February how its product Sora could create movie-like videos based on text prompts.
Technicians in the entertainment industry are eager to use such tools to enhance and speed up film production, while others are wary of adopting systems trained on copyrighted works without permission.
Lawmakers also highlighted concerns about how artificial intelligence-generated fakes, or deepfakes, could be used in elections around the world, including the United States, Pakistan, India and Indonesia.
A Meta spokesperson said that the company is unlikely to release Movie Gen for open use by developers like the Llama series of large language models, and said that the risks of each model will be considered individually. They declined to comment specifically on Meta’s assessment of Movie Gen.
Instead, they said Meta is working directly with the entertainment community and other content creators on the use of Movie Gen and will incorporate it into Meta’s own products sometime next year.
According to a blog post published by Meta and a research paper on the tool, the company used licensed and publicly available datasets to build Movie Gen.
OpenAI has been meeting with Hollywood executives and agents this year to discuss possible partnerships involving Sora, although those talks have reportedly not resulted in any agreement. Concerns about the company’s practices intensified in May when actor Scarlett Johansson accused the makers of ChatGPT of impersonating her voice in its chatbot without permission.
Lions Gate Entertainment, the company behind “The Hunger Games” and “The Twilight Saga,” announced in September that it would allow artificial intelligence startup Runway to access its library of movies and TV shows to train artificial intelligence models. In return, studios and their filmmakers can use the model to enhance their work.
Follow us On Social Media Twitter/X
