Meta announces Movie Gen, an AI-powered video generator

A new AI-powered video generator from Meta produces high-definition footage complete with sound, the company announced today. The announcement comes several months after competitor OpenAI unveiled Sora, its text-to-video model — though public access to Movie Gen isn’t happening yet.

Movie Gen uses text inputs to automatically generate new videos, as well as edit existing footage or still images. The New York Times reports that the audio added to videos is also AI-generated, matching the imagery with ambient noise, sound effects, and background music. The videos can be generated in different aspect ratios.

In addition to generating new clips, Meta says Movie Gen can also create custom videos from images or take an existing video and change different elements of it. One example shared by the company shows a still headshot of a woman; the added video depicts her sitting in a pumpkin patch sipping a drink.

Movie Gen can also be used to edit existing footage and change the style and transitions or add things that didn’t previously exist. In one example shared by Meta, a relatively innocuous video of what appears to be an illustrated runner is edited using AI in different ways: in one frame, he’s holding pompoms. In another, the background has been edited to depict a desert. In a third, the runner is wearing a dinosaur costume. Changes can be made using text prompts.

Nearly two years after powerful AI image and video generators hit the mainstream, AI companies have pushed the technology further: in just the last six months, major tech companies like Google and OpenAI are working on similar tools, along with smaller startups. OpenAI’s Sora, first announced in February, still hasn’t launched publicly; this week, a co-lead working on the video generator left the company for Google.

Meta’s chief product officer, Chris Cox, writes on Threads that the company “[isn’t] ready to release this as a product anytime soon,” as it’s still expensive and generation time is too long.

AI image generators raised concerns about ownership and potentially harmful use cases, and AI video generators have only exacerbated them. Reports indicate that AI startup Runaway trained its video generator on thousands of scraped YouTube videos, something YouTube CEO Neal Mohan has already said would violate the platform’s terms of use. In a blog post, Meta says it trained Movie Gen on “a combination of licensed and publicly available datasets” but didn’t specify which.

Creatives like filmmakers, photographers, artists, writers, and actors also worry about how AI generators will affect their livelihoods, and AI has been a central part of several strikes, including the historic joint Hollywood strikes by the Screen Actors Guild - American Federation of Television and Radio Artists (SAG-AFTRA) and Writers Guild of America (WGA) last year.