Meta Movie Gen capable of producing 16-second clips

The announcement comes several months after competitor OpenAI unveiled its text-to-video model Sora.

Meta Movie Gen is not publicly available yet.

Samples of Movie Gen’s creations showcased by Meta included videos of animals swimming and surfing, as well as videos using people’s real photos to depict them spinning a record next to a cheetah.

Meta said Movie Gen is part of the third wave of its generative AI work, which started with the Make-A-Scene series of models that enabled the creation of image, audio, video, and 3D animation, followed by the Llama Image foundation models, which enabled higher quality generation of images and video, as well as image editing.

In a statement, Meta said: “Movie Gen is our third wave, combining all of these modalities and enabling further fine-grained control for the people who use the models in a way that’s never before been possible. Similar to previous generations, we anticipate these models enabling various new products that could accelerate creativity.”

The company said Movie Gen has four capabilities: video generation, personalised video generation, precise video editing, and audio generation. It explained that it had trained these models on a combination of licensed and publicly available datasets, and also shared more technical detail in a research paper.

It can generate videos of up to 16 seconds at a rate of 16 frames per second, and create audio clips of up to 45 seconds. Movie Gen can also make personalised videos. “We take as input a person’s image and combine it with a text prompt to generate a video that contains the reference person and rich visual details informed by the text prompt,” explained Meta.