Generative AI's Next Frontier Is Video

AI has made remarkable progress with still images. For months, services like Dall-E and Stable Diffusion have been creating beautiful, arresting and sometimes unsettling pictures. Now, a startup called Runway AI is taking the next step: AI-generated video. From a report: On Monday, New York-based Runway announced the availability of its Gen 2 system, which generates short snippets of video from a few words of user prompts. Users can type in a description of what they want to see, for example: "a cat walking in the rain," and it will generate a roughly 3-second video clip showing just that, or something close. Alternately, users can upload an image as a reference point for the system as well as a prompt. The product isn't available to everyone. Runway, which makes AI-based film and editing tools, announced the availability of its Gen 2 AI system via a waitlist; people can sign up for access to it on a private Discord channel that the company plans to add more users to each week.

The limited launch represents the most high-profile instance of such text-to-video generation outside of a lab. Both Alphabet's Google and Meta Platforms showed off their own text-to-video efforts last year -- with short video clips featuring subjects like a teddy bear washing dishes and a sailboat on a lake -- but neither has announced plans to move the work beyond the research stage. Runway has been working on AI tools since 2018, and raised $50 million late last year. The startup helped create the original version of Stable Diffusion, a text-to-image AI model that has since been popularized and further developed by the company Stability AI. In an exclusive live demo last week with Runway co-founder and Chief Executive Officer Cris Valenzuela, this reporter put Gen 2 to the test, suggesting the prompt "drone footage of a desert landscape." Within minutes, Gen 2 generated a video just a few seconds long and a little distorted, but it undeniably appeared to be drone footage shot over a desert landscape.

Read more of this story at Slashdot.