We Made a Film With AI. You’ll Be Blown Away—and Freaked Out.

02,06,25
We tried to direct an AI film with Veo and Runway. The tools are magic. The process is madness. By Joanna Stern and Jarrard Cole https://video-api.wsj.com/api-video/player/v3/iframe.html?guid=D17B233B-1E06-400D-9095-B5247306DD38 Welcome to the premiere of “My Robot & Me.” Please silence your phones, chew your popcorn quietly and remember: Every visual you’re about to see was generated with AI. Most of […]

We tried to direct an AI film with Veo and Runway. The tools are magic. The process is madness.

Joanna Stern

By Joanna Stern and Jarrard Cole

https://video-api.wsj.com/api-video/player/v3/iframe.html?guid=D17B233B-1E06-400D-9095-B5247306DD38

Welcome to the premiere of “My Robot & Me.” Please silence your phones, chew your popcorn quietly and remember: Every visual you’re about to see was generated with AI. Most of the audio too, except my voice.

Some of it’s totally wild. You won’t believe that no real cameras were used. Some of it, you’ll laugh at, because it’s clearly not real. I promise you, I did not have facial reconstructive surgery between scenes.

But enough from me. Hopefully by now you’ve watched the film above, complete with its behind-the-scenes look. Just come back—we’ve got some lessons to share.

Yes, we. To make this film, I teamed up with Jarrard Cole, a real human producer. We met over a decade ago here at WSJ, experimenting with helmet cams and new video formats like VR. These days, he’s become obsessed with AI video tools.

So I challenged him to make a totally AI video. How hard could it be?

Very hard.

Over a thousand clips, days of work and who knows how much data-center computing power later, we ended up with a three-minute film—about my life with a new kind of efficiency robot. Even if you don’t care about camera angles or storyboards, you might care about what this says about using AI in any job.

#1 You can make anything.

Just a few years ago, an AI-generated clip of Will Smith eating spaghetti went viral because it was terrifyingly bad. Now, these tools can pop out scenes that look nearly flawless—at least at first glance. 

After testing a bunch of options, we landed on Google’s Veo and a tool from startup Runway AI. They gave us the best mix of quality and control. OpenAI’s Sora was nowhere near as good. On May 20, Google dropped Veo 3, which adds AI-powered audio, including dialogue and sound effects. Check out Will Smith now!

Yep, if you can think it, you can generate it. A podcasting baby, a SWAT team storming a bounce house, a futuristic Mayan metropolis. But we weren’t going for sight gags—we wanted to tell a story, something with characters, humor and meaning. That proved to be a lot harder.

#2 You still put in the work.

Think you can paste in a script and out pops a Netflix hit? Cute. Every shot of ours was the result of many prompts and generation attempts. And to keep characters and sets consistent from scene to scene, Jarrard invented a whole production pipeline.

The quick version: We used AI image generator Midjourney to generate our sets (a suburban neighborhood, a newsroom) and to design our robot star. Then we used photos of real me to create AI me. We uploaded those to Runway or Veo, where we wrote prompts. Here’s a short one:

Low angle shot: Joanna does push-ups at a brisk pace, maintaining a straight line from head to heel. The robot stands above, monitoring and guiding.

That careful, specific wording made a huge difference. As a filmmaker, Jarrard could break down scenes beat by beat, specifying camera angles, lighting styles and movement. That nail-biter ending? Every shot was carefully described to build suspense.

And it still took us over 1,000 clips. Some were complete disasters, with anatomical nightmares and random new characters. Even in “good” scenes, my face looks different in almost every shot.

#3 You need human creativity.

There’s a popular term for AI-generated content these days: slop. And yeah, our film has some slop vibes. Some shots are overly smooth and parts look undeniably fake. But if we did our jobs right, it still hopefully accomplished what a film should. Maybe it made you laugh, maybe it made you think.

SHARE YOUR THOUGHTS

How do you think AI tools will impact the creative process? Join the conversation below.

And we did it without a massive budget, props departments and special-effects teams. The total cost would have been a few thousand dollars for Google and Runway’s AI video tools. (We paid for some of it, and the companies gave us special access to the rest.)

As two seasoned video producers, we can say AI opens up new ways to create things we never could before. But it can’t replace the craft.

These tools are nothing without human input, creativity and original ideas. As the film hopefully reminded you, we are not robots. Live a little.

Write to Joanna Stern at joanna.stern@wsj.com

Source: https://www.wsj.com/tech/ai/ai-film-google-veo-runway-3918ae28?st=Wm38QL&reflink=article_imessage_share

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram