Get your look in Comfy. Make it move in Veo. Fix your slops in comp

It’s important to me that gen Ai work is integrated & direct-able. There’s not much use describing your shot to Veo and letting it blend your instruction with it’s generic training data; you’ll never get what you want.

I started this test with Red camera log plates, courtesy of Action VFX. The mission was to insert zombies, but to generate them as an asset we can comp. Going full frame instant video is always going to look sloppy. We can compromise, retaining some creative freedoms by hiding the many flaws of gen Ai by ‘borrowing’ the quality of the pristine hi-end plates. This is gen Ai as bespoke asset generation.

You rarely find a ready made asset on the internet, with the correct perspective & texture. Mix up the look you want by art directing your asset in Comfy UI

First I got comfy to invent the correct kind of zombie image, then match the perspective, and finally tweak your descriptive prompt.

I wanted 4 zombies correctly set with perspective. You wont find a zombie plate with the correct perspective on google. Comfy’s Depth Anything gives Comfy a complete understanding of the spatial layout. This means you can drop nearly any zombie plate into your scene and the perspective will match your shot plate. Once you got the look you want in Comfy, use it to drive motion in Veo.

Mock up your compositions then ‘re-texture’ them

At first i grabbed any hand, and created the composition I wanted. I used Comfy’s IP Adapter to mix in the zombie skin, and backed of the strength until the composition remained unaffected.

The image on the left is generated only to show layout —the composition reference. The image on the right is the Comfy ‘re-texture.’ Note that the fingers do still move; precision isn’t easy

Lastly, slops. All current gen Ai has serious artefacts. In fact, it’s unlikely you could sit gen Ai next to pro camera plates in an edit — but we can try…

On the left, the image has been encoded & decoded to latent sapce. No work was done, but all gen Ai material will have this. On the right is the camera original.

This generative Ai hand moves with a little motion blur, whilst the camera plate is largely still but comes with high resolution ‘ambience’ in the form of soft focus rustling trees. I used the generative hand, and kept all of the camera back plate. It’s a win this time, but not every shot is this forgiving.

Sometimes gen Ai can provide the perfect asset. If we can use a little motion blur on the hand, then we can borrow the quality from the shot plate

Here’s the finished video, with a little track i wrote too. See you soon!