Get your look in Comfy. Make it move in Veo. Fix your slops in Flame
It’s important to me, as Flame op, that my gen Ai work is integrated & direct-able. There’s not much use describing your shot to Veo and letting it blend your instruction with it’s generic training data; you’ll never get what you want.
I started this test with Red Camera Log plates, courtesy of Action VFX. The mission was to insert zombies [it’s what the plates wanted], but to keep what was shot too. Going 100% full frame instant video is always going to be extremely low quality; the trick is knowing where to hide it. This is gen Ai as ‘asset generation’ - the objective is to hide the many flaws and mix in the benefits with your pristine hi end plates.
Mix up the look you want in Comfy UI, create the image you will never find on the internet. Force the right perspective and texture — use it to art direct your video
Adjust the balance of the images you mix. Once you’ve nailed the look, Veo becomes a lot more fun
I wanted 4 zombies correctly set with perspective. You wont find a zombie plate with the correct perspective on google. Comfy’s Depth Anything gives Comfy a complete understanding of the spatial layout, when you got that, you can drop any zombie plate into the mix and it will magically fit your shot plate. Without this plate you have nothing specific to drive to Veo. You can do some of this with Photoshop. Comfy is fine control.
Re-texture Your Mock-up Compositions
At first i grabbed any hand, and got the layout right. I used Comfy’s IP Adapter to mix in the zombie skin look. Once you have the look you want, we are still largely reliant on gen Ai engines like Google Veo for motion.
Left is the hand composition reference. Right is after ‘re-texture’ in Comfy
Lastly, slops. All gen Ai has serious artefacts. It makes a lot of sense to carve up your generated clips and combine with camera original where possible.
On the left, the image has been encoded & decoded to latent sapce. No work was done, but all gen Ai material will have this. On the right is the camera original.
Sometimes gen Ai can provide the perfect asset. If we can use a little motion blur on the hand, then we can borrow the quality from the shot plate