A diffusion based generative AI in theory isn’t doing any of the same processing to determine the end result. So it’s just effectively eyeing it and saying “this looks right” and nailing what’s a non-trivial rendering technique.
I’d hung out with a vision machine learning researcher at Nvidia for a while discussing broad concepts around what was necessary to render for the human eye and what was not.
But I’m increasingly wondering as the tech progresses if we may end up reaching an inflection point where generative AI for complex scenes ends up being effectively cheaper than actually rendering the scene from scratch in the first place. Almost like a fidelity instead of geometry focused version of foveated rendering where the game software spits out a scene with no frills and a dedicated AI unit in the GPU generates a ray-traced, subsurface scattered, anti-aliased frame that looks awesome and yet wasn’t accurately rendered at all.
NGL I was thinking the same thing. That 700+ dimensional latent space is a powerhouse. Bryan Catanzaro, Vice President of Applied Deep Learning Research at Nvidia, talks a bit about this.
The lighting in particular is great in this. The transparency of the ear and nose.
They call that subsurface scattering in the biz’.
Which is part of why it’s crazy.
A diffusion based generative AI in theory isn’t doing any of the same processing to determine the end result. So it’s just effectively eyeing it and saying “this looks right” and nailing what’s a non-trivial rendering technique.
I’d hung out with a vision machine learning researcher at Nvidia for a while discussing broad concepts around what was necessary to render for the human eye and what was not.
But I’m increasingly wondering as the tech progresses if we may end up reaching an inflection point where generative AI for complex scenes ends up being effectively cheaper than actually rendering the scene from scratch in the first place. Almost like a fidelity instead of geometry focused version of foveated rendering where the game software spits out a scene with no frills and a dedicated AI unit in the GPU generates a ray-traced, subsurface scattered, anti-aliased frame that looks awesome and yet wasn’t accurately rendered at all.
NGL I was thinking the same thing. That 700+ dimensional latent space is a powerhouse. Bryan Catanzaro, Vice President of Applied Deep Learning Research at Nvidia, talks a bit about this.
Great link! Appreciate it. And yes, very exciting to think about.