Rendering + AI Experiment

2023

Been experimenting with how using diffusion based image generators (i.e. Stable Diffusion XL in this case) to cross the uncanny valley when creating photorealistic digital humans. Below are some renders I made using this workflow.

Here, this is the raw, untouched render straight from Unreal Engine 5's path-tracer. It looks very realistic, but it just looks slightly off and we know it isn't real. There are artifacts from the rendering such as the depth of field on her right shoulder and some polygonal shapes in some of her strands of hair .

Here, I enhanced the render with a very light layer of AI (added a subtle layer of noise that Stable Diffusion XL (guided by a prompt I designed) denoised) which almost nearly brings my render over from the uncanny valley. It adds small details like dust in her hair, denoises the image (noise from the path-traced rendering technique) and even adds details like tendons in her neck which I didn't even think to add. It's facinating how it understands the image and makes the proper adjustments to match what it knows a human, in this position, should look like. Another interesting thing is how it seems to "fix" the bokeh effect in the background creating more defined bokehs and also fixing an artifact on her right sholder from the render. However, there are new, but subtle, artifacts present now such as misshapen pupils.