Simulon Adds Reality-Aware Rendering for Virtual Characters in Real Footage

If you've ever wanted to film a tiny dragon following you around the kitchen without learning a full VFX pipeline, Simulon just shipped something worth a look. Founder Divesh Naidoo announced Reality-Aware Rendering, a new way of making mixed real-and-virtual clips inside the Simulon app.

How it works

The idea is simple to describe. You place what Simulon calls neural assets (characters or objects) into a scene with your phone, then act out the moment yourself. Once you stop recording, the app renders the take by reading what happened in it: what you did, what you said, and the space around you. The assets then respond to all of that.

The part I find most interesting is that the finished clip is not your footage with a character pasted on top. According to the announcement, the final video is regenerated from the recording as a new performance of the scene. That's why the early examples people have been posting show characters reacting to the room and to the person filming, instead of just floating in front of the camera.

Availability

Reality-Aware Rendering is available now on iPhone, with Android listed as coming soon.

Why I'm watching it

For fan creators, this sits in a fun spot between cosplay skits, stop-motion and full 3D work. Being able to block out a scene by literally walking through it could make short character pieces much faster to try. The open questions are the ones that always come with generated video: how consistent characters stay across several takes, how much control you have when the regenerated performance drifts from what you intended, and how rights work for characters you didn't design yourself. I'd love to see a few longer, uncut tests before calling it a new standard, but as a first look it's a clever take on mixing real footage with imaginary characters.

Source: Divesh Naidoo's announcement on X.