Render Texture using FBO

[Nyubug]

Hello!

So, my goal is to create a material that can do the following. Render a camera view and specify the collection of objects into an image in real time and display it on a plane object. The idea is to mimic 2D animation parallax effects by creating different layers such as a mountain and forest.

The Portal BSDF node sounds perfect on paper, but the problem is the node is limited to Cycles, doesn't have a way to isolate objects (unless we use geo nodes), and doesn't seem to have transparency allowing the portals to overlap each other properly.

For example.

So, I ended up making a python script that does the following. Sets the view layer of the isolated collection, draws the camera view into an offscreen, read the offscreen texture color, then set the pixels of an image.

This is the result of the python script. Each collection is rendered into its own image and displayed on a plane object. This is the result I'm aiming for. Animation playback is around 4 FPS for 7 images each at 1024 pixels which is very problematic. Technically the final render will be around 512 pixels because we are dividing the camera FOV in half to add overscan for parallax effects.

So those were my failed attempts, but I'm curious if this can be done in Malt with better performance. I found this website mentioning FBO and was wondering if this is possible to implement on a mesh material? Or perhaps there is another method and I'm just overthinking it.
https://www.songho.ca/opengl/gl_fbo.html

[pragma37]

You can certainly do this by rendering each layer to its own texture.

However if instead of that you scale down the mesh in the axis you want to flatten the object, and scale back the position used for shading, you get the same result with much less trouble (and much faster).

Here I'm skipping the scaling in the shadow pass, so shadows still work normally.
malt-parallax.zip

Note that there was a bug in the Transform node (actually I think is a driver bug) that I just fixed. So download the latest Malt release before opening the file.

[Nyubug]

Hmm.. I did see the Object Info node, but I didn't see an option to specify an object which led me to believe it only works on the object that is using the material. In geo nodes there is an input allowing you to choose an object.
I'm not familiar with how Malt works yet, It's a bit of a learning curve. 😐

[pragma37]

Ah, I misunderstood what you meant. Yes, the Malt's Object Info is limited to the object being rendered.
For other objects you would have to use drivers.

[Nyubug]

So even though I can preserve shaders after deforming the vertices by storing the vertex positions, how can I preserve other things such as shadows?

I found out that if I don't flatten some of the Suzanne's, the shadows are saved on the floor. But the moment I apply the vertex transformation to the Suzanne's the shadows are removed from the floor.

There could be other issues such as reflections(normals), bloom, line rendering, ambient occlusion, etc all of which might look different after transforming the vertices.

So, my question is how can I be sure that everything remains the same after transforming the vertices?
I still think the render texture method would be the best combined with the scene filter node to filter out collections into the texture. I just need a way to expose the camera view as a mesh shader node color output but I'm unsure where to begin.

render0000-1112.mp4

[pragma37]

So even though I can preserve shaders after deforming the vertices by storing the vertex positions, how can I preserve other things such as shadows?

On top of not flattening the mesh in shadow maps, you would have to displace it to its "original" position.
Speculars are view depended so you would have to compute them manually too.

I still think the render texture method would be the best combined with the scene filter node to filter out collections into the texture. I just need a way to expose the camera view as a mesh shader node color output but I'm unsure where to begin.

If you want to take that route, you can use this as a reference since it's somewhat similar:
#559 (comment)