What is the use of Render Flow function ?

[sauradip]

Hi,

I observed Render Flow method in

SC-GS/train_gui.py

Line 1139 in 26cd57d

loss = loss + lambda_optical * optical_flow_loss

, and how is this 2D optical flow being projected back to 3D space ? I did not find this in the paper !

[yihua7]

Hello,

• The render_flow function related to what you mentioned is indeed optional and not typically used; it is primarily useful for flow supervision when RAFT flow or FlowFormer flow data is available.
• Rendered 2D optical flow would not be projected back to 3D but can be supervised by estimated optical flow.
• While this feature remains unhighlighted in our paper, we have made the implementation accessible in the hope that it might benefit those in the community who require it. Although another paper, GaussianFlow makes it one of the main contributions, we hold that the point is too tiny to build a contribution.
• Interestingly, some features like the dynamic mask and embedding for dynamic Gaussians are highlighted as significant innovations in the papers GauFRe and Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting. In contrast, these components were already utilized in our SC-GS framework but were considered too minor to be emphasized separately in our publications. A deeper look into our code would reveal that these are essentially the dynamic mask and hyper coordinate parameters.