Questions about PoseNet

[DevikalyanDas]

Hello, thanks for your great work,

I want to know where is the PoseNet CNN that you used to train the sheep. The pre-trained weights are available but I am not finding the network that uses these weights. Can you please point out where is the training pipeline of the PoseNet as I need it to train a 3D model in a similar setup?

Thank you.

[DevikalyanDas]

Ok it is the 'nerf_root_rts' in banmo.py in line 332. I wanted to know if you have released the random viewpoint generation and random masks augmentation part?

[gengshan-y]

Hi, the training forward function is here, where render_dp(...) contains the data augmentation.

The posenet training is activated here once you pass warmup_pose_ep > 0 (i think we used 10) and pose_cnn_path as "".

Let me know if there are further questions

[DevikalyanDas]

Hello, thanks for your reply. I have one more question about this setup of posenet, if I use Dino rendered dino feature image instead of CSE rendered image, will this setup work?

[gengshan-y]

Note here the assumption to train the posenet is we have a mesh, where each vertex has a K-dimensional semantic features, as learned in the CSE paper.

If you have a geometry (either volume/3dgs/mesh) with learned 3D dino fields, this setup can be applicable.