Successful neuralsim and nr3d_lib version for street-view surface reconstruction

[haoyuhsu]

Hi everyone,

I have tested multiple versions of neuralsim and nr3d_lib on street-view surface reconstruction and encountered some issues due to version mismatches between the two packages. These issues include: (1) an "unrecognized key" error when loading the preprocessed dataset, and (2) excessively long compilation times for nr3d_lib.

After several attempts, I found that the combination of commit d0bee89 in neuralsim and commit b3d9627 in nr3d_lib worked for me.

I hope this helps anyone who might be stuck running the repo!

[lhp121]

@haoyuhsu Hello, may I ask which version of neuralsim you are using? Is it neuralsim-dev, neuralsim-0.5.1, or neuralsim-pre-releases? Thank you.

[haoyuhsu]

@lhp121 I am not sure what the neutralism versions mean. I simply checkout to the commit point b3d9627 in nr3d_lib.

[lhp121]

@haoyuhsu Okay, thank you very much! I have now installed the third-party libraries and can start training. I found the version of the neuralsim code you mentioned. However, I'm currently facing this issue that I can't resolve, and I wonder if you've encountered it before? AssertionError: If capturable=True, params and state_steps must be CUDA tensors. (Also, I'd like to ask if you are in this research direction, and can we exchange contact information to have more discussions?)

[haoyuhsu]

@lhp121 Do you remember to also checkout to the commit point d0bee89 in neuralsim? Unfortunately, I am not an expert in this research direction; I just use this repo as a surface reconstruction toolkit in my project.

[lhp121]

@haoyuhsu Yes, I have switched to the commit of neuralsim that you successfully ran. The previous error "If capturable=True, params and state_steps must be CUDA tensors." The author said it was due to the PyTorch version, and I resolved this issue by switching PyTorch from 1.13.0 to 2.0.0. However, when training with the data provided by the author, a new error occurred at 3%: y = ((y - prev) / (cdf_y[frame_ind, h] - prev) + h) / res_y Runtime Error: CUDA error: device-side assert triggered. I am currently debugging and making modifications, and I greatly appreciate your help and response!

[haoyuhsu]

@lhp121 For my project, I need to re-run the data preprocessing. I suspect there may be some discrepancies between the processed data provided by the author and the output generated directly by the code.

[lhp121]

@haoyuhsu Okay, thank you very much! Your guess makes a lot of sense because the data I used for training was directly downloaded from the authors without any preprocessing of my own. I feel that there are no issues with the logic of the code, and I am quite concerned about making changes to the code that might affect subsequent training. I will try preprocessing the data myself!