During traing process in carla town01, I get " IndexError: index 4082 is out of bounds for axis 1 with size 3470".

[MichaelChoum]

• OS: [Ubuntu 20.04]
• GPU: [RTX 4060]
• CUDA: [12.4]
• GPU Driver: [550.54.14.]

Hello, thanks a lot for your nice work! During I try to train a simple network with only carla town01 dataset, I face some problem. Everything goes right during data_generation and cost map build. But when I try to train the network with just the carla town01 dataset, I get " IndexError: index 4082 is out of bounds for axis 1 with size 3470".

The 3D point cloud is like:

The generated cost map is like:

And the shell outputs "cost map size set to: 3570 x 3470". I'm not sure whether the size is right, I just feel the builded cost map look fine.

And finally when I start to train, I get:

[Warning] Pre-trained ResNet50 models cannot be used since detectron2 not found
[Warning] Pre-trained ResNet50 models cannot be used since mask2former not found
[INFO] Trainer initialized
[INFO] Start Training
wandb: Currently logged in as: mmmicha. Use wandb login --relogin to force relogin
wandb: wandb version 0.18.7 is available! To upgrade, please run:
wandb: $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.12.21
wandb: Run data is saved locally in /home/znh/disk/logs/wandb/run-20241129_131759-iif0lv4m
wandb: Run wandb offline to turn off syncing.
wandb: Syncing run plannernet_envtown01_ep100_inputDepSem_costSem_optimSGD_combi_more_data
wandb: ⭐️ View project at https://wandb.ai/mmmicha/viplanner
wandb: 🚀 View run at https://wandb.ai/mmmicha/viplanner/runs/iif0lv4m
Available GPU list: [0]
Running on GPU: 0
[INFO] MODEL LOADED (71045584 parameters)
[INFO] OPTIMIZER AND SCHEDULER CONFIGURED
/home/znh/IsaacLab-1.2.0/_isaac_sim/exts/omni.isaac.ml_archive/pip_prebundle/torch/optim/lr_scheduler.py:60: UserWarning: The verbose parameter is deprecated. Please use get_last_lr() to access the learning rate.
warnings.warn(
[INFO] PlannerDataGenerator init with semantics=True, rgb=False for ENV town01
[INFO] Loading odom data... DONE!
/home/znh/miniforge3/envs/isaaclab/lib/python3.10/site-packages/pypose/lietensor/lietensor.py:918: UserWarning: Tensor Shape Invalid by calling <slot wrapper 'getitem' of 'torch._C.TensorBase' objects>, go to https://pypose.org/docs/main/generated/pypose.LieTensor
warnings.warn('Tensor Shape Invalid by calling {}, '
[INFO] Filter odom points within the inflation range of the obstacles in the cost map...DONE!
[INFO] odom points outside obs inflation : 9584 (47.92 %)
Traceback (most recent call last):
File "/home/znh/projects/viplanner/viplanner/train.py", line 42, in
trainer.train()
File "/home/znh/projects/viplanner/viplanner/utils/trainer.py", line 95, in train
self._load_data(train=True)
File "/home/znh/projects/viplanner/viplanner/utils/trainer.py", line 258, in _load_data
generator = PlannerDataGenerator(
File "/home/znh/projects/viplanner/viplanner/utils/dataset.py", line 461, in init
self.get_odom_goal_pairs()
File "/home/znh/projects/viplanner/viplanner/utils/dataset.py", line 635, in get_odom_goal_pairs
self.get_graph()
File "/home/znh/projects/viplanner/viplanner/utils/dataset.py", line 736, in get_graph
collision = occupancy_map[
IndexError: index 4082 is out of bounds for axis 1 with size 3470
Traceback (most recent call last):
File "/home/znh/projects/viplanner/viplanner/train.py", line 42, in
trainer.train()
File "/home/znh/projects/viplanner/viplanner/utils/trainer.py", line 95, in train
self._load_data(train=True)
File "/home/znh/projects/viplanner/viplanner/utils/trainer.py", line 258, in _load_data
generator = PlannerDataGenerator(
File "/home/znh/projects/viplanner/viplanner/utils/dataset.py", line 461, in init
self.get_odom_goal_pairs()
File "/home/znh/projects/viplanner/viplanner/utils/dataset.py", line 635, in get_odom_goal_pairs
self.get_graph()
File "/home/znh/projects/viplanner/viplanner/utils/dataset.py", line 736, in get_graph
collision = occupancy_map[
IndexError: index 4082 is out of bounds for axis 1 with size 3470.

And my file structure is like:

Could you plese tell me where the problem is. I've tried a lot but no worked out. Thank a lot! I think the main clue is "IndexError: index 4082 is out of bounds for axis 1 with size 3470.", could you please confirm the size of carla town01 cost map size is (3570, 3470) right? I think that may help.

[MichaelChoum]

And here is my learning_cfg.py:
learning_cfg.txt

And my costmap_fg.py:
costmap_cfg.txt

I've changed the setting according to the comment in these two file to suit the carla town01 dataset.