Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added segmentation based imitation learning model as a new brain #632

Merged
merged 8 commits into from
Oct 3, 2023

Conversation

Qi-Zha0
Copy link
Contributor

@Qi-Zha0 Qi-Zha0 commented Sep 12, 2023

No description provided.

@sergiopaniego
Copy link
Member

Thanks for opening the PR @Qi-Zha0 !

It integrates correctly with the rest of the codebase.
Running the config file provided, the vehicle drives slightly moved to the left of the lane and it's prone to crossing to the other lane. Is this the expected behavior?

If this is not the expected behavior, I recommend checking the camera position and vehicle compared to the one used to extract the dataset. Is it the same one and the camera position is exactly the same?

image

@Qi-Zha0
Copy link
Contributor Author

Qi-Zha0 commented Sep 17, 2023

@sergiopaniego Good catch! The drifting is not an intended behavior. It was because I did not correctly set the pitch values of the cameras. It took me a while to realize that ros-bridge uses a right-handed coordinate system while CARLA uses a left-handed coordinate system. Thus the pitch/yaw values should become -pitch/-yaw. I have updated the object file with the correct camera settings. Please let me know if there are any further issues : )

@sergiopaniego
Copy link
Member

Great work @Qi-Zha0!! The performance now is much better.
I've run into a small bug when trying to obtain evaluation metrics from the model in GUI mode. Running the evaluation pipeline (using the top-left play button) and stopping the recording of metrics after a while (using the top-left play button again), the software generates an error:
image
This error appears because the inference_times are not recorded but the variable is stated. Saving the inference times should solve the problem (https://github.com/JdeRobot/BehaviorMetrics/blob/a80c972664070988f3ca7df21125696aecbcc2c6/behavior_metrics/brains/CARLA/tensorflow/brain_carla_bird_eye_deep_learning.py#L156C3-L156C66). Can you take a look at it?

@sergiopaniego
Copy link
Member

Please update this branch with the latest updates from the main branch (noetic-devel).
I've separated the files from brains, configs, and models based on the simulator that they use since we're mainly using CARLA, and having files from gazebo can cause confusion.

@Qi-Zha0
Copy link
Contributor Author

Qi-Zha0 commented Sep 25, 2023

I've resolved the merge conflicts. Let me know if there are any further changes I should make. Thanks!

@sergiopaniego sergiopaniego merged commit de77dd2 into JdeRobot:noetic-devel Oct 3, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants