Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SKILLs] Spot open drawer #1803

Open
wants to merge 54 commits into
base: main
Choose a base branch
from
Open

[SKILLs] Spot open drawer #1803

wants to merge 54 commits into from

Conversation

jimmytyyang
Copy link
Contributor

Motivation and Context

Training low-level skills for Spot to open drawers

This PR introduces several important functions
(1) Use gaze approach to determine if Spot open the drawers or not
(2) Update the opening drawer's function

How Has This Been Tested

Running interactive_play.py to verify its functions.

Types of changes

  • [Development] A pull request that add new features to the habitat-lab task and environment codebase. Development Pull Requests must be small (less that 500 lines of code change), have unit testing, very extensive documentation and examples. These are typically new tasks, environments, sensors, etc... The review process for these Pull Request is longer because these changes will be maintained by our core team of developers, so make sure your changes are easy to understand!

Checklist

  • My code follows the code style of this project.
  • I have updated the documentation if required.
  • I have read the CONTRIBUTING document.
  • I have completed my CLA (see CONTRIBUTING)
  • I have added tests to cover my changes if required.

Jimmy Yang added 26 commits January 13, 2024 22:08
make place spot task
0c8e654
place the object at the receptale. Use this to check if the object is…
2f3c1b7 
… properly placed
consider x,y location
aa2a66a
add image pre-train features
4dccb03
add goal textual features
d78ea08
add a flag to force the model run in the cpus to prevent out of memor…
50155ee 
…y issue
change model from float to float16
94a0c93
refine #1: add ee location and orientation function
db7cd34
refine #2: add snap down function to place the target
a0e75eb
refine #3: add orientation reward/sensor/measurement for the ee
174c535
refine #4: init the arm pose based on fine-grained pick
64f10c0
refine #5: refine the place yaml
c834e75
refine #6 add semantic yaml
2af66fb
fix sensor name
38e872c
refine #7: add check for orientation when dropping the object, add me…
38eefe3 
…trics for object_at_goal
refine #8 fix snap_down_height_diff round issue
4145cb0
refine #8: add _prev_obj_at_receptacle to register prev distance
b3b3bcc
bug fixed for the place reward: change from self._config.ee_orientati…
334ee17 
…on_to_initial_threshold != -1 to self._config.ee_orientation_to_initial_threshold == -1
add textual info for target object that the robot is holding
b5a214b
refine #9: add semantic sensor for the receptacle; add receptacle ids…
10c1a29 
… in sim
refine #10 add hssd and replica_cad training yamls with obj feat and …
1cad55a 
…rep mask; fix task.object_id
refine #1: add object to target sensor with the new yaml
8e76e4a
refine #2 add jaw camera
3f879eb
add jaw sesnor
5d8cc12
refine #3: add jaw semantic sensor
fbc38f2
refine #4 remove unnessary sensors
f38b50b
@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Feb 10, 2024
@jimmytyyang jimmytyyang changed the base branch from main to spot_semantic_place February 10, 2024 22:27
@jimmytyyang jimmytyyang changed the base branch from spot_semantic_place to main March 25, 2024 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed Do not delete this pull request or issue due to inactivity.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jimmytyyang @facebook-github-bot