Code for controlling Spot in simulation

[AayushManShrestha]

In the YouTube demo video, a Spot robot was being controlled via ROSA. I couldn't find the code used in the demo video. It would be helpful if code used to control a Spot robot in simulation was available. Thank you.

[RobRoyce]

Setting up ROSA for physical robot control is implementation-specific. That's one of the reasons we decided to be un-opinionated when it comes to uplink command and control.

For the demo, we basically gave ROSA an Xbox controller. It publishes joystick messages (axes and button presses) to the joy topic, which gets interpreted by the teleop_twist_joy node, which then publishes twists to cmd_vel. We also have a node that multiplexes any /joy publishers to ensure ROSA commands can always be overridden by human input (JoyMux will override ROSA commands when it detects ESTOP button on Xbox controller).

  graph TD;
      ROSA --> |/joy| JoyMux{JoyMux};
      XboxCtrl --> |/joy| JoyMux{JoyMux};
      JoyMux --> |/joy_cmd| TeleopJoy;
      TeleopJoy --> |/cmd_vel| Spot

Loading

---

Can you tell me how you're currently controlling the robot (e.g. publishing to /cmd_vel)? And which capabilities you're interested in from the demo? We will consider creating a default Spot embodiment if it can be generalized to cover most implementations.

Alternatively, you can check out the publish_twist_to_cmd_vel tool implemented for the turtle_agent, which can easily be adapted as a custom tool for your use case.

[AayushManShrestha]

I am currently controlling the Spot robot by publishing velocity commands to the /spot/cmd_vel topic, which are then processed by a ROS node to execute the desired movements using the Spot SDK. If you are considering to create a general implementation, then using Spot with arm would be better.

I am interested in the following capabilities:

1. Controlling Spot's movement using natural language instructions
2. Controlling Spot arm to manipulate object
3. Capturing visual data from the simulation environment
4. I don't know if this goes beyond the scope of the project but integrating VLMs would be nice