[RFC] ROS 2 + Flojoy

[IsabelParedes]

Note: In the following ROS refers to ROS 2

---

Target Users

• Roboticists:
  • Know how to program and use ROS
  • Have access to robots using ROS or a local ROS installation for simulations
  • Expectations:
    • Prototype ROS nodes with Flojoy
    • Access ROS data from Flojoy
      • Subscriptions to any ROS topic
      • (Optional) blocks for visualizing ROS data, similar to Foxglove studio
        • Camera images or other sensor data
        • Live plots
        • etc.
    • Control robots from Flojoy
      • Publishers to /cmd_vel for example
    • Create services and actions in Flojoy

---

ROS as a dependency

Options:

• Native ROS installation $\rightarrow$ point the user to the ROS installation instructions. The user would then use their system's package manager to install the binary packages if they are supported or install ROS from source.
• conda environment $\rightarrow$ the ROS packages needed to use the ROS blocks in Flojoy could be maintained in an environment.yaml file which Flojoy could use to create a conda environment for ROS during launch.

For non-standard ROS packages such as rxros2, it is not reasonable to expect the user to follow separate installation instructions and hope that it works with Flojoy. Instead, these additional dependencies should be updated and released (if permitted) so that they can be easily included in the conda environment.

---

ROS Blocks

Simulink implements a few ROS blocks which can serve as a basis for Flojoy.

Source: https://www.youtube.com/watch?v=BPj1bsnlDcQ

Blocks:

• Connect / Disconnect
• Publisher
  • User selects topic and message type
• Subscriber
  • User selects topic and message type
• Service client
• Service server
• (Optional) Action clients and servers

The idea would be that a Flojoy flowchart would represent one ROS node. The user could add subscribers to this node to retrieve data from the ROS network. The data could then be processed with any of the other blocks that Flojoy provides. And at the end of the flowchart, the user can chose to publish the processed data back to the ROS network.

A simplified example:
The flowchart has a subscriber block which subscribes to a /distance measurement topic; it receives measurements from a distance sensor on the robot. Other blocks (not ROS specific) are added to the flowchart to compare this distance to a constant. Say if the distance is less than $0.1m$, then the robot should stop moving. At the end of the flowchart, there would be a ROS publisher block. If the comparison outputs true, then the publisher will send a message to the /cmd_vel topic (or equivalent) to stop the robot.

---

(Optional) Code Generation

It should be possible to make the flowchart exportable as a ROS package. It would start off with a bare-bones package template with a single executable for launching the ROS node. As blocks are added to the flowchart, the template will be populated.

Once the user has a working flowchart in Flojoy, which has been live tested with a robot. This flowchart is exported as a package to be carried over and installed on the robot itself. This would be the ideal situation for prototyping ROS nodes with Flojoy.

[TheBigSasha]

Installation & Dependencies

It looks like all the ROS libraries we could find require ROS to already be installed, so it may be OK if there is a requirement for the user to have ROS installed. For the Python interop library side, we can support any Python library of your choice being installed, with the same annotaion syntax we had before.

[TheBigSasha]

Blocks:
Connect / Disconnect
Publisher
User selects topic and message type
Subscriber
User selects topic and message type
Service client
Service server
(Optional) Action clients and servers

This looks good, we can definitely work the types of message and topic in how we handle dynamic types in the stateful blocks RFC. We are still looking for how to do this best. One option is a getIns() / getOuts() / getParameters() methods in FJBlock which is used to get the types of the block's I/O, another way is to infer types at flowchart build time (so types are locked from the time the flowchart has run). We made a new issue / RFC for handling of flowchart typing and parameters - flojoy-ai/studio#1015

[39bytes]

It should be possible to make the flowchart exportable as a ROS package. It would start off with a bare-bones package template with a single executable for launching the ROS node. As blocks are added to the flowchart, the template will be populated.
Once the user has a working flowchart in Flojoy, which has been live tested with a robot. This flowchart is exported as a package to be carried over and installed on the robot itself. This would be the ideal situation for prototyping ROS nodes with Flojoy.

Do you have any plan as to how this might work? I was a bit confused reading the docs for ROS packages, and how they integrate with ROS nodes. From what I understand, a ROS package just specifies dependencies required for a ROS environment, then nodes are run inside of a package's environment? Then I guess we would (somehow) provide a frontend for managing the package's dependencies, and the ROS node for the python file would include the flow chart data + Flojoy runtime.

Otherwise, everything looks good! Sasha, Joey and I have a good roadmap for implementing this at the block level, see the stateful blocks RFC and types RFC mentioned by Sasha earlier in the thread; we will be working on pushing that out in the coming months.

[IsabelParedes]

Hi @JeffDotPng
Hopefully this clarifies a few things.

Do you have any plan as to how this might work? I was a bit confused reading the docs for ROS packages, and how they integrate with ROS nodes. From what I understand, a ROS package just specifies dependencies required for a ROS environment, then nodes are run inside of a package's environment?

• ROS package $\rightarrow$ Packages are the atomic build unit for ROS, a ROS installation consists of several ROS packages. A ROS package includes information about dependencies (other ROS packages) and can also include multiple ROS nodes.
• ROS node $\rightarrow$ A node is a runtime process that performs a particular computation. For example, a node can contain a publisher which publishes a sequence of even numbers to the topic /even_nums every other second.
• ROS workspace $\rightarrow$ this is just a directory where you add and install ROS packages.
  • When a user installs ROS initially, the user needs to source the workspace where the ROS packages are installed so that those packages become available to use; this step also sets the environment variables needed for using ROS. For our use case, the most important package coming from the ROS installation will be rclpy (ROS client library for python).
  • A user then creates a new workspace where they can add and install new ROS packages. This would be where the exported ROS package from Flojoy would end up.

Basic structure:

ros_workspace/
└── src
    ├── flojoy_ros_package
    │   ├── flojoy_ros_package/
    │   │   ├── flojoy_ros_node.py
    │   │   └── __init__.py
    │   ├── LICENSE
    │   ├── package.xml
    │   ├── resource
    │   │   └── flojoy_ros_package
    │   ├── setup.cfg
    │   ├── setup.py
    │   └── test
    │       ├── test_copyright.py
    │       ├── test_flake8.py
    │       └── test_pep257.py
    ├── other_pkg1
    └── other_pkg2

Then I guess we would (somehow) provide a frontend for managing the package's dependencies, ...

I'm not sure how easy it would be to implement a frontend for managing dependencies, but it may not be strictly necessary. Maybe the dependencies could be automatically added based on the blocks included in the flowchart. For example, if the flowchart has a ROS publisher block, that needs to depend on rclpy so this dependency can be automatically added to the package when exporting. Or if the flowchart has a block that subscribes to Twist type messages, then geometry_msgs could be automatically added to the package.

The dependencies will be greatly limited by the type of ROS blocks available. So it may not be necessary for the user to select the individual dependencies themselves.

... and the ROS node for the python file would include the flow chart data + Flojoy runtime.

Yes. We would need to find a way to populate the flojoy_ros_node.py file (above) with all this information.

Does Flojoy currently have any options for exporting regular flowcharts? Or are there any plans for this?

[39bytes]

Hi @IsabelParedes, thanks for the detailed explanation! Everything is pretty clear now. @TheBigSasha, @itsjoeoui, and I will be working on the stateful blocks/typing RFCs to lay the foundation for the ROS integrations in the next few weeks.

I'm not sure how easy it would be to implement a frontend for managing dependencies, but it may not be strictly necessary. Maybe the dependencies could be automatically added based on the blocks included in the flowchart. For example, if the flowchart has a ROS publisher block, that needs to depend on rclpy so this dependency can be automatically added to the package when exporting. Or if the flowchart has a block that subscribes to Twist type messages, then geometry_msgs could be automatically added to the package.

This makes a lot of sense, it would be nicer to hide this information from the user. I'm imagining a system where block developers could specify arbitrary metadata for a block (not just for ROS), and this would be a good use case.

Yes. We would need to find a way to populate the flojoy_ros_node.py file (above) with all this information.
Does Flojoy currently have any options for exporting regular flowcharts? Or are there any plans for this?

@izi-on has done something similar with current studio, which was used to run flowcharts on microcontrollers, I imagine we could do something similar with the reactive backend.