Install overview might be missing some things.

[gedalia]

Environment

• OS Version: Ubuntu 22.04
• Source or binary build?
  Top of tree

Description

• Expected behavior: build would go a bit better
• Actual behavior: not so much

Steps to reproduce

Hi I started trying to build this repro from scratch and hit a bunch of errors, wasn't sure if things had just gotten stale or if ubuntu 22.04 was behaving unexpectedly, I tried this on 2 different machines and hit slightly different issues.

• Things I might add to the readme:
  pip3 install -U vcstool
  or sudo pip install -U vcstool

• It seems not super obvious where this repo goes.
  I wound up doing
  git clone https://github.com/gazebosim/gz-bazel.git
  inside of the gz folder, and then having to rename it to bazel.

• gz/sdformat/BUILD.bazel is currently failing for me :
  needed
  load("@gz//bazel/lint:lint.bzl", "add_lint_tests")
  load("@gz//bazel/skylark:gz_py.bzl", "gz_py_binary")
  and I had to removed them from
  load( "@gz//bazel/skylark:build_defs.bzl"

• failed to compile gz/msgs/src/Generator.cc needed to change:
  for (const auto name : ns)
  to:
  for (const auto & name : ns)

The package install script seemed to fail for me on dart, wound up using a shell script:

packages=(
  ./sdformat/.github/ci/packages.apt
....
)
# Iterate through the package lists and install packages
for package_list in "${packages[@]}"; do
  sudo apt-get install -y $(cat "$package_list")
done

• I still needed to install many packages, not sure how these got missed:
  sudo apt-get install libtinyxml2-dev libgts-dev libfreeimage3 libfreeimage-dev libtinyxml2 libtinyxml2-dev libavcodec-dev ffmpeg libavformat-dev libgdal-dev libswscale-dev libfmt-dev libyaml-dev freeglut3-dev libzmq3-dev libzip-dev libjsoncpp-dev
  Needed for dart example
  sudo apt-get install libopenscenegraph-dev openscenegraph

*had to build and install nlopt, it then wound up in /usr/local/include and local lib. the symlink in the bazel externals cache was then invalid.

Thanks for providing this starting point getting gazebo to run with bazel, hopefully these bits might make it easier for the next person to build it.

[mjcarroll]

Hey, I'm in the process of updating the bazel build across the board for garden and harmonic. I will update the instructions to incorporate your suggestions here.

[gedalia]

I now have things building but I'm a bit confused. I'm a bit new to building gazebo and I'm trying to grok the overall purpose of this repo, I was expecting a bunch of top level bazel targets that would establish header file paths and library linking info, I'm gradually realizing that this repo seems independent of the core gazebo sim code in https://github.com/gazebosim/gz-sim. I've got a number of gazebo plugins that are building with bazel that I'm trying to upgrade from classic to ignition, but I'm realizing that those headers seem to be in gz-sim. My apologies if this isn't the best forum for questions wasn't sure where else to ask.

[mjcarroll]

This is the correct place.

When we moved from gazebo classic to new gazebo, part of the work was to break the monolithic code of gz-sim into the component pieces that could be reused in different places. To maintain that while also supporting bazel, this repo was created, as well as BUILD.bazel files were added to each of the libraries.

So how this should look

your_ws/
   WORKSPACE
   BUILD
   bazel
   math
   common
   ...
   sim

The full list of repositories and their branches are kept in the example/bazel.repos file. At this point for garden, this is supporting up to transport/fuel_tools. There is a PR here: #61 that adds sensors, rendering, and gui.

There needs to be one more PR to add sim support. I have it working locally but still need to add a PR.

Once your workspace is checked out and build with bazel, you should be able to do something like the following.

bazel build //transport
bazel build //sim

bazel test //msgs/...

The packages.apt mechanism exists so we can just re-use the package dependencies from upstream Ubuntu. It would have been too tedious to provide BAZEL build files for each of our dependencies, as Gazebo has quite a few.

I haven't considered an example of building plugins with bazel, but I could also add that for you if it helps.

[gedalia]

Thanks for the info, away on vacation this week, but I'll take a look at that PR.

[bd-mdelasa]

@mjcarroll thanks for the context/details.

Out of curiosity, is there a reason you've opted to managed repos/branches with vcs instead of bazel's built-in mechanisms (e.g., git_repository, etc.) for doing the same? We've found the ability to manage many repos as externals to be very powerful.

In terms of landing #61 and supporting a full bazel-based build of gz-sim, how far do you estimate things are?

I'm open to learning about where gaps remains and what you see as the areas needing some assistance/TLC.

Thanks!

[mjcarroll]

Out of curiosity, is there a reason you've opted to managed repos/branches with vcs instead of bazel's built-in mechanisms (e.g., git_repository, etc.) for doing the same? We've found the ability to manage many repos as externals to be very powerful.

Namely because I was actively developing on all of those repos. The git_repository rules seem to be really effective if you are pulling in a dependency at a specific revision/tag, but aren't very helpful if you want to also work on the contents of the repo.

Using VCS gets you closer to "mono-repo" approach that seems to be favored for developing with bazel.

[mjcarroll]

In terms of landing #61 and supporting a full bazel-based build of gz-sim, how far do you estimate things are?

Probably not far off with a little bit of attention. I want to get the remaining updates landed on garden branches before the harmonic feature freeze next week.

[bd-mdelasa]

Out of curiosity, is there a reason you've opted to managed repos/branches with vcs instead of bazel's built-in mechanisms (e.g., git_repository, etc.) for doing the same? We've found the ability to manage many repos as externals to be very powerful.

Namely because I was actively developing on all of those repos. The git_repository rules seem to be really effective if you are pulling in a dependency at a specific revision/tag, but aren't very helpful if you want to also work on the contents of the repo.

Using VCS gets you closer to "mono-repo" approach that seems to be favored for developing with bazel.

Makes sense. In case it's useful, I've found --override_repository to be useful in those cases.

Longer term, do you expect the workflow to continue to use vcs? Perhaps it's too soon to tell?

[bd-mdelasa]

In terms of landing #61 and supporting a full bazel-based build of gz-sim, how far do you estimate things are?

Probably not far off with a little bit of attention. I want to get the remaining updates landed on garden branches before the harmonic feature freeze next week.

Thanks for the context. It sounds like the vision is to have harmonic support a native bazel-based build. Is that correct?

[mjcarroll]

It sounds like the vision is to have harmonic support a native bazel-based build. Is that correct?

Correct!

[mjcarroll]

Makes sense. In case it's useful, I've found --override_repository to be useful in those cases.

I'm unfamiliar with that, I will take a look.

Longer term, do you expect the workflow to continue to use vcs? Perhaps it's too soon to tell?

Probably too soon to tell, I'm open to alternatives. vcs is a pretty common tool in the ROS ecosystem, so it's generally available.

[gedalia]

it appears that the commit can be a branch name as well,
git_repository(
name = "",
remote = "",
commit = "origin/branch_name",
)
commit is fed to: git reset --hard COMIT_STRING
so it can support a sha or a branch name.

[gedalia]

Tried the git_repository approach here:
WORKSPACE.txt
utils works:

bazel test @utils//...

but most of the other packages are unhappy due to wanting BUILD files.

[gedalia]

I suspect all the GZ_ROOT + "subdir_name" paths are now incorrect.

[moriarty]

Oh hey @gedalia and @bd-mdelasa 😄 what a small world...

I've tried twice in the past couple of weeks to get this to build without success.

[moriarty]

There are a few weird things going on that are missing from the install overview @mjcarroll is the README up for suggestions?

Bouncing between the gitlab ci files and the readme and mixing in your two open PRs I've gotten some things compiling to some state but not finished... and the where it's failing is likely on my specific env setup... I'll try on another clean Ubuntu machine when I get a chance... this is just a personal side project

[mjcarroll]

is the README up for suggestions?

Of course! I also started working on migration to bzlmod since it's the future (I'm told), but after running into multiple dependency issues, I back-burnered it for a little while. I would like to get back to working on that at some point, but I don't forsee it happening before ROSCon in October.

Any contributions you would like to put forward, I'm happy to review and take though.