Support Bazel Modules and avoid using internal dependencies

[aminya]

Is your feature request related to a problem? Please describe.

It would be great if Drake used Bazel modules to manage its dependencies and avoid using internal packages. There are many benefits to using bzlmod such as making it easier to use, having control over the dependency chain, and simplifying the build system of Drake. It would also prevent code duplication and large binaries for the users of Drake as Bazel will share those internal packages instead.

Bazel 7.0 by default uses bzlmod.

Describe the solution you'd like

Using bazel modules with dependencies from a regularly maintained external registry

https://registry.bazel.build/all-modules

Describe alternatives you've considered

Using Microsoft vcpkg
#17218

Additional context

[jwnimmer-tri]

We should switch to bzlmod at some point in the future -- it's intended to be the forward-looking workspace mechanism for Bazel.

However, at the moment it is a very new feature of Bazel, and we don't yet want to force a very-new minimum Bazel version upon our users. (We currently support down to Bazel 5.1, and Bazel 7.x is barely a month old currently.)

I imagine sometime in the next six(?) months we'll switch to require Bazel >= 7.0 as our minimum (after we drop Ubuntu 20.04 support), and that would be a good time to start poking at bzlmod.

[aminya]

Note that Bazel 5 supports Bazel modules too. Bazel 7 just enforces the use of modules unless --noenable-bzlmod is passed.

https://docs.bazel.build/versions/5.0.0/bzlmod.html

[jwnimmer-tri]

My impression is that there are a non-trivial number of important fixes and features related to bzlmod that have not been backported to 5.x nor 6.x, such that trying to use bzlmod in <7.x would be a headache. If you have experience to the contrary, I'd be happy to be corrected.

[aminya]

No, I haven't used those versions as I don't see a reason for sticking to old deprecated versions while it is so easy to upgrade Bazel on the user side.

[jwnimmer-tri]

... it is so easy to upgrade Bazel.

We'd welcome a PR with the Bazel 7 upgrade. I did not find it to be easy to get it to pass CI, so if it's easy for you, please go ahead!

[jwnimmer-tri]

Per #21054, Drake supports Bazel 7 now (and requires Bazel >= 6 at minimum).

[jwnimmer-tri]

Progress update:

I've started prototyping this, but the changes to workspace paths described in bazelbuild/bazel#18128 (comment) are causing trouble. Drake relied on e.g. drake.tools.lint being an importable module, but under bzlmod that doesn't exist. It's just tools.lint now. Surely we'll be able to find some kind of a solution, but it's not yet obvious to me which solution path will be best.

Also related bazelbuild/rules_python#1679 explains some of the questions well, and (in time) may have good tips for how to resolve this.

Another question will be announcing a deprecation timeline for when Drake can drop support for its WORKSPACE (forcing all downstream to switch themselves over to bzlmod as well). We will probably do the hybrid mode in the meantime, but it's not yet clear to me exactly how to do that in a way that allows bzlmod use downstream in a meaningful way.

[jwnimmer-tri]

Progress update: #22271 gets us to the point where Drake Developers can use bzlmod and all tests pass, with a MODULE.bazel file that lists some of our externals (but far from all of them). It still has a WORKSPACE.bzlmod file with the residual repository_rule externals, which we will need to incrementally prune down. The initial priority will be converting public dependencies to use bzlmod (starlark rules, fmt, spdlog, eigen, etc.), followed by the internal dependencies.

Soon I'll also need to announce some deprecation plan for Drake's workspace repository rules. Bazel 9 (scheduled for late 2025) will drop support for repository rules entirely, so that's the latest the window could stay open. I imagine we'll deprecate our own workspace to be removed sometime mid-2025.