[build] Fully retire the non-lcmtypes drake module

[jwnimmer-tri]

Towards #20731. See #21453 for prior art.

---

This change is 

[jwnimmer-tri]

+@xuchenhan-tri for feature review, please (low priority).

[jwnimmer-tri]

Reviewable status: 1 unresolved discussion, LGTM missing from assignee xuchenhan-tri(platform), needs at least two assigned reviewers

---

a discussion (no related file):
Working

Ugh, this breaks certain particular kinds of downstream projects (e..g, one of the drake-external-examples). I will need to rework this a little bit.

[jwnimmer-tri]

Reviewable status: LGTM missing from assignee xuchenhan-tri(platform), needs at least two assigned reviewers

---

a discussion (no related file):

Previously, jwnimmer-tri (Jeremy Nimmer) wrote…

Working

Ugh, this breaks certain particular kinds of downstream projects (e..g, one of the drake-external-examples). I will need to rework this a little bit.

OK fixed now, by reverting ~20% of it with some new comments.

Tested via:

cd drake-external-examples/drake_bazel_external
env EXAMPLES_LOCAL_DRAKE_PATH=/path/to/drake bazel test //...

[xuchenhan-tri]

with a big picture question.

Reviewed 4 of 7 files at r1, 6 of 6 files at r2, all commit messages.
Reviewable status: 1 unresolved discussion, needs at least two assigned reviewers

---

-- commits line 5 at r2:
Could you say why (perhaps in reviewable and not necessarily in the commit message).
I have a vague sense that this has something to do with lowered visibility of the packages? but I don't fully get it.

Code quote:

With bzlmod this is no longer practical.

[jwnimmer-tri]

Reviewed 4 of 7 files at r1, 6 of 6 files at r2, all commit messages.
Reviewable status: 1 unresolved discussion, needs at least two assigned reviewers

---

-- commits line 5 at r2:

Previously, xuchenhan-tri wrote…

Could you say why (perhaps in reviewable and not necessarily in the commit message).
I have a vague sense that this has something to do with lowered visibility of the packages? but I don't fully get it.

Previously, when the developer's WORKSPACE had the line workspace(name = "foo") the Python sys.path provided by rules_python would be organized to provide package names like from foo.common import thing.

Later, Bazel people decided this was a Really Bad Move and the paths should have been from common import thing. This is important because "name of an external" with bzlmod is more complicated now -- globally there can be multiple copies of an external, so "my copy of foo" is now a local concept, rather than part of a (necessarily global) sys.path.

This is also more consistent with non-Bazelized Python, where the parent of the directory you cloned the git repository into is never on your sys.path. Take for instance https://github.com/urllib3/urllib3, it has a subdir named src which is (notionally) on the sys.path, and then inside of src there is a directory named urllib3 with a file __init__.py, so users would do from urllib import .... In other words, for anything spelled as from xyz import ... there should be a directory in source control named xyz; the git repository's name to clone doesn't count, since that is not actually a constant (the user can name it anything they want). So that's how Python in the wider universe works, this is just bringing rules_python up to speed to match that.

[jwnimmer-tri]

+@SeanCurtis-TRI for platform review, please.

[xuchenhan-tri]

-- commits line 5 at r2:

Previously, jwnimmer-tri (Jeremy Nimmer) wrote…

Previously, when the developer's WORKSPACE had the line workspace(name = "foo") the Python sys.path provided by rules_python would be organized to provide package names like from foo.common import thing.

Later, Bazel people decided this was a Really Bad Move and the paths should have been from common import thing. This is important because "name of an external" with bzlmod is more complicated now -- globally there can be multiple copies of an external, so "my copy of foo" is now a local concept, rather than part of a (necessarily global) sys.path.

This is also more consistent with non-Bazelized Python, where the parent of the directory you cloned the git repository into is never on your sys.path. Take for instance https://github.com/urllib3/urllib3, it has a subdir named src which is (notionally) on the sys.path, and then inside of src there is a directory named urllib3 with a file __init__.py, so users would do from urllib import .... In other words, for anything spelled as from xyz import ... there should be a directory in source control named xyz; the git repository's name to clone doesn't count, since that is not actually a constant (the user can name it anything they want). So that's how Python in the wider universe works, this is just bringing rules_python up to speed to match that.

Thanks for the explanation. It helps somewhat but not completely. Do you happen to have some reference on this from bzlmod that you can share?

[jwnimmer-tri]

Reviewable status: 1 unresolved discussion, LGTM missing from assignee SeanCurtis-TRI(platform)

---

-- commits line 5 at r2:

Previously, xuchenhan-tri wrote…

Thanks for the explanation. It helps somewhat but not completely. Do you happen to have some reference on this from bzlmod that you can share?

Maybe these help:
bazelbuild/rules_python#1679
bazelbuild/bazel#18128

[SeanCurtis-TRI]

Reviewed 4 of 7 files at r1, 6 of 6 files at r2, all commit messages.
Reviewable status: 2 unresolved discussions

---

__init__.py line 1 at r2 (raw file):

# It brittle to have both drake-the-workspace and drake-the-lcmtypes-package

nit: grammar

(If you've ever seen the movie, "Murder By Death", you should be able to imagine Truman Capote shouting this.)

Suggestion:

It is brittle

[jwnimmer-tri]

Reviewed 1 of 1 files at r3, all commit messages.
Reviewable status: 1 unresolved discussion

[SeanCurtis-TRI]

Reviewed 1 of 1 files at r3, all commit messages.
Reviewable status: 1 unresolved discussion

[jwnimmer-tri]

Reviewable status: complete! all discussions resolved, LGTM from assignees SeanCurtis-TRI(platform),xuchenhan-tri(platform)

---

-- commits line 5 at r2:

Previously, jwnimmer-tri (Jeremy Nimmer) wrote…

Maybe these help:
bazelbuild/rules_python#1679
bazelbuild/bazel#18128

Happy to keep chatting, but for now I'll merge this to keep the train going.