[RFC][bazel] Make python interpreter and pip dependencies usable in repository rules #25697
Conversation
Currently, it is quite tricky to use the python interpreter in the repository rules (like in bitstream_workspace.py) because the repositories created by rules_python are not really meant to be used in the loading phase. With blzmod, this has become quite cumbersome and so this commit introduces a clean mechanism to do so. The first is to create an extension. This extension has a single method to register a python interpret and some packages which are import by pip. The extension will then create a repository containing a single executable which acts as python interpreter taking care of all the detais. Strictly speaking, this could have been achieved without an extension but it would then require to manipulate the pythonpath at every call site and create the required labels every time. By putting everything into an extension, we get to do it once and make it very easy to use everywhere. Signed-off-by: Amaury Pouly <[email protected]>
This python script ideally requires the jsonschema package. Signed-off-by: Amaury Pouly <[email protected]>
pamaury
changed the title
pamaury
changed the title
|
I should note that although I think this is a nice "hack", I still have mixed feelings about it because it depends on semi-documented repository names used by rules_python. Looking at the code of rules_python, I am not very worried because they have to run python script and run into the same problem: this is why the python_host repository exists in the first place and why there is one repo per package. However, the fact that those repo are visible and the names used does not seem to be documented as stable. Another remark: with the extension, it's possible to to do several registrations steps so that each "python interpret" has its own set of packages enabled. What this means is that if we want to, we don't have to created one interpreter which has access to the union of all dependencies of all users. Still, the fact that we need to manually list all transitive dependencies makes this process cumbersome anyway. |
See the commits. The goal of this PR is to define a clean way of using the python interpreter defined by rules_python, as well as the pip dependencies, but the repository rules. This required a bit of trickery and pythonpath manipulation. The rationale for using an extension is detailed in the commit.
A side benefit of all this is that the interpret is also usable (for testing, debugging, whatever) using bazel run:
If you try it with this PR, you'll notice that you can
import jsonschemabut notimport hsjonbecausehjsonis not listed inREPO_RULES_PIP_DEPS