Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pydrake: Reduce size occupied by modules #10288

Open
EricCousineau-TRI opened this issue Dec 20, 2018 · 10 comments
Open

pydrake: Reduce size occupied by modules #10288

EricCousineau-TRI opened this issue Dec 20, 2018 · 10 comments
Assignees
Labels
component: pydrake Python API and its supporting Starlark macros priority: medium type: cleanup

Comments

@EricCousineau-TRI
Copy link
Contributor

EricCousineau-TRI commented Dec 20, 2018

At fdd7553, in the //:install tree, pydrake occupies 112MB of space out of 566MB. For comparison, libdrake.so occupies 26MB.

This amount should be reduced, either via #8433 or by reducing the overall module count.

Will increase priority if this starts to become a bottleneck.

EDIT (2019-03-23): Benchmarking stuff:

$ cd drake
$ git rev-parse --short HEAD
361223416
$ rm -rf ./build/py2
$ bazel run //bindings/pydrake:install -- ${PWD}/build/py2
$ find ./build/py2 -name '*.so*' | xargs du -hsc
...
114M    total

$ rm -rf ./build/py3
$ bazel run --config=python3 //bindings/pydrake:install -- ${PWD}/build/py3
$ find ./build/py3 -name '*.so*' | xargs du -hsc
...
121M    total
@jamiesnape
Copy link
Contributor

Does not seem unreasonable to me. Backlog?

@EricCousineau-TRI
Copy link
Contributor Author

Not yet, depending on what solution we want to take for #10408.

@jwnimmer-tri
Copy link
Collaborator

@EricCousineau-TRI could you re-check the object code sizes, now that #10350 has merged?

@jamiesnape
Copy link
Contributor

jamiesnape commented Jan 15, 2019

Not yet, depending on what solution we want to take for #10408.

I don't think #10408 changes anything. 112MB of 556MB is not unreasonable, and to be honest ~224MB of ~668MB is equally not unreasonable.

@EricCousineau-TRI
Copy link
Contributor Author

All sounds good; knocking down priority to backlog. Will still briefly check sizes, probably tomorrow though.

@jwnimmer-tri
Copy link
Collaborator

If we ever pick this up again, we should try https://github.com/google/bloaty, e.g.,

bloaty -C full -d segments,sections,shortsymbols,symbols $(find bazel-bin/bindings/pydrake -name \*.runfiles -prune -o -name _objs -prune -o -name \*.so -print)

@jamiesnape
Copy link
Contributor

Aside: Running the aforementioned McBloatface on libdrake.so is kind of interesting too (drake::math::GrayCodeToInteger() comes in at almost 20% of the size of the binary on Mac, next highest is 3%).

@EricCousineau-TRI
Copy link
Contributor Author

Posted a brief benchmark script to run from source, specific to these artifacts. Haven't used McBloatface, yet.

Briefly tried using namespace pydrake __attribute__((visibility("hidden"))), but that wasn't the easy fix - size is almost exactly the same :(

$ git diff
diff --git a/bindings/pydrake/pydrake_pybind.h b/bindings/pydrake/pydrake_pybind.h
index cdfa1b5d1..da87675be 100644
--- a/bindings/pydrake/pydrake_pybind.h
+++ b/bindings/pydrake/pydrake_pybind.h
@@ -7,7 +7,7 @@
 // for) these binding capabilities.
 
 namespace drake {
-namespace pydrake {
+namespace pydrake __attribute__((visibility("hidden"))) {
 
 /** @defgroup python_bindings Python Bindings
 @ingroup technical_notes

$ ... # Ran benchmark above for Python 3:
121M    total

@jwnimmer-tri
Copy link
Collaborator

For the record, this is a lot better now that I've been folding modules together. I would advise we finish that first (getting to the point where we only have two shared libraries -- libdrake.so + libpydrake.so), and only after that we could investigate hidden symbols or other linker optimizations within libpydrake.so.

@jwnimmer-tri
Copy link
Collaborator

The change in #20465 helped somewhat (5%). Ongoing module-folding is on deck to continue to improve.

Also #20296 will help a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: pydrake Python API and its supporting Starlark macros priority: medium type: cleanup
Projects
None yet
Development

No branches or pull requests

3 participants