Skip to content

Commit

Permalink
Heroku-24: Add binutils to the run image (#306)
Browse files Browse the repository at this point in the history
Python's `ctypes.util.find_library` is used to find libraries at
runtime. `find_library()` calls out to various external programs to help
with this task, initially trying `ldconfig`, and then falling back to
`gcc` and `ld` if needed - as documented here:
https://docs.python.org/3/library/ctypes.html#finding-shared-libraries

Even after the removal of compilation tools from the run image in #273,
`ldconfig` still exists in the run image, so libraries can still be
found by `ctypes.util.find_library` in the base image.

However, whilst the `find_library()` docs say that it supports
`LD_LIBRARY_PATH`, the docs fail to mention that the env var is only
honoured when `ld` can be found:
https://github.com/python/cpython/blob/e870c852c0ea96fa4e4569e9c39c7ceb80ce858d/Lib/ctypes/util.py#L315-L320

As such, when only `ldconfig` is available, `find_library()` only
returns libraries from the base image, and not any installed via eg the
APT or other buildpacks (which rely upon `LD_LIBRARY_PATH` since they
have to do userland "installs" of libraries due to not having root
permissions).

One such popular use of `ctypes.util.find_library` is in Django's GIS
feature, which uses it to locate the GDAL library at runtime:
https://github.com/django/django/blob/4971a9afe5642569f3dcfcd3972ebb39e88dd457/django/contrib/gis/gdal/libgdal.py#L53-L66

Therefore, we sadly have to include `binutils` in the run image after
all - though thankfully doing so only forgoes 19 MB of the original
203 MB run image size reduction from removing `gcc`, `make` and
`libc6-dev`.

GUS-W-15821115.
  • Loading branch information
edmorley authored May 21, 2024
1 parent 05e8c01 commit ce3c7ee
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 0 deletions.
9 changes: 9 additions & 0 deletions heroku-24/installed-packages-amd64.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ bash
bind9-dnsutils
bind9-host
bind9-libs
binutils
binutils-common
binutils-x86-64-linux-gnu
bsdutils
bzip2
ca-certificates
Expand Down Expand Up @@ -62,6 +65,7 @@ libassuan0
libattr1
libaudit-common
libaudit1
libbinutils
libblkid1
libbpf1
libbrotli1
Expand All @@ -80,6 +84,8 @@ libcfitsio10t64
libcgif0
libcom-err2
libcrypt1
libctf-nobfd0
libctf0
libcurl3t64-gnutls
libcurl4t64
libdatrie1
Expand Down Expand Up @@ -120,6 +126,7 @@ libgnutls-openssl27t64
libgnutls30t64
libgomp1
libgpg-error0
libgprofng0
libgraphite2-3
libgssapi-krb5-2
libharfbuzz-gobject0
Expand All @@ -136,6 +143,7 @@ libicu74
libidn2-0
libimagequant0
libimath-3-1-29t64
libjansson4
libjbig0
libjpeg-turbo8
libjpeg8
Expand Down Expand Up @@ -216,6 +224,7 @@ libselinux1
libsemanage-common
libsemanage2
libsepol2
libsframe1
libsharpyuv0
libsmartcols1
libsodium23
Expand Down
9 changes: 9 additions & 0 deletions heroku-24/installed-packages-arm64.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ bash
bind9-dnsutils
bind9-host
bind9-libs
binutils
binutils-aarch64-linux-gnu
binutils-common
bsdutils
bzip2
ca-certificates
Expand Down Expand Up @@ -62,6 +65,7 @@ libassuan0
libattr1
libaudit-common
libaudit1
libbinutils
libblkid1
libbpf1
libbrotli1
Expand All @@ -80,6 +84,8 @@ libcfitsio10t64
libcgif0
libcom-err2
libcrypt1
libctf-nobfd0
libctf0
libcurl3t64-gnutls
libcurl4t64
libdatrie1
Expand Down Expand Up @@ -120,6 +126,7 @@ libgnutls-openssl27t64
libgnutls30t64
libgomp1
libgpg-error0
libgprofng0
libgraphite2-3
libgssapi-krb5-2
libharfbuzz-gobject0
Expand All @@ -136,6 +143,7 @@ libicu74
libidn2-0
libimagequant0
libimath-3-1-29t64
libjansson4
libjbig0
libjpeg-turbo8
libjpeg8
Expand Down Expand Up @@ -216,6 +224,7 @@ libselinux1
libsemanage-common
libsemanage2
libsepol2
libsframe1
libsharpyuv0
libsmartcols1
libsodium23
Expand Down
1 change: 1 addition & 0 deletions heroku-24/setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ apt-get upgrade -y --no-install-recommends

packages=(
bind9-dnsutils # For `dig`, `host` and `nslookup`.
binutils # Python's `ctypes.util.find_library` requires `ld` to find libraries specified via `LD_LIBRARY_PATH`.
bzip2
curl
file
Expand Down

0 comments on commit ce3c7ee

Please sign in to comment.