How to modify the example to include libpg (postgres) ?

[marvin-hansen]

Hi, I found this repo and the great article while looking for a solution to cross-compile
a crate that uses Diesel and postges client with LLVM.

The postgres client depends on libpg and so the idea is to add libpg lib to a custom sys-root.

My preliminary cross-compile config for Linux aborts with an error:

  = note: ld.lld: error: unable to find library -lpq
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

That is expected, because the required libpg-dev is not installed on the host because it's
needed for ARM64 and Intel.

From reading the Dockerfile that generates the sysroot, I am wondering if a simple

sudo apt-get install libpg-dev

in the Ubuntu base image suffice to make libpg available to different platform sys-roots?

The repo:
https://github.com/marvin-hansen/bazel-diesel-postgres

[scasagrande]

Adding it to the base image would not be sufficient. You need to do something similar to what we do for teh RHEL libs: https://github.com/scasagrande/toolchains_llvm_sysroot/blob/main/Dockerfile#L162

The basic workflow would look like as follows. I have included links to the equivalent examples in the rhel section.

• make a FROM statement
• install libpg-dev
• make an isolated folder . You'll need to find all of the required files so you might need to do some manual exploring first.
• copy the files to an isolated location
• copy these files into your final sysroot_* image

[marvin-hansen]

Okay, thank you. Much appreciated. Let me try this tomorrow and see if I can bake a custom sysroot that makes my project cross compile. Again thank you for writing this up, it fills a real gap.

…

On Thu, Sep 12, 2024 at 22:10 Steven Casagrande ***@***.***> wrote: Adding it to the base image would not be sufficient. You need to do something similar to what we do for teh RHEL libs: https://github.com/scasagrande/toolchains_llvm_sysroot/blob/main/Dockerfile#L162 The basic workflow would look like as follows. I have included links to the equivalent examples in the rhel section. - make a FROM statement <https://github.com/scasagrande/toolchains_llvm_sysroot/blob/main/Dockerfile#L160> - install libpg-dev <https://github.com/scasagrande/toolchains_llvm_sysroot/blob/main/Dockerfile#L162> - make an isolated folder <https://github.com/scasagrande/toolchains_llvm_sysroot/blob/main/Dockerfile#L163-L164> - copy the files to an isolated location <https://github.com/scasagrande/toolchains_llvm_sysroot/blob/main/Dockerfile#L165> - copy these files into your final sysroot_* image <https://github.com/scasagrande/toolchains_llvm_sysroot/blob/main/Dockerfile#L185> — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFYR7XAA3MNO5EBMOTXVR23ZWGOE3AVCNFSM6AAAAABODKP7MKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANRSGY4DCMQ> . You are receiving this because you authored the thread.Message ID: <scasagrande/toolchains_llvm_sysroot/repo-discussions/2/comments/10626812@ github.com>

[scasagrande]

You're welcome! Keep an eye on this line and the --platforms flag. You'll need to remove it to generate a native x86_64 sysroot, or keep it for cross compiling to arm64.

[marvin-hansen]

Okay, makes sense. I’ve looked at the RH UBI image and apparently it doesn’t contain libpg. https://catalog.redhat.com/software/containers/ubi8/ubi/5c359854d70cc534b3a3784e?container-tabs=packages I suppose the next viable option would be to download the source and compile it in the build container?

…

On Thu, Sep 12, 2024 at 22:18 Steven Casagrande ***@***.***> wrote: You're welcome! Keep an eye on this line <https://github.com/scasagrande/toolchains_llvm_sysroot/blob/main/Dockerfile#L160> and the --platforms flag. You'll need to remove it to generate a native x86_64 sysroot, or keep it for cross compiling to arm64. — Reply to this email directly, view it on GitHub <#2 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFYR7XAKF5TZH275PDSFW4TZWGPDPAVCNFSM6AAAAABODKP7MKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANRSGY4TCNI> . You are receiving this because you authored the thread.Message ID: <scasagrande/toolchains_llvm_sysroot/repo-discussions/2/comments/10626915@ github.com>

[scasagrande]

You don't have to use rhel ubi as the image, you can use ubuntu:22.04 or whatever makes you happy. You're just using the image as a platform to download the files for you.

[scasagrande]

rhel ubi was used in the example to show how you can fetch a specific implementation and you're not restricted to ubuntu etc

[scasagrande]

libpq-dev is available in these repos:

https://pkgs.org/download/libpq-devel
https://pkgs.org/download/libpq-dev

[marvin-hansen]

@scasagrande Thank you.

As strange as it sounds, but I actually found a much simpler solution to the problem of cross compiling Diesel.

There is a module that vendored and patched lib-pg so that it statically compiles for all major platforms.

https://github.com/brainhivenl/libpq

With that, you basically just do three things:

1. Add the module:

bazel_dep(name = "libpq", version = "0.0.1")
git_override(
    module_name = "libpq",
    remote = "https://github.com/brainhivenl/libpq",
    commit ="ebdeb0cc9b8e051779e00b37ec0f93fab9ba8dd8",
)

2. Annotated the pq-sys crate:

use_repo(crate, "crates")

crate.annotation(
    crate = "pq-sys",
    build_script_env = {
        "PQ_LIB_STATIC": "1",
    },
    rustc_flags = [
        "-L",
        "$(BINDIR)/external/libpq~~_repo_rules~postgres",
    ],
    data = ["@@libpq~~_repo_rules~postgres//:libpq"],
)

3. Run bazel build //...

And it cross compiles Diesel / Postgres from and to all possible targets,
Mac to Linux, Linux to Mac, Arm to Intel, and intel to Arm. You name it. Totally bonkers.

I am baffled about the mind boggling simplicity but t.b.h it has just solved my problem.

That said, it is possible that I have to come back to the custom sys-root at a later stage
since not every sys dependency is as nice as lib-pg.

[scasagrande]

you'll want to change your target labels because you are using unstable labels that are not designed for direct reference

my suggestion to get around module visibility:

BUILD.bazel

alias(
    name = "libpq",
    actual = "@libpq",
)

and then your annotation, try disabling the build file generation, it will frequently break call its own toolchains and break out of your bazel ecosystem

crate.annotation(
    crate = "pq-sys",
    gen_build_script = "off",
    deps = ["@@//:libpq"],
)

[marvin-hansen]

Man this is brilliant.
No wonder you work in Quantum Computing.

I did the alias and that makes things a lot better.

crate.from_specs()
use_repo(crate, "crates")

crate.annotation(
    crate = "pq-sys",
    build_script_env = {
        "PQ_LIB_STATIC": "1",
    },
    gen_build_script = "off",
    rustc_flags = [
        "-L",
        "$(BINDIR)/@@//alias:libpq",
    ],
    deps = ["@@//alias:libpq"],
)

However, there is no way to build libpq without gen_build_script, so it is just one of those modules.
After looking into the repo, it seems the gen script downloads the source code from the postgres site and then applies patches.
I thought the sources would be vendored since that is what I would have done, but alas, there are many Bazel ways.

On that occasion, by chance do you know how to configure multiple crate universe?

Point is, Diesel with this static libpq cannot be vendored anymore for reasons I cannot figure out,
but afaik, rules_rust allows for multiple crate universe so the idea is to move Diesel with this libpq
into a separate crate universe with something like a sys_deps prefix.

if not no worries, I just wondered.

[scasagrande]

On that occasion, by chance do you know how to configure multiple crate universe?

two ways

first, use the experimental extension isolation flag: https://bazel.build/reference/command-line-reference#flag--experimental_isolated_extension_usages , and then in your MODULE.bazel you supply and extra kwarg like this

crate = use_extension("@rules_rust//crate_universe:extension.bzl", "crates", isolated=True)

I believe that's the correct kwarg. Then you can call it more than once and setup separate universe objects.

alternatively you have to create separate extension files with copy + paste content in order to force bazel to treat them as separate isolated extensions.

So you'd have:

crates_extension.bzl and crates_extension_2.bzl

load("@rules_rust//crate_universe:extension.bzl", "crate")

crates = crate

Then in your MODULE.bazel

crate = use_extension("//:crates_extension.bzl", "crates")
crate.from_cargo(
    name = "crate_index",
    cargo_lockfile = "//:Cargo.lock",
    manifests = ["//:Cargo.toml"],
)
use_repo(crate, "crate_index")

crate_2 = use_extension("//:crates_extension_2.bzl", "crates")
crate_2.from_cargo(
    name = "crate_index_2",
    cargo_lockfile = "//:Cargo_2.lock",
    manifests = ["//:Cargo_2.toml"],
)
use_repo(crate_2, "crate_index_2")

[marvin-hansen]

@scasagrande

Again, thank you so much; this basically solved it;

I ended up with using the isolate = True flag like so:

crate = use_extension("@rules_rust//crate_universe:extension.bzl", "crate", isolate = True)

And then just enable it in bazelrc

common --experimental_isolated_extension_usages

Meanwhile, the example repo builds and my main repo with multiple cross compile targets builds.

In retrospect, making Diesel and Postgres work took some extra steps; for once, there is an issue with the migration macro when the crate is non-root and then there is the cross compilation that takes some extra effort. That said, I didn't expect to see cross compilation without a custom sysroot so all things considered, this is actually a really good solution.

Again, thank you for your amazing help.