Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cross install seems to be broken #1215

Open
4 of 11 tasks
ctron opened this issue Mar 6, 2023 · 21 comments · May be fixed by #1216
Open
4 of 11 tasks

cross install seems to be broken #1215

ctron opened this issue Mar 6, 2023 · 21 comments · May be fixed by #1216

Comments

@ctron
Copy link

ctron commented Mar 6, 2023

Checklist

Describe your issue

cross build works like expected. But using cross install results in:

[cross] warning: unable to get metadata for package
[cross] note: Falling back to `cargo` on the host.
    Updating crates.io index
  Installing trunk v0.16.0
   Compiling libc v0.2.139
[…]
  --- stderr


  error occurred: Failed to find tool. Is `aarch64-linux-gnu-gcc` installed?


error: failed to compile `trunk v0.16.0`, intermediate artifacts can be found at `/tmp/cargo-installQKUg1O`

It looks like as it install just isn't supported, so it falls back to running "plain cargo", which of course fails.

However, running cross --help shows install too.

What target(s) are you cross-compiling for?

other (specify in description)

Which operating system is the host (e.g computer cross is on) running?

  • macOS
  • Windows
  • Linux / BSD
  • other OS (specify in description)

What architecture is the host?

  • x86_64 / AMD64
  • arm32
  • arm64 (including Mac M1)

What container engine is cross using?

  • docker
  • podman
  • other container engine (specify in description)

cross version

0.2.5

Example

cross install trunk --target aarch64-unknown-linux-gnu

Additional information / notes

No response

@Emilgardis
Copy link
Member

Not sure how we would support this, I guess we could make it error instead of falling back. Does that work?

@ctron
Copy link
Author

ctron commented Mar 6, 2023

Not sure how we would support this, I guess we could make it error instead of falling back. Does that work?

I guess the answer to that would be: no. 😁

Joking aside, I think it would be better to fail instead of leading to wrong results and "weird" errors.

Ideally, that should be supported. I am not sure how different that actually is. I can download a tarball from crates.io and build (cross build) it. That is at least a workaround.

@Emilgardis
Copy link
Member

my question then is, where do we put this binary?

Say I do cross install ripgrep --target armv7-unknown-linux-gnueabihf, where would we put this binary that'll not run on the host without something like qemu or another compability layer?

cargo install is basically sugar for

  1. cargo build --release
  2. mv ./target/release/binary $CARGO_HOME/bin/binary

I'm not sure I see the purpose of making this work the same was as cargo install.

we do have something like this in discussion for #716, but its not really the same thing and would most likely not work in that capacity.

@ctron
Copy link
Author

ctron commented Mar 8, 2023

I think there two steps before:

  1. Resolve the version (might be "latest" or some other sem-ver pattern)
  2. Fetch and unpack the tarball

Only then you can build and move.

But I agree, the "move" step isn't that useful. Unless you use --root to redirect the output to a different location.

@Emilgardis
Copy link
Member

Emilgardis commented Mar 8, 2023

I'm sorry, I still don't understand.

There's no need for us to do anything special, we just need to do allow cross install --target <target> crate we just need to add it here

pub enum Subcommand {

I guess my question is, what does cross install do? How is it useful? What is your use-case? My own expectation of a working cross install would be that it places the binary in whatever the default is (and/or overriden with --root), but then there is a conflict, this binary will probably not run on the host.

what would the following do?

$ cargo install ripgrep
....
Installed ripgrep
$ cross install --target armv7-unknown-linux-gnueabihf
....
???

I think the way to solve this is via cross-util compile --target <target> --output-dir . crate and failing on cross install (see #659)

@ctron
Copy link
Author

ctron commented Mar 8, 2023

There's no need for us to do anything special, we just need to do allow cross install --target crate we just need to add it here

Great, if that's so simple, then I think it should be done.

I guess my question is, what does cross install do?

It does what cargo install would do, just with a different target. Like cross does for cargo build.

My own expectation of a working cross install would be that it places the binary in whatever the default is (and/or overriden with --root), but then there is a conflict, this binary will probably not run on the host.

Yes, same as for cross build. The output will not run on the host (unless using some emulation).

My use-case: I want to cross compile binaries which are hosted on crates.io. Without the need to resolve the "correct" version first, and without the need to download a tarball. This is already solved by cargo install and I would like to re-use this.

I am no interested in placing this into the default host bin folder. That's the default with cargo install, if that could be disabled with cross install fine. If not, then maybe add a warning that the resulting binary might not work on the host.

But still, if one want to cross compile a binary publishes in crates.io, capturing the result in a different location using --root, I think this makes sense.

@Emilgardis Emilgardis linked a pull request Mar 8, 2023 that will close this issue
@ofek
Copy link

ofek commented May 10, 2023

This is affecting us actually. The use case is this https://github.com/ofek/pyapp#building

It's basically a wrapper for Python applications that is configured at build time with environment variables. Rather than making users check out the repo we instruct to just install which is much easier.

Has a fix for this been merged yet?

@Emilgardis
Copy link
Member

Emilgardis commented May 11, 2023

@ofek, what would you want cross install --target powerpc64-unknown-linux-gnu to do? I have implemented cross-util install in #1216 which does what one would expect, but differs from cargo install in that it requires a path to place the binaries in, since there is no reasonable way to handle cross install --target <target> in my opinion.

@ofek
Copy link

ofek commented May 11, 2023

I would expect the same interface as Cargo but make --root required

@ofek
Copy link

ofek commented May 11, 2023

I've documented this limitation for now ofek/pyapp#12

Hopefully this can be fixed soon 🙂

@ofek
Copy link

ofek commented May 11, 2023

I am unfamiliar with this code base, how difficult would it be to fix the install command and as you said require --root?

@Emilgardis
Copy link
Member

Emilgardis commented May 11, 2023

most of it has been done in #1216 , what's left in that pr is basically a rebase needed and making it consume a specific Cross.toml. I don't want to add it to cross directly, because of this

cross has the exact same CLI as Cargo but relies on Docker or Podman.

edit: it doesn't currently support --git or any other non-common cargo install args, that can be fixed by just passing them straight to cargo

@ofek
Copy link

ofek commented May 11, 2023

Oh, so it will never be possible to use the same install command interface i.e. the limitation I documented will forever persist when using the cross CLI?

@Emilgardis
Copy link
Member

Emilgardis commented May 11, 2023

correct, kinda, instead of cross you'd use cross-util which is installed alongside cross

I'm willing to change the way it works but I don't see a strong enough reason to do it

@ofek
Copy link

ofek commented May 11, 2023

My use case is essentially the same as the person who opened this issue, our workflow involves cross compiling crates that have been released. Other than the repository I showed you, another use case we have is because some crates come from repositories that are not public so one cannot clone them and build in the way you are prescribing.

@Emilgardis
Copy link
Member

Maybe I'm not explaining the difference well enough.

Instead of cross install, you do cross-util install

cross-util install --root <path> would invoke cargo install --root <path> in the docker container.

Other than the repository I showed you, another use case we have is because some crates come from repositories that are not public so one cannot clone them and build in the way you are prescribing.

what do you mean here? What would prohibit cross-util install from working?

@ofek
Copy link

ofek commented May 11, 2023

The majority of CI users do not install cross but rather pull down an existing binary from a release to save time and money. Most of these tools extract a single binary like this https://github.com/taiki-e/install-action/blob/main/main.sh (Here is a run for proof where I output the path to both and the executable you're talking about does not exist https://github.com/ofek/hatch-showcase/actions/runs/4953629165/jobs/8861304401#step:9:29)

Furthermore, it seems that most users are familiar with just the cross CLI and it would be unnatural to have every project that wants this functionality to document that you actually need a binary that you did not know existed.

@Emilgardis
Copy link
Member

Emilgardis commented May 12, 2023

that's a regression in install-action, it used to work. I'll see what can be done to fix it. cargo binstall cross works correctly, it gets the cross-util binary.

Furthermore, it seems that most users are familiar with just the cross CLI and it would be unnatural to have every project that wants this functionality to document that you actually need a binary that you did not know existed.

I think having cross install emit a warning and suggestion to use cross-util (and together with #661 make it a hard error) is fine, but I'm starting to wonder if maybe it is worth it to break the cross cli = cargo cli for this specific scenario of install. However, I still don't think the semantics match up as, I hope, I described in #1215 (comment)

in my mind, using install for this is a hack for not having access to --out-dir or wanting to skip a step of having to move the binaries or specifically spelling out the path to them. I don't see a scenario where a "regular" user would have to run cross install --target <target> and expect it to work on their system.

@ofek
Copy link

ofek commented May 12, 2023

We would very much appreciate it if the lack of exact semantics would be allowed for this specific case of the install command to keep a consistent experience!

@ofek
Copy link

ofek commented May 14, 2023

Also I would be fine with testing this feature or if you need help implementing you can tell me what to do

@Emilgardis Emilgardis added this to the v0.3.0 milestone May 14, 2023
@ofek
Copy link

ofek commented Jun 13, 2023

Is there anything I can do to help move this along?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants