AVR support

[japaric]

Parent issue: #2

Next steps:

• Wait until AVR support lands in upstream LLVM
• Update rust-lang LLVM fork (Upgrade to LLVM 4.0 rust-lang/rust#37609), implement cabi stuff, etc.
• Experiment with custom targets out of tree (to get the specification right)
• Get libcore compiling without modifications
• Upstream all LLVM patches required for the AVR fork (Upstream all of the patches for llvm-8.0svn support avr-rust/rust-legacy-fork#116)
• Write documentation about writing/compiling/flashing/debugging Rust
  programs for this target
• Add the AVR target to the compiler (as a built-in target) (tracking issue RFC: Merging the avr-rust fork upstream rust-lang/rust#44052)
• 🎉

See also avr-rust/rust for out of tree work towards AVR support in Rust.

[dylanmckay]

Here's an email I sent out to the llvm-dev list very recently regarding the status of the AVR backend.

tl;dr there's really only 3 files needed before the backend will be 80% usable.

[japaric]

Hey, @dylanmckay. I'm wondering what's the current status. I saw that there's only one "PR" (differential revision) in the review queue and that it has already been accepted but has not landed yet.

After that lands, will LLVM be ready to compile Rust programs for AVR? Will you (and @shepmaster) be sending a PR updating Rust's LLVM next?

Happy to help with anything to land this in rust-lang/rust. 😄 (I should finish landing my PR that moves rust-lang/rust to the WIP Rust port of compiler-rt intrinsics; I suspect that it may help here)

[shepmaster]

a PR updating Rust's LLVM next?

Eventually, yeah, except the LLVM 4.0 changes are more annoying than any previous LLVM changes I've had to make.

moves rust-lang/rust to the WIP Rust port of compiler-rt intrinsics

I think it will help. Especially because one "bug" we have at the moment is that I've forgotten to publish the WIP branch of compiler-rt so people are getting build failures.

[japaric]

Eventually, yeah

I'm really looking forward to LLVM 4.0 in rust-lang/rust so if you make it happen you'll have my gratitude.

the WIP branch of compiler-rt

Did you have to implement new intrinsics? I only see build system related changes in your fork.

[shepmaster]

I'm really looking forward to LLVM 4.0

What benefits do you expect to see from it?

implement new intrinsics

I think we might have to, but haven't yet.

[japaric]

What benefits do you expect to see from it?

I'm working on embedding lld, the LLVM linker, into rustc and having LLVM 4.0 lets me use the latest lld release which has better linker script support. (Linker script support in lld 3.9 is minimal) I'm hoping that someday I'll be able to build Rust programs for Cortex M devices without having to install arm-none-eabi-gcc first.

[dylanmckay]

I saw that there's only one "PR" (differential revision) in the review queue and that it has already been accepted but has not landed yet.

I'm about to commit that one now :)

After that lands, will LLVM be ready to compile Rust programs for AVR? Will you (and @shepmaster) be sending a PR updating Rust's LLVM next?

It should be able to compile very trivial instructions. There are two source files that will fix that.

• AVRBranchSelector - The branch relaxation pass (which will be rewritten to use the generic relaxation pass before it is merged)
• AVRExpandPseudoInsts - This is required for basically any 16-bit operation (hint: almost everything). I've put this one off until last because we can compile really basic examples without it. On top of that is that it is essentially a 2000 line hack full of code duplication.

[raphaelcohn]

@japaric Big shout out for lld embedded into rustc. It's not an easy thing to do - I know I'd struggle - but a rust toolchain that's entirely independent of GNU binutils and gcc is an enormous win for rust.

[japaric]

@raphaelcohn Check rust-lang/rust#36120. It's pretty much already working on Linux and it can link Cortex-M programs without arm-none-eabi-gcc. It hasn't landed because I couldn't get lld to build properly on Windows 😞

a rust toolchain that's entirely independent of GNU binutils and gcc is an enormous win for rust.

Indeed! If all goes well we'll also be able to (cross) compile statically linked MUSL binaries without needing glibc installed on the system as well. Ahh, the dream of the "zero installation steps" cross compilation setup may soon come true.

[dylanmckay]

• The machine code test suite is now fully passing and currently in review (D26387)
• About half of the code generation tests are fully passing and are also currently in review (D26388)

[japaric]

The AVR backend has now finally been merged in-tree completely. - @dylanmckay

🎉 Awesome work, @dylanmckay! 🎉

Now, it's time for the (Rust) compiler hackers (hey, don't look at me!) to roll up their sleeves and update the LLVM submodule to version 4.0 to enable this backend in rustc.

[japaric]

update the LLVM submodule to version 4.0

Check rust-lang/rust#37609

[dylanmckay]

There will still be a bunch of code generation bugs that'll cause the compiler to crash, but I/we can fix them as we get them, and I will then be able to cherry-pick the fixes into our fork.

[japaric]

Current status: rust-lang/rust has alread switched to LLVM 4.0 but there are still some bugs in the AVR backend that make it impossible to compile the core crate for the AVR architecture. (I bet 128-bit integers are to blame)

However, you can get away by using a smaller version of the core crate that does compile with the current AVR backend (plus some patches). If you are feeling adventurous, the process is documented in this blog post.

[shepmaster]

I bet 128-bit integers are to blame

One of many things, sadly. But we are making progress!

[blanham]

What's the current status of this? @japaric @dylanmckay

[dsvensson]

@blanham rust-lang/rust#44052

[japaric]

@dylanmckay if core didn't contain any 128-bit integer related code how many codegen bugs would still need to be fixed to be able to build core? If your answer is 0 then I'll try my best to convince the Rust team to temporarily cfg(target_arch = "avr") away all i128 code from libcore and let us enable the AVR backend in rustc.

I should also note that developments like pure MIR .rlibs (cf. rust-lang/rust#48373) may let us bypass the i128 codegen bugs. Since no machine code will be generated when compiling libcore.rlib no codegen bug can be encountered. Still you would probably hit the codegen bug when compiling the final application unless your application doesn't make use of 128 bit integers and rustc had some dead code elimintation pass at the MIR level (which afaik doesn't exist right now) that eliminates all i128 code before it reaches LLVM.

[dylanmckay]

@dylanmckay if core didn't contain any 128-bit integer related code how many codegen bugs would still need to be fixed to be able to build core?

I have rebased the avr-rust fork ontop of Rust master. There I've summarised the results here.

Aside from the removal of the i128 type, the only other change required to get libcore compiling as a static library was to remove the fmt string <-> float conversion routines (tracked by avr-rust/rust-legacy-fork#92).

The blink example fails to compile with LLVM 6 - avr-rust/rust-legacy-fork#37. That bug will need to be fixed before the AVR fork can be updated.

If your answer is 0 then I'll try my best to convince the Rust team to temporarily cfg(target_arch = "avr")

FWIW, it looks like libcore already understands the i128_type feature. In upstream libcore, disabling the feature does not actually disable all of its uses, because there are many missing cfg flags. I've added them in dylanmckay/libcore@6892809. The upside of this is it is much more likely to have it upstreamed using libcore's existing feature flag versus adding `cfg(target_arch = "avr)" all over the place.

[japaric]

FWIW, it looks like libcore already understands the i128_type feature.

That's just the feature gate that prevents you from using i128 on stable (#![feature(i128_type)]), right? I don't see any i128 related cfg attribute in the latest version of libcore; core doesn't have any Cargo feature either.

[dylanmckay]

@japaric good catch, that's my mistake.

I wonder if it worked for me because it always evaluated to true. Easy to replace with #[cfg(target_arch)] though.

[japaric]

@dylanmckay rustc moved to LLVM 7 a few weeks ago. I believe the last upgrade to LLVM 6 ended up increasing the number of codegen bugs encountered when building core. Was wondering if you had a chance to check if moving to LLVM 7 improved things or made them worse?

[brainlag]

There are at least 2 bugs left: avr-rust/rust-legacy-fork#101 and avr-rust/rust-legacy-fork#103

[shepmaster]

FWIW, we moved to LLVM 8 recently, thanks to @TimNN. We've been able to compile the entire core library (probably not using all of it though), removing the need for our special hacked libcore.

[ZhangHanDong]

One of the Rust 2019 Wish lists

[dylanmckay]

I've updated the checklist for merge in the description and raised #282 to track performing a Crater run against AVR.

[jplatte]

@dylanmckay I think you meant #282 ;)

[dylanmckay]

Whoops, is it still a ninja edit 25 minutes later? 🤔

[eldruin]

AVR support was merged into Rustc. Can this be closed now?

[therealprof]

Yes, let's close this.