Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Support "CLS": safely defining arbitrary CPU-local variables (theseus…
…-os#1028) * This commit allows any crate to safely define a CPU-local variable of its choice without needing to include it in a hardcoded struct containing all CPU-local variables. * Previously, the hardcoded approach was used, via the `PerCpuData` struct, but this was inflexible and required some unsafety to define and access a CPU-local variable. * Now, CPU-local variables, or CLS (Cpu-Local Storage), are declared using the `#[cpu_local]` procedural macro, which is implemented in the `cpu_macros` crate. * This macro declares CLS a `#[thread_local]`s (TLS), but places them in a custom link section `.cls`, allowing us to differentiate them from standard, real TLS variables. * These special CLS variables are not accessed using the standard TLS instructions, but rather we implement special accessors for them in order to ensure they use the arch-specific base registers for CPU-locals, not thread-locals. * CPU-local variables are accessed as an offset from the `%gs` segment register on x86_64, and the `TPIDR_EL1` register on aarch64. * The linker scripts have been changed to place the `.cls` section adjacent to the TLS sections, either before or after depending on the target architecture. * Then, in an extra build step, we run the `elf_cls` tool on the ELF object files outputted by the compiler in order to fixup the CLS sections' flags, symbol types, and symbol values. For example, we use an OS-specific ELF section/symbol type for CLS variables in order to properly identify them at runtime. * Separate control flows have been added to `mod_mgmt` to handle loading & linking CLS sections at runtime. * See the crate-level documentaiton of `cls` for more info. * The logic previously in `tls_initializer` is now re-used for CLS, because this new CLS implementation uses an augmented version of traditional TLS. * Thus, `tls_initializer` has been moved into the new `local_storage_initializer` crate, which offers a generic implementation with two specific instantiations: TLS and CLS. * The new `cls_allocator` crate handles allocations of CLS variables and reloading the CLS regions and CLS base register as necessary. Signed-off-by: Klimenty Tsoutsman <[email protected]> Co-authored-by: Kevin Boos <[email protected]> b1a0a14
- Loading branch information
