Support OCaml 5.2 #134
Support OCaml 5.2 #134
Conversation
To contextualize the Solo5 side, I found:
|
|
Thank you for the pointers. How to properly setup interrupt handling on x86_64 is quite some rabbit hole! To sum up what I learnt:
So I’ve dropped the commit disabling the red zone. |
|
Dear @shym, thanks for rebasing. Would you mind to mark it as ready for review? (Unless you still have some things you want to address first -- but in the meeting today it sounded like this is ready from your side) |
|
I’ve rebased on |
|
I dug deeper into the usage of Finally, I also updated the PR to OCaml 5.2 RC1, released on Friday. |
|
Thank you, sorry I've still not managed to test :( |
|
I’ve updated the PR to OCaml 5.2.0 as it was released last week. Note: I’ve reverted to depend on the |
|
Thank you @shym for that update! I can confirm that this PR compiles and qubes-mirage-firewall compiles & runs as with 4.14.2 (first fast.com tests show around 10-15% more bandwidth now but this wasn't conducted with a strict evaluation protocol :) ). I'll also try with longer runs. |
|
On my side, I approve this PR. It took time for several people and I think the review of the various maintainers and authors has been done. Thank you @shym for taking over and considering what may have blocked us in our previous PRs, I consider that this one can be merged 👍. /cc @mirage/core |
|
I also approve this PR, but as it breaks the supported compilers versions, and I think we also need to keep a 4.4 branch corresponding to the current head before merging :) |
|
I’ve just updated the patches to the compiler so that it contains the patches that are already integrated upstream and in the version integrated upstream: it will make it much easier to re-synchronize when updating the patches for the next version.
|
|
I merged the other PRs outstanding, Would you mind to rebase? Also, the cirrus CI should now that solo5 0.9 is released, work fine!? |
|
I rebased on |
|
I think you can remove the pin now, and depend on >= 0.9.0... so, get rid of 95b7fbd should be fine. then the question is: 5.2.1 support? 5.3? what should we target with an initial merge and release? |
|
another remark: we used to depend on the ocaml-src package, now with e6caff2 there's (instead?) and extra-spirce introduced. why? from my point of view, keeping the unnecessary changes minimal would be appreciated. if you're eager for such a change, could you propose such a pr for the main branch? |
README.md
Outdated
| @@ -71,8 +71,7 @@ Run: `solo5-hvt _build/solo5/main.exe` | |||
|
|
|||
| ## Supported compiler versions | |||
|
|
|||
| Tested against OCaml 4.12.1 through 4.14.1. Other versions may require | |||
| changing `configure.sh`. | |||
| Tested against OCaml 5.0. Other versions may require changing `configure.sh`. | |||
Add two scripts to create an .install file:
- `gen_ocaml_install.sh` drives (bends...) OCaml's `make install` in order
to generate two subfiles, namely `_build/ocaml.install.lib{,exec}`,
containing the lines to install the compiler itself, into the `lib`
and `libexec` sections
- `gen_dot_install.sh` generates the complete `.install` file, including
the content of the subfiles for the compiler
The generated `.install` file diverges from the current installation
process in that it tries to follow the standard organisation of the OPAM
directory: the OCaml cross-compiler is installed in `%{_:lib}%` (ie
`%{prefix}%/lib/ocaml-solo5`) instead of `%{prefix}%/solo5-sysroot`.
Note that the paths fed to `configure.sh` must be consistent with that
for the installed compiler to work as expected.
In the standard use case, namely installation through OPAM, this target is no longer necessary, since a `.install` file is generated so that OPAM does the installation per se internally Remove install.sh
Co-authored-by: Fabrice Buoro <[email protected]>
Rephrase the comments to catch up with the evolutions of `signal.h` Co-authored-by: Fabrice Buoro <[email protected]>
Follow a suggestion by Christiano Haesbaert, as Solo5 doesn't provide signals, it might just be ignored As of OCaml 5.2.0 beta1, `sigfillset` is only used in functions creating threads, and so that will abort in any case. Ignoring the function may make it more future-proof, though.
Make extra sure that, if `pthread_create` is called, the caller will know it failed
Since the main Makefile doesn't know the dependencies for the nolibc and openlibm libraries, create intermediate phony targets. That scheme ensures, even when one library already exists, that: - when building a target that depend on that library, a recursive make is invoked to check that it is up-to-date, - if the recursive make doesn't update the library, the dependency on the library is not a reason to rebuild the target.
OCaml calls mmap with a non-null address and with the `MAP_FIXED` flag only on already reserved memory to commit or decommit that memory block, ie to set its protection to `PROT_READ|PROT_WRITE` or to `PROT_NONE`, in the `caml_mem_commit` and `caml_mem_decommit` functions So we accept this particular case without allocating memory that would leak since the OCaml code base simply ignores the returned value (as `MAP_FIXED` enforces the returned value to be either `addr` or `MAP_FAILED`) Accordingly, remove the man page extract that says that `addr` is just a hint since OCaml always sets `MAP_FIXED` when `addr` isn't null
`strerror_r` is used by the runtime to render some system errors so implement it to get useful information in such cases
`strdup` is not used in the OCaml runtime, as it is explicitly reimplemented to use the runtime allocator
I rebased and dropped what had to be dropped. I think this is good to go for me. |
|
thanks. looks fine to me. should we merge and release? |
|
From my POV yes :) 👍 Maybe we can also create a branch for the 4.14 version ? |
Merged, thanks for all you. It was a really collaborative work, which is always very satisfying 🎉. I made a branch as you asked @palainp. Thanks again, it was a huge work! |
This PR brings support for OCaml 5.2 (5.2.0~beta2 at the moment).
It proposes to do so by revising substantially how the cross compiler is built and so, unfortunately, drops support for OCaml 4.x compilers.
This builds upon the great work that has been done in the following previous PRs:
and so I chose to branch it from #129’s branch, before the changes to the memory allocator (which are not necessary for OCaml 5.2), and rebased it all on
main. All the comments in those PRs have been taken into account.This PR is marked as draft as it should be merged only when OCaml 5.2 is released but it can be tested in a 5.2.0~beta2 switch.
Main changes
To support OCaml 5.2, this PR extends
nolibcas necessary to build the runtime. Most of those extensions are inherited from those previous PRs.Apart from that, the main changes of this PR are as follows.
Patches to the compiler sources
This PR proposes to change from the scheme of modifying the OCaml build system by
seds andechos and instead brings a series of (version-specific) patches inpatches/that are applied on the sources when they are fetched.There two main reasons for this change.
git format-patchpatches, so they contain a full explanation message for each change) makes them easier to review. (In my review of the previous PRs, the modified Makefile of the compiler was broken somakewas generating warnings, even if it was still able to build the compiler; using actual patches makes it easier to make sure what modified build system we will actually rely on)One interesting outcome of this deep revision of the compiler build system is the fact that the
.optversions of the compilers are built, which should give better performance, in particular to build large unikernels.Open question for those patches: I kept
git format-patchdefault of using a binary diff (forconfigure, as it is declared binary in the compiler) as what is useful to review is the changes toconfiguresources that are always bundled in the same patch.git format-patch -apatches would be bigger (twice the size, with the current patch series) but they might be preferred for other reasons.A specific toolchain
The package now creates and installs a
{aarch64,x86_64}-solo5-ocaml-*toolchain, in a similar fashion to what Solo5 does. The main reason for creating that toolchain in the first place was to avoid baking the build-time directories containingnolibcandopenlibminto the generated OCaml compilers. Two versions of the toolchain are generated: one with the build-time directories, which is added toPATHduring the build of the compiler; one with the final destination directories, which is installed inbin.Besides making it possible to change the
-Iand-Lbetween build-time and final versions, I further used this idea to:ccall its other required options (such as-include _solo5/overrides.h), since this machinery is rather well-placed there,aarch64-solo5-none...*?aarch64-linux-gnu-*? ...). This fits in nicely with OCaml configure’s toolchain discovery so that stating the--targetis enough to define all the tools of the toolchain.Installation
This PR proposes to delegate the actual installation to OPAM (or
opam-installer) by creating aocaml-solo5.installfile rather than invoking theinstallcommand itself. The initial reason for this choice is that it is fairly easy to bend OCaml’smake installinto generating the list of all the files it could install and then tell OPAM to install them only if they were actually built. So we can skip building parts of the compiler (ocamllex, etc.) just to keepmake installhappy when they don’t make sense.This comes with the natural consequence that it follows OPAM’s conventions when installing files, so the compiler is installed in
%{prefix}%/lib/ocaml-solo5instead of%{prefix}%/solo5-sysroot(I find following the OPAM convention a rather good idea anyhow). If we relax the name of the findlib toolchain, it becomes possible to install simultaneouslyocaml-solo5andocaml-solo5-cross-aarch64(in my testing, I didn’t switch between the two as much as I would have if they had been co-installable), since the only common file between those two packages is the findlib’ssolo5.conf.Smaller changes
Quite a few commits are clean-ups for things that were natural to fix while working on this (such as ensuring that
makewill rebuildnolibcandopenlibmlibraries iff that should be done). I feel that those small changes will make maintenance smoother.Another change from previous PRs is the choice to drop
stdatomic.hfromnolibcand go with the one of the underlying compiler, to make sure the C compiler will compile C atomics as it is intended to do. This only needs a small patch to Solo5 for FreeBSD, to pull it along with the other freestanding headers when installing the Solo5 toolchain.Testing
This has been tested on:
dns-resolverhvt mirage unikernel,stdatomic.hheader, see Fetchstdatomic.hfrom system on FreeBSD Solo5/solo5#574Open question
I have been wondering about how interrupts are set up in Solo5 (without taking the time to really dig, in particular because the precise set up depends on the binding), because the way the OCaml 5 runtime sets up its stacks is really different from OCaml 4.
In particular, the OCaml 5 runtime assumes that only OCaml code should ever be triggered on a fiber stack. Is Solo5 already currently set up so that it can be guaranteed?
Commit “WIP Disable the x86_64 red zone” is strongly related to that question, on a slightly different angle.
To do before merging
READMEshould be updatedocaml-srcshould be restoredTODOs should be addressedDROPMEcommits should be dropped