dev meeting 20220113

Present at the meeting:

• Marek Kubica (@Leonidas-from-XIV)
• Nathan Rebours (@NathanReb)

Current plan

• Marek keeps working on opam-overlays CI
• Marek creates a temporary github actions with the lint checks for opam-overlays
• Nathan finishes the work on explicit repo list for opam-monorepo solver
• Nathan finishes building tezos with opam-monorepo

Meeting notes

Marek worked on a linter for the opam-overlays CI. It checks for basic things for now but it already useful. Marek ran it on the repo and found some packages that needed to be fixed. We discussed how to properly do that without breaking existing lockfiles and he has now submitted new, fixed ports for the impacted packages. Nathan also tried the tool recently and we discussed possible improvements which Marek started implementing.

It would be nice to start benefiting from the linter so we decided it might be good to add a simple github action that runs the linter on new opam-overlays submissions. That way we can iterate quickly and get immediate feedback until we have a proper ocurrent based CI for opam-overlays.

Marek also worked further on the hybrid mode. Nathan got a chance to try it out on tezos and it worked quite well. Initially we wanted to add tests but those require finishing the explicit repos feature. We'll likely review and merge the hybrid mode as it is and add the tests later.

Finally Marek helped with a few dune ports required for tezos!

Nathan has been focusing on the opam-monorepo build for Tezos, fixing a few opam constraint and other small adjustement to ease the opam-monorepo workflow within tezos. Most of the work was dune ports of some packages, most of which are actually upstreamed now!

We had a chat with Raphael Proust last week which cleared out a few things. Most importantly, some opam files in the tezos repo contains strict = constraints on some of their dependencies which prevented +dune version from being picked. Raphael clarified that some of those constraints might be very important to the performances. We'll open PRs to tezos relaxing some bounds and will discuss which ones can and which one can't. It's not a big blocker as the hard constraint will be turned into a pair of close upper and lower bounds, allowing only the +dune version to be picked but no higher versions. That might require more dune ports though as some packages had more recent versions building versions which Nathan used for the opam-monorepo build.

Nathan also worked on the explicit repo feature. It's not finished yet but he extracted a few PR with intermediate steps to keep the final PR focused and easy to review!

We discussed the plan for maintaining dune ports. We'd like to avoid working on automating ports of new releases and spend that energy and ressources to upstream the ports instead. For now we're focusing on opam-overlays CI which will help increasing the confidence in the quality of the dune ports, which is useful not matter how we proceed next. Most of the ports we do are accepted upstream, there are only two exceptions to this:

• package that have complex rules and are not easy to migrate to dune (hacl-star, zarith, etc...)
• Daniel Buenzli's packages We'd like to try and figure out a solution with Daniel to upstream the dune files and add an optional dune dependency to the opam metadata so we don't have to maintain ports. For this to be efficient we would need to setup a CI on the projects to help maintaining the dune files. Even if we have to do that maintainenance, it would be more satisfying to do it upstream than to have to setup automated workflows and CIs to attempt porting new releases when they hit opam. If this doesn't work out we'll reconsider automating the ports but this is not super satisfying as it is unlikely that those automated ports or the CI will catch failures (addition of new libs, modules, tests won't necessarily trigger failure with the old dune files for instance) thus resulting in breakage and reduced trust in the quality of opam-overlays dune ports. For the other cases, the complex ports, we obviously won't be able to automate these very well and therefore would rather spend our ressources working with the maintainers to upstream a satisfying port.