Refactor via Knuth: "Just a typesetting language"

[jfine2358]

I was at Peter WIlliams' Tectonic talk at the 2022 TeX Conference. I was impressed by his talk. He's doing things I've long wanted to do.

I'd like to discuss extracting only a small subset of XeTeX for use in Tectonic. Perhaps done well this will make it easier for Tectonic to follow XeTeX, and branch for the new stuff, such as native HTML output.

The basis for this small subset is something Don Knuth said about his design goals for TeX. In 1996, in a Q+A session Nethlerland TeX Group (NTG), Don Knuth said (PDF, p47)

I’m not going to design a programming language; I wanted to have just a typesetting language.

And here Don meant by typesetting the creation of DVI (DeVice Independent) files. So no need to generate PDF at this stage. Just a typesetting language could be quite small, and still contain all the typesetting functionality required.

I discussed Knuth's small language goal in my talk at the 2005 EuroTeX meeting (PDF).

[pkgw]

I'll chime in here to mention that the way that the Rust language and packaging systems work, it is much easier to pursue this kind of refactoring as compared to the C/C++ world, and some level of this refactoring has already happened in Tectonic.

The primary Tectonic codebase is separated into a number of interlocking Rust "crates", findable in the crates/ directory of this repository:

https://github.com/tectonic-typesetting/tectonic/tree/master/crates

For instance, the engine_xetex crate is essentially just the XeTeX engine. It depends on (among others) the xetex_layout crate, which encapsulates the XeTeX-specific font management and layout routines.

Cargo's "features" system provides a systematic way to manage, well, features, and other optional behavior in situations where code is divided into relatively fine-grained crates. For instance, the HTTP retrieval framework (the geturl crate) has selectable backends, including a "null" backend if you want to build Tectonic without that functionality at all. I'd also like to translate the font-discovery implementation to likewise have selectable backends, or a "null" backend if you want to avoid system-specific dependencies for font discovery.

Cargo's workspace feature makes it practical to divide the code into these fine-grained crates because we can work on all of them in one repository, even though they can all be published independently — a "monorepo" model. The final piece of the puzzle is my Cranko release automation tool, which provides infrastructure that makes it practical to issue formal releases of all of these crates.