Alternative to kpathsea/kpsewhich for external tools

[let-def]

Hi everybody,

I am writing my own DVI/XDV tooling and previewer. Currently, I am building it on top of TeXlive, but I would like to move to tectonic which I find more convenient.

However, with TeXlive I can use either kpsewhich command or kpathsea library to lookup files, but I can't find a Tectonic equivalent.
More precisely, XDV files with truetype fonts are fine as the absolute path is stored, but traditional TeX fonts require looking up many files:

• a fontmap (*.map)
• a TeX font metric (*.tfm)
• sometimes, a font encoding (*.enc)
• based on the definition of the font, either a virtual font file (.vf) or a postscript font file (.pfa/*.pfb)

What I would like is some command that takes a file name and returns its path in the store or directly the contents of the file.
The closest I have found is tectonic -X show user-cache-dir, but this is a raw repository and I am not sure what is the scheme to map a filename to a path.

Some thoughts on this problem:

• there is usually a need to lookup many files, it would be nice for the command to work as a stream processor (reading queries line by line from stdin, outputting paths line by line on stdout), to avoid the overhead of starting a process each time
• since tectonic download files on demand, it might be necessary to distinguish between querying the state of the repository (is the file locally available, remotely available or unknown?) and actually asking for the contents, downloading if necessary
• is the tectonic repository safe wrt concurrent accesses? are there some necessary precautions to not mess up the store if a long-running tectonic process is already downloading files in the background?

Thanks,
Fred

[pkgw]

Hi Fred,

Thanks for asking!

First of all, I'll mention that Tectonic includes an xdv crate that does some basic parsing of the DVI/XDV format. It doesn't handle everything, and doesn't properly handle some things like RTL text segments, but depending on your needs, it might come in handy.

As for your main question, I can think of at least a few different ways to approach the problem that you want to solve. But if it's what you want, it wouldn't be that complicated to put together some kind of program that takes "TeX paths" as inputs and responds with the file contents, using Tectonic's on-the-fly fetching and caching framework. The first step is to set up the "bundle" to use, and you can get the "sensible default" with tectonic::config::PersistentConfig::default_bundle(). (With a bit more work, you can avoid the dependency on the kitchen-sink tectonic crate by reproducing some of its logic and depending on the lower-level tectonic_bundles crate.) Then you use that object's implementation of the IoProvider trait to fetch files by filename. If that bundle happens to involve a local caching layer, it is possible to figure out the filesystem path corresponding to the locally cached version of the given file, but I'd strongly recommend working with streams rather than computing and passing around filesystem paths that point into the cache.

As for concurrent accesses, there isn't any super clever locking, but the caching layer should allow them. The cache is basically append-only, which reduces the number of possible pitfalls. Basically the worst failure mode might be that the index of downloaded files gets multiple entries for the same file, although I can't say that I've really hammered on the implementation to look for edge cases.