-
Notifications
You must be signed in to change notification settings - Fork 670
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
support 64-bit file offsets on 32-bit glibc/Linux
Adds large file support (64-bit file positions) to the existing Nix API when used with glibc on 32-bit Linux. On many platforms that support 64-bit file positions, this support is present in the standard I/O functions. On 32-bit Linux, there have historically been two APIs: the standard API, which limits file size to 2**31-1 bytes, and a separate API that suffixes function names with "64" and supports 64-bit file sizes. Other platforms do not provide this API. As a result, using the *64 API will make programs non-portable, whereas using the standard API will result in programs that cannot handle large files on 32-bit Linux, even when the system is actually capable of handling such files. To support large files with a consistent API across platforms, this change makes Nix functions call the 64-bit capable equivalents on Linux when using glibc. Other C libraries may not provide the *64 API, so we continue to call the standard functions on those. Broadly, the change consists of 5 parts: 1. Uses of libc::off_t have are replaced by i64. This provides a consistent API across platforms and allows 64-bit offsets to be used even when libc::off_t is 32-bit. This is similar to std::fs, which uses u64 for file positions. Nix uses i64 because off_t is a signed type. Into<i64> is used where appropriate so that existing code that uses libc::off_t will continue to work without changes. 2. A largefile_fn macro that is used to select the large-file-capable version of a function. E.g. largefile_fn![pwrite] is equivalent to libc::pwrite64 on glibc/Linux and plain libc::pwrite everywhere else. 3. A new require_largefile macro that is used to skip tests that require libc::off_t to be larger than 32 bits. 4. Changes to fallocate, ftruncate, lseek, mmap, open, openat, posix_fadvise, posix_fallocate, pread, preadv, pwrite, pwritev, sendfile, and truncate, making them support large files. 5. A set of test_*_largefile tests to verify that each of the previously mentioned functions now works with files whose size requires more than 32 bits to represent. A few functions are still limited to 32-bit file sizes after this change. This includes the aio* functions, the *stat* functions, and mkstemp(). Adding large file support to those requires a bit more work than simply calling a different function and is therefore left for later.
- Loading branch information
Showing
14 changed files
with
409 additions
and
84 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.