-
Notifications
You must be signed in to change notification settings - Fork 670
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
support 64-bit file offsets on systems with 32-bit off_t
Adds large file support (64-bit file positions) to the existing Nix API when targetting systems that have separate 32-bit and 64-bit APIs. On many platforms that support 64-bit file positions, this support is present in the standard I/O functions. On others, there have historically been two APIs: the standard API, which limits file size to 2**31-1 bytes, and a separate API that suffixes function names with "64" and supports 64-bit file sizes. This "transitional interface for 64-bit file offsets" is not present on every platform. As a result, using the *64 API will make programs non-portable, whereas using the standard API will result in programs that cannot handle large file offsets on some systems, even if those systems actually do have a way to handle large file offsets. This change enables 64-bit file offsets on the following platforms where the standard API only provides 32-bit offsets: - 32-bit Linux with glibc. - 32-bit Android. To support large files with a consistent API across platforms, this change makes Nix functions call the 64-bit capable equivalents where necessary. Other C libraries may not provide the *64 API, so we continue to call the standard functions on those. Broadly, the change consists of 5 parts: 1. Uses of libc::off_t have are replaced by i64. This provides a consistent API across platforms and allows 64-bit offsets to be used even when libc::off_t is 32-bit. This is similar to std::fs, which uses u64 for file positions. Nix uses i64 because off_t is a signed type. Into<i64> is used where appropriate so that existing code that uses libc::off_t will continue to work without changes. 2. A largefile_fn macro that is used to select the large-file-capable version of a function. E.g. largefile_fn![pwrite] is equivalent to libc::pwrite64 on glibc/Linux and plain libc::pwrite on systems that don't have or need libc::pwrite64. 3. A new require_largefile macro that is used to skip tests that require libc::off_t to be larger than 32 bits. 4. Changes to fallocate, ftruncate, lseek, mmap, open, openat, posix_fadvise, posix_fallocate, pread, preadv, pwrite, pwritev, sendfile, and truncate, making them support large files. 5. A set of test_*_largefile tests to verify that each of the previously mentioned functions now works with files whose size requires more than 32 bits to represent. A few functions are still limited to 32-bit file sizes after this change. This includes the aio* functions, the *stat* functions, and mkstemp(). Adding large file support to those requires a bit more work than simply calling a different function and is therefore left for later.
- Loading branch information
Showing
14 changed files
with
435 additions
and
100 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.