Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support getting file metadata on Windows #4067

Open
wants to merge 22 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
9b61c05
Refactor `Handle` slightly
CraftSpider Dec 2, 2024
ab0e138
Implement trivial file operations - opening and closing handles. Just…
CraftSpider Dec 2, 2024
e19deaa
Fix MAIN_THREAD handle in windows_join_main
CraftSpider Dec 2, 2024
f1bb77d
Try fix for Windows paths on non-Windows machines
CraftSpider Dec 2, 2024
9a76a85
Most review comments - still missing shim tests
CraftSpider Dec 6, 2024
cc6eae5
Don't leak miri implementation details
CraftSpider Dec 6, 2024
3653169
Fix clippy
CraftSpider Dec 6, 2024
a64dbd1
Test file creation and metadata shims directly
CraftSpider Dec 6, 2024
274d90f
Move windows-fs to pass-dep and use windows_sys
CraftSpider Dec 8, 2024
51273f5
Move FdNum to shims/files.rs, use it in FdTable definitions
CraftSpider Dec 8, 2024
aa6bbf6
Slightly improve flag handling - parse and validate in one place
CraftSpider Dec 8, 2024
938430f
Fixup imports, compile
CraftSpider Dec 8, 2024
3c2ed8a
Make metadata handle store the metadata, instead of just a path. Add …
CraftSpider Dec 8, 2024
ebfc768
Improve extract_windows_epoch impl
CraftSpider Dec 8, 2024
d989984
Improve extract_windows_epoch impl comments
CraftSpider Dec 8, 2024
e5ada76
Add invalid handle encoding test
CraftSpider Dec 8, 2024
52c1676
Add tests for CREATE_ALWAYS and OPEN_ALWAYS error behavior. Add comme…
CraftSpider Dec 8, 2024
5ac99da
Extract Windows epoch helpers from GetSystemTimeAsFileTime and use th…
CraftSpider Dec 8, 2024
ef5ab7f
Merge FileHandle implementation between Unix and Windows
CraftSpider Dec 9, 2024
6ee99aa
Use u32::MAX constant
CraftSpider Dec 13, 2024
a61daf2
Some fs improvements
CraftSpider Dec 16, 2024
92f41ce
Use FdNum more places
CraftSpider Dec 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ measureme = "11"
chrono = { version = "0.4.38", default-features = false }
chrono-tz = "0.10"
directories = "5"
bitflags = "2.6"

# Copied from `compiler/rustc/Cargo.toml`.
# But only for some targets, it fails for others. Rustc configures this in its CI, but we can't
Expand Down
125 changes: 116 additions & 9 deletions src/shims/files.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
use std::any::Any;
use std::collections::BTreeMap;
use std::io::{IsTerminal, Read, SeekFrom, Write};
use std::fs::{File, Metadata};
use std::io::{IsTerminal, Read, Seek, SeekFrom, Write};
use std::ops::Deref;
use std::rc::{Rc, Weak};
use std::{fs, io};
Expand Down Expand Up @@ -72,7 +73,7 @@ pub trait FileDescription: std::fmt::Debug + Any {
false
}

fn as_unix(&self) -> &dyn UnixFileDescription {
fn as_unix<'tcx>(&self, _ecx: &MiriInterpCx<'tcx>) -> &dyn UnixFileDescription {
panic!("Not a unix file descriptor: {}", self.name());
}
}
Expand Down Expand Up @@ -178,6 +179,105 @@ impl FileDescription for io::Stderr {
}
}

#[derive(Debug)]
pub struct FileHandle {
pub(crate) file: File,
pub(crate) writable: bool,
}

impl FileDescription for FileHandle {
fn name(&self) -> &'static str {
"file"
}

fn read<'tcx>(
&self,
_self_ref: &FileDescriptionRef,
communicate_allowed: bool,
ptr: Pointer,
len: usize,
dest: &MPlaceTy<'tcx>,
ecx: &mut MiriInterpCx<'tcx>,
) -> InterpResult<'tcx> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
let mut bytes = vec![0; len];
let result = (&mut &self.file).read(&mut bytes);
match result {
Ok(read_size) => ecx.return_read_success(ptr, &bytes, read_size, dest),
Err(e) => ecx.set_last_error_and_return(e, dest),
}
}

fn write<'tcx>(
&self,
_self_ref: &FileDescriptionRef,
communicate_allowed: bool,
ptr: Pointer,
len: usize,
dest: &MPlaceTy<'tcx>,
ecx: &mut MiriInterpCx<'tcx>,
) -> InterpResult<'tcx> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
let bytes = ecx.read_bytes_ptr_strip_provenance(ptr, Size::from_bytes(len))?;
let result = (&mut &self.file).write(bytes);
match result {
Ok(write_size) => ecx.return_write_success(write_size, dest),
Err(e) => ecx.set_last_error_and_return(e, dest),
}
}

fn seek<'tcx>(
&self,
communicate_allowed: bool,
offset: SeekFrom,
) -> InterpResult<'tcx, io::Result<u64>> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
interp_ok((&mut &self.file).seek(offset))
}

fn close<'tcx>(
self: Box<Self>,
communicate_allowed: bool,
_ecx: &mut MiriInterpCx<'tcx>,
) -> InterpResult<'tcx, io::Result<()>> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
// We sync the file if it was opened in a mode different than read-only.
if self.writable {
// `File::sync_all` does the checks that are done when closing a file. We do this to
// to handle possible errors correctly.
let result = self.file.sync_all();
// Now we actually close the file and return the result.
drop(*self);
interp_ok(result)
} else {
// We drop the file, this closes it but ignores any errors
// produced when closing it. This is done because
// `File::sync_all` cannot be done over files like
// `/dev/urandom` which are read-only. Check
// https://github.com/rust-lang/miri/issues/999#issuecomment-568920439
// for a deeper discussion.
drop(*self);
interp_ok(Ok(()))
}
}

fn metadata<'tcx>(&self) -> InterpResult<'tcx, io::Result<Metadata>> {
interp_ok(self.file.metadata())
}

fn is_tty(&self, communicate_allowed: bool) -> bool {
communicate_allowed && self.file.is_terminal()
}

fn as_unix<'tcx>(&self, ecx: &MiriInterpCx<'tcx>) -> &dyn UnixFileDescription {
assert!(
ecx.target_os_is_unix(),
"unix file operations are only available for unix targets"
);
self
}
}

/// Like /dev/null
#[derive(Debug)]
pub struct NullOutput;
Expand Down Expand Up @@ -275,6 +375,9 @@ impl VisitProvenance for WeakFileDescriptionRef {
}
}

/// Internal type of a file-descriptor - this is what [`FdTable`] expects
pub type FdNum = i32;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also use this type in the definition of FdTable


/// A unique id for file descriptions. While we could use the address, considering that
/// is definitely unique, the address would expose interpreter internal state when used
/// for sorting things. So instead we generate a unique id per file description is the name
Expand All @@ -285,7 +388,7 @@ pub struct FdId(usize);
/// The file descriptor table
#[derive(Debug)]
pub struct FdTable {
pub fds: BTreeMap<i32, FileDescriptionRef>,
pub fds: BTreeMap<FdNum, FileDescriptionRef>,
/// Unique identifier for file description, used to differentiate between various file description.
next_file_description_id: FdId,
}
Expand Down Expand Up @@ -320,17 +423,21 @@ impl FdTable {
}

/// Insert a new file description to the FdTable.
pub fn insert_new(&mut self, fd: impl FileDescription) -> i32 {
pub fn insert_new(&mut self, fd: impl FileDescription) -> FdNum {
let fd_ref = self.new_ref(fd);
self.insert(fd_ref)
}

pub fn insert(&mut self, fd_ref: FileDescriptionRef) -> i32 {
pub fn insert(&mut self, fd_ref: FileDescriptionRef) -> FdNum {
self.insert_with_min_num(fd_ref, 0)
}

/// Insert a file description, giving it a file descriptor that is at least `min_fd_num`.
pub fn insert_with_min_num(&mut self, file_handle: FileDescriptionRef, min_fd_num: i32) -> i32 {
pub fn insert_with_min_num(
&mut self,
file_handle: FileDescriptionRef,
min_fd_num: FdNum,
) -> FdNum {
// Find the lowest unused FD, starting from min_fd. If the first such unused FD is in
// between used FDs, the find_map combinator will return it. If the first such unused FD
// is after all other used FDs, the find_map combinator will return None, and we will use
Expand All @@ -356,16 +463,16 @@ impl FdTable {
new_fd_num
}

pub fn get(&self, fd_num: i32) -> Option<FileDescriptionRef> {
pub fn get(&self, fd_num: FdNum) -> Option<FileDescriptionRef> {
let fd = self.fds.get(&fd_num)?;
Some(fd.clone())
}

pub fn remove(&mut self, fd_num: i32) -> Option<FileDescriptionRef> {
pub fn remove(&mut self, fd_num: FdNum) -> Option<FileDescriptionRef> {
self.fds.remove(&fd_num)
}

pub fn is_fd_num(&self, fd_num: i32) -> bool {
pub fn is_fd_num(&self, fd_num: FdNum) -> bool {
self.fds.contains_key(&fd_num)
}
}
Expand Down
36 changes: 26 additions & 10 deletions src/shims/time.rs
Original file line number Diff line number Diff line change
Expand Up @@ -218,16 +218,8 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {

let filetime = this.deref_pointer_as(LPFILETIME_op, this.windows_ty_layout("FILETIME"))?;

let NANOS_PER_SEC = this.eval_windows_u64("time", "NANOS_PER_SEC");
let INTERVALS_PER_SEC = this.eval_windows_u64("time", "INTERVALS_PER_SEC");
let INTERVALS_TO_UNIX_EPOCH = this.eval_windows_u64("time", "INTERVALS_TO_UNIX_EPOCH");
let NANOS_PER_INTERVAL = NANOS_PER_SEC / INTERVALS_PER_SEC;
let SECONDS_TO_UNIX_EPOCH = INTERVALS_TO_UNIX_EPOCH / INTERVALS_PER_SEC;

let duration = system_time_to_duration(&SystemTime::now())?
+ Duration::from_secs(SECONDS_TO_UNIX_EPOCH);
let duration_ticks = u64::try_from(duration.as_nanos() / u128::from(NANOS_PER_INTERVAL))
.map_err(|_| err_unsup_format!("programs running more than 2^64 Windows ticks after the Windows epoch are not supported"))?;
let duration = this.system_time_since_windows_epoch(&SystemTime::now())?;
let duration_ticks = this.windows_ticks_for(duration)?;

let dwLowDateTime = u32::try_from(duration_ticks & 0x00000000FFFFFFFF).unwrap();
let dwHighDateTime = u32::try_from((duration_ticks & 0xFFFFFFFF00000000) >> 32).unwrap();
Expand Down Expand Up @@ -276,6 +268,30 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
interp_ok(Scalar::from_i32(-1)) // Return non-zero on success
}

#[allow(non_snake_case, clippy::arithmetic_side_effects)]
fn system_time_since_windows_epoch(&self, time: &SystemTime) -> InterpResult<'tcx, Duration> {
let this = self.eval_context_ref();

let INTERVALS_PER_SEC = this.eval_windows_u64("time", "INTERVALS_PER_SEC");
let INTERVALS_TO_UNIX_EPOCH = this.eval_windows_u64("time", "INTERVALS_TO_UNIX_EPOCH");
let SECONDS_TO_UNIX_EPOCH = INTERVALS_TO_UNIX_EPOCH / INTERVALS_PER_SEC;

interp_ok(system_time_to_duration(time)? + Duration::from_secs(SECONDS_TO_UNIX_EPOCH))
}

#[allow(non_snake_case, clippy::arithmetic_side_effects)]
fn windows_ticks_for(&self, duration: Duration) -> InterpResult<'tcx, u64> {
let this = self.eval_context_ref();

let NANOS_PER_SEC = this.eval_windows_u64("time", "NANOS_PER_SEC");
let INTERVALS_PER_SEC = this.eval_windows_u64("time", "INTERVALS_PER_SEC");
let NANOS_PER_INTERVAL = NANOS_PER_SEC / INTERVALS_PER_SEC;

let ticks = u64::try_from(duration.as_nanos() / u128::from(NANOS_PER_INTERVAL))
.map_err(|_| err_unsup_format!("programs running more than 2^64 Windows ticks after the Windows epoch are not supported"))?;
interp_ok(ticks)
}

fn mach_absolute_time(&self) -> InterpResult<'tcx, Scalar> {
let this = self.eval_context_ref();

Expand Down
6 changes: 3 additions & 3 deletions src/shims/unix/fd.rs
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
throw_unsup_format!("unsupported flags {:#x}", op);
};

let result = fd.as_unix().flock(this.machine.communicate(), parsed_op)?;
let result = fd.as_unix(this).flock(this.machine.communicate(), parsed_op)?;
drop(fd);
// return `0` if flock is successful
let result = result.map(|()| 0i32);
Expand Down Expand Up @@ -251,7 +251,7 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
let Ok(offset) = u64::try_from(offset) else {
return this.set_last_error_and_return(LibcError("EINVAL"), dest);
};
fd.as_unix().pread(communicate, offset, buf, count, dest, this)?
fd.as_unix(this).pread(communicate, offset, buf, count, dest, this)?
}
};
interp_ok(())
Expand Down Expand Up @@ -291,7 +291,7 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
let Ok(offset) = u64::try_from(offset) else {
return this.set_last_error_and_return(LibcError("EINVAL"), dest);
};
fd.as_unix().pwrite(communicate, buf, count, offset, dest, this)?
fd.as_unix(this).pwrite(communicate, buf, count, offset, dest, this)?
}
};
interp_ok(())
Expand Down
102 changes: 3 additions & 99 deletions src/shims/unix/fs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@

use std::borrow::Cow;
use std::fs::{
DirBuilder, File, FileType, Metadata, OpenOptions, ReadDir, read_dir, remove_dir, remove_file,
rename,
DirBuilder, File, FileType, OpenOptions, ReadDir, read_dir, remove_dir, remove_file, rename,
};
use std::io::{self, ErrorKind, IsTerminal, Read, Seek, SeekFrom, Write};
use std::io::{self, ErrorKind, Read, Seek, SeekFrom, Write};
use std::path::{Path, PathBuf};
use std::time::SystemTime;

Expand All @@ -14,106 +13,11 @@ use rustc_data_structures::fx::FxHashMap;

use self::shims::time::system_time_to_duration;
use crate::helpers::check_min_arg_count;
use crate::shims::files::{EvalContextExt as _, FileDescription, FileDescriptionRef};
use crate::shims::files::{EvalContextExt as _, FileHandle};
use crate::shims::os_str::bytes_to_os_str;
use crate::shims::unix::fd::{FlockOp, UnixFileDescription};
use crate::*;

#[derive(Debug)]
struct FileHandle {
file: File,
writable: bool,
}

impl FileDescription for FileHandle {
fn name(&self) -> &'static str {
"file"
}

fn read<'tcx>(
&self,
_self_ref: &FileDescriptionRef,
communicate_allowed: bool,
ptr: Pointer,
len: usize,
dest: &MPlaceTy<'tcx>,
ecx: &mut MiriInterpCx<'tcx>,
) -> InterpResult<'tcx> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
let mut bytes = vec![0; len];
let result = (&mut &self.file).read(&mut bytes);
match result {
Ok(read_size) => ecx.return_read_success(ptr, &bytes, read_size, dest),
Err(e) => ecx.set_last_error_and_return(e, dest),
}
}

fn write<'tcx>(
&self,
_self_ref: &FileDescriptionRef,
communicate_allowed: bool,
ptr: Pointer,
len: usize,
dest: &MPlaceTy<'tcx>,
ecx: &mut MiriInterpCx<'tcx>,
) -> InterpResult<'tcx> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
let bytes = ecx.read_bytes_ptr_strip_provenance(ptr, Size::from_bytes(len))?;
let result = (&mut &self.file).write(bytes);
match result {
Ok(write_size) => ecx.return_write_success(write_size, dest),
Err(e) => ecx.set_last_error_and_return(e, dest),
}
}

fn seek<'tcx>(
&self,
communicate_allowed: bool,
offset: SeekFrom,
) -> InterpResult<'tcx, io::Result<u64>> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
interp_ok((&mut &self.file).seek(offset))
}

fn close<'tcx>(
self: Box<Self>,
communicate_allowed: bool,
_ecx: &mut MiriInterpCx<'tcx>,
) -> InterpResult<'tcx, io::Result<()>> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
// We sync the file if it was opened in a mode different than read-only.
if self.writable {
// `File::sync_all` does the checks that are done when closing a file. We do this to
// to handle possible errors correctly.
let result = self.file.sync_all();
// Now we actually close the file and return the result.
drop(*self);
interp_ok(result)
} else {
// We drop the file, this closes it but ignores any errors
// produced when closing it. This is done because
// `File::sync_all` cannot be done over files like
// `/dev/urandom` which are read-only. Check
// https://github.com/rust-lang/miri/issues/999#issuecomment-568920439
// for a deeper discussion.
drop(*self);
interp_ok(Ok(()))
}
}

fn metadata<'tcx>(&self) -> InterpResult<'tcx, io::Result<Metadata>> {
interp_ok(self.file.metadata())
}

fn is_tty(&self, communicate_allowed: bool) -> bool {
communicate_allowed && self.file.is_terminal()
}

fn as_unix(&self) -> &dyn UnixFileDescription {
self
}
}

impl UnixFileDescription for FileHandle {
fn pread<'tcx>(
&self,
Expand Down
Loading