Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support memory sandboxing #632

Merged
merged 2 commits into from
Sep 30, 2023
Merged

Support memory sandboxing #632

merged 2 commits into from
Sep 30, 2023

Conversation

udoprog
Copy link
Collaborator

@udoprog udoprog commented Sep 24, 2023

This change allows anything which uses the new allocator-based APIs to limit memory use in any part of Rune through rune::alloc::memory::with:

rune::alloc::limit::with1024 * 10, || {
    /* code that runs here is limited to only use 10KB of memory */
    Ok(())
})?;

Closures and futures can be limited this way, and if the limit is reached for a given thread an error will be raised (not panic) which is propagated up through regular error handling facilities. Effectively this means that any user of Rune should be able to implement a rigorous memory sandbox that can take untrusted input and stay within defined memory limits.

CC LemmyNet/lemmy#3277

Caveats and missing pieces

The following is a non-exhaustive list of types and operations which are still not covered by the allocator:

  • The stack is currently not limited. So sufficiently nested code could use up unsupervised amounts of memory.
  • Allocation metadata is not accounted for, so in particular many small allocations might end up using up more system memory than what is being accounted for. This plugs in at the Rust allocated level, and uses the global allocation APIs which are currently provided in stable. This could eventually be remedied using a custom allocator, or by using an allocator which reports overhead.
  • CLI arguments. This is not critical since the CLI is usually not fed untrusted input.
  • Sorting using [_]::sort, which uses timsort that might allocate a cache up to half the size of the elements being sorted.
  • Path / PathBuf.
  • RelativePath / RelativePathBuf.
  • I/O related functions that returns a buffer, such as std::fs::read_to_string.
  • Anything using serde. While any types we convert into will take allocated memory into account (and error through serde accordingly) serializations might use internal buffering which bypasses this, such as serde_json using a string buffer. Owned types being passed in are not accounted for until they've been converted to internal types.
  • So far Rc and Arc have not been forked. Mainly because their memory overhead is fixed and relative to other collections which store them internally, We might still decide to fork them to ensure that we get full coverage.
  • A few more internal uses of std / alloc types such as String and Vec which just haven't been ported yet because they're not considered important.

All of these can easily be dealt with except serde. It's up to the serialization format to construct owned buffers and this is not something we can control. When this feature is released there will be a warning associated with all serialization modules (currently json and toml).

The other APIs will be copied into rune_alloc and given appropriate implementations which uses the rune_alloc allocator API.

@udoprog udoprog added the enhancement New feature or request label Sep 24, 2023
@udoprog udoprog force-pushed the memory-limits branch 20 times, most recently from ae60003 to 8092604 Compare September 30, 2023 19:54
@udoprog udoprog changed the title Implement memory limits in Global allocator Implement memory sandboxing Sep 30, 2023
@udoprog udoprog changed the title Implement memory sandboxing Support memory sandboxing Sep 30, 2023
@udoprog udoprog merged commit 6a4b4ce into main Sep 30, 2023
18 checks passed
@udoprog udoprog deleted the memory-limits branch September 30, 2023 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant