[RFC 0141] Private Derivations

[poscat0x04]

Summary

This RFC proposes to add a special type of derivation called private derivation, which, upon being built, will have their file permissions set to 000 instead of the usual 444.

Rendered

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/draft-rfc-private-derivations-derivations-without-read-access/25184/1

[SuperSandro2000]

I think you mean the right thing but the module system can handle that the credentials are in files but most config files do not handle this to well.

[SuperSandro2000]

What about substituter and caches?

[SuperSandro2000]

How to handle backwards compatibility?

[poscat0x04]

Hmm, what is the main challenge of backwards compatibility? A machine with outdated nix versions trying to build private derivations?

[poscat0x04]

We should look into how content-addressed paths are implemented, since they also change the store model.

[SuperSandro2000]

This is more general: how to prevent copying derivations onto machines with older nix versions or caches?

[poscat0x04]

Hmm, I'm not very familiar with how store and copying works. Do we need assistance on the receiving end or is it just like scp-ing some files?

[SuperSandro2000]

and hydra, harmonia, etc

[SuperSandro2000]

how to prevent leaking credentials in build logs?

[poscat0x04]

They definitely might leak, but for the usual use case, which is to write some string into a file like writeText, this shouldn't be a problem. Maybe we can add this to the future works section?

[Ericson2314]

See also https://gist.github.com/edolstra/afa5a41d4acbc0d6c8cccfede7fd4792

[poscat0x04]

I don't quite like the design of store ACLs presented in the gist.

1. It is very reasonable for a public derivation to reference a private derivation. For example, systemd unit files (which should be public) can reference private configurations in their ExecStart= option and a public configuration file (itself a derivation) might also reference credentials.
2. Building a derivation is no longer a simple process of converting a build specification to a build product, it now involves filesystem permissions. The end product also differs depending on the user issuing the build command, which IMO breaks purity.

[kevincox]

What about starting a build to access this path? It could just copy out the value. Or would private derivations not be able to be used as input to other derivations? But in that case how would you use these in NixOS? (Maybe the path can be referenced in a build but the actual contents isn't available? That would probably require sandboxing.)

[poscat0x04]

Yes, the path can be referenced but the contents are not accessible during build. This is possible because the build is carried out by nixbld* users which do not have read access of private derivations.

[kevincox]

I think it would be good to explicitly list this in the RFC. Along with other security properties such as nix copy, and interfacing with binary caches.

[edolstra]

I think this needs a lot more thought. To mention a couple of issues:

• This proposal talks about changing the permission on .drv files, but doesn't say much about the output paths. Should those also be private?
• If it's only about encryption of secrets to be decrypted at runtime, then it's probably easier to add some encryption primitive to the Nix evaluator (see [RFC 0005] Nix encryption #5).
• "Private" is a misnomer. What is really meant is "only readable by root". (The ACL proposal has truly private, i.e. user-specific, closures.)
• An important property of Nix is the closure property, i.e. if a store path is valid and readable, its references should be valid and readable as well. This proposal breaks that, i.e. you can have "holes" in closures that you can't access. This means e.g. that you can't do a nix copy to build a .drv on a remote builder, or to copy a NixOS system closure to a remote machine.
• It's not clear what it means for LoadCredential= to "load the private derivation".

BTW I don't understand the objections to the ACL proposal:

Building a derivation is no longer a simple process of converting a build specification to a build product, it now involves filesystem permissions.

... but this RFC also involves filesystem permissions.

The end product also differs depending on the user issuing the build command, which IMO breaks purity.

Quite the opposite: two users who instantiate the same derivation can both build it and get the same output closure, which they can both access.

[poscat0x04]

This proposal talks about changing the permission on .drv files, but doesn't say much about the output paths. Should those also be private?

This is mentioned in the design details section. To quote, "The only difference between a private derivation and a normal derivation, apart from the hash, is that upon instantiation and after building, the read bit of the .drv file and the output path will be removed."

If it's only about encryption of secrets to be decrypted at runtime, then it's probably easier to add some encryption primitive to the Nix evaluator

This RFC is about storing plaintext data in nix store, maybe I should've made it more clear in the RFC?

"Private" is a misnomer. What is really meant is "only readable by root". (The ACL proposal has truly private, i.e. user-specific, closures.)

Yeah. Maybe a better name would be "non-readable derivations"?

An important property of Nix is the closure property, i.e. if a store path is valid and readable, its references should be valid and readable as well. This proposal breaks that, i.e. you can have "holes" in closures that you can't access.

Again, I really don't see how the closure property is desirable. To quote my previous response, "It is very reasonable for a public derivation to reference a private derivation. For example, systemd unit files (which should be public) can reference private configurations in their ExecStart= option and a public configuration file (itself a derivation) might also reference credentials.".

If you're still not convinced, consider the following scenario: suppose I'm using NixOS and I want to store a configuration file for service A that contain sensitive information in store without it being able to be readable by everyone. Obviously, the system derivation is world readable. If we have the closure property, then since the configuration file is in the closure of the system derivation, it must also be world readable.

This means e.g. that you can't do a nix copy to build a .drv on a remote builder, or to copy a NixOS system closure to a remote machine.

Yes, that is by design. The user is encouraged to build the private derivations locally. Also, you can still copy it if you have root access of the current machine. IMO this is a reasonable limitation, since if you don't even have root access of the machine then you'll probably not want to store credentials on that machine.

It's not clear what it means for LoadCredential= to "load the private derivation".

I meant adding a LoadCredential = <private-derivation-path> attribute to the systemd service config.

... but this RFC also involves filesystem permissions.

I forgot what I was thinking while I wrote that down (facepalms).

Quite the opposite: two users who instantiate the same derivation can both build it and get the same output closure, which they can both access.

Well. I think we a different view of what it means for two outputs to be equal. IMO filesystem metadata such as permissions, ACL, atime and mtime are all part of the derivation. For two derivation outputs to be equal, not only do they need to have the same content, but also the same metadata. This is also what nix currently enforces: it always sets the mtime to unix time 0, the permissions to 444 or 555 and the ownership to root:root. Under this view, the store ACL proposal produces different outputs when different users invoke the nix build command.

[Kiskae]

I've previously tinkered with a solution that solves a lot of the same issues as this PR, so I'd like to offer it as an alternative:

Ephemeral derivations

• derivations that produce a "private" output when it is requested as input, this output never becomes part of the nix store
• this output is never allowed to be referenced, making it safe regarding future nix closures.
• since it can never be referenced it does not need to exist in the nix store and can exist temporarily elsewhere in the file system, which means it can be implemented on macos and linux without mount namespaces.
• since outputs are generated each time they are used and don't need to be persistent, it could be directly integrated with a secrets provider like vault or systemd-creds.

Comparison to this RFC

• As long as there is something providing the required credentials, secret-to-secret transformations can be done as part of the nix build.
• This would not allow configuration with embedded secrets, but that is probably a sign of bad configuration and can be solved by performing runtime substitution of secrets rather than using the copy in the store. This would essentially result in the same results this RFC would accomplish while being compatible with all ways a secret can be provisioned.
• Rather than requiring a change to this nix store this probably requires quite a comprehensive change to the nix builder.

In my honest opinion any secrets that exist in plaintext while data is at rest are an issue, but with this RFC this includes all referenced secrets, while existing solutions would be restricted to only the secrets that are compromised by the plaintext decryption key.

[infinisil]

Please also check out #143!

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-05-14/45414/1

[poscat0x04]

Again, my objection to the ACL model still applies:

1. The permission of a derivation and its dependencies should be entirely independent
2. ACLs are part of the file content, and hence any changes to it should also change the hash

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-05-28/46113/1

[infinisil]

RFCSC:

This RFC has not acquired enough shepherds. This typically shows lack of interest from the community. In order to progress a full shepherd team is required. Consider trying to raise interest by posting in Discourse, talking in Matrix or reaching out to people that you know.

If not enough shepherds can be found in the next month we will close this RFC until we can find enough interested participants. The PR can be reopened at any time if more shepherd nominations are made.

See more info on the Nix RFC process here

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-06-10/46817/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-06-24/47589/1

[nyabinary]

This would be nice with eventual systemd-creds (maybe it should rebrand as an RFC focused on secrets management?)

[GetPsyched]

RFCSC:

This RFC is being closed due to lack interest. If enough shepherds are found this issue can be reopened. If you don't have permission to reopen please open an issue for the NixOS RFC Steering Committee linking to this PR.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-07-08/48678/1