Improve support for subflakes

[edolstra]

Motivation

Subflakes are flakes in the same tree, accessed in flake inputs via relative paths (e.g. inputs.foo.url = "path:./subdir"). Previously these didn't work very well because they would be separately copied to the store, which is inefficient and makes references to parent directories tricky or impossible. Furthermore, they had their own NAR hash in the lock file, which is superfluous since the parent is already locked. In fact, if subflakes worked at all, it was by accident.

Now subflakes are accessed via the accessor of the calling flake. This avoids the unnecessary copy and makes it possible for subflakes to depend on flakes in a parent directory (so long as they're in the same tree).

Lock file nodes for relative flake inputs now have a new parent field:

{
  "locked": {
    "path": "./subdir",
    "type": "path"
  },
  "original": {
    "path": "./subdir",
    "type": "path"
  },
  "parent": [
    "foo",
    "bar"
  ]
}

which denotes that ./subdir is to be interpreted relative to the directory of the bar input of the foo input of the root flake.

Extracted from the lazy-trees branch.

Depends on #10088.

Context

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[dpulls]

🎉 All dependencies have been resolved !

[thufschmitt]

Discussed during the Nix maintainers meeting on 2024-03-04.

• Idea approved
• Document that directly accessing .. in subflakes is likely to break in the future
• Use path literals as the url for subflakes

• Great overall
• Right now, the PR allows import ../. in a subflake. Do we want that?
  • Directly accessing .. is convenient, but breaks the encapsulation property of flakes
    • Relates to nix flakes: flakes in subdirectories can access files in parent directories (breaking the implied seal of flake.nix) #4414
  • We can however access other parts of the tree with relative inputs (that can go up until the root)
    • inputs.something_else = ../foo
  • From an implementation perspective, hard to forbid without lazy trees
  • Decision:
    • Don't formally forbid direct access to .. since it's hard without lazy trees
    • Once lazy trees lands, make that conditional on a flag
    • Potentially make the default restrictive. Would be a breaking change, but
      • can be mitigated by only making that change effective after a flake update
      • small breakage as getting the old behavior back would only be a one-line change
• Syntax for relative flakes?
  • Currently reuses url = "path:./foo" that already kinda works on master
  • A more natural syntax would be url = ./foo
    • This is actually what most people seem to try first when they want subflakes

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-03-04-nix-team-meeting-minute-130/40830/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-11-27-nix-team-meeting-minutes-198/56691/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-12-18-nix-team-meeting-minutes-204/57602/1

[MagicRB]

Does this also mean that subflakes will auto-update? In other words, nix flake update subflake ought to be unnecessary with this right?