Cleanup and Stability Effort about bramble HOT 5 CLOSED

Note on lazy derivation evaluation:

Calling the derivation function does input validation on params and args. Looks at all derivations used as input. If there is a function the function is called to check if the return value is a derivation (or list of derivations). These derivations are children of this derivation. Add them to the graph and move on to the next derivation. Connect all future derivations to the graph.

Once we hit a cmd that wants to run the derivation, or if we return the final derivation from a function, we check where this last derivation is on the graph. then we start from the leaves of the tree and start building in parallel but crawling the DAG. If there is an error we stop all jobs and exit with the error for that derivation (unless another mode is specified).

from bramble.

the build path must be in the bramble store in case the path is used in the output

from bramble.

https://edolstra.github.io/pubs/phd-thesis.pdf
6.7

The outputs of a derivation can have references to each other, and in fact this is quite common. For instance, it can be expected that the programs in the bin output of Glibc depend on the libraries in the out output. This means that the out output is in the closure of the bin output, but not vice versa. But what happens when there are mutually recursive references, e.g., when out also refers to bin? These must be forbidden, since the hash rewriting scheme from Section 6.3.2 cannot handle them. For instance, when we copy out and bin to their content-addressable locations, we must rewrite in both FSOs the hashes of both paths. The function hashModulo only handles direct self-references, and it can do so because the hashes to be disregarded in the hash computation are encoded into the file name.

Fortunately, banning mutually recursive outputs is not a very onerous restriction, since they are pointless. After all, mutual recursion between output paths requires them to be deployed and garbage collected as a unit, negating the granularity advantages that multiple outputs are intended to achieve.

from bramble.

5.4.1

If a fetchurl derivation followed the normal translation scheme, the output paths of the derivation and all derivations depending on it would change. For instance, if we were to change the URL of the Glibc source distribution—a component on which almost all other components depend—massive rebuilds will ensue. This is unfortunate for a change which we know cannot have a real effect as it propagates upwards through the dependency graph.

Fixed-output derivations solve this problem by allowing a derivation to state to Nix that its output will hash to a specific value. When Nix builds the derivation (Section 5.5), it will hash the output and check that the hash corresponds to the declared value. If there is a hash mismatch, the build fails and the output is not registered as valid. For fixed-output derivations, the computation of the output path only depends on the declared hash and hash algorithm, not on any other attributes of the derivation.

This is extremely interesting, I wonder if it's possible to take advantage of this pattern now that outputs are known and attempt to be consistent between systems. Could we store different versions of derivations on disk and link them to different outputs? Could we shortstop rebuilding in certain situations?

from bramble.

closing this since the framing is a little outdated and general work has resumed on the main branch

from bramble.