Comments (5)
Note on lazy derivation evaluation:
Calling the derivation function does input validation on params and args. Looks at all derivations used as input. If there is a function the function is called to check if the return value is a derivation (or list of derivations). These derivations are children of this derivation. Add them to the graph and move on to the next derivation. Connect all future derivations to the graph.
Once we hit a cmd that wants to run the derivation, or if we return the final derivation from a function, we check where this last derivation is on the graph. then we start from the leaves of the tree and start building in parallel but crawling the DAG. If there is an error we stop all jobs and exit with the error for that derivation (unless another mode is specified).
from bramble.
the build path must be in the bramble store in case the path is used in the output
from bramble.
https://edolstra.github.io/pubs/phd-thesis.pdf
6.7
The outputs of a derivation can have references to each other, and in fact this is quite common. For instance, it can be expected that the programs in the bin output of Glibc depend on the libraries in the out output. This means that the out output is in the closure of the bin output, but not vice versa. But what happens when there are mutually recursive references, e.g., when out also refers to bin? These must be forbidden, since the hash rewriting scheme from Section 6.3.2 cannot handle them. For instance, when we copy out and bin to their content-addressable locations, we must rewrite in both FSOs the hashes of both paths. The function hashModulo only handles direct self-references, and it can do so because the hashes to be disregarded in the hash computation are encoded into the file name.
Fortunately, banning mutually recursive outputs is not a very onerous restriction, since they are pointless. After all, mutual recursion between output paths requires them to be deployed and garbage collected as a unit, negating the granularity advantages that multiple outputs are intended to achieve.
from bramble.
5.4.1
If a fetchurl derivation followed the normal translation scheme, the output paths of the derivation and all derivations depending on it would change. For instance, if we were to change the URL of the Glibc source distribution—a component on which almost all other components depend—massive rebuilds will ensue. This is unfortunate for a change which we know cannot have a real effect as it propagates upwards through the dependency graph.
Fixed-output derivations solve this problem by allowing a derivation to state to Nix that its output will hash to a specific value. When Nix builds the derivation (Section 5.5), it will hash the output and check that the hash corresponds to the declared value. If there is a hash mismatch, the build fails and the output is not registered as valid. For fixed-output derivations, the computation of the output path only depends on the declared hash and hash algorithm, not on any other attributes of the derivation.
This is extremely interesting, I wonder if it's possible to take advantage of this pattern now that outputs are known and attempt to be consistent between systems. Could we store different versions of derivations on disk and link them to different outputs? Could we shortstop rebuilding in certain situations?
from bramble.
closing this since the framing is a little outdated and general work has resumed on the main branch
from bramble.
Related Issues (20)
- Switch to using a single docker container for builds
- url hashes and the lockfile
- Absolute path in symlink leads to "invalid tar header" HOT 1
- Figure out how to run things HOT 1
- Decouple parse/starlark step and build step HOT 2
- Fix fetch builders so that they write to the lockfile again
- Why yet another nix? HOT 5
- macOS sandbox
- Derivations that return derivations
- Public build/package cache HOT 1
- Derivation output as OCI/Docker image
- Documentation generation
- Remote build
- Add bramble version to bramble.toml
- Enable discussions? HOT 2
- Check derivation and build output cache during builds
- Document cache-folder strategy for run invocations
- Implement user-global package state and aliases
- Implement the nixos darwin case insensitivity hack HOT 2
- Implement fetch_git builtin
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bramble.