grouping / tagging of sources

niv

Painless dependencies for Nix projects. Read more in the Getting started section below.

Install
Build
Usage
FAQ

Install

niv is available in nixpkgs as niv:

$ nix-env -iA nixpkgs.niv

Alternatively, run the following command to install the development version:

$ nix-env -iA niv -f https://github.com/nmattia/niv/tarball/master \
    --substituters https://niv.cachix.org \
    --trusted-public-keys niv.cachix.org-1:X32PCg2e/zAm3/uD1ScqW2z/K0LtDyNV7RdaxIuLgQM=

Build

Inside the provided nix shell:

$ repl

Run the test suite with this command:

$ ./script/test

Usage

niv simplifies adding and updating dependencies in Nix projects. It uses a single file, nix/sources.json, where it stores the data necessary for fetching and updating the packages.

Add: inserts a package in nix/sources.json.
Update: updates one or all packages in nix/sources.json.
Drop: deletes a package from nix/sources.json.

niv has some utility functions:

Init: bootstraps a Nix project, in particular creates a nix/sources.json file containing niv and nixpkgs as well as a nix/sources.nix file that returns the sources as a Nix object.
Show: shows the packages' information.
Modify: modifies attributes without performing an update.

Configuration

The following environment variables are read by niv:

Name	Note
GITHUB_TOKEN or NIV_GITHUB_TOKEN	When set, the value is used to authenticate GitHub API requests.
GITHUB_HOST or NIV_GITHUB_HOST	The GitHub host to use when fetching packages. Port may be appended here.
GITHUB_API_HOST or NIV_GITHUB_API_HOST	The host used when performing GitHub API requests. Use `GITHUB_API_PORT` for specifying the port.
GITHUB_API_PORT or NIV_GITHUB_API_PORT	The port used when performing GitHub API requests. Defaults to `443` for secure requests. Defaults to `80` for insecure requests. See also: `GITHUB_INSECURE`.
NIV_GITHUB_INSECURE	When set to anything but the empty string, requests are performed over `http` instead of `https`.
NIV_GITHUB_PATH	The base path used when performing GitHub API requests.

The next two sections cover common use cases and full command description.

Getting started

Nix is a very powerful tool for building code and setting up environments. niv complements it by making it easy to describe and update remote dependencies (URLs, GitHub repos, etc). It is a simple, practical alternative to Nix flakes.

This section covers common use cases:

Bootstrapping a Nix project.
Tracking a different nixpkgs branch.
Importing packages from GitHub.
Fetching packages from custom URLs.

Bootstrapping a Nix project

Use the init command when starting a new Nix project or when porting an existing Nix project to niv:

$ niv init
...
$ tree
.
└── nix
    ├── sources.json
    └── sources.nix

1 directory, 2 files

The file nix/sources.json is the file used by niv to store versions and is initialized with nixpkgs:

{
    "nixpkgs": {
        "branch": "nixos-unstable",
        "description": "Nix Packages collection",
        "homepage": null,
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "6c43a3495a11e261e5f41e5d7eda2d71dae1b2fe",
        "sha256": "16f329z831bq7l3wn1dfvbkh95l2gcggdwn6rk3cisdmv2aa3189",
        "type": "tarball",
        "url": "https://github.com/NixOS/nixpkgs/archive/6c43a3495a11e261e5f41e5d7eda2d71dae1b2fe.tar.gz",
        "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
    }
}

To use this dependency, import the file nix/sources.nix, e.g.:

{ sources ? import ./sources.nix }:     # import the sources
import sources.nixpkgs                  # and use them again!
  { overlays = [] ; config = {}; }

For more information about importing sources to your nix files, check the frequently asked questions.

Tracking a nixpkgs branch

The init command sets the nix/sources.json to the content of the file data/nixpkgs.json. Currently, you would be tracking the nixos-unstable branch. Run the following command to update it to the last commit of the configured branch:

$ niv update nixpkgs

To change the branch being tracked run this command:

$ niv update nixpkgs -b master     # equivalent to --branch master

Importing packages from GitHub

The add command will infer information about the package being added, when possible. This works very well for GitHub repositories. Run this command to add jq to your project:

$ niv add stedolan/jq

The following data was added in nix/sources.json for jq:

{
  "homepage": "http://stedolan.github.io/jq/",
  "url": "https://github.com/stedolan/jq/archive/9fa2e51099c55af56e3e541dc4b399f11de74abe.tar.gz",
  "owner": "stedolan",
  "branch": "master",
  "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz",
  "repo": "jq",
  "sha256": "0819rvk8057qgcqvgn7fpldvly2pfdw9fxcjrlqa8gr59p8a1cic",
  "description": "Command-line JSON processor",
  "rev": "9fa2e51099c55af56e3e541dc4b399f11de74abe"
}

Using custom URLs

It is possible to use niv to fetch packages from custom URLs. Run this command to add the Haskell compiler GHC to your nix/sources.json:

$ niv add ghc   \
    -v 8.4.3    \
    -t 'https://downloads.haskell.org/~ghc/<version>/ghc-<version>-i386-deb8-linux.tar.xz'

The option -v sets the "version" attribute to 8.4.3. The option -t sets a template that can be reused by niv when fetching a new URL (see the documentation for add and update).

The type of the dependency is guessed from the provided URL template, if -T is not specified.

For updating the version of GHC used run this command:

$ niv update ghc -v 8.6.2

Commands

niv - dependency manager for Nix projects

version: 0.2.22

Usage: niv [-s|--sources-file FILE] [--no-colors] COMMAND

Available options:
  -s,--sources-file FILE   Use FILE instead of nix/sources.json
  --no-colors              Don't use colors in output
  -h,--help                Show this help text
  --version                Print version

Available commands:
  init                     Initialize a Nix project. Existing files won't be
                           modified.
  add                      Add a GitHub dependency
  show                     
  update                   Update dependencies
  modify                   Modify dependency attributes without performing an
                           update
  drop                     Drop dependency

Add

Examples:

  niv add stedolan/jq
  niv add NixOS/nixpkgs -n nixpkgs -b nixpkgs-unstable
  niv add my-package -v alpha-0.1 -t http://example.com/archive/<version>.zip

Usage: niv add PACKAGE [-n|--name NAME] 
                 [(-a|--attribute KEY=VAL) | (-s|--string-attribute KEY=VAL) | 
                   (-b|--branch BRANCH) | (-o|--owner OWNER) | (-r|--rev REV) | 
                   (-v|--version VERSION) | (-t|--template URL) | 
                   (-T|--type TYPE)]

  Add a GitHub dependency

Available options:
  -n,--name NAME           Set the package name to <NAME>
  -a,--attribute KEY=VAL   Set the package spec attribute <KEY> to <VAL>, where
                           <VAL> may be JSON.
  -s,--string-attribute KEY=VAL
                           Set the package spec attribute <KEY> to <VAL>.
  -b,--branch BRANCH       Equivalent to --attribute branch=<BRANCH>
  -o,--owner OWNER         Equivalent to --attribute owner=<OWNER>
  -r,--rev REV             Equivalent to --attribute rev=<REV>
  -v,--version VERSION     Equivalent to --attribute version=<VERSION>
  -t,--template URL        Used during 'update' when building URL. Occurrences
                           of <foo> are replaced with attribute 'foo'.
  -T,--type TYPE           The type of the URL target. The value can be either
                           'file' or 'tarball'. If not set, the value is
                           inferred from the suffix of the URL.
  -h,--help                Show this help text

Experimental commands:
  git                      Add a git dependency. Experimental.
  github                   Add a GitHub dependency
  local                    Add a local dependency. Experimental.

Update

Examples:
  
  niv update                     # update all packages
  niv update nixpkgs             # update nixpkgs
  niv update my-package -v beta-0.2 # update my-package to version "beta-0.2"

Usage: niv update [PACKAGE 
                    [(-a|--attribute KEY=VAL) | 
                      (-s|--string-attribute KEY=VAL) | (-b|--branch BRANCH) | 
                      (-o|--owner OWNER) | (-r|--rev REV) | 
                      (-v|--version VERSION) | (-t|--template URL) | 
                      (-T|--type TYPE)]]

  Update dependencies

Available options:
  -a,--attribute KEY=VAL   Set the package spec attribute <KEY> to <VAL>, where
                           <VAL> may be JSON.
  -s,--string-attribute KEY=VAL
                           Set the package spec attribute <KEY> to <VAL>.
  -b,--branch BRANCH       Equivalent to --attribute branch=<BRANCH>
  -o,--owner OWNER         Equivalent to --attribute owner=<OWNER>
  -r,--rev REV             Equivalent to --attribute rev=<REV>
  -v,--version VERSION     Equivalent to --attribute version=<VERSION>
  -t,--template URL        Used during 'update' when building URL. Occurrences
                           of <foo> are replaced with attribute 'foo'.
  -T,--type TYPE           The type of the URL target. The value can be either
                           'file' or 'tarball'. If not set, the value is
                           inferred from the suffix of the URL.
  -h,--help                Show this help text

Modify

Examples:

  niv modify nixpkgs -v beta-0.2
  niv modify nixpkgs -a branch=nixpkgs-unstable

Usage: niv modify PACKAGE [-n|--name NAME] 
                  [(-a|--attribute KEY=VAL) | (-s|--string-attribute KEY=VAL) | 
                    (-b|--branch BRANCH) | (-o|--owner OWNER) | (-r|--rev REV) |
                    (-v|--version VERSION) | (-t|--template URL) | 
                    (-T|--type TYPE)]

  Modify dependency attributes without performing an update

Available options:
  -n,--name NAME           Set the package name to <NAME>
  -a,--attribute KEY=VAL   Set the package spec attribute <KEY> to <VAL>, where
                           <VAL> may be JSON.
  -s,--string-attribute KEY=VAL
                           Set the package spec attribute <KEY> to <VAL>.
  -b,--branch BRANCH       Equivalent to --attribute branch=<BRANCH>
  -o,--owner OWNER         Equivalent to --attribute owner=<OWNER>
  -r,--rev REV             Equivalent to --attribute rev=<REV>
  -v,--version VERSION     Equivalent to --attribute version=<VERSION>
  -t,--template URL        Used during 'update' when building URL. Occurrences
                           of <foo> are replaced with attribute 'foo'.
  -T,--type TYPE           The type of the URL target. The value can be either
                           'file' or 'tarball'. If not set, the value is
                           inferred from the suffix of the URL.
  -h,--help                Show this help text

Drop

Examples:

  niv drop jq
  niv drop my-package version

Usage: niv drop PACKAGE [ATTRIBUTE]

  Drop dependency

Available options:
  -h,--help                Show this help text

Init

Usage: niv init [--fast | --latest | --nixpkgs OWNER/REPO
                  (-b|--nixpkgs-branch ARG) |
                  --no-nixpkgs]

  Initialize a Nix project. Existing files won't be modified.

Available options:
  --fast                   Use the latest nixpkgs cached at
                           'https://github.com/nmattia/niv/blob/master/data/nixpkgs.json'.
                           This is the default.
  --latest                 Pull the latest unstable nixpkgs from NixOS/nixpkgs.
  --nixpkgs OWNER/REPO     Use a custom nixpkgs repository from GitHub.
  -b,--nixpkgs-branch ARG  The nixpkgs branch when using --nixpkgs ....
  --no-nixpkgs             Don't add a nixpkgs entry to sources.json.
  -h,--help                Show this help text

show

Usage: niv show [PACKAGE]

Available options:
  -h,--help                Show this help text

Frequently Asked Questions

Can I use private GitHub repositories?
How do I import and use the content of a source?
How do I import a subpath of a source?
How do I import NixOS modules
Can I use local packages?
Can I use git submodules?

Can I use private GitHub repositories?

Yes. There are two ways:

1. Use the git protocol

When using the git protocol, your public SSH keypair is used to authenticate you:

$ niv add git [email protected]:my_user/my_private_repo

2. Use the netrc file

in order to niv add a private github repo you'll need to:

create a .netrc file with the following content

machine github.com
  login YOUR_GITHUB_USER_NAME
  password YOUR_GITHUB_TOKEN

add the path to the above file to /etc/nix/nix.conf:

netrc-file = /PATH/TO/.netrc

set GITHUB_TOKEN env var when calling niv add

GITHUB_TOKEN=$YOUR_GITHUB_TOKEN niv add ...

How do I import and use the content of a source?

The way to import a source depend mainly on the content targetted by this source. A source could be a file, a repository with no knowledge of nix or a repository already in the nix ecosystem.

1. Direct import of a nix based source

In the case of a nix based source, you'll often find a default.nix at the root. Let's take this repository as example. We can add it to our sources.json with the following command.

$ niv add nmattia/niv

We can now import niv to use it a nix expression, e.g.:

{ sources ? import nix/sources.nix }:
let niv = import sources.niv {};
in { inherit niv; } # A glorious expression using the reference to niv

2. Import of a nix based source via an overlay

Rather than use the resulting derivation directly, you can add it to your custom nixpkgs via the overlay system.

{ sources ? import nix/sources.nix}:
let overlay = _: pkgs: {
      niv = (import sources.niv {}).niv;
    };
    nixpkgs = import sources.nixpkgs { overlays = [ overlay ]; config = {}; };
in { inherit (nixpkgs) niv; } # A glorious expression where nixpkgs.niv is referenced

3. Reference to the source's files in the nix store

You can also reference a simple file, a folder or a repo without nix knowledge with niv. In these cases, you can use the source in your nix expression without importing it.

The following exemple will compile gnu hello while using this technique to retrieve the source. First, we need to add the new source.

$ niv add hello-src -v 2.10 -t 'https://ftp.gnu.org/gnu/hello/hello-<version>.tar.gz'

Then, we can use it inside a nix expression.

{ sources ? import nix/sources.nix }:
let hello_src = sources.hello-src;
    nixpkgs =  import sources.nixpkgs {};
in nixpkgs.stdenv.mkDerivation {
    pname = "hello";
    version = "custom";
    src = hello_src;
}

⚠️ If you have problems, consider using the outPath of the source (e.g. sources.hello-src.outPath) instead of the source directly. See this issue for more details.

How do I import a subpath of a source?

In order to use the directory dir of a my-package, use the following pattern:

let
  sources = import ./nix/sources.nix;
in sources.my-package + "/dir"

in this example, sources.my-package becomes my-package's root directory, and + "/dir" appends the subdirectory.

How do I import NixOS modules?

After the package containing the modules has been niv added, importing the modules is straightforward:

let
  sources = import ./nix/sources.nix;
in {
  imports = [ (sources.package + "/path/to/module") ];
}

Can I use local packages?

If you need to use a local path as a source -- especially convenient when modifying dependencies -- niv allows you to override the sources.json via environment variables. To override a source foo with a local path ./bar/baz, set the environment variable NIV_OVERRIDE_foo to ./bar/baz.

Generally, if the environment variable NIV_OVERRIDE_<name> is set and you have a source named <name> then niv will use the value of NIV_OVERRIDE_<name> as the outPath of that source. All non-alphanumeric characters in the source name are escaped to the character _; i.e. to override the package my package-foo you need to set the environment variable NIV_OVERRIDE_my_package_foo.

Can I use a git dependency with submodules?

Yes, however you need to follow some steps.

Add your dependency as git dependency to your sources.json:

niv add git [email protected]:user/repo -n name

Add "submodules": true, to your dependecy in the source.json:

{
    "name": {
        "branch": "main",
        "repo": "[email protected]:user/repo",
        "rev": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
        "submodules": true,
        "type": "git"
    }
}

	prePatch = ''
	sed "s\|GH.executeRequest'\|GH.executeRequest (GH.EnterpriseOAuth \"http://localhost:3333\" \"\")\|" -i app/Niv.hs
	sed "s\|https://github.com\|http://localhost:3333\|" -i app/Niv.hs
	'';

nmattia / niv Goto Github PK

niv's Introduction

niv