tiiuae / sbomnix Goto Github PK

View Code? Open in Web Editor NEW

109.0 8.0 19.0 2.55 MB

A suite of utilities to help with software supply chain challenges on nix targets

Makefile 0.84% Python 89.16% Shell 3.78% Nix 6.22%

cyclonedx nix sbom sbom-generator sbom-tool python bill-of-materials cpe dependencies purl

sbomnix's People

Stargazers

Watchers

Forkers

henrirosten vilvo emattiza fractal-dynamics jayrovacsek nikitawootten flox sarcasticadmin antonminix peterhoeg jdiaz5513 thommahoney olebedev remimimimimi brianmcgee flokli raboof joinemm

sbomnix's Issues

rewrite mapping nix packages to cpe identifiers

NVD plans to retire legacy data feeds on 09/2023: https://nvd.nist.gov/products/cpe

Currently, sbomnix uses NVD "CPE Dictionary" in mapping the nix pakcages to CPE identifiers, see: https://github.com/tiiuae/sbomnix/blob/main/scripts/cpedict/update-cpedict.sh and https://github.com/tiiuae/sbomnix/blob/main/sbomnix/cpe.py.

We need to rethink how to properly do this in sbomnix to make it more accurate and so that it does not rely on the to-be-retired NVD data feed.

All suggestions or ideas how to improve the CPE mapping are welcome.

sbomnix: support aarch64-darwin

PR #58 added support for multiple systems, but @nikitawootten mentioned dropping support for aarch64-darwin due to failing tests.

This issue is a reminder to analyze the issue and, if possible, add sbomnix support for aarch64-darwin.

allow reading runtime dependencies without requiring the package availability in local nix store

Currently, to get the runtime dependencies for a given target, sbomnix requires the target to be available in local nix store.
Note: for buildtime-only dependencies, only the derivation file is required.

The task in this item is to change sbomnix so that runtime dependencies can also be found without requiring the package be installed locally.

Example (nmap is not locally installed):

Buildtime:

sbomnix /nix/store/n54bb3k7p568x2i6abvkavmzc2nwgihh-nmap-7.93.drv --type=buildtime

WARNING  Command line argument '--meta' missing: SBOM will not include license information (see '--help' for more details)
INFO     Loading buildtime dependencies referenced by '/nix/store/n54bb3k7p568x2i6abvkavmzc2nwgihh-nmap-7.93.drv'
INFO     Wrote: sbom.cdx.json
INFO     Wrote: sbom.spdx.json
INFO     Wrote: sbom.csv

Runtime:

sbomnix /nix/store/n54bb3k7p568x2i6abvkavmzc2nwgihh-nmap-7.93.drv --type=runtime
WARNING  Command line argument '--meta' missing: SBOM will not include license information (see '--help' for more details)
INFO     Loading runtime dependencies referenced by '/nix/store/n54bb3k7p568x2i6abvkavmzc2nwgihh-nmap-7.93.drv'
Traceback (most recent call last):
  File "/home/hrosten/projects/sbomnix/venv/bin/sbomnix", line 33, in <module>
    sys.exit(load_entry_point('sbomnix', 'console_scripts', 'sbomnix')())
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/sbomnix/main.py", line 87, in main
    sbomdb = SbomDb(target_path, runtime, buildtime, args.meta)
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 48, in __init__
    self._init_dependencies(nix_path)
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 62, in _init_dependencies
    runtime_dependencies = NixDependencies(nix_path, buildtime=False)
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/nixgraph/graph.py", line 255, in __init__
    self._parse_runtime_dependencies(nix_path)
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/nixgraph/graph.py", line 261, in _parse_runtime_dependencies
    nix_out = exec_cmd(
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/sbomnix/utils.py", line 114, in exec_cmd
    raise error
  File "/nix/store/g7mndp0nh7jy7xc9gxv1jjdl6jxac7hi-python3.10-sbomnix-1.4.5/lib/python3.10/site-packages/sbomnix/utils.py", line 104, in exec_cmd
    ret = subprocess.run(cmd, capture_output=True, encoding="utf-8", check=True)
  File "/nix/store/syz2y6j53y5hpzbs7l0965zwxshi8iyl-python3-3.10.10/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['nix', '--extra-experimental-features', 'nix-command', 'path-info', '/nix/store/n54bb3k7p568x2i6abvkavmzc2nwgihh-nmap-7.93.drv']' returned non-zero exit status 1.

Add option to scan nix profile?

I.e. it can take ~/.nix-profile/manifest.json as input

Currently you get:

% sbomnix ~/.nix-profile              
INFO     Evaluating '/Users/arian/.nix-profile'
INFO     Try force-realising store-path '/Users/arian/.nix-profile'
INFO     Loading runtime dependencies referenced by '/nix/store/pwcgic86vfhhdkpbh03cn7pv7a58vdqh-profile'
CRITICAL No deriver found for: '/nix/store/pwcgic86vfhhdkpbh03cn7pv7a58vdqh-profile

sbomnix: go and rust dependencies

Dependencies from rust and go projects are missing because they fetch their dependencies directly, so the dependencies are not included in the dependency trees produced by nix-store --query --graph, which is what sbomnix uses internally.

For rust, this should be fixed with: NixOS/nixpkgs#217084, but the problem still persists for all other ecosystems that fetch their own dependencies.

All suggestions for how to improve the dependency lookup for sbomnix are welcome.

scanning can take a long time, is there any ability to keep / cache the results for n days?

Running the vulnxscan command twice in a row shows that there is some caching going on, because the second run returns almost immediately.

Rebooting causes the same vulnxscan command to run for a long time again.

I there a way to have finer grained control over this caching behaviour? Say to invalidate the cache after n days.

Any advice (or clarification of how caching is done) is appreciated.

vulnxscan: add severity information

Add severity information to vulnxscan results for all scanners/databases that include the severity information.

how to `nix build` vulnxscan?

nix build .#sbomnix works fine

but nix build .#vulnxscan gives

does not provide attribute 'packages.x86_64-linux.vulnxscan', 'legacyPackages.x86_64-linux.vulnxscan' or 'vulnxscan'

I see in the flake.nix that vulnxscan is an app. And it is clear to me than an app is meant to be nix run-d.

Except that if I run it this way, no gc root is created, and after reboot, all the derivations are gone, and I have to go make coffee while the run command fetches everything again. In my testing, only actually building the app/package will cause a gc root to be created, thereby caching the derivation.

Sorry if this is a stupid question, nix has been just head banging from the get go for me, I've only started using it over the weekend.

add tips&tricks page

add tips&tricks markdown page that would include some interesting non-obvious use-cases of sbomnix such as this:
https://discourse.nixos.org/t/how-to-get-package-maintainers-of-all-installed-packages/27307/12

and this, that creates a list of fetched urls given a result symlink:

# Run sbomnix; include both buildtime and runtime dependencies: 
$ nix run github:tiiuae/sbomnix#sbomnix -- ./result --type=both

# Query the sbom for fetched urls:
$ nix-shell -p csvkit --run \
  'csvsql --verbose --query "select urls from sbom where urls != '\'''\'' " sbom.csv' \
  | tr " " "\n" | grep -v urls | sort | uniq \
  > fetch_urls.txt

nixgraph: allow loading state

Change nixgraph (cli) so that it can load an earlier state (sbom?), to allow visualizing and querying the build target graph ‘postmortem’ without having to re-produce the earlier build environment locally.

only produce `output_path` for the actually-used output

when creating the runtime sbom for the Gnome ISO with sbomnix $(nix-build nixos/release-combined.nix -A nixos.iso_gnome.x86_64-linux) on nixpkgs 0e74ca98a74bc7270d28838369593635a5db3260, the CycloneDX JSON output contains:

{
      "type": "application",
      "bom-ref": "/nix/store/vkx2srall0p9q68hm73zpwq9cmv8pmjg-qemu-host-cpu-only-8.2.1.drv",
      "name": "qemu-host-cpu-only",
      "version": "8.2.1",
      "purl": "pkg:nix/[email protected]",
      "cpe": "cpe:2.3:a:qemu-host-cpu-only:qemu-host-cpu-only:8.2.1:*:*:*:*:*:*:*",
      "description": "A generic and open source machine emulator and virtualizer",
      "licenses": [
        {
          "license": {
            "id": "GPL-2.0-or-later"
          }
        }
      ],
      "properties": [
        {
          "name": "nix:output_path",
          "value": "/nix/store/38nyj07s5k4l7dhnpjap5clqdgx3b38k-qemu-host-cpu-only-8.2.1-ga"
        },
        {
          "name": "nix:output_path",
          "value": "/nix/store/c7cw8hgpb1wks5f91ijkm1xv9nmp2zk8-qemu-host-cpu-only-8.2.1"
        },
        {
          "name": "nix:drv_path",
          "value": "/nix/store/vkx2srall0p9q68hm73zpwq9cmv8pmjg-qemu-host-cpu-only-8.2.1.drv"
        },
        {
          "name": "homepage",
          "value": "https://www.qemu.org/"
        }
      ]
    },

However, looking at the output of nix-store -q --tree $(nix-build nixos/release-combined.nix -A nixos.iso_gnome.x86_64-linux), I would expect only the /nix/store/38nyj07s5k4l7dhnpjap5clqdgx3b38k-qemu-host-cpu-only-8.2.1-ga output path and not /nix/store/c7cw8hgpb1wks5f91ijkm1xv9nmp2zk8-qemu-host-cpu-only-8.2.1

TypeError: sequence item 0: expected str instance, list found

When running sbomnix, I get the following error:

Traceback (most recent call last):
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/bin/.sbomnix-wrapped", line 9, in <module>
    sys.exit(main())
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/lib/python3.10/site-packages/sbomnix/main.py", line 100, in main
    sbomdb = SbomDb(target_path, runtime, buildtime, args.meta, args.depth)
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 55, in __init__
    self._init_sbomdb()
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 105, in _init_sbomdb
    self._sbomdb_join_meta(self.meta_path)
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 116, in _sbomdb_join_meta
    df_meta = _parse_json_metadata(meta_path)
  File "/nix/store/mzck8rim82y9vx4xm18dp312gn88vni2-python3.10-sbomnix-1.4.6/lib/python3.10/site-packages/sbomnix/sbomdb.py", line 471, in _parse_json_metadata
    setcol("meta_maintainers_email", []).append(";".join(emails))
TypeError: sequence item 0: expected str instance, list found

The json metadata was created using the following command:

nix-env -qa --meta --json '.*' > ~/nix-meta.json

In case it's important, I invoked sbomnix using the following command:

nix run github:tiiuae/sbomnix#sbomnix -- $(readlink result) --type=both --meta=nix-meta.json

Can this be done globally - or just per-package?

Is there a way I can pass in a flake target and get a full system-sbom? or is just per-derivation

Packaging latest version in nixpkgs: "error: illegal path references in fixed-output derivation"

Hi,

I tried updating sbomnix (1.4.5 -> 1.6.0) in nixpkgs but hit this weird issue:

$ nix-build -A sbomnix
these 2 derivations will be built:
  /nix/store/g9zk3n7zhj4bgb22llns0y2kh2d89nr1-source.drv
  /nix/store/0y11vlrxgs9zlaxfv6dm6dkbclc0ml6h-sbomnix-1.6.0.drv
building '/nix/store/g9zk3n7zhj4bgb22llns0y2kh2d89nr1-source.drv'...

trying https://github.com/tiiuae/sbomnix/archive/refs/tags/v1.6.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  163k    0  163k    0     0   329k      0 --:--:-- --:--:-- --:--:--  329k
unpacking source archive /build/v1.6.0.tar.gz
error: illegal path references in fixed-output derivation '/nix/store/g9zk3n7zhj4bgb22llns0y2kh2d89nr1-source.drv'
error: 1 dependencies of derivation '/nix/store/0y11vlrxgs9zlaxfv6dm6dkbclc0ml6h-sbomnix-1.6.0.drv' failed to build

Not sure if this is an issue with Nix, nixpkgs or sbomnix, but thought I should start documenting the issue somewhere.

sbomnix: add option to include only direct dependencies

Maybe a --depth argument as seen in nixgraph

vulnxscan fails if the Nix output path is a JSON file

First of all, thanks for you project!

The nix2container project produces container image specification as JSON file. When running vulnxscan on a nix2container output path, it fails such as:

$ nix run github:tiiuae/sbomnix#vulnxscan -- ./result
CRITICAL Specified target is not a nix artifact: 'result'

$ file ./result
./result: symbolic link to /nix/store/7s4y6dcmfc6frqv38j8y6g7ifmazh5hx-image-bash.json

It seems to be because vulnxscan only consider non JSON file as Nix artifact:

sbomnix/scripts/vulnxscan/vulnxscan.py

Line 338 in f763f71

if _is_json(target_path_abs):

Instead of testing the file type, maybe you should run a Nix command on this store path to let Nix deciding if it is a Nix artefact or not. (nix-store -q <FILE> for instance).

Note this is related to this nix2container issue.

Example or documentation on how to use it via an API

Hello,

I'm currently investigating how I could potentially embed SBOMs in OCI containers made with Nix.
I would like to use this project which seems quite mature, but I wish I could use it through a Nix API, it would make sense to use it through it when building the container.

I have opened an issue at https://discourse.nixos.org/t/generate-sbom-from-oci-container-made-with-nix/39430, and I'm currently able to do what I want, but I wish I could also evaluate the use of this project too.

Thanks for shedding some lights on this!

improve the way sbomnix reads pacakge metadata

sbomnix reads the nix package meta info from the json file given via the command-line --meta option.

This needs to be re-done properly, ultimately aiming to get rid of the need for --meta command line argument completely.

tiiuae / sbomnix Goto Github PK

sbomnix's People

Stargazers

Watchers

Forkers

sbomnix's Issues

Recommend Projects

Recommend Topics

Recommend Org