Giter VIP home page Giter VIP logo

malice00 / cdxgen Goto Github PK

View Code? Open in Web Editor NEW

This project forked from cyclonedx/cdxgen

0.0 0.0 0.0 8.69 MB

Creates CycloneDX Software Bill-of-Materials (SBOM) for your projects from source and container images. Supports many languages and package managers. Integrate in your CI//CD pipeline with automatic submission to Dependency Track server.

Home Page: https://cyclonedx.slack.com/archives/C04NFFE1962

License: Apache License 2.0

Shell 0.55% JavaScript 92.84% Ruby 0.31% Python 0.76% Clojure 0.45% Elixir 0.78% TypeScript 0.08% Dockerfile 1.20% Roff 2.92% Starlark 0.12%

cdxgen's Introduction

CycloneDX Generator

cdxgen logo

cdxgen is a cli tool, library, REPL and server to create a valid and compliant CycloneDX Software Bill-of-Materials (SBOM) containing an aggregate of all project dependencies for c/c++, node.js, php, python, ruby, rust, java, .Net, dart, haskell, elixir, and Go projects in JSON format. CycloneDX 1.5 is a lightweight SBOM specification that is easily created, human and machine-readable, and simple to parse.

When used with plugins, cdxgen could generate an SBoM for Linux docker images and even VMs running Linux or Windows operating system. cdxgen also includes a tool called evinse that can generate component evidences for some languages.

NOTE:

CycloneDX 1.5 specification is new and unsupported by many downstream tools. Use version 8.6.0 for 1.4 compatibility or pass the argument --spec-version 1.4.

Why cdxgen?

A typical application might comprise of several repos, components, and libraries linked together. Traditional techniques to generate a single SBoM per language or package manifest do not work in enterprise environments. So we built cdxgen - the universal polyglot SBoM generator!

why cdxgen

Supported languages and package format

Language/Platform Package format Transitive dependencies
node.js npm-shrinkwrap.json, package-lock.json, pnpm-lock.yaml, yarn.lock, rush.js, bower.json, .min.js Yes except .min.js
java maven (pom.xml [1]), gradle (build.gradle, .kts), scala (sbt), bazel Yes unless pom.xml is manually parsed due to unavailability of maven or errors
php composer.lock Yes
python pyproject.toml, setup.py, requirements.txt [2], Pipfile.lock, poetry.lock, pdm.lock, bdist_wheel, .whl, .egg-info Yes using the automatic pip install/freeze. When disabled, only with Pipfile.lock and poetry.lock
go binary, go.mod, go.sum, Gopkg.lock Yes except binary
ruby Gemfile.lock, gemspec Only for Gemfile.lock
rust binary, Cargo.toml, Cargo.lock Only for Cargo.lock
.Net .csproj, packages.config, project.assets.json [3], packages.lock.json, .nupkg Only for project.assets.json, packages.lock.json
dart pubspec.lock, pubspec.yaml Only for pubspec.lock
haskell cabal.project.freeze Yes
elixir mix.lock Yes
c/c++ conan.lock, conanfile.txt Yes only for conan.lock
clojure Clojure CLI (deps.edn), Leiningen (project.clj) Yes unless the files are parsed manually due to lack of clojure cli or leiningen command
swift Package.resolved, Package.swift (swiftpm) Yes
docker / oci image All supported languages. Linux OS packages with plugins [4] Best effort based on lock files
GitHub Actions .github/workflows/*.yml N/A
Linux All supported languages. Linux OS packages with plugins [5] Best effort based on lock files
Windows All supported languages. OS packages with best effort [5] Best effort based on lock files
Jenkins Plugins .hpi files
Helm Charts .yaml N/A
Skaffold .yaml N/A
kustomization .yaml N/A
Tekton tasks .yaml N/A
Kubernetes .yaml N/A
Maven Cache $HOME/.m2/repository/**/*.jar N/A
SBT Cache $HOME/.ivy2/cache/**/*.jar N/A
Gradle Cache $HOME/caches/modules-2/files-2.1/**/*.jar N/A
Helm Index $HOME/.cache/helm/repository/**/*.yaml N/A
Docker compose docker-compose*.yml. Images would also be scanned. N/A
Google CloudBuild configuration cloudbuild.yaml N/A
OpenAPI openapi*.json, openapi*.yaml N/A

NOTE:

  • Apache maven 3.x is required for parsing pom.xml
  • gradle or gradlew is required to parse gradle projects
  • sbt is required for parsing scala sbt projects. Only scala 2.10 + sbt 0.13.6+ and 2.12 + sbt 1.0+ is supported for now.
    • Alternatively, create a lock file using sbt-dependency-lock plugin

Footnotes:

  • [1] - For multi-module application, the BoM file could include components that may not be included in the packaged war or ear file.
  • [2] - Pip freeze is automatically performed to improve precision. Requires virtual environment.
  • [3] - Perform dotnet or nuget restore to generate project.assets.json. Without this file cdxgen would not include indirect dependencies.
  • [4] - See section on plugins
  • [5] - Powered by osquery. See section on plugins

cdxgen tree

Automatic usage detection

For node.js projects, lock files are parsed initially so the SBoM would include all dependencies including dev dependencies. An AST parser powered by babel-parser is then used to detect packages that are imported and used by non-test code. Such imported packages would automatically have their scope property set to required in the resulting SBoM. By passing the argument --no-babel, you can disable this analysis. Scope property would then be set based on the dev attribute in the lock file.

This attribute can be later used for various purposes. For example, dep-scan use this attribute to prioritize vulnerabilities. Tools such dependency track, unfortunately, do not include this feature and hence might over-report the CVEs.

By passing the argument --required-only, you can limit the SBoM to only include packages with the scope "required", commonly referred to as production or non-dev dependencies. Combine with --no-babel to limit this list to only non-dev dependencies based on the dev attribute being false in the lock files.

For go, go mod why command is used to identify required packages. For php, composer lock file is used to distinguish required (packages) from optional (packages-dev).

Usage

Installing

sudo npm install -g @cyclonedx/cdxgen

# For CycloneDX 1.4 compatibility use version 8.6.0 or pass the argument `--spec-version 1.4`
sudo npm install -g @cyclonedx/[email protected]

Deno install is also supported.

deno install --allow-read --allow-env --allow-run --allow-sys=uid,systemMemoryInfo --allow-write --allow-net -n cdxgen "npm:@cyclonedx/cdxgen/cdxgen"

You can also use the cdxgen container image

docker run --rm -v /tmp:/tmp -v $(pwd):/app:rw -t ghcr.io/cyclonedx/cdxgen -r /app -o /app/bom.json

docker run --rm -v /tmp:/tmp -v $(pwd):/app:rw -t ghcr.io/cyclonedx/cdxgen:v8.6.0 -r /app -o /app/bom.json

To use the deno version, use ghcr.io/cyclonedx/cdxgen-deno as the image name.

docker run --rm -v /tmp:/tmp -v $(pwd):/app:rw -t ghcr.io/cyclonedx/cdxgen-deno -r /app -o /app/bom.json

In deno applications, cdxgen could be directly imported without any conversion. Please see the section on integration as library

import { createBom, submitBom } from "npm:@cyclonedx/cdxgen@^9.0.1";

Getting Help

$ cdxgen -h
Options:
  -o, --output                 Output file for bom.xml or bom.json. Default bom.
                               json
  -t, --type                   Project type
  -r, --recurse                Recurse mode suitable for mono-repos. Defaults to
                                true. Pass --no-recurse to disable.
                                                       [boolean] [default: true]
  -p, --print                  Print the SBoM as a table with tree. Defaults to
                               true if output file is not specified with -o
                                                                       [boolean]
  -c, --resolve-class          Resolve class names for packages. jars only for n
                               ow.                                     [boolean]
      --deep                   Perform deep searches for components. Useful whil
                               e scanning live OS and oci images.      [boolean]
      --server-url             Dependency track url. Eg: https://deptrack.cyclon
                               edx.io
      --api-key                Dependency track api key
      --project-group          Dependency track project group
      --project-name           Dependency track project name. Default use the di
                               rectory name
      --project-version        Dependency track project version    [default: ""]
      --project-id             Dependency track project id. Either provide the i
                               d or the project name and version together
      --required-only          Include only the packages with required scope on
                               the SBoM.                               [boolean]
      --fail-on-error          Fail if any dependency extractor fails. [boolean]
      --no-babel               Do not use babel to perform usage analysis for Ja
                               vaScript/TypeScript projects.           [boolean]
      --generate-key-and-sign  Generate an RSA public/private key pair and then
                               sign the generated SBoM using JSON Web Signatures
                               .                                       [boolean]
      --server                 Run cdxgen as a server                  [boolean]
      --server-host            Listen address             [default: "127.0.0.1"]
      --server-port            Listen port                     [default: "9090"]
      --install-deps           Install dependencies automatically for some proje
                               cts. Defaults to true but disabled for containers
                                and oci scans. Use --no-install-deps to disable
                               this feature.           [boolean] [default: true]
      --validate               Validate the generated SBoM using json schema. De
                               faults to true. Pass --no-validate to disable.
                                                       [boolean] [default: true]
      --spec-version           CycloneDX Specification version to use. Defaults
                               to 1.5                             [default: 1.5]
      --version                Show version number                     [boolean]
  -h                           Show help                               [boolean]

All boolean arguments accepts --no prefix to toggle the behavior.

Example

Minimal example.

cdxgen -o bom.json

For a java project. This would automatically detect maven, gradle or sbt and build bom accordingly

cdxgen -t java -o bom.json

To print the SBoM as a table pass -p argument.

cdxgen -t java -o bom.json -p

To recursively generate a single BoM for all languages pass -r argument.

cdxgen -r -o bom.json

To generate SBoM for an older specification version such as 1.4, pass the version using the --spec-version argument.

cdxgen -r -o bom.json --spec-version 1.4

Universal SBoM

By passing the type -t universal, cdxgen could be forced to opportunistically collect as many components and services as possible by scanning all package, container and kubernetes manifests. The resulting SBoM could have over thousand components thus requiring additional triaging before use with traditional SCA tools.

SBoM server

Invoke cdxgen with --server argument to run it in a server mode. By default, it listens to port 9090 which can be customized with the arguments --server-host and --server-port.

cdxgen --server

Or use the container image.

docker run --rm -v /tmp:/tmp -p 9090:9090 -v $(pwd):/app:rw -t ghcr.io/cyclonedx/cdxgen -r /app --server --server-host 0.0.0.0

Use curl or your favourite tool to pass arguments to the /sbom route.

Scanning a local path

curl "http://127.0.0.1:9090/sbom?path=/Volumes/Work/sandbox/vulnerable-aws-koa-app&multiProject=true&type=js"

Scanning a git repo

curl "http://127.0.0.1:9090/sbom?url=https://github.com/HooliCorp/vulnerable-aws-koa-app.git&multiProject=true&type=js"

You can POST the arguments.

curl -H "Content-Type: application/json" http://localhost:9090/sbom -XPOST -d $'{"url": "https://github.com/HooliCorp/vulnerable-aws-koa-app.git", "type": "nodejs", "multiProject": "true"}'

Docker compose

git clone https://github.com/cyclonedx/cdxgen.git
docker compose up

War file support

cdxgen can generate a BoM file from a given war file.

# cdxgen -t java app.war
cdxgen app.war

Resolving class names

Sometimes it is necessary to resolve class names contained in jar files. By passing an optional argument --resolve-class, it is possible to get cdxgen create a separate mapping file with the jar name (including the version) as the key and class names list as a value.

cdxgen -t java --resolve-class -o bom.json

This would create a bom.json.map file with the jar - class name mapping. Refer to these examples to learn about the structure.

Resolving licenses

cdxgen can automatically query the public registries such as maven or npm or nuget to resolve the package licenses. This is a time consuming operation and is disabled by default. To enable, set the environment variable FETCH_LICENSE to true as shown.

export FETCH_LICENSE=true

Dependency Tree

cdxgen can retain the dependency tree under the dependencies attribute for a small number of supported package manifests. These are currently limited to:

  • package-lock.json
  • yarn.lock
  • pnpm-lock.yaml
  • Maven (pom.xml)
  • Gradle
  • Scala SBT
  • Python (requirements.txt, setup.py, pyproject.toml, poetry.lock)

Environment variables

Variable Description
CDXGEN_DEBUG_MODE Set to debug to enable debug messages
GITHUB_TOKEN Specify GitHub token to prevent traffic shaping while querying license and repo information
MVN_CMD Set to override maven command
MVN_ARGS Set to pass additional arguments such as profile or settings to maven
MAVEN_HOME Specify maven home
GRADLE_CACHE_DIR Specify gradle cache directory. Useful for class name resolving
GRADLE_MULTI_PROJECT_MODE Unused. Automatically handled
GRADLE_ARGS Set to pass additional arguments such as profile or settings to gradle (all tasks). Eg: --configuration runtimeClassPath
GRADLE_ARGS_PROPERTIES Set to pass additional arguments only to the gradle properties task, used for collecting metadata about the project
GRADLE_ARGS_DEPENDENCIES Set to pass additional arguments only to the gradle dependencies task, used for listing actual project dependencies
GRADLE_HOME Specify gradle home
GRADLE_CMD Set to override gradle command
GRADLE_DEPENDENCY_TASK By default cdxgen use the task "dependencies" to collect packages. Set to override the task name.
SBT_CACHE_DIR Specify sbt cache directory. Useful for class name resolving
FETCH_LICENSE Set this variable to true or 1 to fetch license information from the registry. npm and golang
USE_GOSUM Set to true or 1 to generate BOMs for golang projects using go.sum as the dependency source of truth, instead of go.mod
CDXGEN_TIMEOUT_MS Default timeout for known execution involving maven, gradle or sbt
CDXGEN_SERVER_TIMEOUT_MS Default timeout in server mode
BAZEL_TARGET Bazel target to build. Default :all (Eg: //java-maven)
CLJ_CMD Set to override the clojure cli command
LEIN_CMD Set to override the leiningen command
SBOM_SIGN_ALGORITHM Signature algorithm. Some valid values are RS256, RS384, RS512, PS256, PS384, PS512, ES256 etc
SBOM_SIGN_PRIVATE_KEY Private key to use for signing
SBOM_SIGN_PUBLIC_KEY Optional. Public key to include in the SBoM signature
CDX_MAVEN_PLUGIN CycloneDX Maven plugin to use. Default "org.cyclonedx:cyclonedx-maven-plugin:2.7.8"
CDX_MAVEN_GOAL CycloneDX Maven plugin goal to use. Default makeAggregateBom. Other options: makeBom, makePackageBom
CDX_MAVEN_INCLUDE_TEST_SCOPE Whether test scoped dependencies should be included from Maven projects, Default: true
ASTGEN_IGNORE_DIRS Comma separated list of directories to ignore while analyzing using babel. The environment variable is also used by atom and astgen.
ASTGEN_IGNORE_FILE_PATTERN Ignore regex to use

Plugins

cdxgen could be extended with external binary plugins to support more SBoM use cases. These are now installed as an optional dependency.

sudo npm install -g @cyclonedx/cdxgen-plugins-bin

Docker / OCI container support

docker type is automatically detected based on the presence of values such as sha256 or docker.io prefix etc in the path.

cdxgen odoo@sha256:4e1e147f0e6714e8f8c5806d2b484075b4076ca50490577cdf9162566086d15e -o /tmp/bom.json

You can also pass -t docker for simple labels. Only the latest tag would be pulled if none was specified.

cdxgen shiftleft/scan-slim -o /tmp/bom.json -t docker

You can also pass the .tar file of a container image.

docker pull shiftleft/scan-slim
docker save -o /tmp/slim.tar shiftleft/scan-slim
podman save -q --format oci-archive -o /tmp/slim.tar shiftleft/scan-slim
cdxgen /tmp/slim.tar -o /tmp/bom.json -t docker

NOTE:

  • Only application related packages are collected by cdxgen. Support for OS installed packages is coming soon.

Podman in rootless mode

Setup podman in either rootless or remote mode

On Linux, do not forget to start the podman socket which is required for API access.

systemctl --user enable --now podman.socket
systemctl --user start podman.socket
podman system service -t 0 &

Generate SBoM for a live system

You can use cdxgen to generate SBoM for a live system or a VM for compliance and vulnerability management purposes by passing the argument -t os.

cdxgen -t os

This feature is powered by osquery which is installed along with the binary plugins. cdxgen would opportunistically try to detect as many components, apps and extensions as possible using the default queries. The process would take several minutes and result in an SBoM file with thousands of components.

Generating component evidence

See evinse mode in the advanced documentation.

SBoM signing

cdxgen can sign the generated SBoM json file to increase authenticity and non-repudiation capabilities. To enable this, set the following environment variables.

  • SBOM_SIGN_ALGORITHM: Algorithm. Example: RS512
  • SBOM_SIGN_PRIVATE_KEY: Location to the RSA private key
  • SBOM_SIGN_PUBLIC_KEY: Optional. Location to the RSA public key

To generate test public/private key pairs, you can run cdxgen by passing the argument --generate-key-and-sign. The generated json file would have an attribute called signature which could be used for validation. jwt.io is a known site that could be used for such signature validation.

SBoM signing

Verifying the signature

Use the bundled cdx-verify command which supports verifying a single signature added at the bom level.

npm install -g @cyclonedx/cdxgen
cdx-verify -i bom.json --public-key public.key

Custom verification tool (Node.js example)

There are many libraries available to validate JSON Web Tokens. Below is a javascript example.

# npm install jws
const jws = require("jws");
const fs = require("fs");
// Location of the SBoM json file
const bomJsonFile = "bom.json";
// Location of the public key
const publicKeyFile = "public.key";
const bomJson = JSON.parse(fs.readFileSync(bomJsonFile, "utf8"));
// Retrieve the signature
const bomSignature = bomJson.signature.value;
const validationResult = jws.verify(bomSignature, bomJson.signature.algorithm, fs.readFileSync(publicKeyFile, "utf8"));
if (validationResult) {
  console.log("Signature is valid!");
} else {
  console.log("SBoM signature is invalid :(");
}

Automatic services detection

cdxgen can automatically detect names of services from YAML manifests such as docker-compose or Kubernetes or Skaffold manifests. These would be populated under the services attribute in the generated SBoM. With evinse, additional services could be detected by parsing common annotations from the source code.

Conversion to SPDX format

Use the CycloneDX CLI tool for advanced use cases such as conversion, diff and merging.

License

Permission to modify and redistribute is granted under the terms of the Apache 2.0 license. See the LICENSE file for the full license.

Integration as library

cdxgen is ESM only and could be imported and used with both deno and Node.js >= 16

Minimal example:

import { createBom, submitBom } from "npm:@cyclonedx/cdxgen@^9.0.1";

See the Deno Readme for detailed instruction.

import { createBom, submitBom } from "@cyclonedx/cdxgen";
// bomNSData would contain bomJson, bomXml
const bomNSData = await createBom(filePath, options);
// Submission to dependency track server
const dbody = await submitBom(args, bomNSData.bomJson);

Node.js >= 20 permission model

Refer to the permissions document

Contributing

Follow the usual PR process but prior to raising a PR run the following commands.

npm run lint
npm run pretty
npm test

Enterprise support

Enterprise support including custom development and integration services are available via AppThreat Ltd. Free community support is also available via discord.

cdxgen's People

Contributors

ajmalab avatar andreyeto avatar anush-cr avatar bobflannigon avatar candrews avatar cerrussell avatar dduarte avatar dependabot-preview[bot] avatar eoftedal avatar gitnachogo avatar heubeck avatar hubertp avatar jkowalleck avatar kro29200 avatar ksg97031 avatar malice00 avatar mikejarrett avatar mistog4n avatar prabhu avatar raineinto avatar rsichnyi avatar rwanauo avatar stevespringett avatar sullivtr avatar tree-chtsec avatar vaaralav avatar visagansanthanam-unisys avatar willienel avatar wr-rmcnamara avatar yutayamate avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.