Giter VIP home page Giter VIP logo

cyclonedx / cdxgen Goto Github PK

View Code? Open in Web Editor NEW
496.0 16.0 143.0 15.96 MB

Creates CycloneDX Bill of Materials (BOM) for your projects from source and container images. Supports many languages and package managers. Integrate in your CI/CD pipeline with automatic submission to Dependency Track server. Slack: https://cyclonedx.slack.com/archives/C04NFFE1962

Home Page: https://cyclonedx.github.io/cdxgen/

License: Apache License 2.0

JavaScript 88.27% Shell 0.31% Ruby 0.18% Starlark 0.07% Elixir 0.46% Roff 1.70% Clojure 0.26% Dockerfile 0.79% Python 1.99% TypeScript 0.05% CMake 5.23% Meson 0.54% Makefile 0.16%
bom sca cyclonedx sbom docker oci containers owasp package-url purl

cdxgen's Introduction

JSR NPM GitHub Releases NPM Downloads GitHub License GitHub Contributors SWH Libraries.io dependency status

CycloneDX Generator (cdxgen)

cdxgen logo

cdxgen is a CLI tool, library, REPL, and server to create a valid and compliant CycloneDX Bill of Materials (BOM) containing an aggregate of all project dependencies in JSON format. CycloneDX is a full-stack BOM specification that is easily created, human and machine-readable, and simple to parse. The tool supports CycloneDX specification versions from 1.4 - 1.6.

Supported BOM formats:

  • Software (SBOM) - For many languages and container images.
  • Cryptography (CBOM) - For Java and Python projects.
  • Operations (OBOM) - For Linux container images and VMs running Linux or Windows operating systems.
  • Software-as-a-Service (SaaSBOM) - For Java, Python, JavaScript, TypeScript, and PHP projects.
  • Attestations (CDXA) - Generate SBOM with templates for multiple standards. Sign the BOM document at a granular level to improve authenticity.
  • Vulnerability Disclosure Report (VDR) - Use cdxgen with OWASP depscan to automate the generation of VDR at scale.

Why cdxgen?

Most SBOM tools are like simple barcode scanners. For easy applications, they can parse a few package manifests and create a list of components only based on these files without any deep inspection. Further, a typical application might have several repos, components, and libraries with complex build requirements. Traditional techniques to generate an SBOM per language or package manifest either do not work in enterprise environments or don't provide the confidence required for both compliance and automated analysis. So we built cdxgen - the universal polyglot SBOM generator that is user-friendly, precise, and comprehensive!

why cdxgen

Our philosophy:

  • Explainability: Don't list, but explain with evidence.
  • Precision: Try using multiple techniques to improve precision, even if it takes extra time.
  • Personas: Cater to the needs of a range of personas such as security researchers, compliance auditors, developers, and SOC.
  • Lifecycle: Support BOM generation for various product lifecycles.

Documentation

Please visit our documentation site for detailed usage, tutorials, and support documentation.

Sections include:

Automatic usage detection

For node.js projects, lock files are parsed initially, so the SBOM would include all dependencies, including dev ones. An AST parser powered by babel-parser is then used to detect packages that are imported and used by non-test code. Such imported packages would automatically set their scope property to required in the resulting SBOM. You can turn off this analysis by passing the argument --no-babel. Scope property would then be set based on the dev attribute in the lock file.

This attribute can be later used for various purposes. For example, dep-scan uses this attribute to prioritize vulnerabilities. Unfortunately, tools such as dependency track, do not include this feature and might over-report the CVEs.

With the argument --required-only, you can limit the SBOM only to include packages with the scope "required", commonly called production or non-dev dependencies. Combine with --no-babel to limit this list to only non-dev dependencies based on the dev attribute being false in the lock files.

For go, go mod why command is used to identify required packages. For php, composer lock file is parsed to distinguish required (packages) from optional (packages-dev).

Usage

Installing

npm install -g @cyclonedx/cdxgen

If you are a Homebrew user, you can also install cdxgen via:

$ brew install cdxgen

Deno and bun runtime can be used with limited support.

deno install --allow-read --allow-env --allow-run --allow-sys=uid,systemMemoryInfo,gid,homedir --allow-write --allow-net -n cdxgen "npm:@cyclonedx/cdxgen/cdxgen"

You can also use the cdxgen container image with node, deno, or bun runtime versions.

The default version uses Node.js 20

docker run --rm -v /tmp:/tmp -v $(pwd):/app:rw -t ghcr.io/cyclonedx/cdxgen -r /app -o /app/bom.json

To use the deno version, use ghcr.io/cyclonedx/cdxgen-deno as the image name.

docker run --rm -v /tmp:/tmp -v $(pwd):/app:rw -t ghcr.io/cyclonedx/cdxgen-deno -r /app -o /app/bom.json

For the bun version, use ghcr.io/cyclonedx/cdxgen-bun as the image name.

docker run --rm -v /tmp:/tmp -v $(pwd):/app:rw -t ghcr.io/cyclonedx/cdxgen-bun -r /app -o /app/bom.json

In deno applications, cdxgen could be directly imported without any conversion. Please see the section on integration as a library

import { createBom, submitBom } from "npm:@cyclonedx/cdxgen@^9.0.1";

Getting Help

cdxgen [command]

Commands:
  cdxgen completion  Generate bash/zsh completion

Options:
  -o, --output                 Output file. Default bom.json                                       [default: "bom.json"]
  -t, --type                   Project type. Please refer to https://cyclonedx.github.io/cdxgen/#/PROJECT_TYPES for supp
                               orted languages/platforms.                                                        [array]
      --exclude-type           Project types to exclude. Please refer to https://cyclonedx.github.io/cdxgen/#/PROJECT_TY
                               PES for supported languages/platforms.
  -r, --recurse                Recurse mode suitable for mono-repos. Defaults to true. Pass --no-recurse to disable.
                                                                                               [boolean] [default: true]
  -p, --print                  Print the SBOM as a table with tree.                                            [boolean]
  -c, --resolve-class          Resolve class names for packages. jars only for now.                            [boolean]
      --deep                   Perform deep searches for components. Useful while scanning C/C++ apps, live OS and oci i
                               mages.                                                                          [boolean]
      --server-url             Dependency track url. Eg: https://deptrack.cyclonedx.io
      --api-key                Dependency track api key
      --project-group          Dependency track project group
      --project-name           Dependency track project name. Default use the directory name
      --project-version        Dependency track project version                                   [string] [default: ""]
      --project-id             Dependency track project id. Either provide the id or the project name and version togeth
                               er                                                                               [string]
      --parent-project-id      Dependency track parent project id                                               [string]
      --required-only          Include only the packages with required scope on the SBOM. Would set compositions.aggrega
                               te to incomplete unless --no-auto-compositions is passed.                       [boolean]
      --fail-on-error          Fail if any dependency extractor fails.                                         [boolean]
      --no-babel               Do not use babel to perform usage analysis for JavaScript/TypeScript projects.  [boolean]
      --generate-key-and-sign  Generate an RSA public/private key pair and then sign the generated SBOM using JSON Web S
                               ignatures.                                                                      [boolean]
      --server                 Run cdxgen as a server                                                          [boolean]
      --server-host            Listen address                                                     [default: "127.0.0.1"]
      --server-port            Listen port                                                             [default: "9090"]
      --install-deps           Install dependencies automatically for some projects. Defaults to true but disabled for c
                               ontainers and oci scans. Use --no-install-deps to disable this feature.         [boolean]
      --validate               Validate the generated SBOM using json schema. Defaults to true. Pass --no-validate to di
                               sable.                                                          [boolean] [default: true]
      --evidence               Generate SBOM with evidence for supported languages.           [boolean] [default: false]
      --spec-version           CycloneDX Specification version to use. Defaults to 1.5           [number] [default: 1.5]
      --filter                 Filter components containing this word in purl or component.properties.value. Multiple va
                               lues allowed.                                                                     [array]
      --only                   Include components only containing this word in purl. Useful to generate BOM with first p
                               arty components alone. Multiple values allowed.                                   [array]
      --author                 The person(s) who created the BOM. Set this value if you're intending the modify the BOM
                               and claim authorship.                               [array] [default: "OWASP Foundation"]
      --profile                BOM profile to use for generation. Default generic.
  [choices: "appsec", "research", "operational", "threat-modeling", "license-compliance", "generic"] [default: "generic"
                                                                                                                       ]
      --exclude                Additional glob pattern(s) to ignore                                              [array]
      --include-formulation    Generate formulation section using git metadata.               [boolean] [default: false]
      --include-crypto         Include crypto libraries found under formulation.              [boolean] [default: false]
      --standard               The list of standards which may consist of regulations, industry or organizational-specif
                               ic standards, maturity models, best practices, or any other requirements which can be eva
                               luated against or attested to.
  [array] [choices: "asvs-4.0.3", "bsimm-v13", "masvs-2.0.0", "nist_ssdf-1.1", "pcissc-secure-slc-1.1", "scvs-1.0.0", "s
                                                                                                     saf-DRAFT-2023-11"]
      --auto-compositions      Automatically set compositions when the BOM was filtered. Defaults to true
                                                                                               [boolean] [default: true]
  -h, --help                   Show help                                                                       [boolean]
  -v, --version                Show version number                                                             [boolean]

All boolean arguments accept --no prefix to toggle the behavior.

Example

Minimal example.

cdxgen -o bom.json

For a java project. cdxgen would automatically detect maven, gradle, or sbt and build bom accordingly

cdxgen -t java -o bom.json

To print the SBOM as a table pass -p argument.

cdxgen -t java -o bom.json -p

To recursively generate a single BOM for all languages pass -r argument.

cdxgen -r -o bom.json

The default specification used by cdxgen is 1.5. To generate BOM for a different specification version, such as 1.6 or 1.4, pass the version number using the --spec-version argument.

# 1.6 is unsupported by most tools
cdxgen -r -o bom.json --spec-version 1.6

# 1.4 is supported by most tools
cdxgen -r -o bom.json --spec-version 1.4

To generate SBOM for C or Python, ensure Java >= 21 is installed.

# Install java >= 21
cdxgen -t c -o bom.json

NOTE: cdxgen is known to freeze with Java 8 or 11, so ensure >= 21 is installed and JAVA_HOME environment variable is configured correctly. If in doubt, use the cdxgen container image.

Universal SBOM

By passing the type argument -t universal, cdxgen could be forced to opportunistically collect as many components and services as possible by scanning all package, container, and Kubernetes manifests. The resulting SBOM could have over a thousand components, thus requiring additional triaging before use with traditional SCA tools.

SBOM server

Invoke cdxgen with --server argument to run it in server mode. By default, it listens to port 9090, which can be customized with the arguments --server-host and --server-port.

cdxgen --server

Or use the container image.

docker run --rm -v /tmp:/tmp -p 9090:9090 -v $(pwd):/app:rw -t ghcr.io/cyclonedx/cdxgen -r /app --server --server-host 0.0.0.0

Use curl or your favorite tool to pass arguments to the /sbom route.

Server arguments

Arguments can be passed either via the query string or as a JSON body. The following arguments are supported.

Argument Description
type Project type. Please refer to Supported Project Types.
multiProject [boolean]
requiredOnly Include only the packages with required scope on the SBOM. [boolean]
noBabel Do not use babel to perform usage analysis for JavaScript/TypeScript projects. [boolean]
installDeps Install dependencies automatically for some projects. Defaults to true but disabled for containers and oci scans. [boolean] [default: true]
project
projectName Dependency track project name. Default use the directory name
projectGroup Dependency track project group
projectVersion Dependency track project version [default: ""]

Health endpoint

Use the /health endpoint to check if the SBOM server is up and running.

curl "http://127.0.0.1:9090/health"

Scanning a local path

curl "http://127.0.0.1:9090/sbom?path=/Volumes/Work/sandbox/vulnerable-aws-koa-app&multiProject=true&type=js"

Scanning a git repo

curl "http://127.0.0.1:9090/sbom?url=https://github.com/HooliCorp/vulnerable-aws-koa-app.git&multiProject=true&type=js"

If you need to pass credentials to authenticate.

curl "http://127.0.0.1:9090/sbom?url=https://<access_token>@github.com/some/repo.git&multiProject=true&type=js"
curl "http://127.0.0.1:9090/sbom?url=https://<username>:<password>@bitbucket.org/some/repo.git&multiProject=true&type=js"

You can POST the arguments.

curl -H "Content-Type: application/json" http://localhost:9090/sbom -XPOST -d $'{"url": "https://github.com/HooliCorp/vulnerable-aws-koa-app.git", "type": "nodejs", "multiProject": "true"}'

Docker compose

git clone https://github.com/cyclonedx/cdxgen.git
docker compose up

War file support

cdxgen can generate a BOM file from a given war file.

# cdxgen -t java app.war
cdxgen app.war

Resolving class names

Sometimes, it is necessary to resolve class names contained in jar files. By passing an optional argument --resolve-class, it is possible to get cdxgen to create a separate mapping file with the jar name (including the version) as the key and class names list as a value.

cdxgen -t java --resolve-class -o bom.json

This would create a bom.json.map file with the jar - class name mapping. Refer to these examples to learn about the structure.

Resolving licenses

cdxgen can automatically query public registries such as maven, npm, or nuget to resolve the package licenses. This is a time-consuming operation and is disabled by default. To enable, set the environment variable FETCH_LICENSE to true, as shown. Ensure that GITHUB_TOKEN is set or provided by built-in GITHUB_TOKEN in GitHub Actions, otherwise rate limiting might prevent license resolving.

export FETCH_LICENSE=true

Dependency Tree

cdxgen can retain the dependency tree under the dependencies attribute for a small number of supported package manifests. These are currently limited to:

  • package-lock.json
  • yarn.lock
  • pnpm-lock.yaml
  • Maven (pom.xml)
  • Gradle
  • Scala SBT
  • Python (requirements.txt, setup.py, pyproject.toml, poetry.lock)
  • .NET (packages.lock.json, project.assets.json, paket.lock)
  • Go (go.mod)
  • PHP (composer.lock)
  • Ruby (Gemfile.lock)
  • Rust (Cargo.lock)

Plugins

cdxgen could be extended with external binary plugins to support more SBOM use cases. These are now installed as an optional dependency.

sudo npm install -g @cyclonedx/cdxgen-plugins-bin

Docker / OCI container support

docker type is automatically detected based on the presence of values such as sha256 or docker.io prefix etc in the path.

cdxgen odoo@sha256:4e1e147f0e6714e8f8c5806d2b484075b4076ca50490577cdf9162566086d15e -o /tmp/bom.json

You can also pass -t docker with repository names. Only the latest tag would be pulled if none was specified.

cdxgen shiftleft/scan-slim -o /tmp/bom.json -t docker

You can also pass the .tar file of a container image.

docker pull shiftleft/scan-slim
docker save -o /tmp/slim.tar shiftleft/scan-slim
podman save -q --format oci-archive -o /tmp/slim.tar shiftleft/scan-slim
cdxgen /tmp/slim.tar -o /tmp/bom.json -t docker

Podman in rootless mode

Setup podman in either rootless or remote mode

Do not forget to start the podman socket required for API access on Linux.

systemctl --user enable --now podman.socket
systemctl --user start podman.socket
podman system service -t 0 &

Generate OBOM for a live system

You can use the obom command to generate an OBOM for a live system or a VM for compliance and vulnerability management purposes. Windows and Linux operating systems are supported in this mode.

# obom is an alias for cdxgen -t os
obom
# cdxgen -t os

This feature is powered by osquery, which is installed along with the binary plugins. cdxgen would opportunistically try to detect as many components, apps, and extensions as possible using the default queries. The process would take several minutes and result in an SBOM file with thousands of components of various types, such as operating-system, device-drivers, files, and data.

Generate Cryptography Bill of Materials (CBOM)

Use the cbom alias to generate a CBOM. This is currently supported only for Java projects.

cbom -t java
# cdxgen -t java --include-crypto -o bom.json .

Generating SaaSBOM and component evidences

See evinse mode in the advanced documentation.

BOM signing

cdxgen can sign the generated BOM json file to increase authenticity and non-repudiation capabilities. To enable this, set the following environment variables.

  • SBOM_SIGN_ALGORITHM: Algorithm. Example: RS512
  • SBOM_SIGN_PRIVATE_KEY: Location to the RSA private key
  • SBOM_SIGN_PUBLIC_KEY: Optional. Location to the RSA public key

To generate test public/private key pairs, you can run cdxgen by passing the argument --generate-key-and-sign. The generated json file would have an attribute called signature, which could be used for validation. jwt.io is a known site that could be used for such signature validation.

SBOM signing

Verifying the signature

Use the bundled cdx-verify command, which supports verifying a single signature added at the bom level.

npm install -g @cyclonedx/cdxgen
cdx-verify -i bom.json --public-key public.key

Custom verification tool (Node.js example)

There are many libraries available to validate JSON Web Tokens. Below is a javascript example.

# npm install jws
const jws = require("jws");
const fs = require("fs");
// Location of the SBOM json file
const bomJsonFile = "bom.json";
// Location of the public key
const publicKeyFile = "public.key";
const bomJson = JSON.parse(fs.readFileSync(bomJsonFile, "utf8"));
// Retrieve the signature
const bomSignature = bomJson.signature.value;
const validationResult = jws.verify(bomSignature, bomJson.signature.algorithm, fs.readFileSync(publicKeyFile, "utf8"));
if (validationResult) {
  console.log("Signature is valid!");
} else {
  console.log("SBOM signature is invalid :(");
}

Automatic services detection

cdxgen can automatically detect names of services from YAML manifests such as docker-compose, Kubernetes, or Skaffold manifests. These would be populated under the services attribute in the generated SBOM. With evinse, additional services could be detected by parsing common annotations from the source code.

Conversion to SPDX format

Use the CycloneDX CLI tool for advanced use cases such as conversion, diff and merging.

License

Permission to modify and redistribute is granted under the terms of the Apache 2.0 license. See the LICENSE file for the full license.

Integration as library

cdxgen is ESM only and could be imported and used with both deno and Node.js >= 20

Minimal example:

import { createBom, submitBom } from "npm:@cyclonedx/cdxgen@^9.0.1";

See the Deno Readme for detailed instructions.

import { createBom, submitBom } from "@cyclonedx/cdxgen";
// bomNSData would contain bomJson
const bomNSData = await createBom(filePath, options);
// Submission to dependency track server
const dbody = await submitBom(args, bomNSData.bomJson);

Contributing

Please check out our contribute to CycloneDX/cdxgen documentation if you are interested in helping.

Before raising a PR, please run the following commands.

corepack enable pnpm
pnpm install
# Generate types using jsdoc syntax
pnpm run gen-types
# Run biomejs formatter and linter with auto fix
pnpm run lint
# Run jest tests
pnpm test

cdxgen's People

Contributors

ajmalab avatar aryan-rajoria avatar candrews avatar cerrussell avatar chenrui333 avatar eoftedal avatar gbennett-squarespace avatar gitnachogo avatar heubeck avatar hubertp avatar jkowalleck avatar ksg97031 avatar lucasljungberg avatar malice00 avatar marob avatar mikejarrett avatar nikemare avatar prabhu avatar puchta avatar raineinto avatar robaliias avatar rogeriobastos avatar saketjajoo avatar scrocquesel avatar sebastianvoss avatar setchy avatar stevespringett avatar sullivtr avatar timmyteo avatar validide avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cdxgen's Issues

Gradle SBOM Issue

cdxgen doesn't seems to work for Gradle projects.

Tried all these for gradle projects:

cdxgen -t java -r --resolve-class
cdxgen -t java -r
cdxgen -t java


execution log:

Executing gradlew dependencies in /
null null
Unable to produce BOM for .


Environment:

OS: MacOS
cdxgen version: 3.1.0 (also tried in 3.0.9 resulted the same)
java version: openjdk 11.0.12 2021-07-20 LTS
npm version: 7.13.0
node version: 16.5.0

HTTP / HTTPS proxy option support

Hi,

I was trying to use the tool under a network where HTTP / HTTPS proxy is required for the Internet access, but it does not work.

Since there is no way to set the proxy on Node globally (e.g. HTTP_PROXY, HTTPS_PROXY), it would be great if the proxy option is offered by this tool.

Thank you.

cdxgen fails to parse gnu license with Error: Invalid character in string

Hello,

I'm trying to create a bom for a python app with the following requirements
airbrake==2.1.0 amqp==2.1.1 backports.ssl-match-hostname==3.4.0.2 blinker==1.3 boto==2.32.1 certifi==2017.7.27.1 chardet==3.0.4 configobj==5.0.6 Django==1.6.6 dnspython==1.11.1 GeoIP==1.3.2 gevent==1.1.1 greenlet==0.4.10 hiredis==0.1.4 idna==2.6 Jinja2==2.7.3 kombu==4.0.0 lockfile==0.9.1 lxml==3.3.5 MarkupSafe==0.23 MySQL-python==1.2.5 Pillow==2.7.0 PyJWT==1.4.0 pymongo==3.6.1 python-daemon==1.5.5 pytz==2014.4 redis==2.10.3 requests==2.18.4 route53==1.0 six==1.7.3 tornado==4.0.1 unittest-xml-reporting==2.1.0 urllib3==1.22 vine==1.1.3

I'm getting the following error:

/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLStringifier.js:208
            throw new Error(`Invalid character in string: ${str} at index ${res.index}`);
                  ^

Error: Invalid character in string: GNU LESSER GENERAL PUBLIC LICENSE
		       Version 2.1, February 1999
...
...
...
That's all there is to it! at index 2932
    at XMLStringifier.assertLegalChar (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLStringifier.js:208:19)
    at XMLStringifier.text (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLStringifier.js:52:21)
    at new XMLText (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLText.js:22:37)
    at XMLElement.text (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLNode.js:308:17)
    at XMLElement.node (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLNode.js:294:17)
    at XMLElement.element (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLNode.js:187:30)
    at XMLElement.element (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLNode.js:163:32)
    at XMLElement.element (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLNode.js:158:27)
    at XMLElement.element (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLNode.js:115:30)
    at XMLElement.element (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/xmlbuilder/lib/XMLNode.js:158:27)

Any ideas on how to resolve this? not sure if it's something the code here can catch and process anyway.

Unable to work with cdxgen

I'm unable to use cdxgen in whatever version.

Here is what I get:

npm install -g @appthreat/cdxgen
npm WARN deprecated [email protected]: request has been deprecated, see request/request#3142
npm WARN deprecated [email protected]: Use cheerio-select instead
npm WARN deprecated [email protected]: this library is no longer supported
/usr/bin/cdxgen -> /usr/lib/node_modules/@appthreat/cdxgen/bin/cdxgen

[email protected] postinstall /usr/lib/node_modules/@appthreat/cdxgen/node_modules/core-js
node -e "try{require('./postinstall')}catch(e){}"

/usr/lib
└─┬ @appthreat/[email protected]
├── @babel/[email protected]
├─┬ @babel/[email protected]
│ ├─┬ @babel/[email protected]
│ │ └─┬ @babel/[email protected]
│ │ ├─┬ [email protected]
│ │ │ ├─┬ [email protected]
│ │ │ │ └─┬ [email protected]
│ │ │ │ └── [email protected]
│ │ │ ├── [email protected]
│ │ │ └─┬ [email protected]
│ │ │ └── [email protected]
│ │ └── [email protected]
│ ├─┬ @babel/[email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ ├─┬ @babel/[email protected]
│ │ ├── @babel/[email protected]
│ │ └── @babel/[email protected]
│ ├── @babel/[email protected]
│ ├─┬ @babel/[email protected]
│ │ ├── @babel/[email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ ├─┬ [email protected]
│ │ │ ├── [email protected]
│ │ │ └── [email protected]
│ │ ├── [email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ ├─┬ [email protected]
│ │ ├── [email protected]
│ │ ├─┬ [email protected]
│ │ │ └─┬ [email protected]
│ │ │ └── [email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ └─┬ [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├── @sindresorhus/[email protected]
│ ├─┬ @szmarczak/[email protected]
│ │ └── [email protected]
│ ├─┬ @types/[email protected]
│ │ ├── @types/[email protected]
│ │ ├── @types/[email protected]
│ │ ├── @types/[email protected]
│ │ └── @types/[email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │ └── [email protected]
│ ├─┬ [email protected]
│ │ ├─┬ [email protected]
│ │ │ └── [email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ └── [email protected]
├── [email protected]
├── [email protected]
├─┬ [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ ├── [email protected]
│ │ ├─┬ [email protected]
│ │ │ ├── [email protected]
│ │ │ ├─┬ [email protected]
│ │ │ │ ├─┬ [email protected]
│ │ │ │ │ └─┬ [email protected]
│ │ │ │ │ └── [email protected]
│ │ │ │ └── [email protected]
│ │ │ ├── [email protected]
│ │ │ └─┬ [email protected]
│ │ │ ├─┬ [email protected]
│ │ │ │ └── [email protected]
│ │ │ └─┬ [email protected]
│ │ │ └── [email protected]
│ │ └── [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ ├─┬ [email protected]
│ │ ├─┬ [email protected]
│ │ │ ├── [email protected]
│ │ │ ├── [email protected]
│ │ │ ├── [email protected]
│ │ │ └── [email protected]
│ │ └── [email protected]
│ ├─┬ [email protected]
│ │ ├── [email protected]
│ │ ├─┬ [email protected]
│ │ │ ├── [email protected]
│ │ │ ├── [email protected]
│ │ │ └─┬ [email protected]
│ │ │ └── [email protected]
│ │ └─┬ [email protected]
│ │ ├── [email protected]
│ │ ├── [email protected]
│ │ ├── [email protected]
│ │ ├── [email protected]
│ │ ├── [email protected]
│ │ ├── [email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ └── [email protected]
├─┬ [email protected]
│ └─┬ [email protected]
│ └── [email protected]
├─┬ [email protected]
│ ├── [email protected]
│ └── [email protected]
├── [email protected]
├─┬ [email protected]
│ └── [email protected]
├── [email protected]
└─┬ [email protected]
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └── [email protected]
│ └─┬ [email protected]
│ └─┬ [email protected]
│ └─┬ [email protected]
│ └── [email protected]
├── [email protected]
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │ └─┬ [email protected]
│ │ └── [email protected]
│ └── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├─┬ [email protected]
│ ├── [email protected]
│ └── [email protected]
├── [email protected]
├── [email protected]
└─┬ [email protected]
└── [email protected]

Then, if I try to execute it :
cdxgen
/usr/lib/node_modules/@appthreat/cdxgen/index.js:414
const createJavaBom = async (
^

SyntaxError: Unexpected token (
at createScript (vm.js:56:10)
at Object.runInThisContext (vm.js:97:10)
at Module._compile (module.js:549:28)
at Object.Module._extensions..js (module.js:586:10)
at Module.load (module.js:494:32)
at tryModuleLoad (module.js:453:12)
at Function.Module._load (module.js:445:3)
at Module.require (module.js:504:17)
at require (internal/module.js:20:19)
at Object. (/usr/lib/node_modules/@appthreat/cdxgen/bin/cdxgen:3:13)

Podfile.lock example

PODS:
  - GRMustache.swift (2.0.0)
  - GzipSwift (5.0.0)
  - Just (0.7.1)
  - PromiseKit (6.8.4):
    - PromiseKit/CorePromise (= 6.8.4)
    - PromiseKit/Foundation (= 6.8.4)
    - PromiseKit/UIKit (= 6.8.4)
  - PromiseKit/CorePromise (6.8.4)
  - PromiseKit/Foundation (6.8.4):
    - PromiseKit/CorePromise
  - Sparkle (1.21.3)

DEPENDENCIES:
  - GRMustache.swift (from `https://github.com/iina/GRMustache.swift`)
  - GzipSwift
  - Just (from `https://github.com/iina/Just`, branch `swift-5`)
  - PromiseKit
  - Sparkle

SPEC REPOS:
  https://github.com/cocoapods/specs.git:
    - GzipSwift
    - PromiseKit
    - Sparkle

EXTERNAL SOURCES:
  GRMustache.swift:
    :git: https://github.com/iina/GRMustache.swift
  Just:
    :branch: swift-5
    :git: https://github.com/iina/Just

CHECKOUT OPTIONS:
  GRMustache.swift:
    :commit: a46e65d00b4753449b90465be813bbdb1a9ffc9d
    :git: https://github.com/iina/GRMustache.swift
  Just:
    :commit: d0ae3f9bc2d6bf247b19217764a096bbac55f007
    :git: https://github.com/iina/Just

SPEC CHECKSUMS:
  GRMustache.swift: 53405b02c374520f9193cf41bfa677f03a6d70aa
  GzipSwift: 5592f4d62b641e04d06443ba471f8ed76b1363e4
  Just: 60c0995e04718eb621a07abef45ce6883c562f2b
  PromiseKit: 51794a832647e7b819336dc2279039ce9f1cc49b
  Sparkle: 3f75576db8b0265adef36c43249d747f22d0b708

PODFILE CHECKSUM: 4969f0d73fe5d871a1a1853ad7807a345443e71a

COCOAPODS: 1.7.5

cannot parse csproj files with only one ItemGroup

If you modify the sample.csproj file to use a single ItemGroup, parsing will fail. For example:

test/sample.csproj:

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <LangVersion>latest</LangVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="3.1.1" />
    <PackageReference Include="Microsoft.Bot.Builder.Dialogs" Version="4.8.0" />
    <PackageReference Include="Microsoft.Bot.Builder.Integration.AspNet.Core" Version="4.8.0" />
    <PackageReference Include="Microsoft.Bot.Builder.Adapters.Facebook" Version="4.8.0-rc2-preview" />
    <PackageReference Include="AdaptiveCards" Version="1.0.3" />
  </ItemGroup>
</Project>
$ npm test
...
 FAIL  ./utils.test.js (17.311s)
  ✓ SSRI test (5ms)
  ✓ finds license id from name (1ms)
  ✓ parse gradle dependencies (6ms)
  ✓ get py metadata (123ms)
  ✓ parseGopkgData (6576ms)
  ✓ get crates metadata (225ms)
  ✕ parse cs proj (18ms)
  ✓ get nget metadata (242ms)
  ✓ parsePomFile (22ms)
  ✓ parsePomMetadata (805ms)
  ✓ get repo license (657ms)
  ✓ get go pkg license (510ms)
  ✓ get licenses (2ms)
  ✓ parsePkgLock (11ms)
  ✓ parseNodeShrinkwrap (9ms)
  ✓ parseSetupPyFile (509ms)
  ✓ parsePnpmLock (137ms)
  ✓ parseYarnLock (357ms)
  ✓ parseComposerLock (5ms)
  ✓ parse requirements.txt with comments (1652ms)
  ✓ parse pipfile.lock with hashes (2693ms)
  ✓ parse scala sbt list (2ms)
...
Test Suites: 1 failed, 1 total
Tests:       1 failed, 21 passed, 22 total
Snapshots:   0 total
Time:        17.563s, estimated 24s
Ran all test suites.
npm ERR! Test failed.  See above for more details.

Problem uploading BOM to OWASP Dependency Track

I was trying to use cdxgen to generate BOM and upload it to Dependency Track for analysis, but it doesn't work. The project is created in Dependency Track (if it doesn't exist) and the BOM file is uploaded, but the analysis fails. In the logs I can see errors like this:

dtrack-apiserver_1 | Caused by: org.datanucleus.exceptions.NucleusUserException: Attempt to store value "d3af366f679d75a6f5eb5e7575ae5b77879fdfcd3cd5ef5bdbb7fc79bdf771ddbd6da7f8d5fe3df7a7787bcf1ee9eef4" in column ""SHA_256"" that has maximum length of 64. Please correct your data!

As you can see, the database expects a maximum of 64 characters, but 96 are sent.

Digging a bit deeper in the code of cdxgen I saw that the component's hashes are encoded in hexadecimal instead of adding directly the base64 hash (function addComponentHash).

let hash = Buffer.from(digest, "base64").toString("hex");

Is this being hex encoded for any particular reason?

Thanks.

clojure example

(defproject foo "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :dependencies [[org.clojure/clojure                "1.10.1"]
                 [org.clojure/data.json              "0.2.6"]
                 [com.amazonaws/aws-lambda-java-core "1.2.0"]
                 [com.novemberain/pantomime          "2.11.0"]
                 [com.cognitect.aws/api              "0.8.456"]
                 [com.cognitect.aws/endpoints        "1.1.11.789"]
                 [com.cognitect.aws/s3               "799.2.682.0"]]
  :aot :all)

Support for spdx output

There might be organizations that prefer one format over another so let's support both the formats ;)

pnpm-lock.yaml example: pnpm support

https://raw.githubusercontent.com/pnpm/benchmarks-of-javascript-package-managers/master/pnpm-lock.yaml

dependencies:
  common-tags: 1.8.0
  cross-spawn: 5.1.0
  fs-extra: 1.0.0
  get-folder-size: 1.0.1
  load-yaml-file: 0.1.1
  mkdirp: 0.5.1
  npm: 6.14.5
  path-key: 2.0.1
  pnpm: 5.0.2
  pretty-bytes: 4.0.2
  pretty-ms: 2.1.0
  rimraf: 2.7.1
  thenify: 3.3.0
  write-yaml-file: 1.0.1
  yarn: 1.22.4
devDependencies:
  standard: 10.0.3
lockfileVersion: 5.1
packages:
  /acorn-jsx/3.0.1:
    dependencies:
      acorn: 3.3.0
    dev: true
    resolution:
      integrity: sha1-r9+UiPsezvyDSPb7IvRk4ypYs2s=
  /acorn/3.3.0:
    dev: true
    engines:
      node: '>=0.4.0'
    hasBin: true
    resolution:
      integrity: sha1-ReN/s56No/JbruP/U2niu18iAXo=
  /acorn/5.7.3:
    dev: true
    engines:
      node: '>=0.4.0'
    hasBin: true
    resolution:
      integrity: sha512-T/zvzYRfbVojPWahDsE5evJdHb3oJoQfFbsrKM7w5Zcs++Tr257tia3BmMP8XYVjp1S9RZXQMh7gao96BlqZOw==
  /ajv-keywords/[email protected]:
    dependencies:
      ajv: 4.11.8
    dev: true
    peerDependencies:
      ajv: '>=4.10.0'
    resolution:
      integrity: sha1-MU3QpLM2j609/NxU7eYXG4htrzw=

issue in mvn and node based single repo

hi,

running the cdxgen I have encounter an issue which I'm not even sure about the cause. My project structure has java and nodejs application in the same repo. Although the bom.xml file not created I'm not sure if it has listed all the requirement dependencies in the project.

project directory structure:
Gruntfile.js README.md bom.xml bom2.xml fonts node_modules package.json pom.xml src

command: cdxgen -o bom2.xml --unhandled-rejections=strict
error output:
Executing 'mvn org.cyclonedx:cyclonedx-maven-plugin:makeAggregateBom' in /Users/path_to_repo (node:91593) UnhandledPromiseRejectionWarning: TypeError: Cannot read property '_' of undefined at Object.parsePom (/usr/local/lib/node_modules/@appthreat/cdxgen/utils.js:41:31) at Object.exports.createBom (/usr/local/lib/node_modules/@appthreat/cdxgen/index.js:343:29) at Object.<anonymous> (/usr/local/lib/node_modules/@appthreat/cdxgen/bin/cdxgen:74:5) at Module._compile (internal/modules/cjs/loader.js:1158:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1178:10) at Module.load (internal/modules/cjs/loader.js:1002:32) at Function.Module._load (internal/modules/cjs/loader.js:901:14) at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12) at internal/main/run_main_module.js:18:47 (node:91593) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict(see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1) (node:91593) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code. BOM file written to bom2.xml
please suggest. Thank you.

cdxgen throws on monorepo input

❯ cdxgen -r -o bom.json
bin/generateManifest.js The "path" argument must be of type string. Received undefined
packages/my-monorepo-lerna-package/tools/generateLazyLoadedIconModule.ts The "path" argument must be of type string. Received undefined
(node:43173) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'packages' of undefined
    at Object.parsePnpmLock (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/utils.js:362:30)
    at createNodejsBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:804:27)
    at async createXBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:1498:12)
    at async Object.exports.createBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:1697:14)
(node:43173) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:43173) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
(node:43173) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'packages' of undefined
    at Object.parsePnpmLock (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/utils.js:362:30)
    at createNodejsBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:804:27)
    at async createXBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:1498:12)
    at async Object.exports.createBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:1697:14)
(node:43173) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:43173) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'packages' of undefined
    at Object.parsePnpmLock (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/utils.js:362:30)
    at createNodejsBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:804:27)
    at async createXBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:1498:12)
    at async Object.exports.createBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:1697:14)
(node:43173) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 3)
(node:43173) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'packages' of undefined
    at Object.parsePnpmLock (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/utils.js:362:30)
    at createNodejsBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:804:27)
    at async createXBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:1498:12)
    at async Object.exports.createBom (/Users/bob/.nvm/versions/node/v12.18.3/lib/node_modules/@appthreat/cdxgen/index.js:1697:14)
(node:43173) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 4)

Implement fallback for nodejs

For nodejs projects, bom generation is failing without node_modules directory. We should be able to fallback to parsing package-lock.json in such cases.

Non-conformat XML generated

The CycloneDX XML schema uses xs:sequence to specify that element order is required.

I haven't reproduced this myself, but just had this issue raised for the CycloneDX CLI Tool CycloneDX/cyclonedx-cli#152

The problem is that the externalReferences element is before the metadata element in the BOM.

BOM validation failed for GO bom

I am receiving the below error while validating the GO bom XML/JSON files.

[swapnil@bharshankar minio]$ cyclonedx-cli validate --input-file minio-bom.xml --fail-on-errors
Validating XML BOM...
Invalid namespace URI: expected http://cyclonedx.org/schema/bom/1.3 actual http://cyclonedx.org/schema/bom/1.2
[swapnil@bharshankar minio]$
[swapnil@bharshankar minio]$ cyclonedx-cli validate --input-file minio-bom.json --fail-on-errors
Validating JSON BOM...
Validation failed:
#/properties/components/items
[swapnil@bharshankar minio]$

Please refer attached bom XML/JSON files.
minio.zip

Golang parsing incorrect

It appears from #42 that you parse the go.mod file itself to detect Go dependencies.

This is incorrect, and results in missing indirect dependencies, as well as potentially incorrect direct dependencies due to build tags. See here for more discussion on the correct ways: CycloneDX/cyclonedx-gomod#20

BOM validation failed due to nested URL element.

cdxgen is adding nested url element in output.

dotnet run -- validate --input-file console-browserify-sbom.xml --input-format xml_v1_2
Validating XML BOM...
Validation failed at line number 36 and position 14: The element 'http://cyclonedx.org/schema/bom/1.2:url' cannot contain child element 'http://cyclonedx.org/schema/bom/1.2:url' because the parent element's content model is text only.
BOM validated successfully.

bom output file. Please refer attachment.

        <reference type="website">
          <url>
            <url>https://github.com/davidchambers/Base64.js#readme</url>
          </url>
        </reference>

console-browserify-sbom.zip

Validation failure with license lookup

I noticed that the xml and json produced after enabling license lookup occasionally are failing cyclonedx validation.

xml is failing due to the ordering. Then there are failures due to non-availability of either id or name for the given license. This is going to take some time to fix.

SyntaxError: Unexpected token

1、npm install -g @appthreat/cdxgen

2、cdxgen -h ,but error
C:\Users\wz\AppData\Roaming\npm\node_modules@appthreat\cdxgen\node_modules\got\dist\source\create.js:157
const paginateEach = (async function* (url, options) {
^
SyntaxError: Unexpected token *
at createScript (vm.js:80:10)
at Object.runInThisContext (vm.js:139:10)
at Module._compile (module.js:617:28)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at Object. (C:\Users\wz\AppData\Roaming\npm\node_modules@appthreat\cdxgen\node_modules\got\dist\source\index.js:14:18)

Parsing go.mod files incorrectly skips files ending with `go `

The overmatching in parseGoModData function causes the scanner to skip modules that end with a go , such as aws-sdk-go.

Example that would be incorrectly skipped:

module cdxgen/test
go 1.14
require (    google.golang.org/grpc v1.32.    github.com/aws/aws-sdk-go v1.38.47    github.com/spf13/viper v1.3.0    github.com/spf13/cobra v1.0.0)
// Having both replace sections is invalid in a go.mod file, but it allows the tests to validate both casesreplace google.golang.org/grpc =&gt; google.golang.org/grpc v1.21.0
replace (    github.com/spf13/viper =&gt; github.com/spf13/viper v1.0.2)

python setup.py support

The documentation states Python packages are supported via setup.py, but there is no documentation (that I can see) on how to use it.

When I run the command below I get an error

$> cdxgen setup.py

Error: ENOTDIR invalid cwd /fully/qualified/path/setup.py
    at GlobSync._readdirError (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/glob/sync.js:322:21)
    at GlobSync._readdir (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/glob/sync.js:288:10)
    at GlobSync._processReaddir (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/glob/sync.js:135:22)
    at GlobSync._process (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/glob/sync.js:130:10)
    at new GlobSync (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/glob/sync.js:46:10)
    at Function.globSync [as sync] (/usr/local/lib/node_modules/@appthreat/cdxgen/node_modules/glob/sync.js:24:10)
    at Object.getAllFiles (/usr/local/lib/node_modules/@appthreat/cdxgen/utils.js:36:17)
    at createXBom (/usr/local/lib/node_modules/@appthreat/cdxgen/index.js:1613:26)
    at Object.exports.createBom (/usr/local/lib/node_modules/@appthreat/cdxgen/index.js:1805:20)
    at /usr/local/lib/node_modules/@appthreat/cdxgen/bin/cdxgen:88:32 {
  path: '/fully/qualified/path/setup.py',
  code: 'ENOTDIR'
}
Unable to produce BOM for setup.py 

if I just try it in the directory

$> cdxgen
Unable to produce BOM for .

How can I build a bom from only a setup.py?

License lookup for npm

Similar to how we fetch licenses for go packages we have request from clients to include licenses for npm packages as well. Perhaps, this should be an opt-in flag since there will be far too many npm packages.

We can also think about a breaking change for go to only fetch licenses with the presence of a flag. Currently, license lookup is performed by default for go and an environment variable SKIP_FETCH_LICENSE is used to disable this lookup.

[Dependency-Track integration] JSON BOM upload fails with Dependency Track 4.2.2

Steps:

  1. Generate JSON BOM with cdxgen for node project
  2. Base64 encode that JSON BOM and added it into another JSON file that has the format required for upload to DT
  3. Upload to DT via the REST API
  4. Got an error while processing it in the DT api server
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `java.lang.String` from Object value (token `JsonToken.START_OBJECT`)
dtrack-apiserver_1  |  at [Source: (byte[])"{
dtrack-apiserver_1  |   "bomFormat": "CycloneDX",
dtrack-apiserver_1  |   "specVersion": "1.2",
dtrack-apiserver_1  |   "serialNumber": "urn:uuid:f70802e7-ceac-495f-b60a-650c52199f0b",
dtrack-apiserver_1  |   "version": 1,
dtrack-apiserver_1  |   "metadata": {
dtrack-apiserver_1  |     "timestamp": "2021-05-12T06:53:34.244Z",
dtrack-apiserver_1  |     "tools": [
dtrack-apiserver_1  |       {
dtrack-apiserver_1  |         "tool": {
dtrack-apiserver_1  |           "vendor": "AppThreat",
dtrack-apiserver_1  |           "name": "cdxgen",
dtrack-apiserver_1  |           "version": "2.3.1"
dtrack-apiserver_1  |         }
dtrack-apiserver_1  |       }
dtrack-apiserver_1  |     ],
dtrack-apiserver_1  |     "authors": [
dtrack-apiserver_1  |       {
dtrack-apiserver_1  |         "author": {
dtrack-apiserver_1  |           "name": {
dtrack-apiserver_1  |             "name": "Team AppThreat"
dtrack-apiserver_1  |           },
dtrack-apiserver_1  |           "email": "cloud@appth"[truncated 618575 bytes]; line: 20, column: 19] (through reference chain: org.cyclonedx.model.Bom["metadata"]->org.cyclonedx.model.Metadata["authors"]->java.util.ArrayList[0]->org.cyclonedx.model.OrganizationalContact["name"])

I suspect that the authors field is causing the error.

I've also tried uploading the raw JSON BOM via the web UI and encountered the same issue.

When I removed the authors field from the BOM, everything seems to be working fine.

Gemfile.lock example

GEM
  remote: https://rubygems.org/
  specs:
    CFPropertyList (3.0.1)
    activesupport (4.2.11.1)
      i18n (~> 0.7)
      minitest (~> 5.1)
      thread_safe (~> 0.3, >= 0.3.4)
      tzinfo (~> 1.1)
    algoliasearch (1.27.1)
      httpclient (~> 2.8, >= 2.8.3)
      json (>= 1.5.1)
    atomos (0.1.3)
    claide (1.0.3)
    cocoapods (1.8.4)
      activesupport (>= 4.0.2, < 5)
      claide (>= 1.0.2, < 2.0)
      cocoapods-core (= 1.8.4)
      cocoapods-deintegrate (>= 1.0.3, < 2.0)
      cocoapods-downloader (>= 1.2.2, < 2.0)
      cocoapods-plugins (>= 1.0.0, < 2.0)
      cocoapods-search (>= 1.0.0, < 2.0)
      cocoapods-stats (>= 1.0.0, < 2.0)
      cocoapods-trunk (>= 1.4.0, < 2.0)
      cocoapods-try (>= 1.1.0, < 2.0)
      colored2 (~> 3.1)
      escape (~> 0.0.4)
      fourflusher (>= 2.3.0, < 3.0)
      gh_inspector (~> 1.0)
      molinillo (~> 0.6.6)
      nap (~> 1.0)
      ruby-macho (~> 1.4)
      xcodeproj (>= 1.11.1, < 2.0)
    cocoapods-core (1.8.4)
      activesupport (>= 4.0.2, < 6)
      algoliasearch (~> 1.0)
      concurrent-ruby (~> 1.1)
      fuzzy_match (~> 2.0.4)
      nap (~> 1.0)
    cocoapods-deintegrate (1.0.4)
    cocoapods-downloader (1.2.2)
    cocoapods-plugins (1.0.0)
      nap
    cocoapods-search (1.0.0)
    cocoapods-stats (1.1.0)
    cocoapods-trunk (1.4.1)
      nap (>= 0.8, < 2.0)
      netrc (~> 0.11)
    cocoapods-try (1.1.0)
    colored2 (3.1.2)
    concurrent-ruby (1.1.5)
    escape (0.0.4)
    fourflusher (2.3.1)
    fuzzy_match (2.0.4)
    gh_inspector (1.1.3)
    httpclient (2.8.3)
    i18n (0.9.5)
      concurrent-ruby (~> 1.0)
    json (2.2.0)
    minitest (5.12.2)
    molinillo (0.6.6)
    nanaimo (0.2.6)
    nap (1.1.0)
    netrc (0.11.0)
    ruby-macho (1.4.0)
    thread_safe (0.3.6)
    tzinfo (1.2.5)
      thread_safe (~> 0.1)
    xcodeproj (1.13.0)
      CFPropertyList (>= 2.3.3, < 4.0)
      atomos (~> 0.1.3)
      claide (>= 1.0.2, < 2.0)
      colored2 (~> 3.1)
      nanaimo (~> 0.2.6)

PLATFORMS
  ruby

DEPENDENCIES
  cocoapods

BUNDLED WITH
   1.17.2

When using cdxgen to generate SBOM for a Python application The hash is stored in the wrong format

I've got a Python application which I used cdxgen on to generate the BOM file, When I try to upload it to DependencyTrack I get an error
43.511 ERROR [BomUploadProcessingTask] Error while processing bom javax.jdo.JDOFatalUserException: Attempt to store value "77c6f6e3aeb8e7ad5dd1ddf875d7da79ce78eb7e9de74d9dedc79ae9edbd73779a7faf1fddd7fad7cd3ceb77b6d7ae9e" in column ""SHA_256"" that has maximum length of 64. Please correct your data!

I believe we've found the issue in the code here: https://github.com/AppThreat/cdxgen/blob/master/utils.js#L597
should be
p._integrity = "sha256-" + toBase64(digest["sha256"]);

I'm going to open a PR to fix this following this issue.

ReferenceError: buildBomString is not defined

Hello,

I have a problem with cdxgen starting with the version 3.1.0.
When trying to create a bom for scala using the command cdxgen -t java -o bom.xml it failed by giving the following message

Executing sbt dependencyList in /project using plugins /tmp/cdxsbtg-7JscxW
(node:254376) UnhandledPromiseRejectionWarning: ReferenceError: buildBomString is not defined
    at createJavaBom (/opt/npm-packages/lib/node_modules/@appthreat/cdxgen/index.js:830:7)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Object.exports.createBom (/opt/npm-packages/lib/node_modules/@appthreat/cdxgen/index.js:1870:14)
    at async /opt/npm-packages/lib/node_modules/@appthreat/cdxgen/bin/cdxgen:88:22
(Use `node --trace-warnings ...` to show where the warning was created)
(node:254376) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:254376) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

That does not appear in version before 3.1.0.

Parsing go.mod with single require statement results in no BOM

cdxgen does not properly parse a go.mod with a single requirement. Example:

module github.com/myuser/myrepo

go 1.16

require github.com/stretchr/testify v1.7.0

results in Unable to produce BOM message if SCAN_DEBUG_MODE=1 is set.

Expected Behavior:
Parser would properly handle either the multi-line require ( ... ) or single line require ...

Enhancement: Protocol Buffers support

XML format of CycloneDX has served us well over many years. However, with ever expanding languages and attributes the format has started showing its age affecting the performance of NextGen applications. XML is awesome for enterprises but is slowing down creative and innovative ideas.

I have decided to support Protocol Buffers with this tool. Given the effort involved it is going to take time. Plus initially we can keep it aligned with CycloneDX format and can even support the upcoming supply chain attributes.

The --recurse option is not supported for maven, ruby, php, rust, or golang

The --recurse option sets options.multiProject, which is used in several places throughout the code in calls to utils.getAllFiles. However, only the following file types use the recursive search:

  • build.gradle
  • build.sbt*
  • *.csproj
  • yarn.lock
  • package-lock.json
  • pnpm-lock.yaml
  • requirements.txt

As a result, creating a single nested directory will cause cdxgen to fail for maven, ruby, php, rust, and golang.

Support for go binary

Few approaches to collect package list from a go binary

go version -m binary

strings <binary> | grep -P "^dep\t"

One could also produce a mini binary containing just go version command it seems.

Better support for android with gradle projects

cdxgen is struggling a bit for android + gradle projects without android sdk installed. Perhaps we could improve the error message to help users install the sdk correctly and set the associated environment variables.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.