Giter VIP home page Giter VIP logo

owasp-dep-scan / dep-scan Goto Github PK

View Code? Open in Web Editor NEW
714.0 13.0 83.0 11.63 MB

OWASP dep-scan is a next-generation security and risk audit tool based on known vulnerabilities, advisories, and license limitations for project dependencies. Both local repositories and container images are supported as the input, and the tool is ideal for integration.

Home Page: https://owasp.org/www-project-dep-scan/

License: MIT License

Dockerfile 1.24% Python 97.30% Shell 0.71% Jinja 0.37% Nix 0.38%
vulnerability-scanners cve dependency-analysis risk-audit containers sbom sca dependency-audit compliance cyclonedx

dep-scan's Introduction

Introduction

OWASP dep-scan is a next-generation security and risk audit tool based on known vulnerabilities, advisories, and license limitations for project dependencies. Both local repositories and container images are supported as the input, and the tool is ideal for integration with ASPM/VM platforms and in CI environments.

Depscan logo

release Discord

Contents

Features

  • Scan most application code - local repos, Linux container images, Kubernetes manifests, and OS - to identify known CVEs with prioritization
  • Perform advanced reachability analysis for multiple languages (See reachability analysis)
  • Package vulnerability scanning is performed locally and is quite fast. No server is used!
  • Generate Software Bill-of-Materials (SBOM) with Vulnerability Disclosure Report (VDR) information
  • Generate a Common Security Advisory Framework (CSAF) 2.0 VEX document (check out the CSAF Readme)
  • Perform deep packages risk audit for dependency confusion attacks and maintenance risks (See risk audit)

Reachable Flows

Dependency Tree with Insights

Dependency Tree with Insights

Vulnerability Data sources

Linux distros

  • AlmaLinux
  • Debian
  • Alpine
  • Amazon Linux
  • Arch Linux
  • RHEL/CentOS
  • Rocky Linux
  • Ubuntu
  • OpenSUSE/SLES
  • Photon
  • Chainguard
  • Wolfi OS

Application vulnerabilities would be reported for all Linux distros and Windows. To download the full vulnerability database suitable for scanning OS, invoke dep-scan with --cache for the first time. dep-scan would also download the appropriate database based on project type automatically.

Usage

dep-scan is ideal for use during continuous integration (CI) and as a local development tool.

OCI Artifacts via ORAS cli

Use ORAS cli to download the vulnerability database for effortless integration. Example workflow is here.

export VDB_HOME=depscan
mkdir -p $VDB_HOME
oras pull ghcr.io/appthreat/vdb:v5 -o $VDB_HOME
# oras pull ghcr.io/appthreat/vdb-10y:v5 -o $VDB_HOME
oras pull ghcr.io/owasp-dep-scan/depscan:v4 -o $VDB_HOME

Use vdb-10y which is a larger database with vulnerability data spanning the last 10 years from 2014. In contrast, vdb with a starting year of 2018 is appropriate for most users.

Single binary executables

Download the executable binary for your operating system from the releases page. These binary bundle the following:

  • dep-scan with Python 3.11
  • cdxgen with Node.js 21
  • cdxgen binary plugins
curl -LO https://github.com/owasp-dep-scan/depscan-bin/releases/latest/download/depscan-linux-amd64
chmod +x depscan-linux-amd64
./depscan-linux-amd64 --help

On Windows,

curl -LO https://github.com/owasp-dep-scan/depscan-bin/releases/latest/download/depscan.exe
.\depscan.exe --help

Server mode

dep-scan and cdxgen could be run in server mode. Use the included docker-compose file to get started.

git clone https://github.com/owasp-dep-scan/dep-scan
docker compose up
depscan --server --server-host 0.0.0.0 --server-port 7070

In server mode, use /cache endpoint to cache the vulnerability database.

# This would take over 5 minutes
curl http://0.0.0.0:7070/cache

Use the /scan endpoint to perform scans.

Note

The type parameter is mandatory in server mode.

  • Scanning a local directory.
curl --json '{"path": "/tmp/vulnerable-aws-koa-app", "type": "js"}' http://0.0.0.0:7070/scan
  • Scanning a SBOM file (present locally).
curl --json '{"path": "/tmp/vulnerable-aws-koa-app/sbom_file.json", "type": "js"}' http://0.0.0.0:7070/scan
  • Scanning a GitHub repo.
curl --json '{"url": "https://github.com/HooliCorp/vulnerable-aws-koa-app", "type": "js"}' http://0.0.0.0:7070/scan -o app.vdr.json
  • Uploading a SBOM file and generating results based on it.
curl -X POST -H 'Content-Type: multipart/form-data' -F 'file=@/tmp/app/sbom_file.json' http://0.0.0.0:7070/scan?type=js

Scanning projects locally (Python version)

sudo npm install -g @cyclonedx/cdxgen
pip install owasp-depscan

This would install two commands called cdxgen and depscan.

You can invoke the scan command directly with the various options.

cd <project to scan>
depscan --src $PWD --reports-dir $PWD/reports

The full list of options is below:

usage: cli.py [-h] [--no-banner] [--cache] [--csaf] [--sync] [--profile {appsec,research,operational,threat-modeling,license-compliance,generic}] [--no-suggest] [--risk-audit] [--private-ns PRIVATE_NS] [-t PROJECT_TYPE] [--bom BOM]
              [-i SRC_DIR_IMAGE] [-o REPORT_FILE] [--reports-dir REPORTS_DIR] [--report-template REPORT_TEMPLATE] [--report-name REPORT_NAME] [--no-error] [--no-license-scan] [--deep] [--no-universal] [--no-vuln-table]
              [--threatdb-server THREATDB_SERVER] [--threatdb-username THREATDB_USERNAME] [--threatdb-password THREATDB_PASSWORD] [--threatdb-token THREATDB_TOKEN] [--server] [--server-host SERVER_HOST] [--server-port SERVER_PORT]
              [--cdxgen-server CDXGEN_SERVER] [--debug] [--explain] [--reachables-slices-file REACHABLES_SLICES_FILE] [-v]

Fully open-source security and license audit for application dependencies and container images based on known vulnerabilities and advisories.

options:
  -h, --help            show this help message and exit
  --no-banner           Do not display banner
  --cache               Cache vulnerability information in platform specific user_data_dir
  --csaf                Generate a OASIS CSAF VEX document
  --sync                Sync to receive the latest vulnerability data. Should have invoked cache first.
  --profile {appsec,research,operational,threat-modeling,license-compliance,generic}
                        Profile to use while generating the BOM.
  --no-suggest          Disable suggest mode
  --risk-audit          Perform package risk audit (slow operation). Npm only.
  --private-ns PRIVATE_NS
                        Private namespace to use while performing oss risk audit. Private packages should not be available in public registries by default. Comma separated values accepted.
  -t PROJECT_TYPE, --type PROJECT_TYPE
                        Override project type if auto-detection is incorrect
  --bom BOM             Examine using the given Software Bill-of-Materials (SBOM) file in CycloneDX format. Use cdxgen command to produce one.
  -i SRC_DIR_IMAGE, --src SRC_DIR_IMAGE
                        Source directory or container image or binary file
  -o REPORT_FILE, --report_file REPORT_FILE
                        DEPRECATED. Use reports directory since multiple files are created. Report filename with directory
  --reports-dir REPORTS_DIR
                        Reports directory
  --report-template REPORT_TEMPLATE
                        Jinja template file used for rendering a custom report
  --report-name REPORT_NAME
                        Filename of the custom report written to the --reports-dir
  --no-error            UNUSED: Continue on error to prevent build from breaking
  --no-license-scan     UNUSED: dep-scan doesn't perform license scanning by default
  --deep                Perform deep scan by passing this --deep argument to cdxgen. Useful while scanning docker images and OS packages.
  --no-universal        Depscan would attempt to perform a single universal scan instead of individual scans per language type.
  --no-vuln-table       Do not print the table with the full list of vulnerabilities. This can help reduce console output.
  --threatdb-server THREATDB_SERVER
                        ThreatDB server url. Eg: https://api.sbom.cx
  --threatdb-username THREATDB_USERNAME
                        ThreatDB username
  --threatdb-password THREATDB_PASSWORD
                        ThreatDB password
  --threatdb-token THREATDB_TOKEN
                        ThreatDB token for token based submission
  --server              Run depscan as a server
  --server-host SERVER_HOST
                        depscan server host
  --server-port SERVER_PORT
                        depscan server port
  --cdxgen-server CDXGEN_SERVER
                        cdxgen server url. Eg: http://cdxgen:9090
  --debug               Run depscan in debug mode.
  --explain             Makes depscan to explain the various analysis. Useful for creating detailed reports.
  --reachables-slices-file REACHABLES_SLICES_FILE
                        Path for the reachables slices file created by atom.
  --purl SEARCH_PURL    Scan a single package url.
  -v, --version         Display the version

Scanning containers locally (Python version)

Scan latest tag of the container shiftleft/scan-slim

depscan --cache --src shiftleft/scan-slim -o containertests/depscan-scan.json -t docker

Include license to the type to perform the license audit.

depscan --cache --src shiftleft/scan-slim -o containertests/depscan-scan.json -t docker,license

You can also specify the image using the sha256 digest

depscan --src redmine@sha256:a5c5f8a64a0d9a436a0a6941bc3fb156be0c89996add834fe33b66ebeed2439e -o containertests/depscan-redmine.json -t docker

You can also save container images using docker or podman save command and pass the archive to depscan for scanning.

docker save -o /tmp/scanslim.tar shiftleft/scan-slim:latest
# podman save --format oci-archive -o /tmp/scanslim.tar shiftleft/scan-slim:latest
depscan --src /tmp/scanslim.tar -o reports/depscan-scan.json -t docker

Refer to the docker tests under the GitHub action workflow for this repo for more examples.

Scanning projects locally (Docker container)

ghcr.io/owasp-dep-scan/dep-scan container image can be used to perform the scan.

To scan with default settings

docker run --rm -v $PWD:/app ghcr.io/owasp-dep-scan/dep-scan --src /app --reports-dir /app/reports

To scan with custom environment variables based configuration

docker run --rm \
    -e VDB_HOME=/db \
    -e GITHUB_TOKEN=<token> \
    -v /tmp:/db \
    -v $PWD:/app ghcr.io/owasp-dep-scan/dep-scan --src /app --reports-dir /app/reports

In the above example, /tmp is mounted as /db into the container. This directory is then specified as VDB_HOME for caching the vulnerability information. This way the database can be cached and reused to improve performance.

Supported languages and package format

dep-scan uses cdxgen command internally to create a Software Bill-of-Materials (SBOM) file for the project. This is then used for performing the scans.

The following projects and package-dependency format is supported by cdxgen.

Language Package format
node.js package-lock.json, pnpm-lock.yaml, yarn.lock, rush.js, bower.json, .min.js
java maven (pom.xml [1]), gradle (build.gradle, .kts), scala (sbt), bazel
php composer.lock
python setup.py, requirements.txt [2], Pipfile.lock, poetry.lock, bdist_wheel, .whl, .egg-info
go binary, go.mod, go.sum, Gopkg.lock
ruby Gemfile.lock, gemspec
rust binary, Cargo.toml, Cargo.lock
.Net .csproj, packages.config, project.assets.json [3], packages.lock.json, .nupkg
dart pubspec.lock, pubspec.yaml
haskell cabal.project.freeze
elixir mix.lock
c/c++ conan.lock, conanfile.txt
clojure Clojure CLI (deps.edn), Leiningen (project.clj)
docker / oci image All supported languages and Linux OS packages
GitHub Actions Workflows .github/workflows/*.yml
Jenkins Plugins .hpi files
YAML manifests docker-compose, kubernetes, kustomization, skaffold, tekton etc

Reachability analysis

Depscan can perform reachability analysis for Java, JavaScript, TypeScript and Python with built-in support for parsing atom reachables slicing. Simply invoke depscan with the research profile and language type to enable this feature.

To receive a verbose output including the reachable flows, pass the argument --explain

--profile research -t language [--explain]

Example analysis for a Java project

depscan --profile research -t java -i <source directory> --reports-dir <reports directory> --explain

Example analysis for a JavaScript project

depscan --profile research -t js -i <source directory> --reports-dir <reports directory> --explain

Example analysis for a PHP project

Ensure PHP > 7.4 is installed. However, we support scanning PHP 5.2 - 8.3. Alternatively, use the depscan container image.

depscan --profile research -t php -i <source directory> --reports-dir <reports directory> --explain

PHP Reachability

PHP NOT Reachability

Customization through environment variables

The following environment variables can be used to customize the behavior.

  • VDB_HOME - Directory to use for caching database. For docker-based execution, this directory should get mounted as a volume from the host
  • VDB_DATABASE_URL - Vulnerability DB URL. Defaults to: ghcr.io/appthreat/vdbgz:v5
  • USE_VDB_10Y - Set to true to use the larger 10-year vulnerability database. Default download url: ghcr.io/appthreat/vdb-10y:v5

GitHub Security Advisory

To download security advisories from GitHub, a personal access token with minimal permissions is necessary.

  • Fine-grained token: Grant no permissions and select the following for repository access: Public Repositories (read-only)
  • Token (classic): Grant no permissions
export GITHUB_TOKEN="<PAT token>"

Suggest mode

Depscan comes with a suggest mode enabled by default to simplify the triaging experience. The fix version for each vulnerability is retrieved from the sources. Sometimes, there might be known vulnerabilities in the fix version reported. Eg: in the below screenshot the fix versions suggested for jackson-databind might contain known vulnerabilities.

Normal mode

Suggest mode

Notice, how the new suggested version is 2.9.10.5 which is an optimal fix version. Please note that the optimal fix version may not be the appropriate version for your application based on compatibility.

Pass --no-suggest to disable this behavior.

Package Risk audit

--risk-audit argument enables package risk audit. Currently, only npm and PyPI packages are supported in this mode. Some risk factors are identified and assigned weights to compute a final risk score. Packages that then exceed a maximum risk score (config.pkg_max_risk_score) are presented in a table.

Use --private-ns to specify the private package namespace that should be checked for dependency confusion type issues where a private package is available on the public npm/pypi registry.

For example, to check if private packages with namespaces @appthreat and @shiftleft are not accidentally made public, use the below argument.

--private-ns appthreat,shiftleft
Risk category Default Weight Reason
pkg_private_on_public_registry 4 Private package is available on a public registry
pkg_min_versions 2 Packages with less than 3 versions represent an extreme where they could be either super stable or quite recent. Special heuristics are applied to ignore older stable packages
mod_create_min_seconds 1 Less than 12 hours difference between modified and creation time. This indicates that the upload had a defect that had to be rectified immediately. Sometimes, such a rapid update could also be malicious
latest_now_min_seconds 0.5 Less than 12 hours difference between the latest version and the current time. Depending on the package such a latest version may or may not be desirable
latest_now_max_seconds 0.5 Package versions that are over 6 years old are in use. Such packages might have vulnerable dependencies that are known or yet to be found
pkg_min_maintainers 2 Package has less than 2 maintainers. Many opensource projects have only 1 or 2 maintainers so special heuristics are used to ignore older stable packages
pkg_min_users 0.25 Package has less than 2 npm users
pkg_install_scripts 2 Package runs a custom pre or post installation scripts. This is often malicious and a downside of npm.
pkg_node_version 0.5 Package supports outdated version of node such as 0.8, 0.10, 4 or 6.x. Such projects might have prototype pollution or closure related vulnerabilities
pkg_scope 4 or 0.5 Packages that are used directly in the application (required scope) gets a score with a weight of 4. Optional packages get a score of 0.25
deprecated 1 Latest version is deprecated

Refer to pkg_query.py::get_category_score method for the risk formula.

Automatic adjustment

A parameter called created_now_quarantine_seconds is used to identify packages that are safely past the quarantine period (1 year). Certain risks such as pkg_min_versions and pkg_min_maintainers are suppressed for packages past the quarantine period. This adjustment helps reduce noise since it is unlikely that a malicious package can exist in a registry unnoticed for over a year.

Configuring weights

All parameters can be customized by using environment variables. For eg:

export PKG_MIN_VERSIONS=4 to increase and set the minimum versions category to 4.

Live OS scan

By passing -t os, depscan can generate an SBOM for a live operating system or a VM with OS packages and kernel information. Optionally, pass the argument --deep to generate an SBOM with both OS and application packages and to check for application vulnerabilities.

All OS packages.

depscan -t os -i . -o reports/depscan.json

All OS and application packages.

depscan -t os --deep -i . -o reports/depscan.json

License scan

dep-scan can scan the dependencies for any license limitations and report them directly on the console log. To enable license scanning set the environment variable FETCH_LICENSE to true.

export FETCH_LICENSE=true

The license data is sourced from choosealicense.com and is quite limited. If the license of a given package cannot be reliably matched against this list it will get silently ignored to reduce any noise. This behavior could change in the future once the detection logic gets improved.

License scan

Kubernetes and Cloud apps

dep-scan could auto-detect most cloud applications and Kubernetes manifest files. Pass the argument -t yaml-manifest to manually specify the type.

PDF reports

Ensure wkhtmltopdf is installed or use the official container image to generate pdf reports. Use with --explain for more detailed reports.

Custom reports

dep-scan can be provided with a Jinja template using the --report-template parameter. Giving it will pass the vulnerability report into your template for rendering the report.

Please find a basic example here:

{% if metadata -%}
Report for {{ metadata.component.group }}:{{ metadata.component.name }}:{{ metadata.component.version }}
{% endif -%}

{% if vulnerabilities -%}
There were {{ vulnerabilities | length }} issues identified:

{% for vuln in vulnerabilities -%}
* {{ vuln['bom-ref'] }} - {{ vuln.recommendation }}
{% endfor -%}
{% else -%}
🏆 _No vulnerabilities found_
{% endif -%}

Severity counts:
* Low: {{ summary.LOW }}
* Medium: {{ summary.MEDIUM }}
* High: {{ summary.HIGH }}
* Critical: {{ summary.CRITICAL }}
* Unspecified: {{ summary.UNSPECIFIED }}

The objects available are taken from the CycloneDX *.vdr.json BOM file generated, just have a look at the file for its full structure:

  • metadata
  • vulnerabilities
  • components
  • dependencies
  • services

summary is a dictionary type with vulnerability severity quantities as shown in the example above. pkg_vulnerabilities - Same as vulnerabilities from the VDR pkg_group_rows - List of vulnerability id and the dependency tree prioritized by depscan.

Furthermore, insights are imaginable to be made available to the template, please reach out or contribute on demand. We appreciate it if you like to contribute your report templates as examples, please add/find them here.

Performance tuning

Use nydus to speed up the initial vdb download

vdb v5 is published in RAFS (Registry Accelerated File System) format with better de-duplication and packing. depscan would automatically use this image if nydus-static binary is available in the PATH.

curl -LO https://github.com/dragonflyoss/nydus/releases/download/v2.2.4/nydus-static-v2.2.4-linux-amd64.tgz
tar -xvf nydus-static-v2.2.4-linux-amd64.tgz
chmod +x nydus-static/*
mv nydus-static/* /usr/local/bin/

Discord support

The developers could be reached via the discord channel for enterprise support.

License

MIT License

This project was donated to the OWASP Foundation in August 2023 by AppThreat Ltd.

dep-scan's People

Contributors

algomaster99 avatar cerrussell avatar deleterepo avatar fabaff avatar heubeck avatar ianonymous3000 avatar matkoniecz avatar mayaa23 avatar prabhu avatar quince-pie avatar saketjajoo avatar timmyteo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dep-scan's Issues

Support for Clojure

I really loved this snyk alternative and has great potential. Just wondering if there is any clojure support we can expect anytime soon.

dep-scan does not detect python `requirements.txt` vulnerabilities

Neat work with the dep-scan :).
I just wanted to file an issue, which seems like a potential bug.

Issue:

While trying out against a simple requirements.txt file with the content, it detects as no vulnerabilities, while not being the same.

Django==2.2.8

Sample Output:

Command Ran:

export SCAN_DEBUG_MODE="debug"
scan -i ./
DEBUG [2020-09-29 22:23:46,548] Scanning using the bom file ./reports/bom-python.json
INFO [2020-09-29 22:23:46,672] No license violation detected ✅
DEBUG [2020-09-29 22:23:49,974] Vulnerability database loaded from /Users/xxxx/Library/Application Support/vdb/data.vdb
INFO [2020-09-29 22:23:49,975] To use GitHub advisory source please set the environment variable GITHUB_TOKEN!
DEBUG [2020-09-29 22:23:53,049] Vulnerability database contains 1948391 records
INFO [2020-09-29 22:23:53,049] Performing regular scan for ./ using plugin python
INFO [2020-09-29 22:23:53,049] Scanning 1 oss dependencies for issues
INFO [2020-09-29 22:24:39,343] No oss vulnerabilities detected ✅

This returns no vulnerabilities, well which is not the case as i did verify with vdb --search Django:2.2.8 and it has vulnerabilities.

Let me know if i'm missing something here or got the options wrong.

P.S: I tried debugging the issue, but it seems to point to the dependency vulnerability-db -> bulk_index_search returning NULL from what i could check.

Scanning BOMs doesn't work

If I scan a repository using docker image shiftleft/sast-scan it will generate a BOM and dep-scan show's results.

If I try to re-run dep-scan using --bom and either the bom-ruby.json or bom-ruby.xml file the output is empty.

[root@e49a4c7cfeba app]# depscan --bom bom-ruby.xml

  ___            _____ _                    _
 / _ \          |_   _| |                  | |
/ /_\ \_ __  _ __ | | | |__  _ __ ___  __ _| |_
|  _  | '_ \| '_ \| | | '_ \| '__/ _ \/ _` | __|
| | | | |_) | |_) | | | | | | | |  __/ (_| | |_
\_| |_/ .__/| .__/\_/ |_| |_|_|  \___|\__,_|\__|
      | |   | |
      |_|   |_|

[root@e49a4c7cfeba app]# depscan --bom bom-ruby.json

  ___            _____ _                    _
 / _ \          |_   _| |                  | |
/ /_\ \_ __  _ __ | | | |__  _ __ ___  __ _| |_
|  _  | '_ \| '_ \| | | '_ \| '__/ _ \/ _` | __|
| | | | |_) | |_) | | | | | | | |  __/ (_| | |_
\_| |_/ .__/| .__/\_/ |_| |_|_|  \___|\__,_|\__|
      | |   | |
      |_|   |_|

An example project that works whilst scanning normally but doesn't work trying to scan the BOM is the ruby-bundler directory in https://gitlab.com/gitlab-org/security-products/tests/dependency-scanning.git

I have also generated SBOMs using other tools and when scanning them with depscan --bom get empty results.

Container image scan for application vulnerabilities

cdxgen 3.0.0 would support application SBoM generation from container images. dep-scan could be enhanced to support reporting application-specific vulnerabilities from this bom.

NOTE:

This is not the same as container image scanning that reports OS-specific vulnerabilities. If you're interested in that feature you can contact me to discuss some funding options.

Unnecessary package import risk

Noticed that some node.js apps have package.json entries for built-in packages, which is risky. Possible issues in the python world too.

Multiple license string is producing multiple entries

jruby-complete is offered under licenses: EPL-2.0, GPL-2.0, LGPL-2.1

https://mvnrepository.com/artifact/org.jruby/jruby-complete

It is not possible to compute an effective license string in this case so the tool is producing multiple entries like this.

| org.jruby:jruby-complete                    | 9.1.17.0  | GPL-2.0      | include-copyright, document-changes, disclose-source, same-license          |
+---------------------------------------------+-----------+--------------+-----------------------------------------------------------------------------+
| org.jruby:jruby-complete                    | 9.1.17.0  | LGPL-2.1     | include-copyright, disclose-source, document-changes, same-license--library |
+---------------------------------------------+-----------+--------------+-----------------------------------------------------------------------------+
| org.jruby:jruby-complete                    | 9.1.17.0  | EPL-2.0      | disclose-source, include-copyright, same-license                            |
+---------------------------------------------+-----------+--------------+-----------------------------------------------------------------------------+

I feel this behavior is correct since the conditions are different. But if anyone has a better idea or expertise with multiple licenses, please post your comments in this issue.

npm: Ignore @types packages

Hi,

Unless I misread the situation and the typing package is actually not secure, dep-scan considers a types package and the "regular" package as the same.

For example I use the latest morgan package (v1.10.0) right now, and the scanner (I use docker run) gives me no vulnerability.

If I install the latest @types/morgan, which happens to be v1.9.0, I then get a medium vulnerability Code Injection.
I did the test of removing [email protected] and @types/morgan and installing [email protected] and I get the same vulnerability.

So I think for npm, the scanner needs to ignore @types/* packages since they are devDependencies anyway.

Thank you !

Support for correction db

dep-scan use a rudimentary vendor and package name alias to fix misfiled CVEs. However, some CVEs are terribly filed with no version number information and even missing CPEs. We need to think about a way to add the corrections data to dep-scan and make it scalable.

Support for suppression

The tool should allow suppression of false positives. We need:

  • A way of defining the suppressions. The xml format used by dependency-check is not suitable for the modern era. We need to think of a better yaml or toml format.
  • A way of representing suppressions in the report. grafeas currently doesn't support this metadata

aerospike python 2 fix version

In aerospike docker image, fix_version is suggested as 3.6.14, which is an interesting edge-case. Need to investigate where this is coming from.

{"id": "CVE-2022-0391", "package": "pkg:pypi/[email protected]", "purl": "pkg:pypi/[email protected]", "package_type": "python", "package_usage": "N/A", "version": "2.7.16", "fix_version": "3.6.14", "severity": "HIGH", "cvss_score": "7.5", "short_description": "A flaw was found in Python, specifically within the urllib.parse module. This module helps break Uniform Resource Locator (URL) strings into components. The issue involves how the urlparse method does not sanitize input and allows characters like '\\r' and '\\n' in the URL path. This flaw allows an attacker to input a crafted URL, leading to injection attacks. This flaw affects Python versions prior to 3.10.0b1, 3.9.5, 3.8.11, 3.7.11 and 3.6.14.", "related_urls": ["https://bugs.python.org/issue43882", "https://lists.fedoraproject.org/archives/list/[email protected]/message/CSD2YBXP3ZF44E44QMIIAR5VTO35KTRB/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/UDBDBAU6HUPZHISBOARTXZ5GKHF2VH5U/", "https://security.netapp.com/advisory/ntap-20220225-0009/", "https://www.oracle.com/security-alerts/cpuapr2022.html"]}
{"id": "CVE-2022-26488", "package": "pkg:pypi/[email protected]", "purl": "pkg:pypi/[email protected]", "package_type": "python", "package_usage": "N/A", "version": "2.7.16", "fix_version": null, "severity": "HIGH", "cvss_score": "7.0", "short_description": "In Python before 3.10.3 on Windows, local users can gain privileges because the search path is inadequately secured. The installer may allow a local attacker to add user-writable directories to the system search path. To exploit, an administrator must have installed Python for all users and enabled PATH entries. A non-administrative user can trigger a repair that incorrectly adds user-writable paths into PATH, enabling search-path hijacking of other users and system services. This affects Python (CPython) through 3.7.12, 3.8.x through 3.8.12, 3.9.x through 3.9.10, and 3.10.x through 3.10.2.", "related_urls": ["https://mail.python.org/archives/list/[email protected]/thread/657Z4XULWZNIY5FRP3OWXHYKUSIH6DMN/", "https://security.netapp.com/advisory/ntap-20220419-0005/"]}
{"id": "CVE-2022-45061", "package": "pkg:pypi/[email protected]", "purl": "pkg:pypi/[email protected]", "package_type": "python", "package_usage": "N/A", "version": "2.7.16", "fix_version": null, "severity": "HIGH", "cvss_score": "7.5", "short_description": "An issue was discovered in Python before 3.11.1. An unnecessary quadratic algorithm exists in one path when processing some inputs to the IDNA (RFC 3490) decoder, such that a crafted, unreasonably long name being presented to the decoder could lead to a CPU denial of service. Hostnames are often supplied by remote servers that could be controlled by a malicious actor; in such a scenario, they could trigger excessive CPU consumption on the client attempting to make use of an attacker-supplied supposed hostname. For example, the attack payload could be placed in the Location header of an HTTP response with status code 302. A fix is planned in 3.11.1, 3.10.9, 3.9.16, 3.8.16, and 3.7.16.", "related_urls": ["https://github.com/python/cpython/issues/98433"]}
{"id": "CVE-2021-23336", "package": "pkg:pypi/[email protected]", "purl": "pkg:pypi/[email protected]", "package_type": "python", "package_usage": "N/A", "version": "2.7.16", "fix_version": "3.6.13", "severity": "MEDIUM", "cvss_score": "5.9", "short_description": "The package python/cpython from 0 and before 3.6.13, from 3.7.0 and before 3.7.10, from 3.8.0 and before 3.8.8, from 3.9.0 and before 3.9.2 are vulnerable to Web Cache Poisoning via urllib.parse.parse_qsl and urllib.parse.parse_qs by using a vector called parameter cloaking. When the attacker can separate query parameters using a semicolon (;), they can cause a difference in the interpretation of the request between the proxy (running with default configuration) and the server. This can result in malicious requests being cached as completely safe ones, as the proxy would usually not see the semicolon as a separator, and therefore would not include it in a cache key of an unkeyed parameter.", "related_urls": ["https://snyk.io/vuln/SNYK-UPSTREAM-PYTHONCPYTHON-1074933", "https://github.com/python/cpython/pull/24297", "https://snyk.io/blog/cache-poisoning-in-popular-open-source-packages/", "http://www.openwall.com/lists/oss-security/2021/02/19/4", "https://lists.debian.org/debian-lts-announce/2021/02/msg00030.html", "https://lists.fedoraproject.org/archives/list/[email protected]/message/HCQTCSP6SCVIYNIRUJC5X7YBVUHPLSC4/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/NODWHDIFBQE5RU5PUWUVE47JOT5VCMJ2/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/MNUN5SOMFL2BBKP6ZAICIIUPQKZDMGYO/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/MP572OLHMS7MZO4KUPSCIMSZIA5IZZ62/", "https://lists.apache.org/thread.html/rf9fa47ab66495c78bb4120b0754dd9531ca2ff0430f6685ac9b07772@%3Cdev.mina.apache.org%3E", "https://lists.fedoraproject.org/archives/list/[email protected]/message/FONHJIOZOFD7CD35KZL6SVBUTMBPGZGA/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/KJXCMHLY7H3FIYLE4OKDYUILU2CCRUCZ/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/3YKKDLXL3UEZ3J426C2XTBS63AHE46SM/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/TFTELUMWZE3KV3JB2H5EE6VFRZFRD5MV/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/OAGSWNGZJ6HQ5ISA67SNMK3CJRKICET7/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/NJSCSN722JO2E2AGPWD4NTGVELVRPB4R/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/HZTM7KLHFCE3LWSEVO2NAFLUHMGYMCRY/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/3EPYWWFDV22CJ5AOH5VCE72DOASZZ255/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/IHQDU7NXA7EWAE4W7VO6MURVJIULEPPR/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/LVNH6Z24IG3E67ZCQGGJ46FZB4XFLQNZ/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/W2LSKBEFI5SYEY5FM6ICZVZM5WRQUCS4/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/46N6A52EGSXHJYCZWVMBJJIH4NWIV2B5/", "https://security.netapp.com/advisory/ntap-20210326-0004/", "https://lists.debian.org/debian-lts-announce/2021/04/msg00005.html", "https://lists.debian.org/debian-lts-announce/2021/04/msg00015.html", "https://lists.fedoraproject.org/archives/list/[email protected]/message/N6VXJZSZ6N64AILJX4CTMACYGQGHHD5C/", "https://security.gentoo.org/glsa/202104-04", "https://lists.apache.org/thread.html/ra8ce70088ba291f358e077cafdb14d174b7a1ce9a9d86d1b332d6367@%3Cusers.airflow.apache.org%3E", "http://www.openwall.com/lists/oss-security/2021/05/01/2", "https://lists.apache.org/thread.html/rc005f4de9d9b0ba943ceb8ff5a21a5c6ff8a9df52632476698d99432@%3Cannounce.apache.org%3E", "https://lists.fedoraproject.org/archives/list/[email protected]/message/SGIY6I4YS3WOXAK4SXKIEOC2G4VZKIR7/", "https://lists.fedoraproject.org/archives/list/[email protected]/message/RSLQD5CCM75IZGAMBDGUZEATYU5YSGJ7/", "https://www.oracle.com/security-alerts/cpuApr2021.html", "https://www.oracle.com//security-alerts/cpujul2021.html", "https://www.oracle.com/security-alerts/cpuoct2021.html", "https://www.oracle.com/security-alerts/cpujan2022.html"]}

Simulate ua-parser attack to assess dep-scan

It is not clear if the risk audit feature of dep-scan caught the ua-parser.js attack. My guess is that it should've checked for the presence of the preinstall script and flagged this up. But will be nice to emulate and validate the feature.

Issue when used within docker for GHA

We're seeing this error when using dep-scan through a container in github actions.

invocation:
/usr/bin/docker run --name quayioappthreatdepscanlatest_e516a4 --label 290506 --workdir /github/workspace --rm -e "IsRunningOnCI" -e "Environment" -e "DOTNET_ROOT" -e "VDB_HOME" -e "GITHUB_TOKEN" -e "INPUT_SRC" -e "INPUT_REPORT_FILE" -e "INPUT_BOM" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/carrot-app/carrot-app":"/github/workspace" quay.io/appthreat/dep-scan:latest "scan" "--src" "/github/workspace" "--report_file" "/github/workspace/reports/depscan.json"

  ___            _____ _                    _
 / _ \          |_   _| |                  | |
/ /_\ \_ __  _ __ | | | |__  _ __ ___  __ _| |_
|  _  | '_ \| '_ \| | | '_ \| '__/ _ \/ _` | __|
| | | | |_) | |_) | | | | | | | |  __/ (_| | |_
\_| |_/ .__/| .__/\_/ |_| |_|_|  \___|\__,_|\__|
      | |   | |
      |_|   |_|

Traceback (most recent call last):
  File "/usr/local/bin/scan", line 33, in <module>
    sys.exit(load_entry_point('appthreat-depscan==2.2.0', 'console_scripts', 'scan')())
  File "/usr/local/lib/python3.9/site-packages/appthreat_depscan-2.2.0-py3.9.egg/depscan/cli.py", line 242, in main
    project_types_list = utils.detect_project_type(src_dir)
  File "/usr/local/lib/python3.9/site-packages/appthreat_depscan-2.2.0-py3.9.egg/depscan/lib/utils.py", line [15](https://github.com/carrotfertility/carrot-app/actions/runs/3447575122/jobs/5753722505#step:6:16)6, in detect_project_type
    if find_files(src_dir, ".github", "workflows", "*.yml", quick=True):
TypeError: find_files() got multiple values for argument 'quick'

"Exception while parsing NVD CVE feed" when no vulnerability found

It runs OK several days before, but recently it went to this exception when scanning the same project:

WARNING [2020-12-28 08:40:27,572] Exception while parsing NVD CVE feed
Traceback (most recent call last):
File "/usr/local/bin/scan", line 33, in
sys.exit(load_entry_point('appthreat-depscan==1.9.2', 'console_scripts', 'scan')())
File "/usr/local/lib/python3.8/site-packages/appthreat_depscan-1.9.2-py3.8.egg/depscan/cli.py", line 293, in main
s.refresh()
File "/usr/local/lib/python3.8/site-packages/appthreat_vulnerability_db-1.5.2-py3.8.egg/vdb/lib/nvd.py", line 95, in refresh
return self.download_all()
File "/usr/local/lib/python3.8/site-packages/appthreat_vulnerability_db-1.5.2-py3.8.egg/vdb/lib/nvd.py", line 51, in download_all
self.store(data)
File "/usr/local/lib/python3.8/site-packages/appthreat_vulnerability_db-1.5.2-py3.8.egg/vdb/lib/nvd.py", line 100, in store
docs = dbLib.store(db, data)
File "/usr/local/lib/python3.8/site-packages/appthreat_vulnerability_db-1.5.2-py3.8.egg/vdb/lib/db.py", line 61, in store
docs = storage.store(datas, db_file=db["db_file"], index_file=db["index_file"])
File "/usr/local/lib/python3.8/site-packages/appthreat_vulnerability_db-1.5.2-py3.8.egg/vdb/lib/storage.py", line 17, in store
data_list = serialize_vuln_list(datas)
File "/usr/local/lib/python3.8/site-packages/appthreat_vulnerability_db-1.5.2-py3.8.egg/vdb/lib/utils.py", line 146, in serialize_vuln_list
for data in datas:
TypeError: 'NoneType' object is not iterable

But all is OK about 10 days before:

image

It seems that it cannot found any vulnerability recently, it should give a message like "No vulnerability detected", but runs into this exception. But I am not sure of it.

Error at start

Hi, I'm using https://github.com/AppThreat/dep-scan-action on some of my repos and it started to fail today with the following error:

  ___            _____ _                    _
 / _ \          |_   _| |                  | |
/ /_\ \_ __  _ __ | | | |__  _ __ ___  __ _| |_
|  _  | '_ \| '_ \| | | '_ \| '__/ _ \/ _` | __|
| | | | |_) | |_) | | | | | | | |  __/ (_| | |_
\_| |_/ .__/| .__/\_/ |_| |_|_|  \___|\__,_|\__|
      | |   | |
      |_|   |_|

INFO [2021-11-10 11:29:56,256] ================================================================================
╭──────────────────────── New Feature ────────────────────────╮
│ Depscan supports OSS Risk audit for this project.           │
│ To enable set the environment variable ENABLE_OSS_RISK=true │
╰─────────────────────────────────────────────────────────────╯
Traceback (most recent call last):
  File "/usr/local/bin/scan", line 33, in <module>
    sys.exit(load_entry_point('appthreat-depscan==2.0.5', 'console_scripts', 'scan')())
  File "/usr/local/lib/python3.8/site-packages/appthreat_depscan-2.0.5-py3.8.egg/depscan/cli.py", line 394, in main
    s.refresh()
  File "/usr/local/lib/python3.8/site-packages/appthreat_vulnerability_db-1.7.2-py3.8.egg/vdb/lib/nvd.py", line 106, in refresh
    return self.download_all()
  File "/usr/local/lib/python3.8/site-packages/appthreat_vulnerability_db-1.7.2-py3.8.egg/vdb/lib/gha.py", line 104, in download_all
    data, page_info = self.fetch(type=lastId)
  File "/usr/local/lib/python3.8/site-packages/appthreat_vulnerability_db-1.7.2-py3.8.egg/vdb/lib/gha.py", line 131, in fetch
    return self.convert(json_data)
  File "/usr/local/lib/python3.8/site-packages/appthreat_vulnerability_db-1.7.2-py3.8.egg/vdb/lib/gha.py", line 189, in convert
    if cve.get("withdrawnAt"):
AttributeError: 'NoneType' object has no attribute 'get'

This is the command launched:

/usr/bin/docker run --name quayioappthreatdepscanlatest_7e77b1 --label e28490 --workdir /github/workspace --rm -e VDB_HOME -e GITHUB_TOKEN -e INPUT_SRC -e INPUT_REPORT_FILE -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_REF_NAME -e GITHUB_REF_PROTECTED -e GITHUB_REF_TYPE -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_ARCH -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/genesis/genesis":"/github/workspace" quay.io/appthreat/dep-scan:latest  "scan" "--src" "/github/workspace/main/" "--report_file" "/github/workspace/reports/depscan.json"

Any clue of what is happening?

Support for multi-project repo

There are repos that currently bundle two or more applications. They could use differerent technologies and frameworks - say a java-based service with pom.xml and an express.js frontend with package.json. cdxgen currently doesn't do a good job of dealing with this situation.

FP: netaddr 0.8.0

CVE-2019-17383 for netaddr ruby gem appears to have been filed under netaddr_project. The suffix _project is also used for pypi reports thus causing a false positive for python projects.

Entry from vulnerability database.

PYTHONPATH=. python vdb/cli.py --search netaddr:0.8.0

  ___            _____ _                    _
 / _ \          |_   _| |                  | |
/ /_\ \_ __  _ __ | | | |__  _ __ ___  __ _| |_
|  _  | '_ \| '_ \| | | '_ \| '__/ _ \/ _` | __|
| | | | |_) | |_) | | | | | | | |  __/ (_| | |_
\_| |_/ .__/| .__/\_/ |_| |_|_|  \___|\__,_|\__|
      | |   | |
      |_|   |_|

INFO [2021-01-25 13:14:38,216] Vulnerability database loaded from /Users/prabhu/Library/Application Support/vdb/data.vdb
+----------------+-------------------------+--------------------+---------------+---------+------------+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Id             | Package                 | Affected Version   | Fix Version   | CWE     | Severity   |   Score | Description                                                                                                                                                |
+================+=========================+====================+===============+=========+============+=========+============================================================================================================================================================+
| CVE-2019-17383 | netaddr_project:netaddr | <2.0.4             | 2.0.4         | CWE-276 | CRITICAL   |     9.8 | The netaddr gem before 2.0.4 for Ruby has misconfigured file permissions, such that a gem install may result in 0777 permissions in the target filesystem. |
+----------------+-------------------------+--------------------+---------------+---------+------------+---------+------------------------------------------------------------------------------------------------------------------------------------------------------------+

Support for storing vulndb data in mysql/postgresql

As of now vulndb data is being used by dep-scan but it's storing the data on storage volume which is not ideal for it's size. Especially in production environment. So I think it would be great to store vulnerabilities data in database.

ASK: Could suggest mode become the default?

This would make the new table structure the default when implementing 63.

#63

I would like to slowly transform this tool to offer actionable ideas rather than merely displaying CVE output.

Stacktrace thrown when scanning cyclonedx SBOM created against Python project

When running depscan against a bom.xml file generated by CycloneDX for a Python project, the following stack trace is thrown:

Traceback (most recent call last):
  File "/home/matthew/.local/bin/depscan", line 8, in <module>
    sys.exit(main())
  File "/home/matthew/.local/lib/python3.10/site-packages/depscan/cli.py", line 260, in main
    os.makedirs(reports_dir)
  File "/usr/lib/python3.10/os.py", line 225, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: ''

How to reproduce:

  1. Clone the Google Microservice demo: git clone https://github.com/GoogleCloudPlatform/microservices-demo.git
  2. Open the email service (A Python project): cd microservices-demo/src/emailservice
  3. Create the SBOM file: cyclonedx-py -r --output bom.xml
  4. Scan the bom.xml file: depscan --bom "bom.xml" --type bom --report_file depscan.json

This is the sample bom.xml file: bom.zip

Implement suggest mode for remote audit

depscan-improvements

As evident in the screenshot, lodash 4.7.11 is recommended as the fix version for a vulnerability. However, subsequently the same version is flagged as vulnerable with 4.7.12 as the fix version. Suggest mode exists to resolve this type of issues.

Suggest mode is only getting triggered while using db based scans here https://github.com/AppThreat/dep-scan/blob/master/depscan/cli.py#L308

While performing audits, the code is taking a different flow here https://github.com/AppThreat/dep-scan/blob/master/depscan/cli.py#L271

Some refactoring is required here https://github.com/AppThreat/dep-scan/blob/master/depscan/cli.py#L131 to create separate methods that will be invoked by both audit and scan

Investigate re-using this project as a container scanning tool

It should be possible to use docker-py and some platform specific ways of listing packages (like rpm -qa) to create a container or OS scanner tool with this repo. Have to think about a way of differentiating this from the likes of trivy and anchore though since the world doesn't need yet another container scanner.

May be grafeas support might help sell things a bit but not sure what else we could do.

Scanning multiproject in java Spring boot

Captura de Pantalla 2021-12-21 a la(s) 16 03 02
I'm trying to scan a java multiproject. But I doesn't obtain a result. I'm using a simple command: depscan --src $PWD --report_file $PWD/reports/depscan.json. How can I scan this kind of project? I have three build.gradle files, one of them in root of folder, and the others inside specific project folder.
Captura de Pantalla 2021-12-21 a la(s) 16 05 26

False positives and false negatives

Hi!
My name is Nacho, and I am working on a research of many security testing tools such as ShiftLeft.
I would like to know if there has been any benchmark test of dep-scan to get the FP and FN of this tool (i am intested in maven dependencies specially).
Thanks for answering!!

Fix -o argument

-o argument currently accepts a single file which is then used to derive different filenames. This should be fixed by accepting a directory name for -o argument.

adoptopenjdk incorrect results

adoptopenjdk package_type is incorrectly detected as debian due to deb making the fix versions incorrect.

{"id": "CVE-2021-3995", "package": "pkg:deb/ubuntu/[email protected]?distro=ubuntu-20.04", "purl": "pkg:deb/ubuntu/[email protected]?distro=ubuntu-20.04", "package_type": "debian", "package_usage": "N/A", "version": "2.34-0.1ubuntu9.1", "fix_version": "2.36.1-8+deb11u1", "severity": "LOW", "cvss_score": "2.0", "short_description": "# util-linux - security update\n\nThe Qualys Research Labs discovered two vulnerabilities in util-linux's\nlibmount. These flaws allow an unprivileged user to unmount other users'\nfilesystems that are either world-writable themselves or mounted in a\nworld-writable directory \n([\\\nCVE-2021-3996](https://security-tracker.debian.org/tracker/CVE-2021-3996)), or to unmount FUSE filesystems that belong to certain other \nusers \n([\\\nCVE-2021-3995](https://security-tracker.debian.org/tracker/CVE-2021-3995)).\n\n\nFor the stable distribution (bullseye), these problems have been fixed in\nversion 2.36.1-8+deb11u1.\n\n\nWe recommend that you upgrade your util-linux packages.\n\n\nFor the detailed security status of util-linux please refer to its\nsecurity tracker page at:\n[\\\nhttps://security-tracker.debian.org/tracker/util-linux](https://security-tracker.debian.org/tracker/util-linux)\n\n\n\n\n## Related CVE(s)\nCVE-2021-3995, CVE-2021-3996\n            \n            ", "related_urls": ["https://www.debian.org/security/2022/dsa-5055"]}
{"id": "CVE-2022-1271", "package": "pkg:deb/ubuntu/[email protected]?distro=ubuntu-20.04", "purl": "pkg:deb/ubuntu/[email protected]?distro=ubuntu-20.04", "package_type": "debian", "package_usage": "N/A", "version": "1.10-0ubuntu4", "fix_version": "1.10-4+deb11u1", "severity": "LOW", "cvss_score": "3.7", "short_description": "# gzip - security update\n\ncleemy desu wayo reported that incorrect handling of filenames by zgrep\nin gzip, the GNU compression utilities, can result in overwrite of\narbitrary files or execution of arbitrary code if a file with a\nspecially crafted filename is processed.\n\n\nFor the oldstable distribution (buster), this problem has been fixed\nin version 1.9-3+deb10u1.\n\n\nFor the stable distribution (bullseye), this problem has been fixed in\nversion 1.10-4+deb11u1.\n\n\nWe recommend that you upgrade your gzip packages.\n\n\nFor the detailed security status of gzip please refer to its security\ntracker page at:\n[\\\nhttps://security-tracker.debian.org/tracker/gzip](https://security-tracker.debian.org/tracker/gzip)\n\n\n\n\n            ", "related_urls": ["https://www.debian.org/security/2022/dsa-5122"]}
{"id": "CVE-2022-1664", "package": "pkg:deb/ubuntu/[email protected]?distro=ubuntu-20.04", "purl": "pkg:deb/ubuntu/[email protected]?distro=ubuntu-20.04", "package_type": "debian", "package_usage": "N/A", "version": "1.19.7ubuntu3", "fix_version": "1.19.8", "severity": "LOW", "cvss_score": "2.0", "short_description": "# dpkg - security update\n\nMax Justicz reported a directory traversal vulnerability in\nDpkg::Source::Archive in dpkg, the Debian package management system.\nThis affects extracting untrusted source packages in the v2 and v3\nsource package formats that include a debian.tar.\n\n\nFor the oldstable distribution (buster), this problem has been fixed\nin version 1.19.8.\n\n\nFor the stable distribution (bullseye), this problem has been fixed in\nversion 1.20.10.\n\n\nWe recommend that you upgrade your dpkg packages.\n\n\nFor the detailed security status of dpkg please refer to its security\ntracker page at:\n<https://security-tracker.debian.org/tracker/dpkg>\n\n\n\n\n            ", "related_urls": ["https://www.debian.org/security/2022/dsa-5147"]}

Fix versions are different for ubuntu.

"focal": {
        "Status": "released",
        "Note": "2.34-0.1ubuntu9.3"
      },
      "impish": {
        "Status": "released",
        "Note": "2.36.1-8ubuntu2.2"
      },
      "jammy": {
        "Status": "released",
        "Note": "2.37.2-4ubuntu2"
      },
      "trusty": {
        "Status": "not-affected",
        "Note": "code not present"
      },

TypeError in `license.py`

Hi!

Tool works fine for some Python projects and Java + Node projects. Whilst testing out the tool for one Python project, found a small bug. Not sure what the root cause is yet, so just adding some context here. Fix should be simple enough.

Command:

scan --src $PWD --report_file $PWD/reports/depscan.json

Output:

INFO [2020-07-22 10:34:14,830] ================================================================================
Traceback (most recent call last):
  File "/usr/local/bin/scan", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/depscan/cli.py", line 234, in main
    build_license_data(license_data_dir), pkg_list=pkg_list
  File "/usr/local/lib/python3.7/site-packages/depscan/lib/license.py", line 35, in bulk_lookup
    pkg_key = pkg["vendor"] + ":" + pkg["name"] + "@" + pkg["version"]
TypeError: can only concatenate str (not "NoneType") to str

A fix would just be to use some string formatting or handle the case of None. Not sure why None is being introduced, if I have more time I will try and track down more. For now I tried this fix which allowed the scan to complete successfully.

pkg_key = f"{pkg['vendor']}:{pkg['name']}@{pkg['version']}"

Cheers

Improve fix version logic

Fix version suggested should be free of known vulnerabilities. The current fix version logic breaks for the test case jackson-databind:2.9.9.3

PYTHONPATH=. python vdb/cli.py --search jackson-databind:2.9.9.3
INFO [2020-07-11 15:04:30,343] Vulnerability database loaded from /home/prabhu/.local/share/vdb/data.vdb
+----------------+---------------------------------------------+--------------------+---------------+---------+------------+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Id             | Package                                     | Affected Version   | Fix Version   | CWE     | Severity   |   Score | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+================+=============================================+====================+===============+=========+============+=========+=================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+
| CVE-2020-14061 | com.fasterxml.jackson.core:jackson-databind | 2.9.0-2.9.10.4     | 2.9.10.5      | UNKNOWN | HIGH       |     7.5 | Deserialization of untrusted data in Jackson Databind                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+----------------+---------------------------------------------+--------------------+---------------+---------+------------+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| CVE-2020-14062 | com.fasterxml.jackson.core:jackson-databind | 2.9.0-2.9.10.4     | 2.9.10.5      | UNKNOWN | HIGH       |     7.5 | Deserialization of untrusted data in Jackson Databind                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |

PYTHONPATH=. python vdb/cli.py --search jackson-databind:2.9.10.5
INFO [2020-07-11 15:05:59,557] Vulnerability database loaded from /home/prabhu/.local/share/vdb/data.vdb
+----------------+----------------------------+--------------------+---------------+---------+------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Id             | Package                    | Affected Version   | Fix Version   | CWE     | Severity   |   Score | Description                                                                                                                                                                                                                                                                                                                                                             |
+================+============================+====================+===============+=========+============+=========+=========================================================================================================================================================================================================================================================================================================================================================================+
| CVE-2020-14060 | fasterxml:jackson-databind | 2.0.0-2.9.10.5     |               | CWE-502 | HIGH       |     8.1 | FasterXML jackson-databind 2.x before 2.9.10.5 mishandles the interaction between serialization gadgets and typing, related to oadd.org.apache.xalan.lib.sql.JNDIConnectionPool (aka apache/drill).                                                                                                                                                                     |
+----------------+----------------------------+--------------------+---------------+---------+------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| CVE-2020-14061 | fasterxml:jackson-databind | 2.0.0-2.9.10.5     |               | CWE-502 | HIGH       |     8.1 | FasterXML jackson-databind 2.x before 2.9.10.5 mishandles the interaction between serialization gadgets and typing, related to oracle.jms.AQjmsQueueConnectionFactory, oracle.jms.AQjmsXATopicConnectionFactory, oracle.jms.AQjmsTopicConnectionFactory, oracle.jms.AQjmsXAQueueConnectionFactory, and oracle.jms.AQjmsXAConnectionFactory (aka weblogic/oracle-aqjms). |
+----------------+----------------------------+--------------------+---------------+---------+------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

2.9.10.5 is shown as fix version in case 1. But the tool already knows that the same version is also vulnerable.

Tool should recommend the start year based on the oldest vulnerability

Currently, NVD_START_YEAR is configurable with a default value of 2018. The tool should recommend a start year based on the oldest CVE found. If a CVE belonging to the year 2018 is found then the scan should recommend a re-scan with start year of 2017 (Previous year)

This can be implemented in the analysis module.

Docs should be updated based on the recommendation.

NOTICE: license-scan would require a new argument

cdxgen requires an environment variable called FETCH_LICENSE to fetch license information from public registries for performance reasons. Therefore, dep-scan would soon require a new type license for performing license scanning.

-t license
--type license

--no-license-scan would become silent since that would become the default.

Node.js depscan limitations

With node.js advisories, one thing that is slowly and surely happening is the growth of github advisories. Many new node.js vulnerabilities are only being reported on github advisories. Malicious packages on the other hand are only being reported on npmjs. Scan can pull the information from both npm and github. However, for performance reasons only npm data is used by depscan by default.

We may have to enhance depscan to check both GitHub and npm data sources for javascript at some point. Some related code:

https://github.com/AppThreat/dep-scan/blob/master/depscan/cli.py#L263
https://github.com/AppThreat/dep-scan/blob/8446797f3bf6c478243fb94ef24820a4c31a814d/depscan/lib/audit.py#L4

Self assessment of dep-scan for node.js projects

This will be the first of a series of self-assessment about the readiness of this project. I evaluated dep-scan against npm audit for NodeGoat project.

How dep-scan work

As a reminder to how dep-scan works, dep-scan asks cdxgen to produce the Bill-of-Materials (BoM) for the given project. This information is then used to extract the package list of the form - vendor, name and version. A local vulnerability database from vulndb would then be queried to identify the matches.

Observations

For node.js projects, npm outright wins - not because of its superiority but because of its weakness and inconsistencies in the ecosystem as described below:

Namespace is missing and inconsistent

It is extremely hard to extract the vendor or group information for packages that were created before npm started mandating namespacing / scoped packages. Let's take the example of node-growl version 1.10.0 with a reported CVE of CVE-2017-16042 and npm advisory 146. This project ideally should have been called @tj/node-growl but such a rename action might probably break-the-internet. So such packages are allowed to be installed and package locked without any namespacing.

When we look at the CPE string the cve however, we find that:

cpe:2.3:a:growl_project:growl:*:*:*:*:*:node.js:*:*

Vendor is growl_project and not tj

dep-scan rightly couldn't identify this vulnerability since the vendor information is wrong. npm audit was able to find this by not using any vendor information at all. When I tried to replicate this behaviour for node.js project (making vendor optional) there were far too much false positives. For example, connect node.js package was being matched to Adobe connect and entirely different product.

Missing CVE

Filing for a CVE is apparently optional in the node.js world. Take the advisory 755 for example. It is rated critical yet there are no CVEs attached to it.

Resolution

I am adding npm as a source to vulndb. Initially this will have a new bulk_search method to search the npm registry for advisories for the given package list. It will be slightly performant than the current local db lookup although we might lose the offline access benefits that we can currently get. This approach is still beneficial compared to folks invoking npm audit directly since the information returned by dep-scan will be normalised into grafeas format. Plus with the upcoming multi-project feature people can view all vulnerability information of all the projects in one place and one format.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.