bbva / patton Goto Github PK

View Code? Open in Web Editor NEW

96.0 8.0 18.0 3.11 MB

The clever vulnerability dependency finder

Home Page: https://owasp.org/www-project-patton/

License: Apache License 2.0

Makefile 6.37% Shell 12.10% Gherkin 50.76% Go 21.67% Python 8.56% Dockerfile 0.54%

security security-tools cve cpe vulnerability-identification

patton's Introduction

Patton - The vulnerability knowledge database

Patton is a set of tools for helping admins and security auditors to search for vulnerabilities in software components and computer systems.

Patton contains, at this moment, the tools needed to analyze vulnerabilities on Ubuntu, Debian and RHEL 8 or newer systems and we are currently working on increasing the targets to include Python dependencies.

Patton is an OWASP project: https://owasp.org/www-project-patton/

Description

Searching for vulnerabilities is a really hard task. Vulnerabilities databases have they own format and they index vulnerabilities in a way that makes difficult to match with the package names you install in your systems or with the libraries your software depends on.

Patton pre-cooks a vulnerabilities database, this database is really small to ease downloading and contains only the needed data to allow the match of the vulnerable component.

Patton contains a series of small epecialized tools that are able of making the match between the vulnerability and the software package or library and offers a single interface to invoke all the tools in a consistent way.

Setup

Prerequisites

zstd
Docker

Install

wget https://raw.githubusercontent.com/BBVA/patton/develop/bin/patton
sudo install patton /usr/local/bin

Usage

$ patton --help
Usage: patton [OPTION]... [PATTERN]
Try '/usr/local/bin/patton -h|--help' for more information
  -h, --help           display this help text and exit
  -V, --version        display version information and exit
  -d, --database-file  path to database file
  -t, --search-type    type of search to execute: product|pkg_debian|pkg_ubuntu|pkg_rhel|fulltext
  -s, --search-subtype for search-type:(debian|ubuntu), sets the suite
    e.g.: buster, potato, fossa, xenial, precise, trusty...
  -v, --pkg-version    cpe version when searching by cpe
  -n, --pkg-name       path to database file
  -w, --pkg-vendor     path to database file

Debian

patton -t pkg_debian < /var/lib/dpkg/status

Ubuntu

patton -t pkg_ubuntu < /var/lib/dpkg/status

Red Hat Enterprise Linux

patton -t pkg_rhel

NOTE: Have to be run on RHEL 8 or newer

Fulltext search

wget https://github.com/BBVA/patton/releases/download/latest/patton.db.zst
patton -d patton.db.zst -t fulltext openssl

TODO

rename scanners/debian to scanners/debian-debsecan consistently

patton's People

Contributors

Stargazers

Watchers

Forkers

pombredanne esedeerre bberastegui jenarvaezg mahakoala iht threatinteltest directionless freegit9527 marciopocebon triplekill l1kw1d laozhudetui cbk914 be-secure sumodgeorge jingqiangliu

patton's Issues

Web service Paton v2

Build the new web service for Patton v2

Patton v2 - Find new logo

Current logo has poorly resolution. This issue is to propose new logo.

Requirement: Logo license must be valid!

install delta DB cve and cpe from xml

Right now the only way to populate the DB is make a full wipe and load up.

For now it's fine since we don't store useful information, but this will age quickly.

Pacman (Arch Linux) package manager dependency parser

Add support for the pacman package manager used by Arch Linux distributions.

As with the others package manager there is a need to try to map as closely as possible from package name to CPE prior to search for vulnerabilities.

Try to use the same approach; look for installed packages and run the CVE search against them or get a list of the desired packages to search for in stdin.

Failure on database update - NIST XML feed retirement

I am not able to update the database due to NIST XML feed retirement.

The following error occured when I try to update the database:

$ patton-server -C postgres://patton:[email protected]:5432/patton update-db
    > official-cpe-dictionary_v2.3.xml.gz: 6.60MB [00:01, 5.74MB/s]                                                                                                                                                                                                                                                                                                                                                                        
    > nvdcve-2.0-recent.xml.gz: 28.7kB [00:00, 3.80MB/s]                                                                                                                                                                                                                                                                                                                                                                                   
Traceback (most recent call last):
  File "/tmp/venv/bin/patton-server", line 8, in <module>
    sys.exit(main())
  File "/tmp/venv/lib/python3.7/site-packages/patton_server/command_line.py", line 129, in main
    update_db(**config)
  File "/tmp/venv/lib/python3.7/site-packages/patton_server/dal/updater.py", line 86, in update_db
    web_hook))
  File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/tmp/venv/lib/python3.7/site-packages/patton_server/dal/updater.py", line 59, in _update_db
    only_last_cve=True)
  File "/tmp/venv/lib/python3.7/site-packages/patton_server/dal/loader.py", line 97, in download_assets
    shutil.copyfileobj(f_in, f_out)
  File "/tmp/venv/lib/python3.7/shutil.py", line 79, in copyfileobj
    buf = fsrc.read(length)
  File "/usr/lib/python3.7/gzip.py", line 276, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.7/_compression.py", line 68, in readinto
    data = self.read(len(byte_view))
  File "/usr/lib/python3.7/gzip.py", line 463, in read
    if not self._read_gzip_header():
  File "/usr/lib/python3.7/gzip.py", line 411, in _read_gzip_header
    raise OSError('Not a gzipped file (%r)' % magic)
OSError: Not a gzipped file (b'<!')

Golang raw dependency parser

Add support for searching for vulnerabilities in Golang projects.

First we have to resolve the final version of all the components on which the the project depends on. After that we need to map the library name and version to a CPE and finally search for vulnerabilities on those CPEs.

No Repository Introduction in README.md

There is no repository introduction to know what this project is about

To Do:

Add Repository Introduction

YUM/DNF package manager dependency parser

Add support for the yum/dnf package manager used by RPM based distributions.

As with the others package manager there is a need to try to map as closely as possible from package name to CPE prior to search for vulnerabilities.

Try to use the same approach; look for installed packages and run the CVE search against them or get a list of the desired packages to search for in stdin.

Javascript raw dependency parser

Add support for searching for vulnerabilities in Javascript projects.

Patton v2 - Snapshot for first launch of database

Automatic create a snapshot in CI/CD system and update it in Github. This is useful for first launch of project.

Also, add this improvement in the load scripts

Consistency issues in documentation / file naming

Hi!

There is a small lack of consistency in the documentation: https://patton.readthedocs.io/deployment.html#deploy-with-internal-database

The docker-compose file is referred in the documentation as docker-compose.yml while the file in the repo is docker-compose.yaml.

Whatever you prefer to change :)

Patton 2 - APT (debian, ubuntu...) raw dependency parser

Windows package manager dependency parser

Add support for the package manager used by Windows.

As with the others package manager there is a need to try to map as closely as possible from package name to CPE prior to search for vulnerabilities.

Try to use the same approach; look for installed packages and run the CVE search against them or get a list of the desired packages to search for in stdin.

Of course, in this case, we need a Windows script

Patton v2 - Fill OWASP Wiki

Fill OWASP Wiki page

Patton v2 - CI/CD tasks

Add project owners EOM

@engapa I can't add assignees, for obvious reasons :)

Patton v2 - CLI

Create a new CLI that doesn't use the REST API, instead, connect to the database and performs the queries

Patton v2: Support RHEL

Estimate effort to support latest RHEL and previous RHEL

APK (Alpine) package manager dependency parser

Add support for the APK package manager used by Alpine distributions.

As with the others package manager there is a need to try to map as closely as possible from package name to CPE prior to search for vulnerabilities.

Try to use the same approach; look for installed packages and run the CVE search against them or get a list of the desired packeges to search for in stdin.

Tests

As today there're very few tests so it is going to be really difficult to work in the project if we don't have a goot test suite

patton-server init-db Killed

Despite what I do when I run the below command, eventually I see Killed. Consequently prodvuln_view is never created.

$ patton-server -v init-db
[INFO] Ensuring database is ready
[INFO] Donwloading CPE and CVE files
    > nvdcve-2.0-2007.xml.gz: 2.04MB [00:00, 4.27MB/s]
    > nvdcve-2.0-2018.xml.gz: 12.2MB [00:00, 13.2MB/s]
    > nvdcve-2.0-2017.xml.gz: 9.15MB [00:00, 12.5MB/s]
    > nvdcve-2.0-2003.xml.gz: 439kB [00:00, 1.41MB/s]
    > nvdcve-2.0-2013.xml.gz: 2.78MB [00:00, 5.45MB/s]
    > nvdcve-2.0-2016.xml.gz: 3.19MB [00:00, 5.72MB/s]
    > nvdcve-2.0-2015.xml.gz: 2.42MB [00:00, 4.40MB/s]
    > official-cpe-dictionary_v2.3.xml.gz: 3.46MB [00:00, 5.79MB/s]
    > nvdcve-2.0-2012.xml.gz: 2.65MB [00:00, 4.58MB/s]
    > nvdcve-2.0-2006.xml.gz: 2.12MB [00:00, 4.31MB/s]
    > nvdcve-2.0-2019.xml.gz: 3.01MB [00:00, 5.30MB/s]
    > nvdcve-2.0-2002.xml.gz: 1.50MB [00:00, 3.60MB/s]
    > nvdcve-2.0-2005.xml.gz: 1.35MB [00:00, 2.84MB/s]
    > nvdcve-2.0-2010.xml.gz: 2.94MB [00:00, 5.45MB/s]
    > nvdcve-2.0-2004.xml.gz: 869kB [00:00, 2.23MB/s]
    > nvdcve-2.0-2009.xml.gz: 2.17MB [00:00, 4.52MB/s]
    > nvdcve-2.0-2014.xml.gz: 2.86MB [00:00, 5.05MB/s]
    > nvdcve-2.0-2011.xml.gz: 6.35MB [00:00, 9.42MB/s]
    > nvdcve-2.0-2008.xml.gz: 2.28MB [00:00, 4.44MB/s]
[INFO] Start loading CPE
[INFO] cpe_loaded
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2002.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2003.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2004.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2005.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2006.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2007.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2008.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2009.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2010.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2011.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2012.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2013.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2014.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2015.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2016.xml
[INFO] loading CVEs from file: /tmp/tmpj_culfcj/nvdcve-2.0-2017.xml
Killed

Error updating the DB in kali

I had problems updating the data base in a Kali 2017.3 machine using a Docker machine as Postgres DB:

The value of the version variable of my machine was:

I solved this patching the line 46 of /usr/local/lib/python3.6/dist-packages/asyncpg/serverversion.py
I changed:
from: [int(p) for p in parts][:3]
to: versions = [int(''.join(s for s in str(p) if s.isdigit())) for p in parts[:3]]

Patton v2 - Update README Github File

Split database and binaries in different repositories

At this moment Patton binaries and database the creation script (Makefile) live in the same repository but both require its own release. It would be better to split the repository so Patton binaries have their own repository separated of the database creation script.

Create a new repository for the database, move Makefile and related github actions.

{
    "method": "auto",
    "source": "auto",
    "libraries" : [
        {
            "library": "Microsoft IIS",
            "version": "7"
        }
    ]
}

Doing this results in an error:

HTTP/1.1 400 Bad Request
Connection: keep-alive
Keep-Alive: 5
access-control-allow-origin: *
Content-Length: 1592
Content-Type: application/json

{
    "message": {
        "cursor": {
            "binary_types": null, 
            "rowcount": -1, 
            "string_types": null, 
            "arraysize": 1, 
            "query": "(Select 'microsoft iis:7' as t0, v.cve as cpe_0, v.cpe as cpe_0, v.cvss as cvss_0 from prodvuln_view as v where to_tsvector('english', v.cpe) @@ to_tsquery('(microsoft iis:* & 7:*)') order by v.cvss desc limit 10) ", 
            "description"
...
        }, 
        "args": [
            "syntax error in tsquery: \"(microsoft iis:* & 7:*)\"\n"
        ], 
        "pgerror": "ERROR:  syntax error in tsquery: \"(microsoft iis:* & 7:*)\"\n", 
        "pgcode": "42601"
    }

But this works:

{
    "method": "auto",
    "source": "auto",
    "libraries" : [
        {
            "library": "IIS",
            "version": "7"
        }
    ]
}

When performing a search, it returns inaccurate CPEs.

For example:

{
    "method": "auto",
    "source": "auto",
    "libraries" : [
        {
            "library": "mod_ssl",
            "version": "2.1"
        }
    ]
}

Returns:

        "cpes": [
            "cpe:/a:mod_ssl:mod_ssl:2.1.8", 
            "cpe:/h:cisco:content_switching_module_with_ssl:2.1", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.5", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.6", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.3", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.4", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.7", 
            "cpe:/h:cisco:content_switching_module_with_ssl:2.1", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.2"
        ]

These would be inaccurate no?

            "cpe:/h:cisco:content_switching_module_with_ssl:2.1", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.5", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.6", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.3", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.4", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.7", 
            "cpe:/h:cisco:content_switching_module_with_ssl:2.1", 
            "cpe:/a:cisco:cisco_content_switching_module_ssl:2.1.2"

Is there any way to specify a more accurate parameter when you know values with greater accuracy?

Patton v2 - scaffolding and finish new version

Put all the new script of Patton v2 together and finish the new scaffolding and setups scripts

Vulnerability feeds are now available only as JSON

Per the timeline provided in the XML feed retirement announcement we will be implementing Phase 3 of the XML retirement plan on Wednesday October 16th.

* This is one week later than originally planned.
* The feeds will no longer be accessible to any users or automated processes. 
* Users will need to begin ingesting the JSON 1.1 Feeds for vulnerability information.

old location as XML:
https://static.nvd.nist.gov/feeds/xml/cve/2.0/nvdcve-2.0-2019.xml.gz

new location as JSON:
https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2019.json.gz

bbva / patton Goto Github PK

patton's Introduction

Patton - The vulnerability knowledge database

Description

Setup

Prerequisites

Install

Usage

Debian

Ubuntu

Red Hat Enterprise Linux

Fulltext search

TODO

patton's People

Contributors

Stargazers

Watchers

Forkers

patton's Issues

Recommend Projects

Recommend Topics

Recommend Org