Giter VIP home page Giter VIP logo

gost's Introduction

gost (go-security-tracker)

MIT License

gost builds a local copy of Security Tracker(Redhat/Debian/Ubuntu/Microsoft).
After you register CVEs to watch list, gost notify via E-mail/Slack if there is an update. The pronunciation of gost is the same as the English word "ghost".

Abstract

gost is written in Go, and therefore you can just grab the binary releases and drop it in your $PATH.

gost builds a local copy of Security Tracker (Redhat or Debian or Ubuntu or Microsoft).

A system administrator always monitor Security Tracker. It can be a burden. For example, after the vulnerability is found, we have to wait until the patch comes out. I hope anyone notifies me if there is an update.

Main features

gost has the following features.

  • Build a local copy of Security Tracker
  • A server mode for easy querying
  • Register CVEs to watch list
  • Notify if there is an update (E-Mail or Slack)
  • Monitoring metric can be specified (e.g. CVSS Score, Severity, etc.)

Usage

$ gost help
Security Tracker

Usage:
  gost [command]

Available Commands:
  completion  generate the autocompletion script for the specified shell
  fetch       Fetch the data of the security tracker
  help        Help about any command
  notify      Notifiy update about the specified CVE
  register    Register CVEs to monitor
  server      Start security tracker HTTP server
  version     Show version

Flags:
      --config string       config file (default is $HOME/.gost.yaml)
      --dbpath string       /path/to/sqlite3 or SQL connection string (default "$PWD/gost.sqlite3")
      --dbtype string       Database type to store data in (sqlite3, mysql, postgres or redis supported) (default "sqlite3")
      --debug               debug mode
      --debug-sql           SQL debug mode
  -h, --help                help for gost
      --http-proxy string   http://proxy-url:port (default: empty)
      --log-dir string      /path/to/log (default "/var/log/gost")
      --log-json            output log as JSON
      --log-to-file         output log to file
      --to-email            Send notification via Email
      --to-slack            Send notification via Slack

Use "gost [command] --help" for more information about a command.

Fetch RedHat

Fetch vulnerability infomation updated after 2016-01-01

$ gost fetch redhat 

INFO[07-27|11:13:27] Initialize Database
INFO[07-27|11:13:27] Opening DB.                              db=sqlite3
INFO[07-27|11:13:27] Migrating DB.                            db=sqlite3
INFO[07-27|11:13:27] Fetch the list of CVEs
INFO[07-27|13:59:33] Fetched 6136 CVEs
 6136 / 6136 [=================] 100.00% 8m25s
INFO[07-27|14:08:00] Insert RedHat into DB                    db=sqlite3
 0 / 6136 [--------------------]   0.00%INFO[07-27|14:08:00] Insert 6136 CVEs
 6136 / 6136 [=================] 100.00% 17s

Fetch Debian

Fetch vulnerability infomation

$ gost fetch debian 

INFO[07-27|15:30:49] Initialize Database
INFO[07-27|15:30:49] Opening DB.                              db=sqlite3
INFO[07-27|15:30:49] Migrating DB.                            db=sqlite3
INFO[07-27|15:30:49] Fetched all CVEs from Debian
INFO[07-27|15:31:09] Insert Debian CVEs into DB               db=sqlite3
 21428 / 21428 [================] 100.00% 5s

Fetch Ubuntu

Fetch vulnerability infomation

$ gost fetch ubuntu 

INFO[05-23|06:28:18] Initialize Database 
INFO[05-23|06:28:18] Fetched                                  CVEs=36737
INFO[05-23|06:28:18] Insert Ubuntu into DB                    db=sqlite3
 36737 / 36737 [============================================================================] 100.00% 55s

Fetch Microsoft

Fetch vulnerability infomation

$ gost fetch microsoft

INFO[02-24|02:13:41] Initialize Database 
INFO[02-24|02:13:41] Fetched all CVEs from Microsoft 
INFO[02-24|02:13:43] Insert Microsoft CVEs into DB            db=sqlite3
INFO[02-24|02:13:43] Inserting cves                           cves=11609
11609 / 11609 [----------------] 100.00% 3281 p/s
INFO[02-24|02:13:47] Insert KB Relation                       relations=6016
6016 / 6016 [----------------] 100.00% 5462 p/s

Server mode

$ gost server
[Aug 15 21:38:44]  INFO Opening DB (sqlite3)
[Aug 15 21:38:44]  INFO Migrating DB (sqlite3)
[Aug 15 21:38:44]  INFO Starting HTTP Server...
[Aug 15 21:38:44]  INFO Listening on 127.0.0.1:1325

$ curl http://127.0.0.1:1325/redhat/cves/CVE-2017-1000117 | jq .                                                                                                                     [~]
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1755  100  1755    0     0   243k      0 --:--:-- --:--:-- --:--:--  285k
{
  "ID": 12,
  "ThreatSeverity": "Important",
  "PublicDate": "2017-08-10T00:00:00Z",
  "Bugzilla": {
    "RedhatCVEID": 12,
    "description": "CVE-2017-1000117 git: Command injection via malicious ssh URLs",
    "id": "1480386",
    "url": "https://bugzilla.redhat.com/show_bug.cgi?id=1480386"
  },
  "Cvss": {
    "RedhatCVEID": 0,
    "cvss_base_score": "",
    "cvss_scoring_vector": "",
    "status": ""
  },
  "Cvss3": {
    "RedhatCVEID": 12,
    "cvss3_base_score": "6.3",
    "cvss3_scoring_vector": "CVSS:3.0/AV:N/AC:L/PR:N/UI:R/S:U/C:L/I:L/A:L",
    "status": "draft"
  },
  "Iava": "",
  "Cwe": "",
  "Statement": "",
  "Acknowledgement": "",
  "Mitigation": "",
  "AffectedRelease": [],
  "PackageState": [
    {
      "RedhatCVEID": 12,
      "product_name": "Red Hat Software Collections for Red Hat Enterprise Linux",
      "fix_state": "Affected",
      "package_name": "rh-git29-git",
      "cpe": "cpe:/a:redhat:rhel_software_collections:2"
    },
    {
      "RedhatCVEID": 12,
      "product_name": "Red Hat Enterprise Linux 6",
      "fix_state": "Affected",
      "package_name": "git",
      "cpe": "cpe:/o:redhat:enterprise_linux:6"
    },
    {
      "RedhatCVEID": 12,
      "product_name": "Red Hat Enterprise Linux 7",
      "fix_state": "Affected",
      "package_name": "git",
      "cpe": "cpe:/o:redhat:enterprise_linux:7"
    }
  ],
  "Name": "CVE-2017-1000117",
  "DocumentDistribution": "Copyright © 2016 Red Hat, Inc. All rights reserved.",
  "Details": [
    {
      "RedhatCVEID": 12,
      "Detail": "Details pending"
    },
    {
      "RedhatCVEID": 12,
      "Detail": "A shell command injection flaw related to the handling of \"ssh\" URLs has been discovered in Git. An attacker could use this flaw to execute shell commands with the privileges of the user running the Git client, for example, when performing a \"clone\" action on a malicious repository or a legitimate repository containing a malicious commit."
    }
  ],
  "References": [
    {
      "RedhatCVEID": 12,
      "Reference": "https://lkml.org/lkml/2017/8/10/757\nhttp://blog.recurity-labs.com/2017-08-10/scm-vulns"
    }
  ]
}

Installation

You need to install selector command (fzf or peco).

$ go get github.com/vulsio/gost

Docker Setup, Fetch, Run as Serer and Curl

Fetch Debian, Ubuntu, and RedHat then start as a server mode

$ docker run --rm -i \ 
	 -v $PWD:/gost \
	 -v $PWD:/var/log/gost \
	 vuls/gost fetch debian
   $ docker run --rm -i \ 
	 -v $PWD:/gost \
	 -v $PWD:/var/log/gost \
	 vuls/gost fetch ubuntu
$ docker run --rm -i \
	-v $PWD:/gost \
	-v $PWD:/var/log/gost \
	vuls/gost fetch redhat 
$ ls 
access.log      gost.log        gost.sqlite3

$ docker run --rm -i \
        -v $PWD:/gost \
        -v $PWD:/var/log/gost \
        -p 1325:1325 \
        vuls/gost server --bind=0.0.0.0

HTTP Get to the server on Docker

$ curl http://127.0.0.1:1325/debian/9/pkgs/expat/unfixed-cves | jq "."                                                                                         Fri Jul 27 16:03:15 2018
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   970  100   970    0     0  60308      0 --:--:-- --:--:-- --:--:-- 60625
{
  "CVE-2013-0340": {
    "ID": 8452,
    "CveID": "CVE-2013-0340",
    "Scope": "remote",
    "Description": "expat 2.1.0 and earlier does not properly handle entities expansion unless an application developer uses the XML_SetEntityDeclHandler function, which allows remote attackers to cause a denial of service (resource consumption), send HTTP requests to intranet servers, or read arbitrary files via a crafted XML document, aka an XML External Entity (XXE) issue.  NOTE: it could be argued that because expat already provides the ability to disable external entity expansion, the responsibility for resolving this issue lies with application developers; according to this argument, this entry should be REJECTed, and each affected application would need its own CVE.",
    "Package": [
      {
        "ID": 9829,
        "DebianCVEID": 8452,
        "PackageName": "expat",
        "Release": [
          {
            "ID": 32048,
            "DebianPackageID": 9829,
            "ProductName": "stretch",
            "Status": "open",
            "FixedVersion": "",
            "Urgency": "unimportant",
            "Version": "2.2.0-2+deb9u1"
          }
        ]
      }
    ]
  }
}

Contribute

  1. fork a repository: github.com/vulsio/gost to github.com/you/repo
  2. get original code: go get github.com/vulsio/gost
  3. work on original code
  4. add remote to your repo: git remote add myfork https://github.com/you/repo.git
  5. push your changes: git push myfork
  6. create a new Pull Request

License

MIT

Author

Teppei Fukuda

gost's People

Contributors

dependabot[bot] avatar directionless avatar knqyf263 avatar kotakanbe avatar lance0 avatar mainek00n avatar qbi avatar sadayuki-matsuno avatar segatomo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

gost's Issues

make install failure

$ mkdir -p $GOPATH/src/github.com/knqyf263
$ cd $GOPATH/src/github.com/knqyf263
$ git clone https://github.com/knqyf263/gost.git
$ cd gost
$ make install

../../go-redis/redis/cluster.go:16:2: cannot find package "github.com/go-redis/redis/v7/internal" in any of:
/usr/local/go/src/github.com/go-redis/redis/v7/internal (from $GOROOT)
/root/go/src/github.com/go-redis/redis/v7/internal (from $GOPATH)
../../go-redis/redis/ring.go:14:2: cannot find package "github.com/go-redis/redis/v7/internal/consistenthash" in any of:
/usr/local/go/src/github.com/go-redis/redis/v7/internal/consistenthash (from $GOROOT)
/root/go/src/github.com/go-redis/redis/v7/internal/consistenthash (from $GOPATH)
../../go-redis/redis/cluster.go:17:2: cannot find package "github.com/go-redis/redis/v7/internal/hashtag" in any of:
/usr/local/go/src/github.com/go-redis/redis/v7/internal/hashtag (from $GOROOT)
/root/go/src/github.com/go-redis/redis/v7/internal/hashtag (from $GOPATH)
../../go-redis/redis/cluster.go:18:2: cannot find package "github.com/go-redis/redis/v7/internal/pool" in any of:
/usr/local/go/src/github.com/go-redis/redis/v7/internal/pool (from $GOROOT)
/root/go/src/github.com/go-redis/redis/v7/internal/pool (from $GOPATH)
../../go-redis/redis/cluster.go:19:2: cannot find package "github.com/go-redis/redis/v7/internal/proto" in any of:
/usr/local/go/src/github.com/go-redis/redis/v7/internal/proto (from $GOROOT)
/root/go/src/github.com/go-redis/redis/v7/internal/proto (from $GOPATH)
../../go-redis/redis/command.go:12:2: cannot find package "github.com/go-redis/redis/v7/internal/util" in any of:
/usr/local/go/src/github.com/go-redis/redis/v7/internal/util (from $GOROOT)
/root/go/src/github.com/go-redis/redis/v7/internal/util (from $GOPATH)
../../labstack/echo/middleware/basic_auth.go:8:2: cannot find package "github.com/labstack/echo/v4" in any of:
/usr/local/go/src/github.com/labstack/echo/v4 (from $GOROOT)
/root/go/src/github.com/labstack/echo/v4 (from $GOPATH)
cmd/debian.go:7:2: cannot find package "github.com/spf13/cobra" in any of:
/usr/local/go/src/github.com/spf13/cobra (from $GOROOT)
/root/go/src/github.com/spf13/cobra (from $GOPATH)
fetcher/microsoft.go:12:2: cannot find package "github.com/tealeg/xlsx" in any of:
/usr/local/go/src/github.com/tealeg/xlsx (from $GOROOT)
/root/go/src/github.com/tealeg/xlsx (from $GOPATH)
make: *** [vet] Error 123

Error while unmarshalling RedHat advisories

Hello, I'm currently facing an error when using gost. It started on September 12th. The error is as follows:

json: cannot unmarshal string into Go struct field RedhatEntry.cvss3_score of type float64

The command I run is: gost fetch redhat --after 2016-01-01.

Have you already faced it? Are you aware of it?

Thanks,
Clément.

Why does git clone execution of gost take long time ?

Hello, I got a problem on using gost.

The problem is unexpectedly long execution of git clone in gost.
I investigated it and it seems caused by git clone of https://github.com/aquasecurity/vuln-list.git .
vuln-list repository is very large.

gost/git/git.go

Lines 27 to 37 in 4f1c3df

// CloneOrPull clone/pull aquasecurity/vuln-list
func CloneOrPull(url, repoPath, osDir string) (map[string]struct{}, error) {
exists, err := util.Exists(filepath.Join(repoPath, ".git"))
if err != nil {
return nil, xerrors.Errorf("failed to check if a file exists: %w", err)
}
updatedFiles := map[string]struct{}{}
if exists {
log15.Debug("git pull")
files, err := pull(repoPath, osDir)

updatedFiles, err := git.CloneOrPull(repoURL, dir, redhatDir)

So, I think it's better for gost to use git clone --depth 1 instead of git clone. I think it helps execution time shorter.
I tested git clone --depth 1 https://github.com/aquasecurity/vuln-list.git on my PC, but it takes 10 minites.

Any other suggestion is welcome.

ref: https://github.blog/jp/2021-01-13-get-up-to-speed-with-partial-clone-and-shallow-clone/

Error when running `gost fetch redhat`

gost fetch redhat

INFO[05-13|14:09:59] Initialize Database
2020-05-13T14:09:59.849+0800 DEBUG db path: db/trivy.db
2020-05-13T14:09:59.857+0800 DEBUG remove an existed directory
[==============> ] It will take a while for the first time... 2020-05-13T14:10:03.177+0800 DEBUG error: while accessing https://github.com/aquasecurity/vuln-list.git/info/refs

fatal: HTTP request failed

error in vulnerability DB initialize: error in vulnsrc clone or pull: failed to clone repository: error in git clone: failed to exec: exit status 128

Behavior of CVEID conflicts in Microsoft

When gost fetches microsoft, it brings XML and XLSX (CSV).
https://github.com/knqyf263/gost/blob/19b31ef5494e92a1ae4a17547218dbb96eb2005d/fetcher/microsoft.go#L16-L18

Looking at uniqCve, the data (models.MicrosoftCVE) is assumed to be 1:1 for CVEID.
https://github.com/knqyf263/gost/blob/19b31ef5494e92a1ae4a17547218dbb96eb2005d/db/microsoft.go#L165

In fact, looking at the XML and XLSX data, there are entries for CVE-2017-0045 in both.
The current implementation overwrites the XLSX data when the CVEID conflicts.
https://github.com/knqyf263/gost/blob/master/db/microsoft.go#L532

Actually, the CVEIDs are the same, but there may be spaces and so on, and they may not collide as shown below.
It's also kind of a bug...

127.0.0.1:6379> Keys CVE#CVE-2017-0045*
1) "CVE#CVE-2017-0045 "
2) "CVE#CVE-2017-0045"
127.0.0.1:6379> HKEYS "CVE#CVE-2017-0045 "
1) "Microsoft"
127.0.0.1:6379> HKEYS "CVE#CVE-2017-0045"
1) "Microsoft"
127.0.0.1:6379> HVALS "CVE#CVE-2017-0045 " // XLSX
1) "{\"title\":\"Security Update for Windows DVD Maker\",...
127.0.0.1:6379> HVALS "CVE#CVE-2017-0045" // XML
1) "{\"title\":\"Windows DVD Maker Cross-Site Request Forgery Vulnerability\",..

Now, when CVEIDs conflict, should we overwrite them with XLSX data, as in the current implementation?
As far as I can see, the XML data seems to have more information.

As a solution.
Do you have an idea of which data should be prioritized and which data should be used (merging XML and XLSX information, keeping both)?
If you prioritize one of the data, do you use either XML or XLSX as a rule, or are you thinking of an indicator based on models.MicrosoftCVE.LastUpdateDate, etc.?

Errors during `gost fetch redhat`

I'm encountering some errors which sound like they may be related to #39 and/or #41 as well as vulsio/vulsctl#26 .

When I attempt to update an existing vuln-list with gost fetch redhat --debug, I get the following error:

[root@vuls-210209 install-host]# gost fetch redhat --debug
INFO[02-16|20:07:26] Initialize Database 
2021-02-16T20:07:26.040Z	DEBUG	db path: db/trivy.db
2021-02-16T20:07:26.041Z	DEBUG	git pull
2021-02-16T20:08:33.239Z	DEBUG	From https://github.com/aquasecurity/vuln-list
 * branch            master     -> FETCH_HEAD
fatal: This operation must be run in a work tree

error in vulnerability DB initialize: error in vulnsrc clone or pull: failed to pull repository: error in git pull: failed to exec: exit status 128

I think part of the reason this may have been difficult to reproduce is that you have to start with a clean vuln-list that is cloned from github in order for the problem to exhibit itself. This is because
https://github.com/aquasecurity/trivy/blob/ab8b73e8f8b63a55559b6001cf1910cb13f06a69/pkg/git/git.go#L95 does
a simple clone when the repoPath directory is missing, but https://github.com/aquasecurity/trivy/blob/ab8b73e8f8b63a55559b6001cf1910cb13f06a69/pkg/git/git.go#L181 is hard-coded to pull the master branch when the directory exists.

The default branch for vuln-list is no longer "master", probably due to github's decision to make the default branch "main" a few months ago.

The import of git.go is from https://github.com/knqyf263/gost/blob/3060e463e183010ebd1a9946718c819e914d408e/go.mod#L11 but [email protected] is quite old and pkg/git/git.go is not even present in trivy after somewhere around v0.2.0.

Troubleshooting this further, I turned on git tracing: export GIT_TRACE_PACKET=1; export GIT_TRACE=1; export GIT_CURL_VERBOSE=1 and ran again:

[root@vuls-210209 install-host]# gost fetch redhat --debug
INFO[02-16|20:10:01] Initialize Database 
2021-02-16T20:10:01.476Z	DEBUG	db path: db/trivy.db
2021-02-16T20:10:01.476Z	DEBUG	git pull
2021-02-16T20:11:09.072Z	DEBUG	trace: exec: 'git-pull' 'origin' 'master'
trace: run_command: 'git-pull' 'origin' 'master'
trace: built-in: git 'rev-parse' '--git-dir'
trace: built-in: git 'rev-parse' '--is-bare-repository'
trace: built-in: git 'rev-parse' '--show-toplevel'
trace: built-in: git 'ls-files' '-u'
trace: built-in: git 'symbolic-ref' '-q' 'HEAD'
trace: built-in: git 'config' '--bool' 'branch.main.rebase'
trace: built-in: git 'config' '--bool' 'pull.rebase'
trace: built-in: git 'rev-parse' '-q' '--verify' 'HEAD'
trace: built-in: git 'fetch' '--update-head-ok' 'origin' 'master'
trace: run_command: 'git-remote-https' 'origin' 'https://github.com/aquasecurity/vuln-list.git'
* Couldn't find host github.com in the .netrc file; using defaults
* About to connect() to github.com port 443 (#0)
*   Trying 140.82.113.4...
* Connected to github.com (140.82.113.4) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* 	subject: CN=github.com,O="GitHub, Inc.",L=San Francisco,ST=California,C=US
* 	start date: May 05 00:00:00 2020 GMT
* 	expire date: May 10 12:00:00 2022 GMT
* 	common name: github.com
* 	issuer: CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US
> GET /aquasecurity/vuln-list.git/info/refs?service=git-upload-pack HTTP/1.1
User-Agent: git/1.8.3.1
Host: github.com
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache

< HTTP/1.1 200 OK
< Server: GitHub Babel 2.0
< Content-Type: application/x-git-upload-pack-advertisement
< Content-Security-Policy: default-src 'none'; sandbox
< Transfer-Encoding: chunked
< expires: Fri, 01 Jan 1980 00:00:00 GMT
< pragma: no-cache
< Cache-Control: no-cache, max-age=0, must-revalidate
< Vary: Accept-Encoding
< X-GitHub-Request-Id: D9B2:1025:F3424:134DA3:602C2699
< X-Frame-Options: DENY
< 
* Connection #0 to host github.com left intact
packet:          git< # service=git-upload-pack
packet:          git< 0000
packet:          git< 2a0830236caa2fada2e48afa0866c42de28f1233 HEAD\0multi_ack thin-pack side-band side-band-64k ofs-delta shallow deepen-since deepen-not deepen-relative no-progress include-tag multi_ack_detailed allow-tip-sha1-in-want allow-reachable-sha1-in-want no-done symref=HEAD:refs/heads/main filter object-format=sha1 agent=git/github-g2ff1cad44179
packet:          git< 2a0830236caa2fada2e48afa0866c42de28f1233 refs/heads/main
packet:          git< 37d8ac3154161acd547d306cd536b8024c4cc251 refs/heads/master
packet:          git< e456cb2229b6cd09b5324032d941ad4dceb7f04c refs/pull/3/head
packet:          git< ba87cbc601817584c7ab9436af41140cfefcd22d refs/pull/3/merge
packet:          git< e0b31936e3c9693e0dc1c969c1de3fbf6f0caa6a refs/pull/4/head
packet:          git< a5420eb5296c30b45147e2c9c6025277bd2c3e11 refs/pull/5/head
packet:          git< bdc7d8cb508dc145634dd4cbdf207523341dfaf2 refs/pull/6/head
packet:          git< bfa3a8fe67c4a924e419c2d33bc80b74087f0992 refs/pull/7/head
packet:          git< 636270cf8003c0f9cd950af71d235c1d5cbf994f refs/pull/9/head
packet:          git< 0000
trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all'
trace: exec: 'git' 'rev-list' '--objects' '--stdin' '--not' '--all'
trace: built-in: git 'rev-list' '--objects' '--stdin' '--not' '--all'
From https://github.com/aquasecurity/vuln-list
 * branch            master     -> FETCH_HEAD
trace: run_command: 'gc' '--auto'
trace: exec: 'git' 'gc' '--auto'
trace: built-in: git 'gc' '--auto'
trace: built-in: git 'rev-parse' '-q' '--verify' 'HEAD'
trace: built-in: git 'fmt-merge-msg'
fatal: This operation must be run in a work tree

error in vulnerability DB initialize: error in vulnsrc clone or pull: failed to pull repository: error in git pull: failed to exec: exit status 128

We can see that gost is pulling origin/master: trace: run_command: 'git-pull' 'origin' 'master'
We can also see that gost should be pulling origin/main:

[root@vuls-210209 install-host]# git --git-dir vuln-list/.git branch -a
* main
  remotes/origin/HEAD -> origin/main
  remotes/origin/main

We also see that the automatic git gc --auto doesn't inherit the git-dir or work-tree parameters (for some reason) because we see the fatal: This operation must be run in a work tree error.

I am able to get gost fetch redhat to work as expected if I modify pullByOSCommand() from:

func pullByOSCommand(repoPath string) ([]string, error) {
        gitDir := filepath.Join(repoPath, ".git")
        commandArgs := []string{"--git-dir", gitDir, "--work-tree", repoPath}

        revParseCmd := []string{"rev-parse", "HEAD"}
        output, err := utils.Exec("git", append(commandArgs, revParseCmd...))
        if err != nil {
                return nil, xerrors.Errorf("error in git rev-parse: %w", err)
        }
        commitHash := strings.TrimSpace(output)

        pullCmd := []string{"pull", "origin", "master"}
        _, err = utils.Exec("git", append(commandArgs, pullCmd...))
        if err != nil {
                return nil, xerrors.Errorf("error in git pull: %w", err)
        }

        diffCmd := []string{"diff", commitHash, "HEAD", "--name-only"}
        output, err = utils.Exec("git", append(commandArgs, diffCmd...))
        if err != nil {
                return nil, xerrors.Errorf("error in git diff: %w", err)
        }
        updatedFiles := strings.Split(strings.TrimSpace(output), "\n")
        return updatedFiles, nil
}

to:

func pullByOSCommand(repoPath string) ([]string, error) {
        gitDir := filepath.Join(repoPath, ".git")
        commandArgs := []string{"--git-dir", gitDir, "--work-tree", repoPath}

        revParseCmd := []string{"rev-parse", "HEAD"}
        output, err := utils.Exec("git", append(commandArgs, revParseCmd...))
        if err != nil {
                return nil, xerrors.Errorf("error in git rev-parse: %w", err)
        }
        commitHash := strings.TrimSpace(output)

        // RDA ...
        os.Chdir(repoPath);
        currDir, err := os.Getwd()
        log.Logger.Debug("Current Working Directory: ", currDir)

        //pullCmd := []string{"pull", "origin", "master"}
        //_, err = utils.Exec("git", append(commandArgs, pullCmd...))
        pullCmd := []string{"pull", "origin"}
        _, err = utils.Exec("git", pullCmd)
        // ... RDA
        if err != nil {
                return nil, xerrors.Errorf("error in git pull: %w", err)
        }

        // RDA ...
        os.Chdir("..");
        currDir, err = os.Getwd()
        log.Logger.Debug("Current Working Directory: ", currDir)
        // ... RDA

        diffCmd := []string{"diff", commitHash, "HEAD", "--name-only"}
        output, err = utils.Exec("git", append(commandArgs, diffCmd...))
        if err != nil {
                return nil, xerrors.Errorf("error in git diff: %w", err)
        }
        updatedFiles := strings.Split(strings.TrimSpace(output), "\n")
        return updatedFiles, nil
}

(My changes are enclosed within "RDA" comments to help point them out)

The update now appears to work:

[root@vuls-210209 install-host]# gost fetch redhat --debug
INFO[02-16|20:19:48] Initialize Database 
2021-02-16T20:19:48.682Z	DEBUG	db path: db/trivy.db
2021-02-16T20:19:48.683Z	DEBUG	git pull
2021-02-16T20:19:48.691Z	DEBUG	Current Working Directory: /home/vagrant/vulsctl/install-host/vuln-list
2021-02-16T20:19:49.506Z	DEBUG	Current Working Directory: /home/vagrant/vulsctl/install-host
DBUG[02-16|20:19:52] Failed to fetch the CVE details.         err=nil
DBUG[02-16|20:19:52] Red Hat updated files: %d                LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
INFO[02-16|20:20:01] Opening DB.                              db=sqlite3
INFO[02-16|20:20:01] Migrating DB.                            db=sqlite3
INFO[02-16|20:20:01] Insert RedHat into DB                    db=sqlite3
 0 / 23692 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------]   0.00%INFO[02-16|20:20:01] Insert 23692 CVEs 
 23692 / 23692 [====================================================================================================================================================================================================================================================] 100.00% 3m41s
[root@vuls-210209 install-host]# 

The output seems to indicate that all 23692 CVEs are added to the database every time because if I run gost fetch redhat --debug again, I see all 23692 CVEs inserted again:

[root@vuls-210209 install-host]# gost fetch redhat --debug
INFO[02-16|20:25:53] Initialize Database 
2021-02-16T20:25:53.422Z	DEBUG	db path: db/trivy.db
2021-02-16T20:25:53.422Z	DEBUG	git pull
2021-02-16T20:25:53.427Z	DEBUG	Current Working Directory: /home/vagrant/vulsctl/install-host/vuln-list
2021-02-16T20:25:54.111Z	DEBUG	Current Working Directory: /home/vagrant/vulsctl/install-host
DBUG[02-16|20:25:55] Failed to fetch the CVE details.         err=nil
DBUG[02-16|20:25:55] Red Hat updated files: %d                LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
INFO[02-16|20:25:57] Opening DB.                              db=sqlite3
INFO[02-16|20:25:57] Migrating DB.                            db=sqlite3
INFO[02-16|20:25:57] Insert RedHat into DB                    db=sqlite3
 0 / 23692 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------]   0.00%INFO[02-16|20:25:57] Insert 23692 CVEs 
 23692 / 23692 [====================================================================================================================================================================================================================================================] 100.00% 3m44s
[root@vuls-210209 install-host]#

Sorry for the long issue posting. I'm not sure if this is the proper project to post this issue to because the issues are tied to the current version of gost, but only because it is using [email protected].

BTW, https://github.com/knqyf263/gost/blob/3060e463e183010ebd1a9946718c819e914d408e/fetcher/redhat.go#L29 always prints the Failed to fetch the CVE details. err=nil error when debug is enabled.

If I can get some feedback on how I should proceed, I'll put together a PR to address as many of the items in this issue as I can.

Thanks!

Failed to fetch the CVE details. err: json: cannot unmarshal object into Go struct field RedhatCVEJSON.mitigation of type string

$ gost fetch redhatapi --dbpath=/home/vulsuser/gost_api.sqlite3 --debug
INFO[06-13|12:48:58] Initialize Database
INFO[06-13|12:48:58] Fetch the list of CVEs
INFO[06-13|12:51:30] Fetched 28436 CVEs
516 / 28436 [->] 1.81% 90 p/s
528 / 28436 [->
] 1.86% 90 p/s

548 / 28436 [-->________________________________________________________________________________________________________] 1.93% 90 p/s
28436 / 28436 [-------------------------------------------------------------------------------------------------------] 100.00% 18 p/s
Failed to fetch the CVE details. err: json: cannot unmarshal object into Go struct field RedhatCVEJSON.mitigation of type string

$ gost version
gost v0.4.4 d498da9

Release file name is wrong

I've gone into releases and found that there is not gost archive but instead it is called go-cve-dictionary.

MySQL DB Error

Hi

I got the following error:

$ gost fetch --dbtype mysql --dbpath "vuls:test@tcp(localhost:3306)/gost" redhat --after 2016-01-01
INFO[01-30|15:28:31] Initialize Database
INFO[01-30|15:28:31] Opening DB. db=mysql
INFO[01-30|15:28:31] Migrating DB. db=mysql
INFO[01-30|15:28:31] Fetch the list of CVEs
INFO[01-30|15:31:03] Fetched 7199 CVEs
7199 / 7199 [=======================================================================================================================================] 100.00% 21m39s
INFO[01-30|15:52:44] Insert RedHat into DB db=mysql
0 / 7199 [-------------------------------------------------------------------------------------------------------------------------------------------------] 0.00%INFO[01-30|15:52:44] Insert 7199 CVEs
1847 / 7199 [==================================>----------------------------------------------------------------------------------------------------] 25.66% 00m19sEROR[01-30|15:52:51] Failed to insert. dbpath=vuls:test@tcp(localhost:3306)/gost err="Failed to insert. cve: CVE-2017-15136, err: Error 1366: Incorrect string value: '\xC5\x99 (Re...' for column 'acknowledgement' at row 1"
Failed to insert. cve: CVE-2017-15136, err: Error 1366: Incorrect string value: '\xC5\x99 (Re...' for column 'acknowledgement' at row 1

Please help.

/Theo

error in vulnerability DB initialize: error in vulnsrc clone or pull: failed to pull repository: error in git rev-parse: failed to exec: exit status 128

docker run --rm -i -v $PWD:/vuls -v $PWD/gost-log:/var/log/gost vuls/gost fetch redhat
t=2020-04-09T00:43:43+0000 lvl=info msg="Initialize Database"
2020-04-09T00:43:43.619Z	DEBUG	db path: db/trivy.db
2020-04-09T00:43:43.622Z	DEBUG	git pull
2020-04-09T00:43:43.652Z	DEBUG	fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

error in vulnerability DB initialize: error in vulnsrc clone or pull: failed to pull repository: error in git rev-parse: failed to exec: exit status 128

security in docker : use an application user instead of root

This issue is an enhancement request. Can the image run as a non-root user?

Currently, the image runs as the root user.

$ docker run --rm -it --entrypoint=/bin/sh vuls/gost:latest
/vuls # id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)
/vuls #

If the user is a non-root user, we would be able to follow best practices of running as a non-root user (e.g. [1] [2]) and add proper restrictions in Pod Security Policies.

Can't get a package

Hello!

I tried to get your code, but i have the error:

 go get github.com/knqyf263/gost
package github.com/labstack/echo/engine/standard: cannot find package "github.com/labstack/echo/engine/standard" in any of:
	/usr/local/go/src/github.com/labstack/echo/engine/standard (from $GOROOT)
	/home/alexander/go/src/github.com/labstack/echo/engine/standard (from $GOPAT 

github.com/labstack/echo/engine/standard - 404 Not found.

Could your check it please?

Thanks

Failed to initialize vulnerability DB . err: error in vulnsrc clone or pull: failed to pull repository: error in git pull: failed to exec: exit status 128

$ gost fetch redhat --debug
DBUG[06-12|20:52:22] git pull
DBUG[06-12|20:52:23] From https://github.com/aquasecurity/vuln-list

  • branch main -> FETCH_HEAD
    hint: Pulling without specifying how to reconcile divergent branches is
    hint: discouraged. You can squelch this message by running one of the following
    hint: commands sometime before your next pull:
    hint:
    hint: git config pull.rebase false # merge (the default strategy)
    hint: git config pull.rebase true # rebase
    hint: git config pull.ff only # fast-forward only
    hint:
    hint: You can replace "git config" with "git config --global" to set a default
    hint: preference for all repositories. You can also pass --rebase, --no-rebase,
    hint: or --ff-only on the command line to override the configured default per
    hint: invocation.
    fatal: refusing to merge unrelated histories

Failed to initialize vulnerability DB . err: error in vulnsrc clone or pull: failed to pull repository: error in git pull: failed to exec: exit status 128

Failed to update Red hat CVE

Since 2019-06-04, I've been having issues getting "gost fetch redhat" to work. Today I tried two things: 1 - a complete reinstall/rebuild of the software just in case and 2. a whole new fresh install on my laptop while connected to a different internet source (I then took my laptop to a third internet connection to verify). It breaks the same every time. But only with Red Hat. Debian works fine every time.

$ gost fetch debian && gost fetch redhat --after 2016-01-01
INFO[06-17|15:48:34] Initialize Database
INFO[06-17|15:48:34] Opening DB.                              db=sqlite3
INFO[06-17|15:48:34] Migrating DB.                            db=sqlite3
INFO[06-17|15:48:46] Fetched all CVEs from Debian
INFO[06-17|15:48:48] Fetched                                  CVEs=2759
INFO[06-17|15:48:48] Insert Debian CVEs into DB               db=sqlite3
 23983 / 23983 [==================================] 100.00% 9s
INFO[06-17|15:48:59] Initialize Database
INFO[06-17|15:48:59] Opening DB.                              db=sqlite3
INFO[06-17|15:48:59] Migrating DB.                            db=sqlite3
INFO[06-17|15:48:59] Fetch the list of CVEs
INFO[06-17|15:50:31] Fetched 7956 CVEs
[==================================] 7956 / 7956 [==================================]] 100.00% 1h40m58s
EROR[06-17|17:32:32] Failed to fetch the CVE details.         err="Failed to fetch cve data from RedHat. err: [HTTP error. errs: [], status code: 504, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2018-14658.json HTTP error. errs: [], status code: 504, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2018-2810.json]"
Failed to fetch cve data from RedHat. err: [HTTP error. errs: [], status code: 504, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2018-14658.json HTTP error. errs: [], status code: 504, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2018-2810.json]

Which CVE it fails on seems to change nearly every time. I can always download the failed CVE via wget though. If I do it month by month, it seems to work. But that is really tedious and harder to script. However, if I do more than about 2 months at a time or if I do it as above (which is what the vuls documentation suggests) then it always fails.

Suggestions? Thanks!

GOST fails to download microsoft CVEs

What did you do? (required. The issue will be closed when not provided.)

gost fetch microsoft --apikey blahblahblah

What did you expect to happen?

MS CVE data to be downloaded

What happened instead?

Retrieval failed as the fetcher is trying to grab data for months in the future e.g Nov-2022 & Dec-2022 - these pages are blank causing the process to fail.

  • Current Output

Failed to fetch cve data from Microsoft. targetURL: https://api.msrc.microsoft.com/cvrf/v2.0/document/2022-Nov: HTTP error. url: https://api.msrc.microsoft.com/cvrf/v2.0/document/2022-Nov, err: %!w([]error=[])

Please re-run the command using -debug and provide the output below.

Steps to reproduce the behaviour

gost fetch microsoft --apikey abcdefgh1234

Configuration (MUST fill this out):

  • Go version (go version):

go version go1.19.2 linux/amd64

  • Go environment (go env):

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/amadeus/.cache/go-build"
GOENV="/home/amadeus/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/amadeus/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/amadeus/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.19.2"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.19.2/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.2"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build4246865751=/tmp/go-build -gno-record-gcc-switches"

  • gost environment:

Hash : 3a15e2a

To check the commit hash of HEAD
$ gost version

gost v0.4.1 3a15e2a

or

$ cd $GOPATH/src/github.com/vulsio/gost
$ git rev-parse --short HEAD

  • command:

Failed to insert Ubuntu CVE data.

What did you do? (required. The issue will be closed when not provided.)

gost fetch ubuntu --dbpath postgres://USER:PASSWORD@POSTGRESHOST:5432/gost?sslmode=disable --dbtype postgres

Note: Replaced values for USER:PASSWORD@POSTGRESHOST.

What did you expect to happen?

gost would insert all of the data into the PostgreSQL database.

What happened instead?

Failed to insert all of the data into the PostgreSQL database.

  • Current Output
err: Failed to insert Ubuntu CVE data. err: Failed to insert. err: ERROR: value too long for type character varying(255) (SQLSTATE 22001)

Please re-run the command using -debug and provide the output below.

/gost # gost fetch ubuntu --dbpath postgres://USER:PASSWORD@POSTGRESHOST:5432/gost?sslmode=disable --dbtype postgres --debug
DBUG[01-04|10:01:34] git pull
DBUG[01-04|10:01:36] Ubuntu updated files: 38776
INFO[01-04|10:01:46] Initialize Database
INFO[01-04|10:01:46] Fetched                                  CVEs=38601
INFO[01-04|10:01:46] Insert Ubuntu into DB                    db=postgres
26745 / 38601 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------->__________________________________________________________________________] 69.29% 449 p/sFailed to insert. dbpath: postgres://USER:PASSWORD@POSTGRESHOST:5432/gost?sslmode=disable, err: Failed to insert Ubuntu CVE data. err: Failed to insert. err: ERROR: value too long for type character varying(255) (SQLSTATE 22001)
/gost #

Steps to reproduce the behaviour

  1. I used this docker version:
$ docker --version
Docker version 20.10.11, build dea9396
  1. Copy/paste these 2 files into an empty directory. Put the initdb.sh into a postgres-scripts subdirectory. Click the right triangle to expand and view the contents.
`docker-compose.yaml`
version: "3.9"
services:
  postgres:
    image: postgres:13.5-alpine
    environment:
      PGDATA: /var/lib/postgresql/data/pgdata
      POSTGRES_PASSWORD: password
    volumes:
      - type: bind
        source: postgres-scripts
        target: /docker-entrypoint-initdb.d

  gost:
    depends_on:
      - postgres
    image: vuls/gost@sha256:5177e78c9bd29eac8901bd109e455473ec81beb5796f951f8787e6d3fb55c8ae
    entrypoint:
      - /bin/sh
      - -c
    command:
      - "gost fetch ubuntu --dbpath postgres://gostuser:password@postgres:5432/gostdb?sslmode=disable --dbtype postgres"
`initdb.sh`
#!/bin/bash
set -e

psql -v ON_ERROR_STOP=1 --username postgres <<-EOSQL
    CREATE USER gostuser PASSWORD 'password';
    CREATE DATABASE gostdb;
    GRANT ALL PRIVILEGES ON DATABASE gostdb TO gostuser;
EOSQL

So your directory structure looks like this:

.
├── docker-compose.yaml
└── postgres-scripts
    └── initdb.sh

1 directory, 2 files

And make the script executable:

chmod +x postgres-scripts/initdb.sh
  1. In the same directory as docker-compose.yaml, run:
docker compose up -d

If you're on Linux and don't have the compose command, you'll need to add the compose plugin. Below I added instructions on how to add it.

How to add the compose plugin on Linux.
mkdir -p ~/.docker/cli-plugins
curl -sSL https://github.com/docker/compose/releases/download/v2.2.2/docker-compose-linux-$(uname -m) -o ~/.docker/cli-plugins/docker-compose
chmod +x ~/.docker/cli-plugins/docker-compose
  1. Observe the error message by running the command below. It may take a minute or so for output to show:
docker compose logs -f gost

Configuration (MUST fill this out):

  • gost environment:

I used the current latest image with this sha:
https://hub.docker.com/layers/vuls/gost/latest/images/sha256-5177e78c9bd29eac8901bd109e455473ec81beb5796f951f8787e6d3fb55c8ae?context=explore

EDIT: Add instructions for adding the compose plugin for Linux + making the initdb.sh script executable.

CVEs from 2016 throw 403 error when running `gost fetch redhat --after 2016-01-01`

Hello,

I am attempting to use gost on a server (CentOS 7.3.1611) I run and am encountering an issue with retrieving 2016 CVEs hosted on access.redhat.com.

Here is the command and output I am attempting to run:

3:26:50 user@server [~/go/src/github.com/knqyf263/gost]$ gost fetch redhat --after 2016-01-01 
INFO[08-29|13:26:55] Initialize Database 
INFO[08-29|13:26:55] Opening DB.                              db=sqlite3
INFO[08-29|13:26:55] Migrating DB.                            db=sqlite3
INFO[08-29|13:26:55] Fetch the list of CVEs 
INFO[08-29|13:27:57] Fetched 6310 CVEs 
 6310 / 6310 [=====================================================================================================] 100.00% 15m46s
EROR[08-29|13:44:12] Failed to fetch the CVE details.         err="Failed to fetch cve data from RedHat. err: [HTTP error. errs: [], status code: 403, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-3728.json HTTP error. errs: [], status code: 403, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-4913.json HTTP error. errs: [], status code: 403, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-4581.json HTTP error. 
... [many more examples snipped for brevity] ...
https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-1026.json HTTP error. errs: [], status code: 403, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-1027.json HTTP error. errs: [], status code: 403, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-1028.json HTTP error. errs: [], status code: 403, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-1029.json HTTP error. errs: [], status code: 403, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-1030.json HTTP error. errs: [], status code: 403, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-1031.json HTTP error. errs: [], status code: 403, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-1032.json HTTP error. errs: [], status code: 403, url: https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-1033.json]

The script seems to run, taking its time gathering the CVEs, and then once it reaches 100%, everything unravels and it spits out this error. Looking through the script output, it is only CVE URLs starting with https://access.redhat.com/labs/securitydataapi/cve/CVE-2016-* that are affected. Changing the script to --after 2017-01-01 does result in the script working correctly:

14:02:25 user@server [~/go/src/github.com/knqyf263/gost]$ gost fetch redhat --after 2017-01-01   
INFO[08-29|14:09:16] Initialize Database 
INFO[08-29|14:09:16] Opening DB.                              db=sqlite3
INFO[08-29|14:09:16] Migrating DB.                            db=sqlite3
INFO[08-29|14:09:16] Fetch the list of CVEs 
INFO[08-29|14:09:52] Fetched 3945 CVEs 
 3945 / 3945 [=====================================================================================================] 100.00% 22m23s
INFO[08-29|14:32:16] Insert RedHat into DB                    db=sqlite3
 0 / 3945 [---------------------------------------------------------------------------------------------------------------]   0.00%INFO[08-29|14:32:16] Insert 3945 CVEs 
 3945 / 3945 [========================================================================================================] 100.00% 35s

But means that I do not get the older CVEs, which would be helpful in my case.

I am not sure if I am being rate-limited by Red Hat here, but going to these URLs in my browser does seem to work and does not resolve in a 403. I am not sure if anyone has come across this error before, but this is my first time using this script and I did not see any similar issues reported in the past.

Attempting to backfill 2016 by fetching --after 2017-01-01 and then go back and fetch --after 2016-01-01 does not seem to work either, as it tries to fetch the same number (6310) as the initial run

Failed to clone repository: error in git clone: failed to exec: exit status 128

What did you do? (required. The issue will be closed when not provided.)

gost fetch redhat

What did you expect to happen?

Insert RedHat into DB Successfully.

What happened instead?

Failed to initialize vulnerability DB . err: error in vulnsrc clone or pull: failed to clone repository: error in git clone: failed to exec: exit status 128

Please re-run the command using -debug and provide the output below.
fatal: 无法访问 'https://github.com/aquasecurity/vuln-list.git/':Empty reply from server

Steps to reproduce the behaviour

Configuration (MUST fill this out):

  • Go version (go version):
    go version go1.16 linux/amd64

  • gost environment:
    Hash : dbf4931

To check the commit hash of HEAD
$ gost version
gost v0.4.0 7ad032a

make fails

"make install" fails

git clone https://github.com/knqyf263/gost.git
Cloning into 'gost'...
remote: Enumerating objects: 16, done.
remote: Counting objects: 100% (16/16), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 414 (delta 4), reused 2 (delta 1), pack-reused 398
Receiving objects: 100% (414/414), 1.32 MiB | 1.80 MiB/s, done.
Resolving deltas: 100% (234/234), done.

cd gost/

make install

GO111MODULE=off go get -u golang.org/x/lint/golint

cd /root/go/src/golang.org/x/tools; git pull --ff-only

fatal: Not possible to fast-forward, aborting.
package golang.org/x/tools/go/ast/astutil: exit status 128
make: *** [lint] Error 1

Red Hat updated files: %d" LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil

I'm using this image: https://hub.docker.com/layers/vuls/gost/latest/images/sha256-1843c878d8c28c989ac41e4e48eba42005134d2bd9628abd27eaa1ac9a3f3fd3?context=explore

# docker inspect d23d2f8abc19 | jq -r '.[] | .RepoDigests'
[
  "vuls/gost@sha256:1843c878d8c28c989ac41e4e48eba42005134d2bd9628abd27eaa1ac9a3f3fd3"
]
]# tail -f /data/vuls/logs/gost-log/gost.log 
t=2020-07-14T12:36:38+0000 lvl=info msg="Initialize Database"
t=2020-07-14T12:38:21+0000 lvl=dbug msg="Failed to fetch the CVE details." err=nil
t=2020-07-14T12:38:21+0000 lvl=dbug msg="Red Hat updated files: %d" LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
t=2020-07-14T12:38:24+0000 lvl=info msg="Opening DB." db=sqlite3
t=2020-07-14T12:38:24+0000 lvl=info msg="Migrating DB." db=sqlite3
t=2020-07-14T12:38:24+0000 lvl=info msg="Insert RedHat into DB" db=sqlite3
t=2020-07-14T12:38:24+0000 lvl=info msg="Insert 22278 CVEs"

fetch ubuntu not working

What did you do? (required. The issue will be closed when not provided.)

docker run vuls/gost fetch ubuntu

What did you expect to happen?

Successfully updating the database

What happened instead?

  • Current Output
INFO[03-21|15:04:41] Fetched                                  CVEs=39577
INFO[03-21|15:04:41] Insert Ubuntu into DB                    db=sqlite3
35850 / 39577 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->___________________] 90.58% 59 p/sFailed to insert. dbpath: /gost/gost.sqlite3, err: Failed to insert Ubuntu CVE data. err: Failed to insert. err: too many SQL variables; too many SQL variables; no valid transaction

Please re-run the command using -debug and provide the output below.

t=2022-03-21T15:27:44+0000 lvl=dbug msg="remove an existed directory"
t=2022-03-21T15:28:31+0000 lvl=dbug msg="Ubuntu updated files: 39764"
t=2022-03-21T15:28:37+0000 lvl=info msg="Initialize Database"
t=2022-03-21T15:28:38+0000 lvl=info msg=Fetched CVEs=39577
t=2022-03-21T15:28:38+0000 lvl=info msg="Insert Ubuntu into DB" db=sqlite3
705 / 39577 [->________________________________________________________________________] 1.78% ? p/s1740 / 39577 [--->_____________________________________________________________________] 4.40% ? p/s2760 / 39577 [----->___________________________________________________________________] 6.97% ? p/s3615 / 39577 [------>_______________________________________________________________] 9.13% 4848 p/s4275 / 39577 [------->_____________________________________________________________] 10.80% 4848 p/s4920 / 39577 [-------->____________________________________________________________] 12.43% 4848 p/s5310 / 39577 [--------->___________________________________________________________] 13.42% 4717 p/s5790 / 39577 [---------->__________________________________________________________] 14.63% 4717 p/s6420 / 39577 [----------->_________________________________________________________] 16.22% 4717 p/s6990 / 39577 [------------>________________________________________________________] 17.66% 4594 p/s7425 / 39577 [------------>________________________________________________________] 18.76% 4594 p/s7785 / 39577 [------------->_______________________________________________________] 19.67% 4594 p/s8175 / 39577 [-------------->______________________________________________________] 20.66% 4425 p/s8610 / 39577 [--------------->_____________________________________________________] 21.76% 4425 p/s8970 / 39577 [--------------->_____________________________________________________] 22.66% 4425 p/s9345 / 39577 [---------------->____________________________________________________] 23.61% 4265 p/s9690 / 39577 [---------------->____________________________________________________] 24.48% 4265 p/s10020 / 39577 [----------------->__________________________________________________] 25.32% 4265 p/s10320 / 39577 [----------------->__________________________________________________] 26.08% 4095 p/s10680 / 39577 [------------------>_________________________________________________] 26.99% 4095 p/s11025 / 39577 [------------------>_________________________________________________] 27.86% 4095 p/s11445 / 39577 [------------------->________________________________________________] 28.92% 3952 p/s11565 / 39577 [------------------->________________________________________________] 29.22% 3952 p/s11865 / 39577 [-------------------->_______________________________________________] 29.98% 3952 p/s12195 / 39577 [-------------------->_______________________________________________] 30.81% 3777 p/s12450 / 39577 [--------------------->______________________________________________] 31.46% 3777 p/s12690 / 39577 [--------------------->______________________________________________] 32.06% 3777 p/s12780 / 39577 [--------------------->______________________________________________] 32.29% 3596 p/s12990 / 39577 [---------------------->_____________________________________________] 32.82% 3596 p/s13215 / 39577 [---------------------->_____________________________________________] 33.39% 3596 p/s13530 / 39577 [----------------------->____________________________________________] 34.19% 3445 p/s13785 / 39577 [----------------------->____________________________________________] 34.83% 3445 p/s14145 / 39577 [------------------------>___________________________________________] 35.74% 3445 p/s14535 / 39577 [------------------------>___________________________________________] 36.73% 3331 p/s14760 / 39577 [------------------------->__________________________________________] 37.29% 3331 p/s15060 / 39577 [------------------------->__________________________________________] 38.05% 3331 p/s15270 / 39577 [-------------------------->_________________________________________] 38.58% 3195 p/s15540 / 39577 [-------------------------->_________________________________________] 39.27% 3195 p/s15825 / 39577 [--------------------------->________________________________________] 39.99% 3195 p/s15990 / 39577 [--------------------------->________________________________________] 40.40% 3066 p/s16095 / 39577 [--------------------------->________________________________________] 40.67% 3066 p/s16380 / 39577 [---------------------------->_______________________________________] 41.39% 3066 p/s16725 / 39577 [---------------------------->_______________________________________] 42.26% 2948 p/s17085 / 39577 [----------------------------->______________________________________] 43.17% 2948 p/s17370 / 39577 [----------------------------->______________________________________] 43.89% 2948 p/s17685 / 39577 [------------------------------>_____________________________________] 44.69% 2861 p/s18015 / 39577 [------------------------------>_____________________________________] 45.52% 2861 p/s18345 / 39577 [------------------------------->____________________________________] 46.35% 2861 p/s18600 / 39577 [------------------------------->____________________________________] 47.00% 2774 p/s18735 / 39577 [-------------------------------->___________________________________] 47.34% 2774 p/s19065 / 39577 [-------------------------------->___________________________________] 48.17% 2774 p/s19425 / 39577 [--------------------------------->__________________________________] 49.08% 2684 p/s19650 / 39577 [--------------------------------->__________________________________] 49.65% 2684 p/s19755 / 39577 [--------------------------------->__________________________________] 49.92% 2684 p/s19965 / 39577 [---------------------------------->_________________________________] 50.45% 2569 p/s20040 / 39577 [---------------------------------->_________________________________] 50.64% 2569 p/s20265 / 39577 [---------------------------------->_________________________________] 51.20% 2569 p/s20520 / 39577 [----------------------------------->________________________________] 51.85% 2463 p/s20835 / 39577 [----------------------------------->________________________________] 52.64% 2463 p/s21090 / 39577 [------------------------------------>_______________________________] 53.29% 2463 p/s21435 / 39577 [------------------------------------>_______________________________] 54.16% 2402 p/s21600 / 39577 [------------------------------------->______________________________] 54.58% 2402 p/s21915 / 39577 [------------------------------------->______________________________] 55.37% 2402 p/s22185 / 39577 [-------------------------------------->_____________________________] 56.06% 2328 p/s22305 / 39577 [-------------------------------------->_____________________________] 56.36% 2328 p/s22560 / 39577 [-------------------------------------->_____________________________] 57.00% 2328 p/s22905 / 39577 [--------------------------------------->____________________________] 57.87% 2255 p/s23250 / 39577 [--------------------------------------->____________________________] 58.75% 2255 p/s23595 / 39577 [---------------------------------------->___________________________] 59.62% 2255 p/s23835 / 39577 [---------------------------------------->___________________________] 60.22% 2210 p/s24045 / 39577 [----------------------------------------->__________________________] 60.75% 2210 p/s24255 / 39577 [----------------------------------------->__________________________] 61.29% 2210 p/s24525 / 39577 [------------------------------------------>_________________________] 61.97% 2141 p/s24690 / 39577 [------------------------------------------>_________________________] 62.38% 2141 p/s25065 / 39577 [------------------------------------------->________________________] 63.33% 2141 p/s25320 / 39577 [------------------------------------------->________________________] 63.98% 2089 p/s25545 / 39577 [------------------------------------------->________________________] 64.55% 2089 p/s25830 / 39577 [-------------------------------------------->_______________________] 65.27% 2089 p/s26205 / 39577 [--------------------------------------------->______________________] 66.21% 2049 p/s26520 / 39577 [--------------------------------------------->______________________] 67.01% 2049 p/s26820 / 39577 [---------------------------------------------->_____________________] 67.77% 2049 p/s27030 / 39577 [---------------------------------------------->_____________________] 68.30% 2006 p/s27480 / 39577 [----------------------------------------------->____________________] 69.43% 2006 p/s27795 / 39577 [----------------------------------------------->____________________] 70.23% 2006 p/s28260 / 39577 [------------------------------------------------>___________________] 71.41% 2008 p/s28425 / 39577 [------------------------------------------------>___________________] 71.82% 2008 p/s28605 / 39577 [------------------------------------------------->__________________] 72.28% 2008 p/s28890 / 39577 [------------------------------------------------->__________________] 73.00% 1947 p/s29250 / 39577 [-------------------------------------------------->_________________] 73.91% 1947 p/s29595 / 39577 [-------------------------------------------------->_________________] 74.78% 1947 p/s29925 / 39577 [--------------------------------------------------->________________] 75.61% 1932 p/s30360 / 39577 [---------------------------------------------------->_______________] 76.71% 1932 p/s30645 / 39577 [---------------------------------------------------->_______________] 77.43% 1932 p/s30825 / 39577 [---------------------------------------------------->_______________] 77.89% 1905 p/s31110 / 39577 [----------------------------------------------------->______________] 78.61% 1905 p/s31170 / 39577 [----------------------------------------------------->______________] 78.76% 1905 p/s31260 / 39577 [----------------------------------------------------->______________] 78.99% 1828 p/s31380 / 39577 [----------------------------------------------------->______________] 79.29% 1828 p/s31560 / 39577 [------------------------------------------------------>_____________] 79.74% 1828 p/s31725 / 39577 [------------------------------------------------------>_____________] 80.16% 1761 p/s32040 / 39577 [------------------------------------------------------->____________] 80.96% 1761 p/s32430 / 39577 [------------------------------------------------------->____________] 81.94% 1761 p/s32700 / 39577 [-------------------------------------------------------->___________] 82.62% 1752 p/s32820 / 39577 [-------------------------------------------------------->___________] 82.93% 1752 p/s33015 / 39577 [-------------------------------------------------------->___________] 83.42% 1752 p/s33075 / 39577 [-------------------------------------------------------->___________] 83.57% 1679 p/s33135 / 39577 [-------------------------------------------------------->___________] 83.72% 1679 p/s33225 / 39577 [--------------------------------------------------------->__________] 83.95% 1679 p/s33420 / 39577 [--------------------------------------------------------->__________] 84.44% 1608 p/s33465 / 39577 [--------------------------------------------------------->__________] 84.56% 1608 p/s33510 / 39577 [--------------------------------------------------------->__________] 84.67% 1608 p/s33585 / 39577 [--------------------------------------------------------->__________] 84.86% 1522 p/s33720 / 39577 [--------------------------------------------------------->__________] 85.20% 1522 p/s33900 / 39577 [---------------------------------------------------------->_________] 85.66% 1522 p/s34035 / 39577 [---------------------------------------------------------->_________] 86.00% 1472 p/s34125 / 39577 [---------------------------------------------------------->_________] 86.22% 1472 p/s34350 / 39577 [----------------------------------------------------------->________] 86.79% 1472 p/s34485 / 39577 [----------------------------------------------------------->________] 87.13% 1426 p/s34545 / 39577 [----------------------------------------------------------->________] 87.29% 1426 p/s34695 / 39577 [----------------------------------------------------------->________] 87.66% 1426 p/s34905 / 39577 [----------------------------------------------------------->________] 88.20% 1379 p/s35100 / 39577 [------------------------------------------------------------>_______] 88.69% 1379 p/s35505 / 39577 [------------------------------------------------------------->______] 89.71% 1379 p/s35850 / 39577 [------------------------------------------------------------->______] 90.58% 1391 p/sFailed to insert. dbpath: /gost/gost.sqlite3, err: Failed to insert Ubuntu CVE data. err: Failed to insert. err: too many SQL variables; too many SQL variables; no valid transaction

Steps to reproduce the behaviour

docker run vuls/gost fetch ubuntu

Configuration (MUST fill this out):

I'm using your latest docker image vuls/gost

   docker image ls
REPOSITORY                                           TAG                   IMAGE ID       CREATED        SIZE
vuls/gost                                            latest                bb2031785acd   13 days ago    48.4MB

too many SQL variables

What did you do? (required. The issue will be closed when not provided.)

fetch ubuntu

What did you expect to happen?

finish with no error

What happened instead?

Same issue #113

$ gost fetch ubuntu --debug
DBUG[11-26|08:12:45] git pull
DBUG[11-26|08:12:56] Ubuntu updated files: 42508
INFO[11-26|08:13:17] Initialize Database
INFO[11-26|08:13:17] Fetched                                  CVEs=42312
INFO[11-26|08:13:17] Insert Ubuntu into DB                    db=sqlite3
34530 / 42312 [----------------------------------------------------------------------->________________] 81.61% 2522 p/sFailed to insert. dbpath: /home/wk/gost.sqlite3, err: Failed to insert Ubuntu CVE data. err: Failed to insert. err: too many SQL variables; too many SQL variables

And docker run vuls/gost fetch ubuntu shows same error

I can insert data into redis

$ gost fetch ubuntu  --dbtype redis --dbpath "redis://localhost/0"
INFO[11-26|08:28:39] Initialize Database
INFO[11-26|08:28:40] Fetched                                  CVEs=42312
INFO[11-26|08:28:40] Insert Ubuntu into DB                    db=redis
42312 / 42312 [-----------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 12148 p/s

Configuration (MUST fill this out):

  • Go version (go version):
$ go version
go version go1.19.3 linux/amd64
  • Go environment (go env):
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/wk/.cache/go-build"
GOENV="/home/wk/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/wk/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/wk/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.3"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build671889390=/tmp/go-build -gno-record-gcc-switches"
  • gost environment:
$ gost version
gost v0.4.1 3a15e2a

No active release

Can you please build the latest release?
All current releases are in archive mode and only archives with sources available, but not with a binary file.

should have comment or be unexported

Hello,

"golint" will output the following logs:

/home/karas/go/src/github.com/knqyf263/gost/config/config.go:3:6: exported type Config should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/config/config.go:9:6: exported type RedhatWatchCve should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/debian.go:13:1: exported method RDBDriver.GetDebian should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/debian.go:39:1: exported method RDBDriver.InsertDebian should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/debian.go:79:1: exported function ConvertDebian should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/debian.go:125:1: exported method RDBDriver.GetUnfixedCvesDebian should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/redhat.go:15:1: exported method RDBDriver.GetAfterTimeRedhat should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/redhat.go:29:1: exported method RDBDriver.GetRedhat should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/redhat.go:47:1: exported method RDBDriver.GetRedhatMulti should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/redhat.go:55:1: exported method RDBDriver.GetUnfixedCvesRedhat should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/redhat.go:115:1: exported method RDBDriver.InsertRedhat should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/redhat.go:182:1: exported function ConvertRedhat should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/db/redhat.go:236:1: exported function ClearIDRedhat should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/fetcher/debian.go:11:1: comment on exported function RetrieveDebianCveDetails should be of the form "RetrieveDebianCveDetails ..."
/home/karas/go/src/github.com/knqyf263/gost/fetcher/redhat.go:22:1: exported function FetchRedHatVulnList should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/debian.go:3:6: exported type DebianJSON should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/debian.go:5:6: exported type DebianCveMap should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/debian.go:7:6: exported type DebianCveJSON should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/debian.go:14:6: exported type DebianReleaseJSON should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/debian.go:21:6: exported type DebianCVE should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/debian.go:29:6: exported type DebianPackage should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/debian.go:36:6: exported type DebianRelease should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:10:6: exported type RedhatEntry should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:24:6: exported type RedhatCVEJSON should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:46:6: exported type RedhatCVEJSONAffectedReleaseArray should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:50:6: exported type RedhatCVEJSONAffectedReleaseObject should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:54:6: exported type RedhatCVEJSONPackageStateArray should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:58:6: exported type RedhatCVEJSONPackageStateObject should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:62:6: exported type RedhatCVE should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:85:1: exported method RedhatCVE.GetDetail should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:93:1: exported method RedhatCVE.GetPackages should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:107:6: exported type RedhatDetail should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:112:6: exported type RedhatReference should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:117:6: exported type RedhatBugzilla should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:125:6: exported type RedhatCvss should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:132:6: exported type RedhatCvss3 should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:139:6: exported type RedhatAffectedRelease should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/models/redhat.go:148:6: exported type RedhatPackageState should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/notifier/slack.go:23:1: exported function SendSlack should have comment or be unexported
/home/karas/go/src/github.com/knqyf263/gost/util/redhat.go:12:1: exported function DiffRedhat should have comment or be unexported

Thanks.

Failed to update microsoft CVE data

I fetched the Microsoft's CVE to update gost.sqlite3 DB.
gost fetch microsoft --apikey=xxxxxxxxxxxxx
It said following error.

EROR[06-12|16:20:49] Failed to insert. dbpath=/something/gost.sqlite3 err="Failed to insert Microsoft CVE data. err: Failed to delete old records. err: FOREIGN KEY constraint failed"
Failed to insert Microsoft CVE data. err: Failed to delete old records. err: FOREIGN KEY constraint failed

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.