Comments (7)
My current workaround is indeed to
- run nist-data-mirror
- remove offending file nvdcve-1.1-2013.json.gz
- manual download "wget https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2013.json.gz"
from nist-data-mirror.
Additional info:
We are using nist-data-mirror 1.6.0 since June 2022, but this issue has started last week and has hit us now three times in a row in the last days.
from nist-data-mirror.
I did some further checks and the checksum on the unzipped 2013 is not matching:
$ cat nvdcve-1.1-2013.meta
lastModifiedDate:2022-09-30T03:01:57-04:00
size:45431615
zipSize:2449884
gzSize:2449748
sha256:4DF6DAF5270EEA9F79D316297EBCC70352A0BFF40F49A8715A7E6C621B55CBAA
$ sha256sum nvdcve-1.1-2013.json
95a2e870cc5865c11fcc4b63e98d633ca249899011244f74bcd2254127e39f62 nvdcve-1.1-2013.json
$ stat -c %s nvdcve-1.1-2013.json.gz
2449748
So I now extracted the manual downloaded file and now the sha256 matches and nist-data-mirror is not complaining about 2013:
...
Downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2012.meta
Download succeeded nvdcve-1.1-2012.meta
File 2012 is valid.
Downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2013.meta
Download succeeded nvdcve-1.1-2013.meta
File 2013 is valid.
Downloading https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2014.meta
Download succeeded nvdcve-1.1-2014.meta
File 2014 is valid.
...
I still need to find out how both the nvdcve-1.1-2013.json.gz
and the nvdcve-1.1-2013.json
got corrupted on my system.
from nist-data-mirror.
We suffer the same issue with nist-data-mirror
1.5.0 but with the nvdcve
files for first 2020 and then 2014. A check that only valid files are replaced would be great.
from nist-data-mirror.
After deleting both (extracted) json and json.gz file, the nist-data-mirror is working as expected.
I did not find out, what has caused the corruption of the 2013 file.
from nist-data-mirror.
We suffer the same issue with 1.5.3, and now upgraded to docker image nvd-mirror
1.6.0. But I expect the problem to show again in a few days. This is quite a problem for our CI system, as the maven dependency-check plugin caches the files itself but does not handle corrupt .json.gz files very well, i.e. it does not try to fetch again when I've already fixed the nvd-mirror manually...
I don't think the issue is solved. I moved my other comments to #39
from nist-data-mirror.
I agree, I closed my issue, but the core problem is not solved.
As we are doing the distribution of the downloaded files with another tool, we changed our distribution tool to check the integrity of the cve files (again) and block the upload in case of a file corruption.
from nist-data-mirror.
Related Issues (20)
- Docker container doesn't mirror all files HOT 1
- k8s deprecation 'networking.k8s.io/v1beta1' in v1.22
- NistDataMirror.java has hardcoded Windows file separators HOT 2
- Download and uncompression are slower than expected. HOT 7
- Unable to build with IBM Java 8 HOT 2
- GitHub action - Unable to access jarfile nist-data-mirror.jar HOT 1
- Cron running twice
- Some downloads fail with ZipException HOT 2
- Unable to run nist-data-mirror on OpenShift HOT 3
- release 1.4.0 compiled against newer jdk HOT 2
- Docker Image 1.4.0 is throwing "Permission denied" issues HOT 1
- Allow to add volume and configure DNS in HELM chart. HOT 3
- HELM Chart - Allow http proxy values to be supplied through values.yaml. HOT 1
- Don't start the update process if one is already running HOT 1
- Introduce a logger HOT 1
- json 1.0 feeds no longer supported? HOT 5
- No .jar file for release 1.5.0 in releases HOT 1
- Using proxy with account
- java.io.EOFException: Unexpected end of ZLIB input stream HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nist-data-mirror.