Comments (2)
This would be useful indeed but there may be some gremlins along the way as this is the general processing flow:
0. Run pre-scan plugins
- Run all scanners on one file. Cache the results on disk in a temp JSON file, one such file for each file scanned (this is stored on disk as soon as you have more than 10,000 files scanned)
- Run the "process_codebase" step of each scanner one after the other that iterate on the whole codebase and can amend the results from step 1.
- Run post-scan plugins one after the other that iterate on the whole codebase and can amend the results from step 1 and step 2.
- Run output plugins to create JSON/SBOM outputs
We could technically output some JSON lines early, or some pseudo JSON-like lines, and output the headers and summaries at the end, but there no way to make sure the results are correct and would not be amended after the initial scanner step.
Now, what is your concern? getting some details of the scans early for diagnostic purpose?
Or stream processing large scans?
from scancode-toolkit.
Now, what is your concern? getting some details of the scans early for diagnostic purpose? Or stream processing large scans?
Mostly the latter. Of course, if there are other ways to speed up scancode that would also help a lot.
Another thing is that right now it isn't too easy to extract results for a single file from the larger scancode file. So what I might actually be after is the temp JSON file from the cache (one per file).
from scancode-toolkit.
Related Issues (20)
- Update/add Oracle Java/JDK and other JDK licenses HOT 1
- Consider adding ESA licenses and other misc. licenses.
- scancode does not properly handle multiple explicit paths
- Consider new license for Yara rules
- False positive `gpl-1.0-plus` license reported for `node-forge` HOT 2
- Any SPDX-Identifier for this license? HOT 3
- treeware license (option 2)
- Use proper apk package type for Alpine HOT 1
- Incorrect VCS URL for alpine package
- Software for Open Models License (SOM) to be added HOT 2
- Add new ChilliCream License 1.0 HOT 2
- Incorrect copyright detection
- v32.1.0_p3 requests python 3.10 when python 3.11 already installed HOT 4
- AGPL false possitive: in codec2 the file version.h.in HOT 4
- Surprising false positives of mit_or_gpl-3.0_17.RULE HOT 2
- Ripper source code misclassified as lgpl-2.1 HOT 2
- Rule to detect agpl-3-0-plus_or_commercial actually triggers on AGPL-3.0-plus without commercial remarks HOT 1
- Detect "All rights reserved." on subsequent line HOT 1
- Wrong LGPL version detection
- Extend copyright output with raw data option
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scancode-toolkit.