jfalken / github_commit_crawler Goto Github PK

There is no input validation on the config entry ui page.

This should be added so people don't accidently submit blank entries.

Currently, if you wish to add/remove keywords from the search, you need to manually edit the keywords.py file, then restart the process.

The GUI should be updated to allow one to edit this list, and then auto restart the process.

Keep getting this error in the ghcc proc logs (see screenshot)

Error: (<class 'pymongo.errors.ConnectionFailure'>, ConnectionFailure('[Errno 111] Connection refused',), <traceback object at 0x2cdc5f0>)

Everything installs fine and I'm able to successfully run one scan. Then subsequent scans run (verified through Crawler Logs), but the scan data does not populate via the ui. I've been able to reproduce this error several times. Thanks.

After initially running the docker container and running a ghcc scan, the data from subsequent scans is not updated in the UI. I've experienced this problem numerous times. The first scan always works fine, but subsequent scans often does not update in the UI. Reviewing the Crawler Logs confirms the subsequent scans actually ran with new findings, but nothing is updated in the UI (not even the date of the last scan). I've even waited weeks between scans to ensure new commits occurred, but still the UI does not update.

For instances where we know our dev's handles, but they're not in any official org.

The docker container was initially created for ease-of-use for trial/demo purposes. After discussions, some people want to use their actual deployment, instead of using just the base python scripts.

This will work fine as is, however the current crawler process runs once. When complete, supervisord will try to restart it, but the behavior is undefined as it depends upon whether you still have GitHub API credits for that hour and how many retries have occurred.

This feature request is to add the ability for the Docker container version of this script to re-try / re-cycle the crawler every X hours, as specified by the user.

When I try to kickoff a new scan, I get the following error

$ python ghcc.py
[Err]: User has too much swag

Please advise how to move forward.