pentestfail / ta-github Goto Github PK

View Code? Open in Web Editor NEW

2.0 3.0 6.0 4.03 MB

Provides modular inputs & framework to ingest formatted or raw data from Github APIs.

License: MIT License

Python 99.75% Ruby 0.01% CSS 0.08% HTML 0.07% C 0.11%

ta-github's Introduction

Github Addon

Provides modular inputs & framework to ingest JSON data from Github APIs.

App Setup:

Store Github credentials to be used on the "Configuration" page and "Account" tab, then click "Add" to add new credentials. Since an account is specific to a GitHub instance, you must specify the server for the account (will be used for all inputs leveraging the account credentials). Specify if account is for a Github Enterprise instance via the checkbox (uses different API path and likely will fail otherwise).
- App supports authentication via username & password or account "personal access tokens"
- DOES NOT support 2-factor tokens
- GitHub Personal Access Tokens
Add inputs via the add-on's "Inputs" page, clicking "Create New Input", then clicking an input type you wish to create and entering the repository "owner" and "repository" from which the input will collect data.

Name of user or Github organization which owns the repository. Example: https://api.github.com/[OWNER]/repo

Name of repository. Example: https://api.github.com/owner/[REPOSITORY]

APIs supported:

Repository Stats (https://developer.github.com/v3/repos/statistics/#get-contributors-list-with-additions-deletions-and-commit-counts)
Repository Commits (https://developer.github.com/v3/repos/commits/#list-commits-on-a-repository)
Repository Issues (https://developer.github.com/v3/issues/#list-issues-for-a-repository)

Release Notes:

###v1.0.1 Maintenance & Updates for Github.com API

Adds input parameter and logic to handle differences beteween Github's public and enterprise API paths.
Additional logic for API "still processing" (202 response) for stats API
Improved logging and error handling.

###v1.0.0 Initial release. Documentation will be included in future releases.

Submit issues or requests via Github:

TA-Github: https://github.com/pentestfail/TA-Github

ta-github's People

Contributors

Stargazers

Watchers

Forkers

ispg101 superyarick yarick dbreddyai ffeldmann edro15

ta-github's Issues

Initial Configuration page hangs during initial setup

Steps to reproduce - Install App:Github Addon on Splunk Enterprise 7

Commits are not pulling

Retrieval of commits is failing with error
GlobalConfigError: Config Not Found for Input
Log file
base_modinput.py

Typo in the field names

TA-Github/default/props.conf:TIMESTAMP_FIELDS = committers.date

It should be TA-Github/default/props.conf:TIMESTAMP_FIELDS = committer.date

Add additional configuration parameters

Hi Evan,

Just downloaded your add-on and it works great. Would be nice though if you add an ability to provide additional parameters in the inputs configurations. In my case, I need to pull only closed issues into the splunk however github's default api pulls open issues only.

To work around, I had to change input_module_github_api_repos_issues and to add parameter['state'] = "closed", see below

    # Create API request parameters    
    auth = base64.b64encode(git_username + ":" + git_password).decode("ascii")
    header =  {'Authorization': 'Basic {}'.format(auth)}
    parameter = {}
    parameter['since'] = last_status
    parameter['per_page'] = git_pagesize
    parameter['state'] = "closed"
    method = 'GET'

Commits call returns a http 409

Please could you tell me why we are seeing this error?

2019-01-16 14:24:24,007 ERROR pid=20699 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events. Traceback (most recent call last): File "/opt/splunk/etc/apps/TA-Github/bin/ta_github/modinput_wrapper/base_modinput.py", line 127, in stream_events self.collect_events(ew) File "/opt/splunk/etc/apps/TA-Github/bin/github_api_repos_commits.py", line 72, in collect_events input_module.collect_events(self, ew) File "/opt/splunk/etc/apps/TA-Github/bin/input_module_github_api_repos_commits.py", line 135, in collect_events raise error HTTPError: 409 Client Error: Conflict for url: https://git.xxx.xxx/api/v3/repos/Foo/repo/commits?since=2018-01-25&per_page=50

2019-01-16 14:24:15,844 ERROR pid=19785 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events. Traceback (most recent call last): File "/opt/splunk/etc/apps/TA-Github/bin/ta_github/modinput_wrapper/base_modinput.py", line 127, in stream_events self.collect_events(ew) File "/opt/splunk/etc/apps/TA-Github/bin/github_api_repos_commits.py", line 72, in collect_events input_module.collect_events(self, ew) File "/opt/splunk/etc/apps/TA-Github/bin/input_module_github_api_repos_commits.py", line 135, in collect_events raise error HTTPError: 409 Client Error: Conflict for url: https://git.xxx.xxx/api/v3/repos/BAR/prj/commits?per_page=50&since=2018-01-25

Accessing a local github appliance

How do I configure the api to access a local github repo. that is accessible as a https URL?

Specifically where do I put the TLS cert?

And should I configure the switch to use the v3 api?