Giter VIP home page Giter VIP logo

vinifera's Introduction

Vinifera - Monitor Internal Leaks on Github

Github Monitoring Tool ๐Ÿค–

Vinifera Logo


We have been using Vinifera in production since Dec 2019 and has helped us prevent security incidents. Vinifera started out as an internal project to ensure Security hygiene of our public contributions and monitor potential leaks on Github. We believe this will help other companies to strengthen their security hygiene when it comes to public sources like Github.

Stats

What is Vinifera?

Vinifera allows Companies/Organizations to monitor public assets to find references to internal code leaks and potential breaches.

Sometimes developers might leak internal code and credentials by accident. Vinifera aims to help companies detect those breaches in due time and respond to the incident.

How does it work?

Vinifera monitors developers belonging to the organization, monitors and scans public contributions to look for potential violations and breach of internal/secret/proprietary code by looking for references defined.

  • Vinifera works by syncing org users.
  • For each user, all the public assets are registered (if not already tracked).
  • Each asset (Repo, Gist) is then scanned for any differences.
  • Each difference is then scanned, stored, and reported (if contains any leaks)

Vinifera Workflow

What does the name mean?

During the development and inception of the tool, the Security team consumed lots of grapes, so we named it after the fruit we love :)

Vinifera is inspired from the Bionomial name of Grapes

Setup and Usage.

Requirements

Vinifera requires the installation of the following tools:

  1. PostgreSQL
  2. Redis
  3. Docker
  4. Ruby (Install via rbenv/rvm )

Setup

Github Access Token

To scan your organization members, Vinifera requires a token with the ability to read Organization members.

Generate a new token https://github.com/settings/tokens/new with no special scope

You would want to use the token of an admin user (with no special scope), since the admin can list all users of an organization.

https://docs.github.com/en/rest/reference/orgs#list-organization-members

If the authenticated user is also a member of this organization then both concealed and public members will be returned.

Docker

  • Create .docker_env with needed variables
GITHUB_ACCESS_TOKEN=<REDACTED>
VINIFERA_ORG_NAME=<Your_org_name>

RAILS_MAX_THREADS=60 # This also controls DB pool

RAILS_MASTER_KEY=<REDACTED>


SLACK_UPDATES_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE>
SLACK_TARGETS_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE>
SLACK_USER_TRACKING_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE>
SLACK_VINIFERA_VIOLATION_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE>
SLACK_ERROR_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE>

# By default fork and big fork scanning is disabled
VINIFERA_ENABLE_FORK_SCANNING=false
VINIFERA_ENABLE_BIG_FORK_SCANNING=false
  • Build
docker-compose build
  • Run
docker-compose up
  • Re-building after any changes
docker-compose up --build

Docker Compose Commands for Reference


Manual

  • Install required dependencies
cd <location_of_cloned_repo>
bundle install
  • Setup DB and migrations
bundle exec rails db:create
bundle exec rails db:migrate
  • Setup Environment Variables and Slack WebHook Sample env file is available at .example_env
GITHUB_ACCESS_TOKEN=<REDACTED>
VINIFERA_ORG_NAME=<Your_org_name> # Name of org for which token was generated.

VINIFERA_DATABASE_HOST=<localhost>
VINIFERA_DATABASE_PASSWORD=<YOUR_PASSWORD>

RAILS_MAX_THREADS=60 # This also controls DB pool

RAILS_MASTER_KEY=<REDACTED>


SLACK_UPDATES_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE>
SLACK_TARGETS_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE>
SLACK_USER_TRACKING_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE>
SLACK_VINIFERA_VIOLATION_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE>
SLACK_ERROR_GROUP_URL=https://hooks.slack.com/services/<YOUR_CONFIG_HERE>



# Add this only if you are using Docker over TLS, recommended way, if, on the same host as vinifera, you may skip it

DOCKER_CLIENT_CERT_PATH=/home/deployer/.docker
DOCKER_HOST=tcp://<IP>:<PORT>

# By default fork and big fork scanning is disabled
VINIFERA_ENABLE_FORK_SCANNING=false
VINIFERA_ENABLE_BIG_FORK_SCANNING=false

REDIS_URL=redis://<REDIS_URI>
SIDEKIQ_REDIS_URL=redis://<REDIS_URI>

# For PagerDuty Integration
DEFAULT_PD_INTEGRATION_KEY=<xxxxxxxx>
ENABLE_PAGER_DUTY_TRIGGER=true
  • Setup Cron Jobs
bundle exec whenever --update-crontab
[[rules]]
# Insert your rules here
# description = "Internal References"
# regex = '''(?i)((.*)<COMPANY_INTERNAL_REFERENCES>(.*))'''
# tags = ["internal", "company","references"]
  • Start Sidekiq
bundle exec sidekiq
Datadog

Additionally, to get the metrics on Datadog like in the above screenshot, you can use the DataDog agent - https://docs.datadoghq.com/agent/

PagerDuty

To ensure the team never misses any violation, PagerDuty integration option is there:

PagerDuty Incident

For PagerDuty integration, following environment variable needs to be set to true

ENABLE_PAGER_DUTY_TRIGGER=true

Then create a new service and a integration key as described in following doc - https://support.pagerduty.com/docs/services-and-integrations#create-a-new-service

PagerDuty Service

DEFAULT_PD_INTEGRATION_KEY=<xxxxxxxx>

Contributing

We are open to contributions/bug fixes/performance improvements to our project :)

Donations

If you appreciate the tool we have built, feel free to contribute/donate to the projects on the top of which Vinifera was built :)

Vinifera is built on top of other open-source software:

  1. Rails (Our Favourite Web Framework)
  2. Sidekiq (Handles Job LifeCycle, Scheduling, and Retries)
  3. Gitleaks (Gitleaks powers the code scanning via Docker Images)
  4. Docker (For running Scans in an isolated environment)
  5. Docker-api (Ruby Client to interact with Docker Remote API)
  6. Sidekiq Throttled (For Throttling workers)
  7. Octokit (Ruby Toolkit for Github API)
  8. Whenever (Cron Jobs in Ruby)

.... (list will go on ..., you get the gist ;) )

You can also donate to Feeding India

Disclaimer

Neither Zomato nor the developers of this tool are responsible for any damage caused by this tool or usage of the same. Use responsibly. Refer to LICENSE for more details.

License

Vinifera is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

vinifera's People

Contributors

jatindhankhar avatar shrey-sinha avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.