turbot / steampipe-plugin-code Goto Github PK

View Code? Open in Web Editor NEW

15.0 11.0 2.0 362 KB

Use SQL to instantly query secrets and more from source code. Open source CLI. No DB required.

Home Page: https://hub.steampipe.io/plugins/turbot/code

License: Apache License 2.0

Makefile 0.51% Go 99.32% PLSQL 0.17%

steampipe steampipe-plugin sql postgresql postgresql-fdw code-scanner secrets-detection hacktoberfest backup etl

steampipe-plugin-code's Introduction

Code Plugin for Steampipe

Use SQL to query secrets and more from source code.

Get started →
Documentation: Table definitions & examples
Community: Join #steampipe on Slack →
Get involved: Issues

Quick start

Install the plugin with Steampipe:

steampipe plugin install code

Run a query:

select
  secret_type,
  secret,
  authenticated,
  line,
  col
from
  code_secret
where
  src =
    'Mixed secrets are matched:\n'
    '* Slack: xoxp-5228148520-5228148525-1323104836872-10674849628c43b9d4b4660f7f9a7b65\n'
    '* AWS: AKIA4YFAKFKFYXTDS353\n'
    '* Basic auth: https://joe:[email protected]/secret'
    '* Stripe: sk_live_tR3PYbcVNZZ796tH88S4VQ2u';

Engines

This plugin is available for the following engines:

Engine	Description
Steampipe	The Steampipe CLI exposes APIs and services as a high-performance relational database, giving you the ability to write SQL-based queries to explore dynamic data. Mods extend Steampipe's capabilities with dashboards, reports, and controls built with simple HCL. The Steampipe CLI is a turnkey solution that includes its own Postgres database, plugin management, and mod support.
Postgres FDW	Steampipe Postgres FDWs are native Postgres Foreign Data Wrappers that translate APIs to foreign tables. Unlike Steampipe CLI, which ships with its own Postgres server instance, the Steampipe Postgres FDWs can be installed in any supported Postgres database version.
SQLite Extension	Steampipe SQLite Extensions provide SQLite virtual tables that translate your queries into API calls, transparently fetching information from your API or service as you request it.
Export	Steampipe Plugin Exporters provide a flexible mechanism for exporting information from cloud services and APIs. Each exporter is a stand-alone binary that allows you to extract data using Steampipe plugins without a database.
Turbot Pipes	Turbot Pipes is the only intelligence, automation & security platform built specifically for DevOps. Pipes provide hosted Steampipe database instances, shared dashboards, snapshots, and more.

Developing

Prerequisites:

Clone:

git clone https://github.com/turbot/steampipe-plugin-code.git
cd steampipe-plugin-code

Build, which automatically installs the new version to your ~/.steampipe/plugins directory:

make

Configure the plugin:

cp config/* ~/.steampipe/config
vi ~/.steampipe/config/code.spc

Try it!

steampipe query
> .inspect code

Credits

The code_secret table is based on Yelp's detect secrets project. The general matching approach and regular expressions are copied and based on their amazing work.

Open Source & Contributing

This repository is published under the Apache 2.0 (source code) and CC BY-NC-ND (docs) licenses. Please see our code of conduct. We look forward to collaborating with you!

Steampipe is a product produced from this open source software, exclusively by Turbot HQ, Inc. It is distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.

Get Involved

Join #steampipe on Slack →

Want to help but don't know where to start? Pick up one of the help wanted issues:

steampipe-plugin-code's People

Contributors

Stargazers

Watchers

Forkers

adarshnair094 pdecat

steampipe-plugin-code's Issues

Testing bug report with workflow

N/A - Testing

Can we use this plugin to scan project source code (Java, .Net, Nodejs)

Is it possible to scan project source code with this plugin ? Let's say I have a project whose source code is written in Java, can I use this plugin to scan that source code ? Just like -
select
secret_type,
secret,
line,
col
from
code_secret
where
src = Directory of my code base

If this feature is not available, is there any possibility to add support for it ?

Add additional secret types with verification methods from token-spray for code_secret table

Is your feature request related to a problem? Please describe.
I'd like to be able to check and test for additional secret types, as described by projectnuclei's token-spray - https://github.com/projectdiscovery/nuclei-templates/tree/master/token-spray

Describe the solution you'd like
Add more secret types and verification methods for use with the code_secret table

Describe alternatives you've considered
N/A

Additional context
Add any other context or screenshots about the feature request here.

Enhancements to pattern matching

Issue(s)

When attempting to use these for redactions, we noticed the following

Incomplete matching for aws_access_key (sso credentials aren't matched)
Slack regex doesn't match the whole token (thus we only redact a subset of the token)

AWS SSO Access Key Example

pipeline "example" {

  output "normal" {
    value = credential.aws["normal"].access_key
  }

  output "sso" {
    value = credential.aws["sso"].access_key
  }
  
}

fpr example
[flowpipe] Execution ID: exec_cnc<snipped>
[show_imported_creds] Starting pipeline
[show_imported_creds] Output normal = REDACTED
[show_imported_creds] Output sso = ASIA<snipped>G

Slack Example

Tokens can be quite varying for example:

User Token: xoxp-0000000000000-0000000000000-0000000000000-000x00x000x00xx00x000x0x0xxxx00x
Bot Token: xoxb-0000000000000-0000000000000-0xxXx00XxXX0xxxXxXXxxXXx

When these are matched for redaction we get:

User Token: REDACTED
Bot Token: REDACTEDXx00XxXX0xxxXxXXxxXXx

Expected Behavior

These should match the entire secret for redaction purposes

seemingly unnecessary cte (but no order by) is required

Similar to #4.

This fails.

select 
  timestamp,
  message
from 
  aws_cloudwatch_log_event cw,
  code_secret s
where 
  cw.log_group_name = '/aws/lambda/jon-aws-lambda-hello-world'
  and s.src = cw.message

'List' call is missing required quals: column:'src' operator: =

This works.

with data as (
  select 
    timestamp,
    message
  from 
    aws_cloudwatch_log_event a
  where 
    a.log_group_name = '/aws/lambda/jon-aws-lambda-hello-world'
)
select 
  d.*,
  s.*
from 
  data d,
  code_secret s
where 
  s.src = d.message

seemingly unnecessary order by is required

This works.

select
  task_definition_arn,
  secret
from
    aws_ecs_task_definition,
    code_secret s
  where
    src = '[{"Cpu": 256, "Name": "sample-app", "User": null, "Image": "httpd:2.4", "Links": [], "Memory": null, "Command": ["#https://joe:passwd123\n#mllhBNrG467B7Q5iT+ePFr6eLCE24ij9vT/fCeckOunfqz\n#45ab6f911111f9f376a5b52c25d22113f2b45fa1\n#00Am7B2M_U-63q_Ppd6tDzAbBOkvcCht-kDG-baM7t\n#AKIA4YFAKFKFYXTDS353\n\n\n/bin/sh -c \"echo hello\""], "Secrets": null, "Ulimits": null, "Hostname": null, "DependsOn": null, "Essential": true, "DnsServers": null, "EntryPoint": ["sh", "-c"], "ExtraHosts": null, "Privileged": null, "Environment": [], "HealthCheck": null, "Interactive": null, "MountPoints": [], "StopTimeout": null, "VolumesFrom": [], "DockerLabels": null, "PortMappings": [{"HostPort": 80, "Protocol": "tcp", "ContainerPort": 80}], "StartTimeout": null, "PseudoTerminal": null, "SystemControls": null, "LinuxParameters": null, "DnsSearchDomains": null, "EnvironmentFiles": null, "LogConfiguration": {"Options": {"awslogs-group": "/ecs/first-run-task-definition", "awslogs-region": "us-west-1", "awslogs-stream-prefix": "ecs"}, "LogDriver": "awslogs", "SecretOptions": null}, "WorkingDirectory": null, "DisableNetworking": null, "MemoryReservation": 512, "ResourceRequirements": null, "DockerSecurityOptions": null, "FirelensConfiguration": null, "RepositoryCredentials": null, "ReadonlyRootFilesystem": null}]'

This fails when the same data comes as JSONB from aws_ecs_task_definition.

select
  task_definition_arn,
  secret
from
    aws_ecs_task_definition,
    code_secret s
  where
    src = (container_definitions)::text;

ERROR: operator does not exist: text = jsonb
LINE 7: src = '[{"Cpu": 256, "Name": "sample-app", "User": null,...
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.

This (cast the literal text to jsonb) fails in the same way.

select
  task_definition_arn
  from
    aws_ecs_task_definition,
    code_secret s
  where
    src = '[{"Cpu": 256, "Name": "sample-app", "User": null, "Image": "httpd:2.4", "Links": [], "Memory": null, "Command": ["#https://joe:passwd123\n#mllhBNrG467B7Q5iT+ePFr6eLCE24ij9vT/fCeckOunfqz\n#45ab6f911111f9f376a5b52c25d22113f2b45fa1\n#00Am7B2M_U-63q_Ppd6tDzAbBOkvcCht-kDG-baM7t\n#AKIA4YFAKFKFYXTDS353\n\n\n/bin/sh -c \"echo hello\""], "Secrets": null, "Ulimits": null, "Hostname": null, "DependsOn": null, "Essential": true, "DnsServers": null, "EntryPoint": ["sh", "-c"], "ExtraHosts": null, "Privileged": null, "Environment": [], "HealthCheck": null, "Interactive": null, "MountPoints": [], "StopTimeout": null, "VolumesFrom": [], "DockerLabels": null, "PortMappings": [{"HostPort": 80, "Protocol": "tcp", "ContainerPort": 80}], "StartTimeout": null, "PseudoTerminal": null, "SystemControls": null, "LinuxParameters": null, "DnsSearchDomains": null, "EnvironmentFiles": null, "LogConfiguration": {"Options": {"awslogs-group": "/ecs/first-run-task-definition", "awslogs-region": "us-west-1", "awslogs-stream-prefix": "ecs"}, "LogDriver": "awslogs", "SecretOptions": null}, "WorkingDirectory": null, "DisableNetworking": null, "MemoryReservation": 512, "ResourceRequirements": null, "DockerSecurityOptions": null, "FirelensConfiguration": null, "RepositoryCredentials": null, "ReadonlyRootFilesystem": null}]'::jsonb;

This (wrap a jsonb cast around the text cast) works as in the first example.

select
  task_definition_arn
  from
    aws_ecs_task_definition,
    code_secret s
  where
    src = ('[{"Cpu": 256, "Name": "sample-app", "User": null, "Image": "httpd:2.4", "Links": [], "Memory": null, "Command": ["#https://joe:passwd123\n#mllhBNrG467B7Q5iT+ePFr6eLCE24ij9vT/fCeckOunfqz\n#45ab6f911111f9f376a5b52c25d22113f2b45fa1\n#00Am7B2M_U-63q_Ppd6tDzAbBOkvcCht-kDG-baM7t\n#AKIA4YFAKFKFYXTDS353\n\n\n/bin/sh -c \"echo hello\""], "Secrets": null, "Ulimits": null, "Hostname": null, "DependsOn": null, "Essential": true, "DnsServers": null, "EntryPoint": ["sh", "-c"], "ExtraHosts": null, "Privileged": null, "Environment": [], "HealthCheck": null, "Interactive": null, "MountPoints": [], "StopTimeout": null, "VolumesFrom": [], "DockerLabels": null, "PortMappings": [{"HostPort": 80, "Protocol": "tcp", "ContainerPort": 80}], "StartTimeout": null, "PseudoTerminal": null, "SystemControls": null, "LinuxParameters": null, "DnsSearchDomains": null, "EnvironmentFiles": null, "LogConfiguration": {"Options": {"awslogs-group": "/ecs/first-run-task-definition", "awslogs-region": "us-west-1", "awslogs-stream-prefix": "ecs"}, "LogDriver": "awslogs", "SecretOptions": null}, "WorkingDirectory": null, "DisableNetworking": null, "MemoryReservation": 512, "ResourceRequirements": null, "DockerSecurityOptions": null, "FirelensConfiguration": null, "RepositoryCredentials": null, "ReadonlyRootFilesystem": null}]'::jsonb)::text;

I will take another look at the format of what actually comes out of the aws_ecs_task_definition table.

the matcher thinks an aws lambda request id is a heroku api key