Giter VIP home page Giter VIP logo

aws-auto-remediate's Introduction

AWS Auto Remediate

Build Status Codacy Badge Codacy Badge

Release Pre-release Date

Language serverless Python Black code style: prettier

Open source application to instantly remediate common security issues through the use of AWS Config.

auto-remediate

Table of Contents

About

Auto Remediate

The Auto Remediate function is triggered via an SQS Queue auto-remediate-config-compliance. The SQS Queue is populated with a compliance payload from AWS Config via a CloudWatch Event auto-remediate-config-compliance. The purpose of the CloudWatch Event is to filter out all non-compliance related messages that AWS Config generates.

Once the Lambda function has been triggered it will attempt to remediate the security concern. If the remediation was unsuccessful, the event payload will be sent to the dead letter queue (DQL) SQS Queue auto-remediate-dlq. Each time a payload is sent is sent to the DLQ, an attribute try_count is incremented to the SQS message. Once that count exceeds RETRYCOUNT variable attached to the Lambda Function, the message will no longer be sent to the DLQ.

If no remediation exists for the incoming AWS Config event, the AWS Config payload will be sent to an SNS Topic auto-remediate-missing-remediation which can be subscribed to by administrators or other AWS services.

Auto Remediate DLQ

The Auto Remediate DLQ function is triggered on a schedule (defined in the serverless.yml file). When the function is run, it will retrieve messages from SQS Queue auto-remediate-dlq and sends the message to the compliance SQS Queue auto-remediate-config-compliance.

Auto Remediate Setup

The Auto Remediate Setup function is triggered manually by the user. The purpose of this function is to invoke CloudFormation Stacks for each of the AWS Config Rules that will monitor for security issues as well as create/insert records into the DynamoDB settings table used to control the actions of the Auto Remediate function.

Setup

New Account

Proceed to the Deployment section below.

Existing Account

Auto Remediate utilises the compliance event triggers made by AWS Config. Due to the fact that AWS Config will trigger a compliance event only when the compliance status of a resource changes state (i.e., COMPLIANT to NON_COMPLIANT or vice versa) it is advised that you disable the CIS AWS Foundations compliance standards within AWS Security Hub (and ensure all AWS Config rules starting with securityhub are removed from your account) before proceeding.

Once AWS Config is cleared of all AWS Security Hub related rules, you may proceed to deploy Auto Remediate and enable the CIS AWS Foundations compliance standards within AWS Security Hub.

Deployment

  1. Install the Serverless Framework
npm install serverless --global
  1. Install AWS CLI
pip3 install awscli --upgrade --user
  1. Configure the AWS CLI following the instruction at Quickly Configuring the AWS CLI. Ensure the user you're configuring has the appropriate IAM permissions to create Lambda Functions, S3 Buckets, IAM Roles, and CloudFormation Stacks. It is best for administrators to deploy Auto Remediate.

  2. Install Auto Remediate

serverless create --template-url https://github.com/servian/aws-auto-remediate --path aws-auto-remediate
  1. Change into the Auto Remediate directory
cd aws-auto-remediate
  1. Install Serverless plugins needed for deployment
serverless plugin install --name serverless-python-requirements
npm install serverless-iam-roles-per-function
  1. Deploy Auto Remediate to your AWS account
serverless deploy [--region <AWS region>] [--aws-profile <AWS CLI profile>]
  1. Invoke Auto Remediate Setup for the first time to create the necessary AWS Config rules and settings
serverless invoke -f AutoRemediateSetup [--region <AWS region>] [--aws-profile <AWS CLI profile>]
  1. Check Auto Remediate Setup logs
serverless logs -f AutoRemediateSetup [--region <AWS region>] [--aws-profile <AWS CLI profile>]

Update

  1. Remove existing Auto Remediate directory

  2. Install Auto Remediate

serverless create --template-url https://github.com/servian/aws-auto-remediate --path aws-auto-remediate
  1. Deploy Auto Remediate update to your AWS account
serverless deploy [--region <AWS region>] [--aws-profile <AWS CLI profile>]
  1. Invoke Auto Remediate Setup to deploy new AWS Config rules and settings
serverless invoke --function AutoRemediateSetup [--region <AWS region>] [--aws-profile <AWS CLI profile>]

Removal

Auto Remediate is deployed using the Serverless Framework which under the hood creates an AWS CloudFormation Stack allowing for a clean and simple removal process.

To remove Auto Remediate from your AWS account, follow the below steps:

  1. Change into the Auto Remediate directory
cd aws-auto-remediate
  1. Remove Auto Remediate from your AWS account
serverless remove [--region <AWS region>] [--aws-profile <AWS CLI profile>]

Settings

Auto Remediate uses a DynamoDB settings table auto-remediate-settings that allows the user to control which rule should be remediated by the tool. Once Auto Remediate Setup has been run, head on over to DynamoDB and inspect the rules key where you can then set the remediate key to false if you'd like to disable automatic remediate.

For rules deployed by Auto Remediate Setup (e.g., auto-remediate-rds-instance-public-access-check) an extra key deploy can be found in the settings table. Although not functional at the moment, this will allow users to control which Auto Remediate deployed rules should be deployed and which should be skipped.

Config Rules

The tables below detail the auto remediated rules and scenarios.

⚠️ All remediations tagged with a warning symbol may break existing functionality.

AWS Config Managed Rules

Database

Rule Description Remediation
RDS Instance Public Access Check Check whether the Amazon Relational Database Service instances are not publicly accessible.
The rule is NON_COMPLIANT if the publiclyAccessible field is true in the instance configuration item.
⚠️ Sets publiclyAccessible field to False

Storage

Rule Description Remediation
S3 Bucket Service Side Encryption Enabled Checks that your Amazon S3 bucket either has Amazon S3 default encryption enabled or that the S3 bucket policy explicitly denies put-object requests without server side encryption. Enables SSE
S3 Bucket SSL Requests Only Checks whether S3 buckets have policies that require requests to use Secure Socket Layer (SSL). Adds Bucket Policy to force SSL only connections

AWS Security Hub Rules

Compute

Rule Description Remediation
Restricted RDP Checks whether the incoming RDP traffic is allowed from 0.0.0.0/0 or ::/0. This rule is compliant when incoming RDP traffic is restricted. ⚠️ Deletes offending inbound rule
Restricted SSH Checks whether the incoming SSH traffic is allowed from 0.0.0.0/0 or ::/0. This rule is compliant when incoming SSH traffic is restricted. ⚠️ Deletes offending inbound rule

Management and Governance

Rule Description Remediation
CloudTrail CloudWatch Logs Enabled Checks whether AWS CloudTrail trails are configured to send logs to Amazon CloudWatch logs. Enables CloudWatch logs to Log Group cloudtrail/<CloudTrail Name>
CloudTrail Encryption Enabled Ensure CloudTrail logs are encrypted at rest using KMS CMKs. Enables CloudWatch encryption with KMS CMK cloudtrail/<CloudTrail Name>
CloudTrail Log File Validation Enabled Checks whether AWS CloudTrail creates a signed digest file with logs. AWS recommends that the file validation must be enabled on all trails. The rule is NON_COMPLIANT if the validation is not enabled. Enables CloudTrail Validation
Multi Region Cloud Trail Enabled Checks that there is at least one multi-region AWS CloudTrail. The rule is NON_COMPLIANT if the trails do not match inputs parameters. Enables Multi Region CloudTrail

Network and Content Delivery

Rule Description Remediation
VPC Flow Logs Enables Checks whether Amazon Virtual Private Cloud flow logs are found and enabled for Amazon VPC. Creates new S3 Bucket <Account Number>-<Region>-flow-logs for logging with a prefix of <VPC ID>/
VPC Default Security Group Closed Checks that the default security group of any Amazon Virtual Private Cloud (VPC) does not allow inbound or outbound traffic. The rule is NON_COMPLIANT if the default security group has one or more inbound or outbound traffic. Deletes all egress and ingress rules

Security, Identity & Compliance

Rule Description Remediation
Access Keys Rotated Checks whether the active access keys are rotated within the number of days specified in 90 days. ⚠️ Deletes Access Key
Customer Managed Key Rotation Enabled Checks that key rotation is enabled for customer created customer master key (CMK). Enables key rotation
IAM Password Policy Ensure Expires Checks whether the IAM password policy ensures that passwords expire. Enables password expiration
IAM Password Policy Lowercase Letter Check Checks whether the IAM password policy enforces the inclusion of a lowercase letter. Enables "Require at least one lowercase letter" option
IAM Password Policy Minimum Length Check Checks whether the IAM password policy enforces a minimum length. Sets minimum password length to 14.
IAM Password Policy Number Check Checks whether the IAM password policy enforces the inclusion of a number. Enables "Require at least one number" option
IAM Password Policy Prevent Reuse Check Checks whether the IAM password policy prevents password reuse. Sets number of passwords to remember to 24.
IAM Password Policy Symbol Check Checks whether the IAM password policy enforces the inclusion of a symbol. Enables "Require at least one non-alphanumeric character" option
IAM Password Policy Uppercase Letter Check Checks whether the account password policy for IAM users requires at least one uppercase character in password. Enables "Require at least one uppercase letter" option
IAM Policy No Statements with Admin Access Checks whether the default version of AWS Identity and Access Management (IAM) policies do not have administrator access.
If any statement has "Effect": "Allow" with "Action": "*" over "Resource": "*", the rule is NON_COMPLIANT.
⚠️ Creates new Policy with offending Statements removed
IAM User No Policies Check Checks that none of your IAM users have policies attached. IAM users must inherit permissions from IAM groups or roles. Detaches Managed Policies from offending IAM User
IAM User Unused Credentials Check Checks whether AWS Identity and Access Management (IAM) users have passwords or active access keys that have not been used within 90 days. ⚠️ Deletes Access Key / Login Profile
MFA Enabled for IAM Console Access Checks whether AWS Multi-Factor Authentication (MFA) is enabled for all AWS Identity and Access Management (IAM) users that use a console password. ⚠️ Deletes user's Login Profile only. Deleting a user's password does not prevent a user from accessing AWS through the command line interface or the API.

Storage

Rule Description Remediation
S3 Bucket Logging Enabled Checks whether logging is enabled for your S3 buckets. Creates new S3 Bucket <Account Number>-<Region>-access-logs for logging with a prefix of <Bucket Name>/
S3 Bucket Public Read Prohibited Checks to see if S3 buckets are publicly readable. ⚠️ Sets S3 Bucket ACL to private
S3 Bucket Public Write Prohibited Checks to see if S3 buckets allow public write. ⚠️ Sets S3 Bucket ACL to private

Resources

The table below details all AWS resources created when deploying the application.

Service Resource ID
CloudFormation Stack auto-remediate
CloudWatch Event Rule auto-remediate-config-compliance
DynamoDB Table auto-remediate-settings
Lambda Function auto-remediate
auto-remediate-dlq
auto-remediate-setup
SNS Topic auto-remediate-log (not functional #19)
auto-remediate-missing-remediation
SQS Queue auto-remediate-config-compliance
auto-remediate-dlq

Coverage

Full list of development and automated testing coverage found here.

aws-auto-remediate's People

Contributors

dependabot[bot] avatar jihygk avatar mlevit avatar tristanmorgan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-auto-remediate's Issues

Use Boto3 paginator for list_users

There could be an issue in running client.list_users() for accounts with potentially a very large number of users. To ensure the function doesn't run out of memory, we can use Boto3's paginator:

paginator = iam.get_paginator('list_users')
for response in paginator.paginate():
    print(response)

Impact function: iam_user_unused_credentials_check

Exception handling refactor

I think this gives a clue for how we should refactor the error handling - it should raise the NoSuchBucket exception (or whatever) directly and we should assert that that exception is raised and caught here.

That means we should have the try except in the lambda_handler remediate() function, I think

Originally posted by @jihygk in #30

SNS logging seems to interfere with console logging

Extending the Python logging interface and allowing all logs to be pushed to an SNS Topic seems to sometimes cause the console logs to not show up. For now, SNS logging has been disabled inside the lambda_handler function within auto-remediate.

More investigation/testing needs to occur to understand the problem (if any).

Prevent infinite loops from occurring

When a remediation fails and is sent to the DLQ, it will be resent to for remediate and the loop will continue.

To prevent this from happening a counter SQS message attribute needs to be added. Once that attribute reaches n the messages should no longer be sent to DLQ.

iam_user_no_policies_check does not cover all scenarios

As it currently stands the newly implemented pull request #26 for iam_user_no_policies_check remediation only removes IAM managed policies attached to the user however users can also have inline policies.

The current paginator used list_attached_user_policies will not capture inline policies which can only be captured using list_user_policies and deleted using delete_user_policy.

Refactor remediation function selection

I think this section is getting too messy. Can we try to refactor the core of this function (routing of the remediation based on rule name) like this

Class Remediate:
    def __init__ ...

        self.remediations = {
            "securityhub-restricted-rdp": self.security_hub.restricted_rdp,
            ...
        }

    def remediate(remediation_rule):
        return self.remediations.get(remediation_rule, rem_not_found_function)()

Originally posted by @jihygk in #20

More permissive license

Hi,

What are the chances of moving the project to a more permissive license structure like MIT? The current license is incompatible for use with some...let's say more "restrictive" organisations.

Thanks for your consideration,
Ian.

serverless plugin install --name serverless-python-requirements failed

Describe the bug
A clear and concise description of what the bug is.
serverless plugin install --name serverless-python-requirements failed with the following errors:

Fetch Error --------------------------------------------

FetchError: request to https://raw.githubusercontent.com/serverless/plugins/master/plugins.json failed, reason: getaddrinfo ENOTFOUND 8080 8080:80
at ClientRequest. (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\node_modules\node-fetch\index.js:133:11)
at ClientRequest.emit (events.js:198:13)
at ClientRequest.EventEmitter.emit (domain.js:448:20)
at onerror (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\node_modules\https-proxy-agent\node_modules\agent-base\src\index.ts:214:9)
at callbackError (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\node_modules\https-proxy-agent\node_modules\agent-base\src\index.ts:236:5)
at process._tickCallback (internal/process/next_tick.js:68:7)
From previous event:
at PluginInstall.install (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\lib\plugins\plugin\install\install.js:53:8)
From previous event:
at Object.plugin:install:install [as hook] (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\lib\plugins\plugin\install\install.js:41:12)
at BbPromise.reduce (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\lib\classes\PluginManager.js:490:55)
From previous event:
at PluginManager.invoke (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\lib\classes\PluginManager.js:490:22)
at getHooks.reduce.then (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\lib\classes\PluginManager.js:525:24)
From previous event:
at PluginManager.run (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\lib\classes\PluginManager.js:525:8)
at variables.populateService.then (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\lib\Serverless.js:133:33)
at runCallback (timers.js:705:18)
at tryOnImmediate (timers.js:676:5)
at processImmediate (timers.js:658:5)
at process.topLevelDomainCallback (domain.js:126:23)
From previous event:
at Serverless.run (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\lib\Serverless.js:120:74)
at serverless.init.then (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\bin\serverless.js:82:30)
at C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\node_modules\graceful-fs\graceful-fs.js:136:16
at C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\node_modules\graceful-fs\graceful-fs.js:57:14
at FSReqWrap.args [as oncomplete] (fs.js:140:20)
From previous event:
at initializeErrorReporter.then (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\bin\serverless.js:82:8)
at runCallback (timers.js:705:18)
at tryOnImmediate (timers.js:676:5)
at processImmediate (timers.js:658:5)
at process.topLevelDomainCallback (domain.js:126:23)
From previous event:
at Object. (C:\Users\p1jxc01\AppData\Roaming\npm\node_modules\serverless\bin\serverless.js:71:4)
at Module._compile (internal/modules/cjs/loader.js:776:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:829:12)
at startup (internal/bootstrap/node.js:283:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)

Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues
Issues: forum.serverless.com

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Stacktrace
If applicable, add stacktraces to help explain your problem.

Versions (please complete the following information):

  • Serverless Framework: [e.g. 1.42.3]
  • boto3: [e.g. 1.9.156]
  • botocore: [e.g. 1.12.156]
  • moto: [e.g. 1.3.8]
  • pytest: [e.g. 4.4.1]

AWS (please complete the following information):

  • Region: [e.g. ap-southeast-2]

Additional context
Add any other context about the problem here.

Proxy has been setup, can you help to fix it

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.