I followed the readme and installed scoutsuite using pip
$ pip install scoutsuite
This installed without any error but I am unable to find the command scout* or anything similar on my machine.
Is this working with pip install or only though git clone?

 Ingress Rules 5

    TCP
        Ports:
            20
                EC2 security groups:
                    app-prd (sg-123456abc)
            21
                EC2 security groups:
                    app-prd (sg-123456abc)
...

The FTP port is indeed open, but I believe that this is not a security risk because only instances in sg-123456abc can connect to it. The rule should be improved.

I'm trying a test run using a service account, using the command outlined in the readme:

python Scout.py --provider gcp --service-account --key-file /path/to/my/key/file

I get a number of errors thrown:

Fetching Cloud Resource Manager config...
projects
string indices must be integers
string indices must be integers
string indices must be integers
0/0
Fetching Cloud Storage config...
buckets
string indices must be integers
string indices must be integers

and finally:

AttributeError: 'GCPProvider' object has no attribute 'aws_account_id'

I've tried both the pipenv install as well as cloning it from here, on osx. I work with a few different AWS accounts, but none are actively in env when I run this. GCP sdk is local to the folder, have tried brew cask as well. Now to be fair there are only like 3 accounts in the project and no systems/buckets/etc. What is going on here?

The current implementation for authentication could/should rely solely on boto3 and not on any custom code, AKA remnants of Opinel.

It should be validated that this wouldn't create any issues with MFA.

The following have been observed:

• Scout shows that Flow Logs are missing for subnets even though the VPC that contains them has it activated.
• Scout indicated that "Global services logging disabled" even though there is CloudTrail log.
• EC2 attack surface doesn't filter duplicate IPs/ranges
• In "Unused security groups", Scout does not include some SGs even though they have no usage
• Typo in "Role passed to stack" description for CloudFormation ("beause")

Some of these might be FPs.

Viewing the report for Route 53 and am unable to view each hosted zone separately. Clicking the link to each individual zone results in blank page showing the top bar and options on the left. Clicking show all results in viewing all of the zones on the same page.

Implement privilege checks similar to:
https://github.com/RhinoSecurityLabs/Security-Research/blob/master/tools/aws-pentest-tools/aws_escalate.py
https://rhinosecuritylabs.com/aws/aws-privilege-escalation-methods-mitigation/

Also check if IAM roles are assigned with "excessive" permissions.

It isn't possible to open links in new tabs as the anchor isn't included (it always opens the <report>.html file). Implementing this would help with navigation.

e.g. https://stackoverflow.com/questions/15678511/opening-tab-with-anchor-link

I'm attempting to migrate from (the now deprecated) AWSScout2, and in some cases, a pip install of scoutsuite can fail due to a smart-quote character in README.md. Using the following Dockerfile:

FROM bitnami/minideb:stretch
RUN install_packages curl jq patch uuid-runtime python3-pip && pip3 install setuptools wheel && pip3 install awscli boto3
RUN pip3 install scoutsuite

yields:

$ sudo docker build -t scoutsuite .
Sending build context to Docker daemon 22.53 kB
Step 1/3 : FROM bitnami/minideb:stretch
 ---> 2a1cee458dac
Step 2/3 : RUN install_packages curl jq patch uuid-runtime python3-pip && pip3 install setuptools wheel && pip3 install awscli boto3
 ---> Using cache
 ---> b25d3c56f96e
Step 3/3 : RUN pip3 install scoutsuite
 ---> Running in 24d0638f56e6

Collecting scoutsuite
  Downloading https://files.pythonhosted.org/packages/8e/96/d87a0b6910ca51f3def0d623c586bb9eaed385ea90e74047a2f51c5b9a16/ScoutSuite-4.0.3.tar.gz (384kB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-5xzyk4e7/scoutsuite/setup.py", line 24, in <module>
        long_description=open('README.md').read(),
      File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4377: ordinal not in range(128)
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-5xzyk4e7/scoutsuite/
The command '/bin/sh -c pip3 install scoutsuite' returned a non-zero code: 1

It appears that line 118 (or 119, depending on the version) of README.md contains a smart quote character that's not handled in the docker build environment.

If I manually run pip within a virtual environment on CentOS 7, the module installation completes successfully. Not sure if it's related to LANG or LC_ALL being set/unset, but it seems that replacing the smart quote with an apostrophe is the most straightforward fix.

Also, the release management process is a bit confusing, as the GitHub repo only has a version 4.0.2 tag, while PyPI shows 4.0.0, 4.0.1, 4.0.2, and 4.0.3. As a result, it's not clear what commits are present in the various PyPI versions.

Hi,
I'm trying to develop and contribute to this project with my windows machine.
But following files have invalid characters(colon and asterisk) as the filename and failed to clone property.

Can I rename these files and make pull request for this project?
My suggestion is replacing space and colon to hyphen, asterisk to "all"

ScoutSuite/providers/aws/rules/findings/policy allows sts:AssumeRole
ScoutSuite/providers/aws/rules/findings/policy allows sts:AssumeRole *

Ref:
https://docs.microsoft.com/en-us/windows/desktop/msi/filename

here is my clone log. I have changed nothing after clone, but there is difference.

$ git clone [email protected]:mitsuo0114/ScoutSuite.git
Cloning into 'ScoutSuite'...
remote: Enumerating objects: 9961, done.
remote: Counting objects: 100% (9961/9961), done.
remote: Compressing objects: 100% (2488/2488), done.
remote: Total 9961 (delta 7402), reused 9905 (delta 7346), pack-reused 0
Receiving objects: 100% (9961/9961), 1.65 MiB | 2.54 MiB/s, done.
Resolving deltas: 100% (7402/7402), done.

$ cd ScoutSuite/

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        deleted:    ScoutSuite/providers/aws/rules/findings/policy allows sts:AssumeRole
        deleted:    ScoutSuite/providers/aws/rules/findings/policy allows sts:AssumeRole *

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        ScoutSuite/providers/aws/rules/findings/policy allows sts

no changes added to commit (use "git add" and/or "git commit -a")

I can setup my AWS EC2 instance with the role that has the permissions to assume any role it wants, as described in this post. Specifically, if I have a policy like the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole"
            ],
            "Resource": "*"
        }
    ]
}

I can run aws cli commands with just the role_arn and external_id. But I cannot run Scout2 with a profile containing just the role_arn and external_id, since it requires the presence of aws_secret_access_key and aws_access_key_id. Some shallow surfing of the source code tells me that this line seems to be the problem. ( This line also seems related )

Scout2 should have similar behavior as aws cli in this case, i.e. it should be okay without aws_secret_access_key.

1 character passwords were authorized when no password policy exists, even though the console displays 6. It would be good to show this in the report in order to avoid confusion.

Loving this tool! It would be great if you could sort the security groups alphabetically.

At the moment the external attack surface overview shows the external IP's and the exposed protocol/ports. However, we now need to map this external IP back to an EC2/ELB interface associated with it. Can more information be shown about the user/assigned system/DNS related with the IP? Maybe also add a click-through to the associated instance/rules?

Was --check-s3-encryption deprecated?

Hi,

I am unable to set a bucket exception for the Bucket world-listable (anonymous) rule.

Creating the exception in the UI generates

exceptions = 
{
    "s3": {
        "s3-bucket-AllUsers-read": [
            ""
        ]
    }
}

I've tried entering:

1. services.s3.buckets.GUID
2. s3.buckets.GUID
3. buckets.GUID
4. GUID

Same output when using a custom ruleset to bump this rule to Danger. Using Scout2 from the master branch.

When I create an exception list from the UI and run the scan again with the exception file, it seems that RuleExceptions are not applied because of the following line.

Changing the line to the following seems to fix the issue.

cloud_provider.services[service]['findings'][rule]['flagged_items'] = len(cloud_provider.services[service]['findings'][rule]['items'])

Scout indicates Last Backup: N/A even though [automated] backups are present.

Unsure if this FP is due to not taking into account automated backups - to confirm.

When the application runs with errors, it should generate an anonymized GH-friendly template that can be pasted in a new issue and help remediation.

Filtering by region is not supported on the dashboard view. Adding/Removing a region should result in updated numbers.

Add this explanation to the README for GCP.
https://cloud.google.com/sdk/gcloud/reference/auth/activate-service-account

Ensure that PRs are pushed to another branch, e.g. dev.

References:
https://help.github.com/articles/setting-guidelines-for-repository-contributors/
https://gist.github.com/PurpleBooth/b24679402957c63ec426
https://github.com/nayafia/contributing-template
https://gist.github.com/briandk/3d2e8b3ec8daf5a27a62

Two runs ScoutSuite v4.0.3 using an organisation ID and the project ID where the logging sink resides returned no results for logging sinks under the Management->Stack Driver Logging dashboard, which is incorrect as there is one.

Confirmation there is a logging sink:

gcloud logging sinks list --organization=999999999

NAME       DESTINATION                                                         FILTER

mysink  pubsub.googleapis.com/projects/logging-project-215613/topics/mytopic  logName:logs/cloudaudit.googleapis.com%2Factivity

Using organisation ID:

Scout.py --provider gcp --organization-id 999999999 --exceptions ../gcp-audit/ --report-dir ../gcp-audit/ --user-account

Using the project ID where the sink is:

Scout.py --provider gcp --project-id logging-project-215613 --exceptions ../gcp-logging-project-audit/ --report-dir ../gcp-logging-project-audit/ --user-account`

No exceptions are reported although now I realise I should have specified a file rather than directory.

One of the findings is titled MySQL port open to all:

The Usage section is empty, meaning that (I assumed but had to confirm) the security group is not used anywhere.

In order to ease the report review, it would be nice to see "No usages of this security group were found" in that section. It makes it explicit and easier to understand.

Also, if the SG is insecure (mysql open to all) but is not in use, then its risk should be warning instead of danger.

ec2-default-security-group-in-use.json

• Checks that the default SG is in use, but not that the SG hasn't been modified to be secure.
• Add a is_default_configuration parameter to validate.

ec2-default-security-group-with-rules.json:

• Check that the default SG is in use and that is isn't empty, but not that it has the default configuration. Apply same check as previous.

Got an error when being asked whether I wanted to overwrite the existing report.

File '/report/inc-awsconfig/exceptions.js' already exists. Do you want to overwrite it (y/n)?
EOF when reading a lineCreating /report/report.html ...

File '/report/report.html' already exists. Do you want to overwrite it (y/n)? Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/AWSScout2/output/utils.py", line 22, in prompt_4_yes_no
    choice = raw_input().lower()
NameError: name 'raw_input' is not defined

Maybe just needs raw_input to become input in the code, then have this section at the top?

from __future__ import print_function
import sys
if sys.version_info.major < 3:
    input = raw_input

The No Export Sinks finding simply checks that there is a sink configured. When running Scout for an Organization or against multiple projects (i.e. with a Service Account that has access to multiple Projects), if there is 1 sink in the whole Organization/Project list, then the finding will not be flagged.

The finding should check that there is no Project that doesn't export logs to a sink.

If a read replica's backup period is modified, the following message can appear: "DB Backups not supported on a read replica for engine postgres"

Found a FP case:

When testing AssumeRole for * resources, this policy gets flagged (belong to PowerUserAccess Managed policy):

 "Statement":  [
{
  "Effect": "Allow",
  "NotAction": [
    "iam:*",
    "organizations:*"
  ],
  "Resource": [
    "*"
  ]
} ,
{
  "Action": [
    "iam:CreateServiceLinkedRole",
    "iam:DeleteServiceLinkedRole",
    "iam:ListRoles",
    "organizations:DescribeOrganization"
  ],
  "Effect": "Allow",
  "Resource": [
    "*"
  ]
}
  ] ,
  "Version":  "2012-10-17"

Error:

invalid literal for int() with base 10: 'N/A'
Path: ['services', 'ec2', 'regions', 'us-west-2', 'vpcs', 'vpc-xxx', 'instances', 'i-xxx', 'network_interfaces', 'eni-xxx', 'PrivateIpAddresses']
Key = PrivateIpAddresses
Value = {'Association': {'IpOwnerId': 'amazon', 'PublicDnsName': 'xxx.us-west-2.compute.amazonaws.com', 'PublicIp': 'xxx'}, 'Primary': True, 'PrivateDnsName': 'ip-xxx.us-west-2.compute.internal', 'PrivateIpAddress': 'xxx'}
Path = []

Caused by services/ec2.py L107.

Hi,

Although various export are available and the html/js report offers an impressive user experience, a lack of easy to use, human readable csv export of scout2 findings is perhaps the one thing that keeps it from being, hands down, the best aws audit tool.

I may be missing something, even though i've spent many hours on the issue throwing myself at the export wiki entry .
Or perhaps it was never the intention to make scout2 easy to use for mere mortals.

Regardless, i'd love to walk out of this experience a bit wiser.
Enlighten me, would you kindly :)
-A.G.

While reviewing the report I find myself going back and forth between the AWS console and the report itself. For many findings this might get annoying.

It would be nice if the report would have links to the AWS console section associated with the finding. For example a finding related to a security group in ap-northeast-2 should include a link to:

https://ap-northeast-2.console.aws.amazon.com/vpc/home?region=ap-northeast-2#securityGroups:

So that the user can click on that and review it easily.

If the AWS console supports URLs with filters (for example to shown only one of the security groups) that should also be included in the link.

We've started seeing the error "Global services logging disabled" in the CloudTrail section. When we click through on the report, the page is blank.

However I think this finding is a mistake, we do have "Management events" set to "Read/Write events
All". Which I think is the relevant setting. We have recently moved over to the Organizations integration with CloudTrail though - perhaps this is causing confusion?

This rule checks the iam._ARG_0_s.id.inline_policies.id.PolicyDocument.Statement.id.Resource key. The issue is that some policies will use NotResource instead of Resource, which generates an error.

An additional condition should be added to check Statement for Resource and NotResource. This issue might exist in multiple additional rules.

This tool seems to gracefully handle "rate-limit exceeded" responses (https://github.com/nccgroup/opinel/blob/master/opinel/utils/aws.py#L142)

However Scout exceeding API rate-limits can have an impact on the rest of the assets deployed in an AWS account.

Could we add a flag to manually throttle to a some n requests per second?

When running Scout with the aws provider, get these types of warnings:

WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: iam.amazonaws.com

This is potentially a boto3 issue and not a Scout one.

Clicking on the 12 does nothing, and Empty string passed to getElementById() is shown in the browser JS console.

JS console line is: Empty string passed to getElementById(). jquery-1.11.2.min.js:2:24473

Implement the ability to change between light & dark themes.

This should allow switching themes within a report.

Example BS themes: https://bootswatch.com/

Hi All,

I'm not a programmer by any means and this is all new. I only realised there is a newer version available instead of the Scout2 deprecated package. I have successfully installed it using

pip install ScoutSuite

Run the application:
Scout --provider aws --profile <profile name>

The application starts doing its thing and then:
Error: could not fetch directconnect configuration.
'NoneType' object has no attribute 'get_caller_identity'

Error: could not fetch elasticache configuration.
'NoneType' object has no attribute 'get_caller_identity'

Error: could not fetch route53 configuration.
'NoneType' object has no attribute 'get_caller_identity'

Error: could not fetch sns configuration.
'NoneType' object has no attribute 'get_caller_identity'
Traceback (most recent call last):
  File "c:\python27\Scripts\Scout-script.py", line 11, in <module>
    load_entry_point('ScoutSuite==4.0.5', 'console_scripts', 'Scout')()
  File "c:\python27\lib\site-packages\ScoutSuite\__main__.py", line 101, in main
    cloud_provider.fetch(regions=args.regions)
  File "c:\python27\lib\site-packages\ScoutSuite\providers\base\provider.py", line 99, in fetch
    self.services = report.jsrw.to_dict(self.services)
  File "c:\python27\lib\site-packages\ScoutSuite\output\js.py", line 77, in to_dict
    return json.loads(json.dumps(config, separators=(',', ': '), cls=Scout2Encoder))
  File "c:\python27\lib\json\__init__.py", line 251, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "c:\python27\lib\json\encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "c:\python27\lib\json\encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
MemoryError

Not too sure what's going on.

Is this an issue on my end or something else?

Thanks.

Scout2 have any rules to find out any particular cloudwatch / cloudtrail metrics exist, if not then raise a flag.

For example, to check if any metrics related to IAM policy changes exist. If not exist, then need to raise a flag.

Not sure if it is possible but if it is, it would be nice to have the option to see which Security Policy is in use on the ELB/ALB for TLS Specfications

1

In the top bar, clicking once opens the menu but there's no way to close it without selecting an option:

There should be a way to close this (either a specific button and/or by clicking outside of the menu).

2

In the initial view, the run information (last run, datetime, ruleset) are shown at the bottom. This should be moved to an additional tab of the "about" section (top right) and should include some additional info (account ID, scope (account/project), provider, etc.).

3

In the Dashboard section, clicking "+" expands the issue, should then show "-" to collapse:

4

In the service issue pages, it's unclear that issue titles are links:

Moving the mouse over an issue should underline it (as for the main dashboard), to show it's a link.

5

This left sidebar looks a bit heavy

• Update Bootstrap to 4.1.3 (currently uses 3.0.3)
• Update jQuery to 3.3.1 (currently uses 1.11.2)

Use Bootstrap colors for titles (https://getbootstrap.com/docs/4.0/utilities/colors/). This is especially important as text-on-titles doesn't show very well:

One of the issues flagged by ScoutSuite is "Bucket world-listable (anonymous)". The way to verify the issue is to access the S3 bucket through the web browser, so it would be helpful to have a direct link to the bucket in the issue report.

A large percentage of issues do not have descriptions. A description should be added to all issues.

Hi,

we are using the latest master version from GIT. ( checked out yesterday).
However, we are not able to see the regions in CloudTrail.
Neither in the top navigation (Management->CloudTrail) where the count is just empty "Regions ()"
nor on the CloudTrail page itself in the left side menu.

It looks like something does not get properly propagated.
I checked the metadata object in the aws_config JS file and determined, that the count attribute in metadata.management.cloudtrail.resources.regions seems to be missing.
(it does exist for the trails resource).

Additionally, when inspecting the elements in the left side navigation, it kind of propagates the list entries, but as the regions are missing, it won't display in the browser.
services.cloudtrail.regions..link is found within all those list elements, as you can see the region part is missing.

Thanks for your support!

Florian

Hi, I'm attempting to extend Scout2 to pull data from ResourceGroupsTaggingAPI (see: http://boto3.readthedocs.io/en/latest/reference/services/resourcegroupstaggingapi.html). Following the documentation on the wiki to add a new service, there are no results. Here's the code I've added in my attempts:

diff --git a/AWSScout2/configs/data/metadata.json b/AWSScout2/configs/data/metadata.json
index ab814a8..3c0d9ff 100644
--- a/AWSScout2/configs/data/metadata.json
+++ b/AWSScout2/configs/data/metadata.json
@@ -15,6 +15,18 @@
             }
         }
     },
+    "api": {
+        "resourcegroupstaggingapi": {
+            "resources": {
+                "ResourceARN": {
+                    "api_call": "get_resources",
+                    "cols": 2,
+                    "response" : "ResourceARN",
+                    "path": "services.resourcegroupstaggingapi.regions.id.tags"
+                }
+            }
+        }
+    },
     "management": {
         "cloudformation": {
             "resources": {

diff --git a/AWSScout2/configs/services.py b/AWSScout2/configs/services.py
index 8862254..4f10433 100644
--- a/AWSScout2/configs/services.py
+++ b/AWSScout2/configs/services.py
@@ -13,6 +13,7 @@ from AWSScout2.services.elasticache import ElastiCacheConfig
 from AWSScout2.services.elb import ELBConfig
 from AWSScout2.services.elbv2 import ELBv2Config
 from AWSScout2.services.emr import EMRConfig
+from AWSScout2.services.resourcegroupstaggingapi import ResourceGroupsTaggingApiConfig
 from AWSScout2.services.iam import IAMConfig
 from AWSScout2.services.awslambda import LambdaConfig
 from AWSScout2.services.rds import RDSConfig
@@ -44,6 +45,7 @@ class ServicesConfig(object):
     def __init__(self, metadata, thread_config = 4):

         self.cloudformation = CloudFormationConfig(metadata['management']['cloudformation'], thread_config)
+        self.resourcegroupstaggingapi = ResourceGroupsTaggingApiConfig(metadata['api']['resourcegroupstaggingapi'], thread_config)
         self.cloudtrail = CloudTrailConfig(metadata['management']['cloudtrail'], thread_config)
         self.cloudwatch = CloudWatchConfig(metadata['management']['cloudwatch'], thread_config)
         self.directconnect = DirectConnectConfig(metadata['network']['directconnect'], thread_config)

resourcegroupstaggingapi.py:

# -*- coding: utf-8 -*-

import json

from AWSScout2.configs.regions import RegionalServiceConfig, RegionConfig



########################################
# ResourceGroupsTaggingApiRegionConfig
########################################

class ResourceGroupsTaggingApiRegionConfig(RegionConfig):


     pass


########################################
# ResourceGroupsTaggingApiConfig
########################################

class ResourceGroupsTaggingApiConfig(RegionalServiceConfig):
    region_config_class = ResourceGroupsTaggingApiRegionConfig

    def __init__(self, service_metadata, thread_config = 4):
        super(ResourceGroupsTaggingApiConfig, self).__init__(service_metadata, thread_config)

Example execution:

# ./Scout2.py --services  resourcegroupstaggingapi
Fetching ResourceGroupsTaggingAPI config...
             regions        ResourceARN
                 0/0                0/0
Processing CloudTrail config...
Matching EC2 instances and IAM roles...
Saving data to scout2-report/inc-awsconfig/aws_config.js
Saving config...
Saving data to scout2-report/inc-awsconfig/exceptions.js
Saving config...
Creating scout2-report/report.html ...
Opening the HTML report...
# cat scout2-report/inc-awsconfig/aws_config.js  | tail -n +2 | jq '.services.resourcegroupstaggingapi'
{
  "ResourceARN_count": 0,
  "filters": {},
  "findings": {},
  "regions": {},
  "regions_count": 0,
  "resource_types": {
    "global": [],
    "region": [
      "ResourceARN"
    ],
    "vpc": []
  },
  "service": "resourcegroupstaggingapi",
  "targets": {
    "first_region": [
      [
        "ResourceARN",
        "ResourceARN",
        "get_resources",
        {},
        false
      ]
    ],
    "other_regions": [
      [
        "ResourceARN",
        "ResourceARN",
        "get_resources",
        {},
        false
      ]
    ]
  },
  "thread_config": {
    "list": 10,
    "parse": 20
  }
}

The id of the subnet as well as that of the instances in that subnet are empty: