clario-tech / s3-inspector Goto Github PK

Traceback (most recent call last):
File "s3inspector.py", line 106, in install_and_import
importlib.import_module(pkg)
File "C:\Program Files (x86)\Python\Python37-32\lib\importlib_init_.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'termcolor'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "s3inspector.py", line 109, in install_and_import
pip.main(["install", pkg])
AttributeError: module 'pip' has no attribute 'main'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "s3inspector.py", line 335, in
main()
File "s3inspector.py", line 329, in main
install_and_import(package)
File "s3inspector.py", line 111, in install_and_import
globals()[pkg] = importlib.import_module(pkg)
File "C:\Program Files (x86)\Python\Python37-32\lib\importlib_init_.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'termcolor'

Hi

Thanks for the awesome code, all works a treat when there are public buckets in the accounts, however, when there are none I receive the following error:

[Errno 2] No such file or directory: '/tmp/report.txt': IOError
Traceback (most recent call last):
File "/var/task/s3inspector.py", line 236, in lambda_handler
send_report(report_path)
File "/var/task/s3inspector.py", line 253, in send_report
with open(path, "r") as f:
IOError: [Errno 2] No such file or directory: '/tmp/report.txt'

Lines 236, and 253 respectively...

236 send_report(report_path)
253 with open(path, "r") as f:

Is this something someone can help with?

Cheers, Jay

It's possible to create a machine parsable output ?

It's "Northern Virginia", not "North Virginia"

Hello and thanks for your code!

Do you plan to release as an open source license?

We use tags to indicate what project an S3 bucket belongs to (for cost analysis) and who owns or is responsible for it. It would be useful to print out a list of tags for public buckets.

Add header to Python script:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

It's possibile to add a "compliance mode" , which only raises warning only if a public bucket has been found .
Paired with #8 it would be great for automation

I ran this against my account. I have a few public buckets set up to be static web sites. That means the bucket name is something like www.example.com and so it's URL is https://www.example.com, not https://www.example.com.s3.amazonaws.com. I got this error when it tried to report on my bucket:

Bucket www.example.com: PUBLIC!
Location: eu-west-1
Permission: readable by Everyone
Traceback (most recent call last):
  File "s3inspector.py", line 332, in <module>
    main()
  File "s3inspector.py", line 328, in main
    analyze_buckets(s3, s3_client)
  File "s3inspector.py", line 197, in analyze_buckets
    urls = scan_bucket_urls(bucket.name)
  File "s3inspector.py", line 132, in scan_bucket_urls
    content = requests.get(url).text
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/adapters.py", line 497, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: hostname 'www.example.com.s3.amazonaws.com' doesn't match either of '*.s3.amazonaws.com', 's3.amazonaws.com'

Sometimes public buckets are intended to be public because they're the static content part of a serverless web site. So

1. This script needs to not blow up on them
2. This script needs to account for that. Sometimes public buckets are deliberate. They have some good use cases.

I think this is a limitation please correct me if I am wrong, this tool only checks for the ACL of the bucket for public access and does not check the bucket policy?

It's possible to add one option to whitelist some already known public buckets?

S3 buckets that are Public showing not public.

Is there any option to use profiles, so we can assume a role from another account, or can it only call default credentials?

Steps to reproduce:

• python ./s3inspector.py

Seen:

  File "./s3inspector.py", line 336, in <module>
    main()
  File "./s3inspector.py", line 331, in main
    s3, s3_client = get_s3_obj()
  File "./s3inspector.py", line 47, in get_s3_obj
    access_key = raw_input("Enter your AWS access key ID: ")
NameError: name 'raw_input' is not defined

Other info:
Using python 3.6.5. I can see in main() (line 326) that raw_input should have the built-in function input() assigned to it. assert(raw_input) in main() does not fail but in get_s3_obj() the assertion fails.

Possible solution:
Put global raw_input at the start of def main(). Assignments in a function scope do not always propigate to sub-functions.

Hi,

I have created public bucket to test the script and observed that public bucket is showing as Not public in scan.

What could be wrong here ?

Just scanned my buckets, all are ok, but failed on the last one www.gemshelf.com

Looks like regex issue? Here is the stacktrace

Traceback (most recent call last):
  File "s3inspector.py", line 109, in <module>
    urls = scan_bucket_urls(bucket.name)
  File "s3inspector.py", line 47, in scan_bucket_urls
    content = requests.get(url).text
  File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 506, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.gemshelf.com.s3.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'www.gemshelf.com.s3.amazonaws.com' doesn't match either of '*.s3.amazonaws.com', 's3.amazonaws.com'",),))