Stack is going to Create_Complete but there is no output in the built csv files in the s3 bucket about cost-optimization-ec2-right-sizing HOT 4 CLOSED

Added pull request suggestions and pushed changes. Also, logs are now exported to CloudWatch Logs to the cost-optimization-ec2-right-sizing log group to make troubleshooting easier.

from cost-optimization-ec2-right-sizing.

Please try and run the version located here: https://s3.amazonaws.com/solutions-reference/cost-optimization/latest/cost-optimization-ec2-right-sizing.template

If you look in the template, the script that runs the commands is /tmp/perform-rightsizing.sh, So log into the EC2 instance, sudo su - and run the script yourself to see where it fails.

from cost-optimization-ec2-right-sizing.

I suspect that issue #5 may be related, though I don't know for certain. Since there has been no traffic on this issue, I am adding what I am seeing, as it seems that I get stuck on EC2 instance generation, and when it does make it through to CREATE_COMPLETE, nothing ends up in the S3 bucket (this appears to be intermittent behavior; sometimes it completes, sometimes it does not, and when it does complete, nothing is created in the S3 bucket).

Below is the last 10 entries of output from the script (and logged to /tmp/run-rightsizing-redshift.log as well):

INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): monitoring.us-west-2.amazonaws.com INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): monitoring.us-west-2.amazonaws.com INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): monitoring.us-west-2.amazonaws.com INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): monitoring.us-west-2.amazonaws.com INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): monitoring.us-west-2.amazonaws.com INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): monitoring.us-west-2.amazonaws.com INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): monitoring.us-west-2.amazonaws.com INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): monitoring.us-west-2.amazonaws.com Traceback (most recent call last): File "./run-rightsizing-redshift.py", line 423, in <module> ls_cwfile = callgcw.call_gcw(CW_REGION, CW_ACCOUNT, CW_MODE, CW_STATISTICS, CW_PERIOD, CW_STARTTIME, CW_ENDTIME, CW_OUTPUT) File "/tmp/callgcw.py", line 64, in call_gcw getcloudwatchmetrics.download_metrics(ls_region_array, account, mode, statistics, period, startTime, endTime, ls_outputfile_name) File "/tmp/getcloudwatchmetrics.py", line 196, in download_metrics response = p2.map(func, response) File "/usr/lib64/python2.7/multiprocessing/pool.py", line 251, in map return self.map_async(func, iterable, chunksize).get() File "/usr/lib64/python2.7/multiprocessing/pool.py", line 567, in get raise self._value KeyError: 'DiskReadOps' INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): monitoring.us-west-2.amazonaws.com

Hopefully there is enough information available to assist in debugging the issue whatever it is.

from cost-optimization-ec2-right-sizing.

After debugging, I have submitted pull request #6 which seems to address the issue for me. It appears that some of my metrics do not have a key 'DiskReadOps' in them (maybe others, this was the one that errored out for me). When Python attempts to access a key in a dict and that key does not exist, it throws a KeyError, which is what appears to be going on.

I suspect that this issue and issue #5 are likely the same issue, as my CloudFormation stack is stuck on AWS::EC2::Instance CREATE_IN_PROGRESS, as is the entire stack formation, and no files ever make it to the S3 bucket. In my case, I suspect that the metrics in question relate to an EC2 instance that recently had it's instance type changed to allocate more memory to it, and that the metrics refer to the old instance type, which is no longer accumulating any metrics against it, though the instance type being reported in the metric is the new instance type. The metric in question is missing the DiskReadOps and DiskWriteOps keys, which is causing the script to bomb out when getcloudwatchmetrics.py blindly attempts to access them, assuming they are there.

from cost-optimization-ec2-right-sizing.