Frequent ConcurrentModificationException on running SCP updates about aws-control-tower-customizations HOT 7 CLOSED

@stumins
I had the same error as above. Thanks for the improvement in v2.7.0.

However, in v2.6.0, the same error rarely occurs not in EnablePolicyType operation but in UpdatePolicy operation, as shown in the error statement below.

{
  "errorMessage": "An error occurred (ConcurrentModificationException) when calling the UpdatePolicy operation: AWS Organizations can't complete your request because it conflicts with another attempt to modify the same entity. Try again later.",
  "errorType": "ConcurrentModificationException",
  "stackTrace": [
    "  File \"/var/task/state_machine_router.py\", line 218, in lambda_handler\n    return service_control_policy(event, function_name)\n",
    "  File \"/var/task/state_machine_router.py\", line 77, in service_control_policy\n    response = scp.update_policy()\n",
    "  File \"/var/task/cfct/state_machine_handler.py\", line 882, in update_policy\n    response = scp.update_policy(\n",
    "  File \"/var/task/cfct/aws/services/scp.py\", line 80, in update_policy\n    response = self.org_client.update_policy(\n",
    "  File \"/var/task/botocore/client.py\", line 391, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n",
    "  File \"/var/task/botocore/client.py\", line 719, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"
  ]
}

Since there appears to be no modification to the update_policy method in this update, I fear that the UpdatePolicy operation will continue to generate ConcurrentModificationException in v2.7.0.

v2.6.0

v2.7.0

I haven't been able to update yet and have not been able to confirm this issue with v2.7.0, but are you aware of this issue?

Are you aware of this issue and if so, do you plan to fix it?

from aws-control-tower-customizations.

Further thoughts for your consideration:

• Please consider not running EnablePolicyType at all. IMHO, it should be obvious that SCP support is supposed to be enabled in Organizations before starting to employ this solution. It could be in the installation docs.
• The loop keeps going after this failure occurs and does all the SCPs anyway. I'd expect it to cut short and fail instead. At least that way we can retry a lot sooner. (we have 35 SCPs b.t.w.)

from aws-control-tower-customizations.

Hi @Cihl28,

Thanks for reporting this bug. We are aware of the issue and have scheduled a patch for the next release.

from aws-control-tower-customizations.

Hi @Cihl28,

CFCT v2.7.0 was just released with a patch for this issue. Please upgrade to v2.7.0 and let us know if you continue to experience this error.

from aws-control-tower-customizations.

Just noticed this. I'll make plan to have this update applied soon.

from aws-control-tower-customizations.

I can now confirm that this issue is no longer present for me in v2.7.0.
Thanks for fixing it!

from aws-control-tower-customizations.

I'll just go ahead and close this issue.

from aws-control-tower-customizations.