RequestLimitExceeded: EC2 fleet plugin making too many requests to DescribeSpotFleetRequests and DescribeInstances about ec2-fleet-plugin HOT 13 CLOSED

Thanks, will take a look.

from ec2-fleet-plugin.

Thanks! fyi, we're on latest.

Jenkins ver. 2.164.1
EC2 fleet plugin: 1.1.9

from ec2-fleet-plugin.

We've got same problem more than 45,693,448 Requests in April
Jenkins ver. 2.164.2 LTS
EC2 fleet plugin: 1.1.9

from ec2-fleet-plugin.

@jcalvilloitrs sorry to hear that you're also running into the same issue as us. For us, this is becoming a critical issue since all of our AWS calls are being used up.

Do you have more than one fleet setup?

from ec2-fleet-plugin.

@akennealy We had till yesterday 3 fleet and reduced to just only one

from ec2-fleet-plugin.

@schmutze, have you had a chance to take a look at this one?

I've cloned the repo and started looking at the code. Looks like a large number of the api requests are being made in com.amazon.jenkins.ec2fleet.EC2FleetCloud#updateStatus. I've added some logging and rebuilt the plugin, planning to deploy this to Jenkins.

from ec2-fleet-plugin.

@akennealy we updated all plugins (there is an update for another ec2 plugin) and restart jenkins, issue seems to be solved. there is no increase api petitions and bill keeps stable.

from ec2-fleet-plugin.

Thanks @jcalvilloitrs! I'll take a look.

I'm currently running a custom build of the plugin with a change I made to reduce the fleet check from running every 10 seconds to run every 60 seconds. As a workaround, this greatly reduced the AWS API requests being called!

from ec2-fleet-plugin.

@akennealy have you tried update suggested to @jcalvilloitrs or increased interval works fine for you? I think it makes sense to increase refresh interval by default and allow to configure it if someone really need 10 sec

from ec2-fleet-plugin.

I tried reverting my custom build with a increased interval from 10 sec to 60 sec. We very soon after starting seeing builds failing with AWS request limit exceeded. I'm going back to my custom build to increase the limit...

from ec2-fleet-plugin.

Hi @akennealy,

I think I found the problem and have a fix for it, the plugin doesn't need to reduce refresh time from 10 sec to 1 min instead it can use batch mode for EC2 API. Details in PR #75

It will be really nice if you can try it before release, branch check-terminated-in-batch thx

from ec2-fleet-plugin.

Hi @akennealy, @jcalvilloitrs fix released 1.2.2. I'm going to close issue, feel free to reopen it if you still see problems or have questions, thx

from ec2-fleet-plugin.

just in case if this problem still valid plugin

• 1.11.2 has significant reduction for API call rate
• 1.12.0 allows to configure interval time

from ec2-fleet-plugin.