Possible DB Performance Issue about mailtrain HOT 11 CLOSED

I'll try to replicate this. There are two possible reasons for slow sending, either there's something wrong in database or delivering to STMP. Btw. which SMTP provider do you use?

from mailtrain.

We use AWS SES for sending and our rate allowed supports up to 1m day.

I was able to notice the app server was not that busy during this period. It was running less than 10% CPU on a AWS EC2 t2.nano. We were using AWS RDS MySQL t2.micro for the database and increasing to t2.medium definitely improved performance. However, the rate seemed far below what you'd expect thus, my feeling it being a unintended bottleneck of some kind in the DB like a missing index or untuned query.

from mailtrain.

Ok, sounds like a db issue indeed. I'll create a large subscriber list in my test instance and send messages to a local SMTP that forwards these to /dev/null. This way I should end up in the same situation if there's something going on with the db

from mailtrain.

Yes, I can definitely confirm that the sending gets slower in time, here's a graph for a 21 minute run:

I'll try to find out from where the delay actually comes from

from mailtrain.

The culprit turned out to be the query to find next unsent message. I updated it to fetch several messages at once instead of doing it one by one. Once you have sent a large amount of messages this query gets slower since checking unsent status takes more time. The updated code in master is now optimized. I did not tag it yet since I broke "pause" button – pause stops fetching new messages but if you have already cached a list of unsent messages then these cached messages are sent regardless if you pressed paused or not.

from mailtrain.

Excellent! I'll pull the latest and give it a try. Thank you.

from mailtrain.

I did some additional tweaks and tagged Mailtran as version v1.10.1. The database performance is much better and such slow downs should not happen anymore

from mailtrain.

Excellent! I've updated and will send another campaign through this coming weekend.

from mailtrain.

Good news. The performance was increased and I was able to send entire list within one day using the new version.

from mailtrain.

Awesome!

from mailtrain.

Hi @andris9 ,

We are facing this exact same issue. We sent out a campaign to 800k subscribers, initial 60-70k mails got sent really fast, but after that it was like 2-3 mails per second. What could be the issue? Version in package.json is 1.23.2

from mailtrain.