Comments (13)
@excid3 how to improve the bulk insert with the database option using Rails 5.2?
Trying to send 11K notifications without slow down the app
from noticed.
I'm leaving for a couple weeks on vacation, but I'll try and check this out when I get back.
from noticed.
I think this would make more sense to be a bit more like:
delivery_by :postmark # Sends individual emails
delivery_by :postmark, bulk: true # Sends them in bulk
class DeliveryMethods::Postmark
def deliver
end
def bulk_deliver
recipients.each_slice(1000) do |group|
# send group
end
end
end
That way you could still use Postmark to delivery individually or in bulk.
The question I have is...how does Postmark know when to actually send in bulk? It'd need to queue up the recipients somehow know when it was ready to run.
from noticed.
@excid3 yeah that's nice, I like the idea of being able to support both single and batch delivery in a single delivery method. 👍
One consideration is resiliency to failure. By putting the each_slice
inside the bulk_deliver
method, if one of the API calls was to fail, due to network issues or something, it would break the subsequent slices of the batch, and also make it very difficult to retry.
If the each_slice
was to happen before enqueueing, then each slice would be independent of one another and could be caught with ActiveJobs retry_on
in the base job class and retried.
As for your question, Postmark, Sendgrid, Mailgun and just about every other mail provider I've come across deal with bulk sending via a single POST request, with up to 1,000 email addresses in the payload, they'll then split that list, queue and deliver everything once it's on their server.
Does that make sense?
from noticed.
If the
each_slice
was to happen before enqueueing, then each slice would be independent of one another and could be caught with ActiveJobsretry_on
in the base job class and retried.
Agreed. 👍
Maybe the option becomes something like this where you can specific the size of each slice.
delivery_by :postmark, bulk: { group_size: 1000 }
Then this would queue up X number of Postmark bulk delivery job (3 jobs if it had 3000 recipients).
In theory, this wouldn't change too much of the existing functionality, just add to it.
from noticed.
@SirRawlins I added an implementation in #127 🙇. Let me know your thoughts!
from noticed.
@matteomelotti I saw this same issue when sending to a large number of recipients, the deliver_later method runs very slow while it adds all the jobs to the queue.
I pulled the call to deliver_later
into its own background task, so it doesn't slow down the request for the user.
from noticed.
@manojmj92 awesome, I'm sorry for the slow reply, I've not been working with Noticed over the past few weeks. Your PR looks good on the face of things but will leave it for @excid3 to review properly when he gets time.
from noticed.
@excid3 great. Have an awesome vacation, it's well deserved.
from noticed.
@excid3 Any updates on the review? 😄 Would love to use noticed in a project of mine, #127 will really help.
from noticed.
Any update?
from noticed.
I would love to see this, as I'm implements Expo notifications, which supports batch delivery.
Just one caveat, that I haven't seen considered - it might be useful to consider batching by i18n language, so you can render the notification by user language. Possibly something to add to a future documentation of #127 ?
from noticed.
@excid3 Could you review this? Seems like an essential feature!
from noticed.
Related Issues (20)
- [BUG] Can't use Pathname credentials in FCM delivery method, when defining them in a method
- V2 Delivery Method Config Options Presence Validation? HOT 4
- Not able to get recipient in the mailer method HOT 1
- recipient id vary HOT 5
- Noticed v2 Update Instructions / Sample Code HOT 29
- Twilio messaging will no longer work after their migration to content templates for Whatsapp HOT 1
- [BUG] Rspec running infinitely HOT 5
- [BUG] failing zeitwerk check
- [BUG] FCM Delivery Method errors when config[:json] is a method HOT 1
- [BUG] Inaccurate recipient_type class name for STI model, when delivering the notification
- Rails crashes when not using ActionCable on 2.0.2 and above. HOT 6
- [BUG] Second migration file not necessary when installing noticed on a new app
- [BUG] Updating to ~> 2.1 - Unknown migration version "7.1"
- [BUG] Getting an undefined method gsub error when trying to use the twilio_messaging delivery method upon upgrade to V2 HOT 2
- [BUG] Params not being forwarded to mailer HOT 12
- `notification_methods` not working when used on `ApplicationNotifier` HOT 1
- [BUG?] Cannot create Noticed::Event in MariaDB HOT 1
- Unclear section in the upgrade guide about polymorphic record association
- [BUG] Forbidden nil access when using action_cable default channel HOT 14
- [BUG] NoMethodError .to_notifier HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from noticed.