Data import: maintenance mode during data update about gbif-alert HOT 8 CLOSED

I definitely agree with you.
If we run the data import at night (I would say midnight) as you said AND the early warning system alerts are sent at 2pm for example, then we have time to check what is going wrong.

from gbif-alert.

Yeah. But I'll also implement also a check in the "sending alert" code to make sure they just it just waits if maintenance mode is on (so we don't break everything because we decided to change hours or because the server time is incorrect)

from gbif-alert.

Yes, for sure! 👍

from gbif-alert.

The occurrence import is now atomic (#62). If all goes well, we should be able to tackle this simply and elegantly using the following logic:

1. new data is downloaded from GBIF
2. maintenance mode is set to ON (website inaccessible + no emails sent)
3. the occurrence import is done (atomically: all changes or nothing in the database)
4. if the transaction was aborted: send an email to administrators to check what happened
5. in all cases, the maintenance is set to OFF (in worse case - transaction aborted - no new data will be visible in the website but at least the data is still perfectly consistent and the website isn't stuck in maintenance mode)

from gbif-alert.

I find it a good workflow, it's defensive enough, I think. One small question, maybe trascurable problem: what to do with downloaded data in 1 if 4 happens for many consecutive days (e.g. you and your hpothetic back-up person are both on holidays)? Probably memory it's not a big issue, right?

from gbif-alert.

@damianooldoni: Indeed, I don't expect any disk space problem:

• The downloads are quite small (with the current species list, we're at 7 Mb / day)
• In addition to that, the download happens in a temporary directory: as soon as the processes finish, the system knows this disk space can be freed when necessary.

from gbif-alert.

Update: I'm considering using https://github.com/fabiocaccamo/django-maintenance-mode for the maintenance mode itself.

from gbif-alert.

This is working fine for years, IU think we can close this!

from gbif-alert.