fixthestatusquo / lookup-lc Goto Github PK

https://github.com/fixthestatusquo/lookup-lc/blob/main/src/fetch.ts#L57

no need to read, if not there write, then write, then check

just write

this should be initialised once and used by fetch and update

const db = new Level('./emails.db', { valueEncoding: 'json' });

so simplify fetch

• get the source file from trust
• check if correct (eg not empty)
• db.clear
• loop over every item and write them (no need to check, it should be there already)

and voila, we have a tiny (<0.3 sec) between db.clear and the end of the loop where a lookup can happen AND possibly not be paused until the loop is finished AND be an email that should be there AND that hasn't been already

it's an acceptable limitation

now you are running the upgrade on cron, it's the good production solution, but you need to be able to trigger the upgrade manually, otherwise dev will be painful.

so add a new http endpoint (next to the one used for lookup) "fetch" (post) that calls the same code than the one in the cronjob

We use Level DB for lookup. The problem is - two processes can't access it.
Processes:

• Lookup - can run any time
• Update database - once a day.

How to solve this?