digdag can't delete project about digdag HOT 2 CLOSED

Digdag should have a background procedure that deletes old sessions periodically. it's not implemented but expected by design.

from digdag.

implementation idea 1:

• add state_flags and obsolete_name to projects table.
• make projects.name nullable.
• when user deletes a project, digdag does following works in a single transaction:
  • copies name to obsolete_name.
  • set NULL to name.
  • set ProjectStateFlags.DELETED to state_flags.
  • delete all schedules associated to the project.
• adding a new revision should be rejected if state_flags is DELETED (at ProjectControl.insert* methods because having ProjectControl ensures that the record is locked with FOR UPDATE).

This architecture makes it easy to have UNIQUE INDEX on name column. Looking up a project by name doesn't return logically deleted projects but looking up by id returns one. This is a naturally expected behavior.

Future plan 1:

• add a background thread that does following work:
  • delete old sessions and session_attempts older than timestamp T at most N records
  • if number of deleted records is less than N, there're no sessions or session_attempts older than timestamp T. in this case, start deleting projects whose state_flags is DELETED and updated_at is older than T.
  • deleting project consists of following work:
    • delete task log files from storage
    • delete revision archive files from storage
    • delete workflow_configs using multiple transactions
    • delete workflow_definitions using multiple transactions
    • delete revisions using multiple transactions
    • delete project

from digdag.