`tilt ci` does not recreate failed jobs about tilt HOT 1 OPEN

oof, this is tricky. The short version is that this is currently working as designed.

re: "I can't find any docs that say it should work this way" - here's a good doc on tilt's execution model - https://docs.tilt.dev/controlloop. Basically, you can think of it as docker build && kubectl apply && kubectl wait. tilt ci mainly adds exit conditions.

The fundamental problem is that if you kubectl apply a job, and the spec of the Job hasn't changed, then (from Kubernetes' perspective), there's no reason to re-run the job. From the apiserver perspective, the whole contract of apply if that if the spec of an object hasn't changed, then the system should do nothing.

Tilt inherits this behavior -- if the Job hasn't changed, then the Job shouldn't be re-run.

There have been discussions of this over the years (e.g., kubernetes/kubernetes#77396), but lots of stuff relies on this behavior.

I guess the simple workaround right now is to add to your tiltfile like:

if config.tilt_subcommand == 'ci'):
  local('./clean-up-old-jobs.sh')

though i agree that's unsatisfying :(

from tilt.