Proposal: Pipeline-loops Custom Tasks to support common loops cases about community HOT 12 CLOSED

Thanks @fenglixa - this custom controller is more feature reach than the existing task loop, and it allows us to experiment looping over pipelines using a single custom controller. I look forward to experimenting with this!

from community.

I think it's useful feature. although we already have pipeline-pipeline and task-loop.
This feature is specific for pipeline-loop and consider more scenarios which is only for pipeline-loop, and handle many gaps in task-loop and pipeline-in-pipeline, I like it, thanks @fenglixa

from community.

/cc @dibyom

from community.

I want to stress that in the long run I think the use cases that are being expressed here should be supported by a top level looping feature in Pipelines + pipelines in pipelines - looking at the outlined loops "typical loop", "multi args loop" and "condition loop" I don't see anything that we wouldn't want to support with a generic "looping" concept that could be applied to task as well.

My one concern about pursuing this as an experimental project is that folks who want this functionality will start to rely on the experimental project and will not be motivated to continue improving the existing experimental looping project - is there any chance that instead of pursing this experimental project, you might be interested in helping get the existing loops project promoted as a top level feature in Tekton Pipelines @fenglixa ? (which would cover the use case you have outlined as well) Totally understand if not since that's probably more work and would mean a bunch of back and forth discussions with the community to nail down the features - but it WAS a possibility, I would definitely prefer that to introducing another experimental project that does looping.

(That said I won't block adding this as an experimental project)

from community.

looking at the outlined loops "typical loop", "multi args loop" and "condition loop" I don't see anything that we wouldn't want to support with a generic "looping" concept that could be applied to task as well.

I just realized that "condition loop" might be an exception - that would be a bit more controversial since it would mean introduce non-deterministic looping and i think might push us over the edge into being turing complete 🤔

from community.

so given that, maybe this does make the most sense as an experimental project 🤔
(sorry for the stream of consciousness, thanks for your patience 🙏 )

from community.

Thanks @bobcatfish @vincent-pli @afrittoli , for your support!

from community.

I'd rather experiment with this, thus a custom task in experimental works fine.

My one concern about pursuing this as an experimental project is that folks who want this functionality will start to rely on the experimental project and will not be motivated to continue improving the existing experimental looping project.

I don't really see this as a problem. Having it as an experimental project, and a custom task is quite the "barrier" for people to use it, which means if there is a high need of this in a supported way, we will have request for it, I have no worries. Having it in an experimental project and in custom task allows us to iterate quickly over it without having to be blocked by the core (saying this because, as of today, before we get feature flags into tektoncd/pipeline any big feature like this would be blocked and would move forward quite slowly).

from community.

@afrittoli do we need to create a TEP for this proposal to unblock tektoncd/experimental#724? thanks.

from community.

I agree to put into experimental first. We want to get some usage on this feature so we can get some feedbacks on the API definition.

from community.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

from community.

Definitely not stale since we've got https://github.com/tektoncd/experimental/tree/main/task-loops !

/remove-lifecycle stale

Also I think since we've got the project now, it makes sense to consider this particular issue handled. Though I am still really really hoping we'll promote this to supported Pipeline syntax and not just an experimental task! 🤞

from community.