'remap jar to gradle' causes quadratic dependency resolution about eclipse-integration-gradle HOT 13 CLOSED

Thanks, will need to take a look at that. We'll have to do something to how these models are cached / refreshed.

from eclipse-integration-gradle.

This is not our 'real' issue tracker so I raised issue there:
https://issuetracker.springsource.com/browse/STS-3952

Closing this as a duplicate.

from eclipse-integration-gradle.

Just in case you are not watching the Jira issue. I have resolved it as 'fixed'. I beleave it is fixed, but I can not be 100% sure my it would great if you can confirm that independently. So if you have a chance to try out a nightly build again, please do.

PS: I have also disabled the Jar -> Gradle mapping option by default now. As I think it is better not to risk serious problems caused by it in the upcoming release.

So you'll have to turn it on in the preferences before you can test it.

from eclipse-integration-gradle.

I've been checking the Jira issue irregularly every few days. Thank you for the ping nonetheless :)

Unfortunately, I cannot say this is solved for 3.6.3.201411192101-CI-B321 (which, guessing it's using UTC timestamps, includes everything up to commit adb05a4). Still, I'm selecting all 4 projects (root and the three sub projects), right-click, Gradle, Refresh Dependencies. What I (maybe) didn't notice before is that I have 5 consoles opened for that: Building EclipseProject Gradle Model '/.../root' and one ProjectPublications for the root and the three subprojects I have. They all have their own "BUILD SUCCESSFUL Total time: x secs" log message.
It looks to me that because I have multiple projects selected, the process of establishing the publications is done for each project individually instead of just once centrally.

If I only refresh the root project's dependencies, all projects are touched once and nothing else. This doesn't update the publications, though.
If, on the other hand, I refresh a single submodule, the root has its EclipseProject information rebuilt and the selected project's ProjectPublications get's renewed as well (second evaluation). Thus, refreshing multiple projects at once causes the complexity because siblings need to be considered in a multi module setup.

Would providing a tarball of the test setup help to resolve this?

To end this on a positive note, the workaround of disabling jar remapping still works flawlessly.

from eclipse-integration-gradle.

It looks to me that because I have multiple projects selected, the process of establishing the publications is done for each project individually instead of just once centrally.

THat is correct and unfortunately it is unavoidable. The publications model can only be built one project at a time, it is just how the tooling API works, there simply is no way to ask it to build models for a hierarchy all at once.

I did fix a number of problems with how builds get scheduled (there was indeed a serious blow up in the same things getting built multiple times).

The question is if you are still seeing the 'quadratic blow up'. I hope not.

To be clear, if you have 4 projects in a hierarchy and refresh them all then this is the expected behavior re model builds:

• 1 build to produce 'EclipseProject' model for all four projects at once, built via the root project
• 4 builds to produce 'ProjectPublications' model. One build for each project.

A total of 5 builds. So the number of builds should be 'linear' (n+1) w.r.t to the number of projects (n).

If that is what you are seeing then it works 'as designed'.

from eclipse-integration-gradle.

Would providing a tarball of the test setup help to resolve this?

Yes, it would help. Assuming it doesn't work as I described above. But I will setup something like you described and try it myself today. See if I get quadratic or linear # of builds.

from eclipse-integration-gradle.

The publications model can only be built one project at a time, it is just how the tooling API works, there simply is no way to ask it to build models for a hierarchy all at once.

That's unfortunate for any multiproject case, but not the this plugin's fault.

A total of 5 builds. So the number of builds should be 'linear' (n+1) w.r.t to the number of projects (n).

Ah, I now see where my confusion stems from. I completely understand the logic and reason behind this. My original way of thinking was that if I have (n=)4 projects, I was looking at how many individual projects get evaluated/executed in total. What is still happening (due to the tooling API, as you pointed out) is that all 4 projects are evaluated wrt their publications. Because they are part of a multiproject build, though, their siblings need to be evaluated as well to get the whole picture. Thus, I come up with (n)_(n+1) = 20 timestamps, whereas (1+1)_n = 8 was what I would have wished for (1 for EclipseProject, 1 for ProjectPublications for all individual modules).

As pointed out by the Gradle 2.2 daemon documentation model caching is a feature to be developed in the future. Maybe this will cover what I was originally looking for.

To sum it up, I can fully agree that the plugin works as designed. Thus, this issue (and STS-3952) can stay closed. Thank you very much for looking into this! 👍

from eclipse-integration-gradle.

I come up with (n)*(n+1) = 20

Okay at first this didn't make sense to me at all, but I think I understand it now. You are putting some code inside the build scripts of your plugins to 'detect' when they get evaluated.

So from STS point of view we have only have 'n+1' builds which is correct. But the 'ProjectPublications' build (n of the builds) each end up evaluating the siblings build scripts. So I see where you come up with 'quadratic' here.

But unfortunately... yes this is still... 'works as designed'.

from eclipse-integration-gradle.

We may be able to improve this by using 'BuildActions' as discussed here:
http://forums.gradle.org/gradle/topics/gradle-eclipseproject-and-buildinvocations-model-for-the-same-project-have-different-tasks-for-projects

Unfortunately 'BuildActions' api is broken for us and as it seems we cannot use is in osgi environment.

However... if could use it... then we could indeed build the ProjectPublications models for a hierarchy in one go. I have a little experiment that does this here:

https://github.com/kdvolder/gradle-tooling-api-buildaction-example

from eclipse-integration-gradle.

You are putting some code inside the build scripts of your plugins to 'detect' when they get evaluated.
Correct. Sorry for the hassle it caused on your part.

BuildActions being available sounds wonderful! Thank you for keeping me posted on this. Running your example for my test setup, everythings seems to indeed only be evaluated once.

I'm guessing it will be some time for gradle 2.3 to appear, let alone STS support for this. From what I can see, it should be fixed by 2.3-20141120230022 wrt an OSGi environment.

from eclipse-integration-gradle.

Correct. Sorry for the hassle it caused on your part.

No problem, it was actually a useful interaction as it made me investigate 'BuildActions' as a possible future solution for this. I do think the current solution is problematic unless you have just a few 'toy' projects in the workspace.

from eclipse-integration-gradle.

Oh, boy! Are you serious about this?
"I do think the current solution is problematic unless you have just a few
'toy' projects in the workspace."
I have more than 300 projects. Do you think it is feasible to handle this
size with the current tooling?
Thanks
On Nov 25, 2014 9:13 PM, "Kris De Volder" [email protected] wrote:

Correct. Sorry for the hassle it caused on your part.

No problem, it was actually a useful interaction as it made me investigate
'BuildActions' as a possible future solution for this. I do think the
current solution is problematic unless you have just a few 'toy' projects
in the workspace.

—
Reply to this email directly or view it on GitHub
#57 (comment)
.

from eclipse-integration-gradle.

I have more than 300 projects. Do you think it is feasible to handle this size with the current tooling?

It will be fine if you don't enable the 'jar remap to gradle project' option. It is actually disabled by default. If you do not enable this option, then no ProjectPublications are needed and all is going to be okay.

If you do enable it then it could take a while building 'ProjectPublications' models for these 300 projects. It all happens in background so it should still be usable. But I personally found it is just a bit too disturbing to turn it on by default. Especially if many people won't care to have 'jar remapping' and it will just be annoying to them to wait for the model builds.

So by all means, give it a try... and you tell me how bad it is :-)

If I can use 'BuildActions' we can do it a lot better (but only if your 300 projects are part of a few large hierarchies).

For an idea of the difference. My little experiment used 'spring-xd' which is a large multi-project (110 projects in the hierarchy).

Each of the 110 ProjectPublications models takes 3 to 4 seconds to build so that is 6 to 7 minutes for all of them. It all happens in 'background' jobs which don't lock things down... but it is long... and disturbing nonetheless.

In contrast, building all 110 models in one go with a single BuildAction takes only 6 seconds in total!

So in light of that, 7 minutes of 'building stuff' is not acceptable really.

from eclipse-integration-gradle.