Fails to load dependency graph when dependency uses Version Range about mill-github-dependency-graph HOT 14 OPEN

Alright, everything is updated @chrsmutti. If you're using v1 of the action you should be able to just try again and it will pull all the new stuff in. If that doesn't seem to, then just directly set the plugin-version in the GitHub action to use the latest 0.1.1.

from mill-github-dependency-graph.

Awesome, glad it works. I'm going to leave this open a bit because I'd still like to understand why that value is still a range in the tree.

from mill-github-dependency-graph.

Hey @chrsmutti, thanks for the report! Yes! I hit on the in scala-cli as well and it was super confusing. It turns out that the submission API doesn't support ranges since you have to have a valid PURL, which don't support ranges. That's totally fine because in reality, we don't want to display a range here, we actually want to display the resolved result. I'm not sure what version of mill-github-dependency-graph you're using, but this should be fixed in 0.1.0 that I released yesterday. If you're not on the latest version, could you check with that? I also plan on actually adding in https://github.com/package-url/packageurl-java to help with this, because there are some other oddities in PURLs.

Also, if you haven't seen I also created https://github.com/ckipp01/mill-dependency-submission which should be bringing in the latest version with this fix.

from mill-github-dependency-graph.

I'm actually using the dependency submission action already:

An excerpt of the generated manifest using ./millw --import ivy:io.chris-kipp::mill-github-dependency-graph::0.1.0 show io.kipp.mill.github.dependency.graph.Graph/generate:

      "com.fasterxml.jackson.core:jackson-databind:[2.7.0,2.12.3]": {
        "metadata": {
          
        },
        "dependencies": [
          "com.fasterxml.jackson.core:jackson-annotations:2.12.3",
          "com.fasterxml.jackson.core:jackson-core:2.12.3"
        ],
        "relationship": "indirect",
        "package_url": "pkg:maven/com.fasterxml.jackson.core/jackson-databind@[2.7.0,2.12.3]"
      }

from mill-github-dependency-graph.

huh, that's very odd. I just tested this with:

import mill._, scalalib._

object minimal extends ScalaModule {
  def scalaVersion = "2.13.8"

  def ivyDeps = Agg(
    ivy"org.jongo:jongo:1.5.0"
  )
}

And then when running generate on it I see that jackson-databind is indeed resolved:

      "com.fasterxml.jackson.core:jackson-databind:2.12.3": {
        "metadata": {

        },
        "dependencies": [
          "com.fasterxml.jackson.core:jackson-annotations:2.12.3",
          "com.fasterxml.jackson.core:jackson-core:2.12.3"
        ],
        "relationship": "indirect",
        "package_url": "pkg:maven/com.fasterxml.jackson.core/[email protected]"
      },

Is this project public by chance that I could take a look at?

from mill-github-dependency-graph.

Yeah, very odd indeed, I just tested a minimal version as well and had the same result as you. The project is internal 😞

But I think I've reproduced it, it may be related to a module dep adding in some other version of the dependency, like so:

import mill._, scalalib._

object minimalDep extends ScalaModule {
  def scalaVersion = "2.13.8"

  def ivyDeps = Agg(
    ivy"com.fasterxml.jackson.core:jackson-core:2.13.3",
  )
}

object minimal extends ScalaModule {
  def moduleDeps = Seq(`minimalDep`)

  def scalaVersion = "2.13.8"

  def ivyDeps = Agg(
    ivy"org.jongo:jongo:1.5.0"
  )
}

This architecture one is closer to the original repo I was testing it, and results in:

      "com.fasterxml.jackson.core:jackson-core:[2.7.0,2.12.3]": {
        "metadata": {

        },
        "dependencies": [

        ],
        "relationship": "indirect",
        "package_url": "pkg:maven/com.fasterxml.jackson.core/jackson-core@[2.7.0,2.12.3]"
      },

If moduleDep depends on com.fasterxml.jackson.core:jackson-core in the same range as org.jongo:jongo, the issue is also resolved. So it seems like an extreme edge case 😅

from mill-github-dependency-graph.

Question is, which version actually ends up in the classpath?

from mill-github-dependency-graph.

Ah perfect, thanks for the reproduction @chrsmutti. This is super helpful. I'll dig in and see what's going on because yea, this shouldn't be ending up here and I actually have no idea how it is since I'm grabbing the reconciled version every time. I'll dig in and report back.

from mill-github-dependency-graph.

Alright, so interesting enough, I think this might be a bug in Coursier's DependencyTrees. If you use resolve to test this out, you can see that it's resolved correctly with 2.12.3:

❯ cs resolve org.jongo:jongo:1.5.0
com.fasterxml.jackson.core:jackson-annotations:2.12.3:default
com.fasterxml.jackson.core:jackson-core:2.12.3:default
com.fasterxml.jackson.core:jackson-databind:2.12.3:default
de.undercouch:bson4jackson:2.12.0:default
org.jongo:jongo:1.5.0:default

However, with the minimal example you have, even if you try a mill minimal.ivyDepsTree you can see the issue:

❯ ./mill minimal.ivyDepsTree
[16/16] minimal.ivyDepsTree
├─ org.jongo:jongo:1.5.0
│  ├─ com.fasterxml.jackson.core:jackson-annotations:[2.7.0,2.12.3] -> 2.12.3
│  ├─ com.fasterxml.jackson.core:jackson-core:[2.7.0,2.12.3]
│  ├─ com.fasterxml.jackson.core:jackson-databind:[2.7.0,2.12.3] -> 2.12.3
│  │  ├─ com.fasterxml.jackson.core:jackson-annotations:2.12.3
│  │  └─ com.fasterxml.jackson.core:jackson-core:2.12.3
│  └─ de.undercouch:bson4jackson:2.12.0
├─ org.scala-lang:scala-library:2.13.8
└─ com.fasterxml.jackson.core:jackson-core:2.13.3

I don't believe I've ever seen a reconciled version in the tree still have the range like it does here. I'd expect core to also have the same -> 2.12.3, but it doesn't. The funny part is that when you dig into the actual tree, when I come across this dep I see:

dep version: [2.7.0,2.12.3]
retained version: [2.7.0,2.12.3]
reconciled version: [2.7.0,2.12.3]

I don't really understand how in this scenario both the retainedVersion and the reconciledVersion is still the range. In other scenarios I've come across the retainedVersion will still contain the range, but not the reconciledVersion.

This leaves me with a couple thoughts.

1. I can dig into Coursier and try to understand what's happening here, but this is pretty time consuming
2. I try to create a valid PURL out of the reconciled version and if it's not valid, just warn and throw it out

For the short term I'm leaning on just trying to achieve 2. This would at least guarantee that something is being submitted and also avoid odd edge cases like this.

from mill-github-dependency-graph.

I'm really out of my league here as I don't have that much knowledge on dependency resolution, but coming from an user's perspective an error/warning when the PURL is invalid seems like a good idea because the only information I have at the moment is this:

All that said, thank you very much for the quick answers!

In this case in particular, I think I'm at fault as well, the upgraded dependency on the minimalDep is not on the original sbt project I was migrating, it should also depend on jackson-*:2.12.3, which would mean I wouldn't have caught this in the first place 😅

from mill-github-dependency-graph.

In this case in particular, I think I'm at fault as well, the upgraded dependency on the minimalDep is not on the original sbt project I was migrating, it should also depend on jackson-*:2.12.3, which would mean I wouldn't have caught this in the first place 😅

Ha, that's ok. These edge cases are helpful and hard to think of when first building something. So it's helpful see what people are hitting on.

from mill-github-dependency-graph.

pfff, they say PURL doesn't support ranges but you can still give one to their java library and end up with

 "package_url": "pkg:maven/com.fasterxml.jackson.core/jackson-core@%5B2.7.0%2C2.12.3%5D"

Awesome.

from mill-github-dependency-graph.

It worked like a charm! Thank you so much for the quick fix

from mill-github-dependency-graph.

Just to link these together, I created a discussion about this in coursier/coursier#2481.

from mill-github-dependency-graph.