Giter VIP home page Giter VIP logo

mill-github-dependency-graph's Introduction

Mill GitHub Dependency Graph

A Mill plugin to submit your dependency graph to GitHub via their Dependency Submission API.

The main benifits of doing this are:

  1. Being able to see your dependency graph on GitHub in your Insights tab. For example you can see this here for this plugin.
  2. If enabled, Dependabot can send you alerts about security vulnerabilities in your dependencies.

Requirements

  • Right now this plugin only supports the Mill 0.10.x series.
  • Make sure in your repo settings the Dependency Graph feature is enabled as well as Dependabot Alerts if you'd like them. (Settings -> Code security and analysis)

Quick Start

The easiest way to use this plugin is with the mill-dependency-submission action. You can add this to a workflow like below:

name: github-dependency-graph

on:
  push:
    branches:
      - main

jobs:
  submit-dependency-graph:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: coursier/cache-action@v6
    - uses: actions/setup-java@v3
      with:
        distribution: 'temurin'
        java-version: '17'
    - uses: ckipp01/mill-dependency-submission@v1

You can also just run the following command from the root of your workspace which will create the file for you:

curl -o .github/workflows/github-dependency-graph.yml --create-dirs https://raw.githubusercontent.com/ckipp01/mill-github-dependency-graph/main/.github/workflows/github-dependency-graph.yml

After you submit your graph you'll be able to view your dependencies.

How's this work?

The general idea is that the plugin works in a few steps:

  1. Gather all the modules in your build
  2. Gather all direct and transitive dependencies of those modules
  3. Create a tree-like structure of these dependencies. We piggy back off coursier for this and use its DependencyTree functionality.
  4. We map this structure to that of a DependencySnapshot, which is what GitHub understands
  5. We post this data to GitHub.

You can use another available task to see what the Manifests look like locally for your project, which are the main part of the DependencySnapshot.

./mill --import ivy:io.chris-kipp::mill-github-dependency-graph::0.1.0 show io.kipp.mill.github.dependency.graph.Graph/generate

Limitation

You'll notice when using this that a lot of dependencies aren't linked back to the repositories where they are located, some may be wrongly linked, and much of the information the plugin is providing (like direct vs indirect) isn't actually displayed in the UI. Much of this is either bugs or limitations on the GitHub UI side. You can follow some conversation on this here.

mill-github-dependency-graph's People

Contributors

ckipp01 avatar ckipp01-scala-steward[bot] avatar dependabot[bot] avatar lolgab avatar scala-steward avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

scala-steward damianreeves lolgab

mill-github-dependency-graph's Issues

Integrating with sbt-dependency-check plugin

Hi there! I'm playing with this plugin in order to validate some of our FINOS incoming projects, which builds with Mill. Part of our process is to ensure that none of direct or transitive dependencies have HIGH or CRITICAL CVEs, and seems like this is the tool for us; however, since Dependabot doesn't work on transitive dependencies, we normally use https://github.com/albuch/sbt-dependency-check to scan the dependency tree (you can read more on https://github.com/maoo/security-scanning#scala .

I was wondering if there is a way to integrate the mill-github-dependency-grapth with the sbt-dependency-check plugin, and skip the whole part that submits data to GitHub.

I'm not sure if this is a use case that is at all relevant for you, feel free to close this issue if it's not, but even in that case, I thought you could give me some useful pointers, and even some code I could reuse in order to achieve my goal.

Thanks in advance, and congrats for contributing this great tool!

Fails to load dependency graph when dependency uses Version Range

First of all, thanks for the amazing work on this tool, really easy to use and it'll be an incredible selling point for me to pitch Mill and a more modern build process for some services at work!

I was testing this in an internal repository and had some issues with the dependency graph not loading, after some debugging (adding projects incrementally and reading the output of generate), I've pinpointed the issue to Version Ranges.

One of the dependencies of this project is org.jongo:jongo:1.5.0, with depends on com.fasterxml.jackson.core:jackson-core:[2.7.0,2.12.3]. When the manifest is generated with this type of version range, the Dependency Graph page fails to load, downgrading to org.jongo:jongo:1.4.1 that does not include version ranges for Jackson works perfectly.

Github has some way of dealing with this, if the project has a pom.xml it shows the version range correctly, as seen here:
image

But I did not find anything on the documentation of the Dependency Submission API.

StackOverflowError when generating graph

Hi, first of all, thanks for this great tool!

Generating dependency graph fails in our project with StackOverflowError:

io.kipp.mill.github.dependency.graph.Graph.generate java.lang.StackOverflowError
    java.base/java.nio.ByteBuffer.limit(ByteBuffer.java:1107)
    java.base/java.nio.MappedByteBuffer.limit(MappedByteBuffer.java:235)
    java.base/java.nio.MappedByteBuffer.limit(MappedByteBuffer.java:67)
    java.base/sun.nio.ch.Util$BufferCache.get(Util.java:171)
    java.base/sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:231)
    java.base/sun.nio.ch.IOUtil.write(IOUtil.java:71)
    java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
    java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
    java.base/java.nio.channels.Channels.writeFully(Channels.java:97)
    java.base/java.nio.channels.Channels$1.write(Channels.java:172)
    mill.util.FileLogger$$anon$1.write(Loggers.scala:190)
    java.base/java.io.PrintStream.write(PrintStream.java:559)
    java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:233)
    java.base/sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:312)
    java.base/sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
    java.base/java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:181)
    java.base/java.io.PrintStream.write(PrintStream.java:606)
    java.base/java.io.PrintStream.print(PrintStream.java:745)
    java.base/java.io.PrintStream.println(PrintStream.java:882)
    mill.util.FileLogger.debug(Loggers.scala:203)
    mill.util.MultiLogger.debug(Loggers.scala:257)
    mill.util.ProxyLogger.debug(Loggers.scala:280)
    io.kipp.mill.github.dependency.graph.ModuleTrees.toNode$1(ModuleTrees.scala:87)
    io.kipp.mill.github.dependency.graph.ModuleTrees.$anonfun$toFlattenedNodes$5(ModuleTrees.scala:121)
    io.kipp.mill.github.dependency.graph.ModuleTrees.$anonfun$toFlattenedNodes$5$adapted(ModuleTrees.scala:121)
    scala.collection.immutable.List.foreach(List.scala:333)
    io.kipp.mill.github.dependency.graph.ModuleTrees.toNode$1(ModuleTrees.scala:121)
    io.kipp.mill.github.dependency.graph.ModuleTrees.$anonfun$toFlattenedNodes$5(ModuleTrees.scala:121)
    io.kipp.mill.github.dependency.graph.ModuleTrees.$anonfun$toFlattenedNodes$5$adapted(ModuleTrees.scala:121)
    scala.collection.immutable.List.foreach(List.scala:333)

It used to work before this change: input-output-hk/armadillo@5ede8aa#diff-c6f64a5895d9f518051cf83f8557f492ec6f5683a34c2914a46a894001e62726 but I don't see anything suspicious there.

Link to the project: https://github.com/input-output-hk/armadillo
Link to a failed build: https://github.com/input-output-hk/armadillo/actions/runs/3631083573/jobs/6125238848

Any idea what could be the reason?

After introducing cross-building there is bug with resolving Graph/submit.

After updating the mill-github-dependency-graph to version 0.2.3 in the scala-cli repository, it is now throwing an error error:

Cannot resolve io.kipp.mill.github.dependency.graph.Graph.submit. Try `mill resolve io.kipp.mill.github.dependency.graph.Graph._` to see what's available.
[60](https://github.com/VirtusLab/scala-cli/actions/runs/5109822211/jobs/9185025628#step:5:61)
Error: Error: The process '/home/runner/work/scala-cli/scala-cli/mill' failed with exit code 1

I suppose that it is issue after introducing cross-building into mill-github-dependency-graph.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.