Giter VIP home page Giter VIP logo

go-libyear's Introduction

go-libyear

Calculate Go module's libyear!

Install

Use pre-built binaries from the latest release or install with Go:

go install github.com/nieomylnieja/go-libyear/cmd/go-libyear@latest

It can also be built directly from this repository:

git clone https://github.com/nieomylnieja/go-libyear.git
cd go-libyear
make build
./bin/go-libyear ./go.mod

Docker

Docker images hosted on GitHub Container Registry are also provided:

docker pull ghcr.io/nieomylnieja/go-libyear:latest

Usage

go-libyear can be used both as a CLI and Go library. The CLI usage is also documented in usage.txt and accessed through go-libyear --help.

Basic usage:

$ go-libyear /path/to/go.mod
package                             version  date        latest   latest_date  libyear
github.com/nieomylnieja/go-libyear           2023-11-06                        2.41
github.com/pkg/errors               v0.8.1   2019-01-03  v0.9.1   2020-01-14   1.03
github.com/urfave/cli/v2            v2.20.0  2022-10-14  v2.25.7  2023-06-14   0.67
golang.org/x/mod                    v0.12.0  2023-06-21  v0.14.0  2023-10-25   0.35
golang.org/x/sync                   v0.3.0   2023-06-01  v0.5.0   2023-10-11   0.36

Calculated metrics

Libyear

What exactly is libyear? Quoting and paraphrasing libyear.com:

Libyear is a simple measure of software dependency freshness.
It is a single number telling you how up-to-date your dependencies are.
Example: pkg/errors v0.8.1 (June 2019) is 1 libyear behind v0.9.0 (June 2020).

Libyear is the default metric calculated by the program.

Example:

Current Current release Latest Latest release Libyear
v1.45.1 2022-10-11 v2.0.5 2023-10-11 1
v1.46.0 2022-12-04 v2.0.5 2023-10-11 0.85
v2.0.0 2023-10-01 v2.0.5 2022-10-11 0.03

Number of releases

Dependencies with short release cycles are penalized by this measurement, as the version sequence distance is relatively high compared to other dependencies.

Example:

Versions
v1.45.1
v1.45.2
v1.46.0
v2.0.0
v2.0.1
Current Latest Delta
v1.45.1 v2.0.5 5

Version number delta

Version delta is a tuple (x,y,z) where:

  • x is major version
  • y is minor version
  • z is patch version

Only highest-order version number is taken into consideration.

Example:

Current Latest Delta
v1.45.1 v2.0.5 (1,0,0)
v1.45.1 v1.47.5 (0,2,0)
v1.45.1 v.45.5 (0,0,4)

Results manipulation

Flag Explanation
--releases Count number of releases between current and latest.
--versions Calculate version number delta between current and latest.
--indirect Include indirect dependencies in the results.
--skip-fresh Skip up-to-date dependencies from the results.
--find-latest-major Use next, greater than or equal to v2 version as the latest.

Module sources

Source Flag Example
File path default ~/my-project/go.mod
URL --url https://raw.githubusercontent.com/nieomylnieja/go-libyear/main/go.mod
Module path --pkg github.com/nieomylnieja/go-libyear@latest

Output formats

Format Flag
Table default
JSON --json
CSV --csv

Historical data

In order to calculate the metrics in a given point in time, use --age-limit flag. Example:

go-libyear --age-limit 2022-10-01T12:00:00Z ./go.mod

The latest version for each package will be appointed as the latest version of the package before or at the provided timestamp.

The flag works any other flag. If using a script to extract a history of the calculated metrics, it is recommended to use --cache flag as well.

Caching

go-libyear ships with a built-in caching mechanism. It is disabled by default but can be enabled and adjusted with the following flags:

Flag Explanation
--cache Enable caching.
--cache-file-path Use the specified file for caching.
--vcs-cache-dir Use custom cache path for VCS modules.

Go versioning

By default go-libyear will fetch the latest version for the current major version adhering to the following rules:

  • If the current major version is equal to 0.x.x and there's version 1.x.x available, set the latest to 1.x.x.
  • If the current major is equal to or greater than 1.x.x, set the latest to 1.x.x.

If you wish to always set the next major version as the latest, you can use the --find-latest-major (short -M) flag. This flag enforces the following rules:

  • If the current major is equal to or greater than x.x.x, set the latest to the latest (by semver) available version.

  • If the latest major version is greater than the current major and the current version has been published after the first version of the latest major, the libyear is calculated as a difference between the first version of the latest major and latest major version.

    Example:

    Current version is 1.21.9 (2024-02-01), latest is 2.0.5 (2024-01-19);
    1.21.9 was a security fix, it still means we're some time behind v2;
    2.0.0 was released on 2024-01-02, this means we're 17 days (2024-01-19 - 2024-01-02) behind the latest v2, despite the fact that we've updated to the latest security patch for v1.

If you wish to not compensate for such cases and leave libyear as is (it won't be ever negative, we round it to 0 if negative), use the --no-libyear-compensation flag.

The modules reference states that:

If the module is released at major version 2 or higher, the module path must end with a major version suffix like /v2.

This is however not always the case, some older projects, usually pre-module, might not adhere to that. The aforementioned flag also works with such scenarios.

Caveats

Accessing private repositories

Currently the default mode of execution only supports git VCS and GitHub source. To access all private modules use --go-list flag. It will instruct the program to utilize go list command instead of GOPROXY API.

Using --go-list flag

If --go-list flag is provided, go-libyear will used go list command to fetch information about modules. Specifically it runs go list -m -mod=readonly. If the program is executed in a project containing a go.mod which go.sum file is out of sync, it will drop the following error:

updates to go.sum needed, disabled by -mod=readonly

Due to that it is advised to stick with default modules information provider.

Development

CLI application is tested with bats framework. The tests are defined in test folder. Only core calculations are covered by unit tests, main paths are tested through CLI tests.

Acknowledgements

Inspired directly by SE Radio episode 587. Further reading through libyear.com and mimicking libyear-bundler capabilities.

All the concepts and theory is based on or directly quoted from Measuring Dependency Freshness in Software Systems.

go-libyear's People

Contributors

crocmagnon avatar nieomylnieja avatar renovate[bot] avatar servusdei2018 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

go-libyear's Issues

BUG: Ignore new versions which were published before current version

Sometimes, due to different reasons a module might have a higher version available, but it is actually older the our current version.
Example of that is https://github.com/aws/aws-sdk-go-v2.
Given this sample go.mod:

module test

go 1.21

require (
  github.com/aws/aws-sdk-go-v2 v1.18.0
)

When we running go list -m -json -versions github.com/aws/aws-sdk-go-v2 we get the following list:

{
	"Path": "github.com/aws/aws-sdk-go-v2",
	"Version": "v1.18.0",
	"Versions": [
		"v0.1.0",
		...
		"v1.18.1",
		...
		"v1.23.5",
		"v2.0.0-preview.1+incompatible",
		"v2.0.0-preview.2+incompatible",
		"v2.0.0-preview.3+incompatible",
		"v2.0.0-preview.4+incompatible"
	],
	"Time": "2023-04-24T21:11:42Z",8.0",
	"GoVersion": "1.15"
}

We can see that supposedly v2.0.0-preview.4+incompatible is the latest version, but it turns out it was just some kind of a preview version published way back in 2017 and the actual latest is v1.23.5.

If a version was published before the current version it should be disregarded.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

dockerfile
Dockerfile
  • golang 1.22-alpine3.18
test/Dockerfile
  • golang 1.22-alpine3.18
  • bats/bats 1.11.0
github-actions
.github/workflows/ci.yml
  • actions/checkout v4
  • jetpack-io/devbox-install-action v0.11.0
.github/workflows/release.yml
  • actions/checkout v4
  • actions/setup-go v5
  • goreleaser/goreleaser-action v5
  • actions/checkout v4
  • docker/login-action v3
  • docker/metadata-action v5
  • docker/build-push-action v5
  • actions/attest-build-provenance v1
.github/workflows/scan.yml
  • actions/checkout v4
  • jetpack-io/devbox-install-action v0.11.0
gomod
go.mod
  • go 1.22
  • github.com/Masterminds/semver v1.5.0
  • github.com/pkg/errors v0.9.1
  • github.com/stretchr/testify v1.9.0
  • github.com/urfave/cli/v2 v2.27.2
  • go.uber.org/mock v0.4.0
  • golang.org/x/mod v0.17.0
  • golang.org/x/sync v0.7.0
npm
package.json
  • cspell 8.8.3
  • markdownlint-cli 0.40.0
  • yaml 2.4.2
  • yarn 1.22.22

  • Check this box to trigger a request for Renovate to run again on this repository

Panic when running

I got the panic bellow on two projects.
Now both of those projects use cgo and a modules in private repositories in go.mod. Bu I'm pretty sure that the modules in private repositories are at fault as I have another project without cgo but with modules in private repository.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x750b3d]

goroutine 38 [running]:
github.com/nieomylnieja/go-libyear/internal.(*GitHandler).listAllTags(0x0?, 0xc000224040?)
	/srv/go/pkg/mod/github.com/nieomylnieja/[email protected]/internal/git_handler.go:194 +0x1d
github.com/nieomylnieja/go-libyear/internal.(*GitHandler).GetInfo(0xc000194040, {0xc000224040, 0x1b}, 0xc000256190)
	/srv/go/pkg/mod/github.com/nieomylnieja/[email protected]/internal/git_handler.go:143 +0x4f
github.com/nieomylnieja/go-libyear.Command.runForModule({{0x8acc80, 0xc00020a300}, {0x8acce0, 0xb95a80}, {0x8afd98, 0xc0002120b0}, {0x8acec0, 0xc000222000}, 0x0, 0xc000220090, ...}, ...)
	/srv/go/pkg/mod/github.com/nieomylnieja/[email protected]/command.go:119 +0x122
github.com/nieomylnieja/go-libyear.Command.Run.func2()
	/srv/go/pkg/mod/github.com/nieomylnieja/[email protected]/command.go:75 +0x3b
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/srv/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
	/srv/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x96

FEAT: Check for v2+ major versions

Since neither go list nor GOPROXY protocol itself return a full list of available module versions which would also include v2+ versions, we need to manually check if v2+ versions exist for each package.

Once v2 is found, we should also check if v3 exists, and so on.

There might be a way to easily check that for public modules using deps.dev API,

Private modules can be fetched with go list

What happens

$ go-libyear --go-list ./go.mod
Error: private module path: 'example.com/module' cannot be handled by any supported VCS [git]

where example.com/module is an internal module, on a domain configured in GOPRIVATE.

Expected behavior

When commenting this block:

go-libyear/command.go

Lines 108 to 115 in 5dc2dd4

// Verify if the module is private.
if c.vcs.IsPrivate(module.Path) {
var err error
repo, err = c.vcs.GetHandler(module.Path)
if err != nil {
return err
}
}

I'm getting the behavior I'd expect:

$ go-libyear --go-list ./go.mod
# ... (prints proper report)

Notes

  • It looks like go-libyear tries to detect whether it can fetch the repo with git without even checking if it's running with go-list.
  • I don't know whether all private repos can be accessed with go list.

FEAT: Handle private git repositories with native mechanism

Since the shortcomings of go list are rendering it hard/impossible to use in some scenarios, we want to bridge the last gap between our raw GOPROXY (default) approach and go list.

The solution will require us fetching a remote repository, cloning it in a temporary (or not?) directory and reading the git tags.
Support of other VCS is left out.

BUG: Handle invalid canonical semver returned by deps.dev

Some packages have defined an invalid (from Go perspective) version.
Example of that is https://github.com/kat-co/vala.
If we have a following go.mod file:

module test

go 1.21

require (
  github.com/kat-co/vala v0.0.0-20170210184112-42e1d8b61f12
)

Since there's no way to get versions list for a package that does not have any canonical semver, we resort to deps.dev in this scenario.
The site lists 3 versions:
image

And since the latest there is v1, we currently take it as the latest and query goproxy for it.
This however results in an error since goproxy does not recognize such version.

Correct behavior should be to discard non-canonical version from the results returned by goproxy.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.