upbound / build Goto Github PK

Description

It seems like when using go 1.19 (not sure it also happens with go 1.18), make lint fails with the following error:

Can't run linter unused: buildir: failed to load package goarch: could not load export data: cannot import "internal/goarch" (unknown iexport format version 2), export data is newer version - update tool

In order to observe this behaviour, run make lint in https://github.com/crossplane/provider-template. Try updating the golangci-lint version in golang.mk to 1.50 from 1.31 and it will work (albeit with deprecation warnings)

Go uses the go mod tidy command to strip out unneeded dependencies from go.mod and go.sum. Several repositories that use this one expose a make reviewable target that ensures a branch is ready for code review. Ideally make reviewable would invoke go mod tidy.

This is a companion issue to crossplane/crossplane#2257, which suggests that we should publish version markers to make identifying various "rolling named versions" in a channel more identifiable. We currently do a similar practice of publishing a current build for each channel, but it only allows for, well, the current latest.

From Crossplane issue:

We could consider publishing "version markers", which are essentially just files that use a vanity name for releases. For instance, we could publish a stable-1.txt which contained the version for the latest patch on the latest minor, and a stable-2.txt and stable-3.txt for the latest patches on the two latest patches on the previous two minors. Similarly, we could also have latest-1.1, latest-1.0, etc. markers. These would be auto-updated in the release process, and would make it easier to identify and consume the latest patch for various maintained versions. It would also enhance automated testing as versions can be referred to generically (i.e. test upgrading from stable-1 to stable-2 rather than having to update the explicit versions in the tests).

What happened?

When using docs.run (against crossplane/crossplane) the local docs folder is copied to the docs work directory and Hugo starts.

The problem is that Hugo is serving a copy of actual content. As a result any changes made to the docs are not reflected live and make/hugo must be stopped and restarted.

How can we reproduce it?

git clone https://github.com/crossplane/crossplane.git
cd crossplane
make docs.run

Now edit and save a file in the /docs directory.

The change is not reflected.

What environment did it happen in?

build @ 207d7d1

What happened?

The image.mk and imagelight.mk make use of shasum to generate the image name. This command is not available on all linux systems (like Fedora server) but many offer sha256sum as alternative.

How can we reproduce it?

Run make build on a system without shasum.

What environment did it happen in?

Linux fedora 5.18.9-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Jul 2 15:56:43 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

I am getting an unknown architecture error while building crossplane on aarch64 platform, which uses upbound/build as a submodule.

I have done the below changes to resolve the issues.

diff --git a/makelib/common.mk b/makelib/common.mk
index 800bacc..8ffae43 100644
--- a/makelib/common.mk
+++ b/makelib/common.mk
@@ -126,6 +126,11 @@ SAFEHOSTARCH := amd64
 TARGETARCH := amd64
 endif

+ifeq ($(HOSTARCH),aarch64)
+SAFEHOSTARCH := arm64
+TARGETARCH := arm64
+endif

It is necessary to add these changes to build packages on arm64 platform which uses upbound/build as a submodule. I will raise a PR if the above changes looks good.

What happened?

When running make build publish in the provider-helm repository on MacOS Monterey the build panics with the following error:

fatal error: runtime: bsdthread_register error

runtime stack:
runtime.throw(0x14b2fa9, 0x21)
        /usr/local/go/src/runtime/panic.go:605 +0x95 fp=0x7ff7bfefdfc0 sp=0x7ff7bfefdfa0 pc=0x1029c35
runtime.goenvs()
        /usr/local/go/src/runtime/os_darwin.go:108 +0x83 fp=0x7ff7bfefdff0 sp=0x7ff7bfefdfc0 pc=0x10274d3
runtime.schedinit()
        /usr/local/go/src/runtime/proc.go:482 +0xa1 fp=0x7ff7bfefe030 sp=0x7ff7bfefdff0 pc=0x102c5e1
runtime.rt0_go(0x7ff7bfefe060, 0x9, 0x7ff7bfefe060, 0x1000000, 0x9, 0x7ff7bfefe410, 0x7ff7bfefe471, 0x7ff7bfefe476, 0x7ff7bfefe480, 0x7ff7bfefe48c, ...)
        /usr/local/go/src/runtime/asm_amd64.s:175 +0x1eb fp=0x7ff7bfefe038 sp=0x7ff7bfefe030 pc=0x10546db
21:23:07 [FAIL]
make[1]: *** [img.release.manifest.publish.provider-helm] Error 1

Related issue: golang/go#49425

It seems to fail when executing the manifest tool. The version used is 0.7.0 which was released 2017. A possible solution would be to upgrade the manifest tool to a later version.

How can we reproduce it?

• Clone provider-helm
• Update DOCKER_REGISTRY in Makefile to your individual docker user
• Commit the change so you have a clean git history to enable publishing
• run make build publish

What environment did it happen in?

provider-helm master
MacOS Monterey 12.2
go version go1.17.7 darwin/amd64

What problem are you facing?

If I build with make build DOCKER_REGISTRY=muvaf, the variable makes it to package/crossplane.yaml

However, if I do export DOCKER_REGISTRY=muvaf && make build it won't be effective and I'll see crossplane/ in package/crossplane.yaml

The provider is https://github.com/crossplane-contrib/provider-jet-linode and this issue describes what we're trying to do thechangelog/changelog.com#399 if it helps.

How could Upbound help solve your problem?

There might be something overriding the env var but not the explicit variable passed to make command.

What problem are you facing?

current kind version v0.9.0 does not work with rootless podman. rootless pod is the default for normal (i.e. non-root) users using podman.
This is hampering the ability to do local development for contributions to crossplane.
Please upgrade the kind version to the latest kind or at least version v0.11.0.

See also here for more info about running kind on rootless docker or podman:
https://kind.sigs.k8s.io/docs/user/rootless/

How could Upbound help solve your problem?

Please upgrade the kind version to the latest kind or at least version v0.11.0.

removed

The default value for DOCKER_REGISTRY is 127.0.0.1:5000 but having : throws the following error:

build/makelib/image.mk:211: *** target pattern contains no '%'.  Stop.

So, there is no way to use a registry whose URL includes port.

When there are no tags in a repo being built with the tools in this repo it uses this line:

This yields versions like 0.0.0-388.4809829 and critically 0.0.0-417.0943445.
The second example is invalid as SemVer 2.0 does not allow leading zeros in a pre-release section. git describe deals with this by prefixing the git sha short hash with a g.

VERSION := $(shell echo "v0.0.0-$$(git rev-list HEAD --count)-$$(git describe --dirty --always)" | sed 's/-/./2' | sed 's/-/./2')

should be:

VERSION := $(shell echo "v0.0.0-$$(git rev-list HEAD --count)-g$$(git describe --dirty --always)" | sed 's/-/./2' | sed 's/-/./2')

The build submodule should support producing sha256 checksums for all binaries it produces so that they can be published and used to verify downloads.

make help shows the following for the go targets:

Go Targets:
    fmt          Build and publish final releasable artifacts
    vendor       Promote a release to a release channel
    vet          Tag a release

Looks like a copy/paste issue at https://github.com/upbound/build/blob/master/makelib/golang.mk#L206

What happened?

crossplane CLI downloaded successfully! Run the following commands to finish installing it:

sudo mv crossplane /usr/local/bin
crossplane --help

Visit https://crossplane.io/ to get started. 🚀
Have a nice day! 👋

Run ./crossplane build configuration -f /home/runner/work/platform-composites/platform-composites/package --name package.xpkg
  ./crossplane build configuration -f /home/runner/work/platform-composites/platform-composites/package --name package.xpkg
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    DOCKER_USR: ***
    PACKAGE_NAME: upbound/platform-composites
crossplane: error: unexpected argument build
Usage: crossplane <command>

A command line tool for interacting with Crossplane.

Commands:
  xpkg build        Build a new package.
  xpkg install      Install a package in a control plane.
  xpkg login        Login to the default package registry.
  xpkg logout       Logout of the default package registry.
  xpkg push         Push a package to a registry.
  xpkg update       Update a package in a control plane.
  beta render       Render a composite resource (XR).
  beta trace        Trace a Crossplane resource to get a detailed output of its
                    relationships, helpful for troubleshooting.
  beta xpkg init    Initialize a new package from a template.

Flags:
  -h, --help       Show context-sensitive help.
      --verbose    Print verbose logging statements.
  -v, --version    Print version and quit.

Run "crossplane <command> --help" for more information on a command.

What happened?

When there is a version tag like v0.17.0, VERSION uses that value directly and VERSION_MAJOR becomes 0 and VERSION_MINOR becomes 17.

However, on non-tag versions such as v0.17.0-4.g47770dd, both VERSION_MAJOR and VERSION_MINOR becomes v0.17.0-4.g47770dd

Epic #

How can we reproduce it?

Create a target like the following:

ola:
	@echo $(VERSION_MAJOR)
	@echo $(VERSION_MINOR)

and run

make ola

What environment did it happen in?

make 3.8 (Apple)

What problem are you facing?

We are currently relying on up for package building and pushing, now that we ported the same commands to the new crossplane CLI, do we want to move to that instead?

How could Upbound help solve your problem?

Epic #

It appears that images from Pull Requests are accidentally being published to docker hub, when only images built from master or release branches should be.

Take for example the conductor images:
https://hub.docker.com/r/upbound/conductor/tags/

Image tag v0.0.0-64.a0f26af was published, which is not a commit from master, it's a commit from an unmerged PR: https://github.com/upbound/conductor/pull/51/commits

What happened?

In Gitlab CI coverage reports can be shown in merge requests. But this needs valid cobertura files. Unfortunately this project uses an outdated library with some issues including the wrong value for filenames. That's why the report is not shown in Gitlab.

How can we reproduce it?

Create coverage xml file and provide as cobertura artifact in Gitlab job. The resulting MR won't have any hit/miss markes (green/red lines).

What environment did it happen in?

Gitlab 16.2, newest version of this project

Possible Solution

Simply use fork https://github.com/boumenot/gocover-cobertura with correct filename attribute.

I could provide an MR but unfortunately it seems like this repo is not maintained properly right now. :(

As a workaround we can use the following in Gitlab for alpine based images:

sed -i.bak 's@'"$CI_SERVER_HOST/$CI_PROJECT_PATH/"'@@g' coverage.xml

Go 1.15 has been released. We should update the cross container to use this new Go version. Previously, we have waited for a subsequent patch version after minor releases, so that may be desirable here as well.

What problem are you facing?

This build submodule attempts to ensure everyone uses a shared set of tools, using Makefiles. https://github.com/cashapp/hermit appears to be purpose built for this.

How could Upbound help solve your problem?

We might consider using https://github.com/cashapp/hermit to simplify our Makefiles.

We recently allowed building on the main branch, but we do not currently allow publishing to channel main.

What problem are you facing?

Our branch naming does not fit any of main, master or release-*, we would need to parametrise the the filter to publish artifacts.

How could Upbound help solve your problem?

Following the TODO comment, I think this could be done by modifying the filter for publish.artifacts and adding a new element to the filter with custom branch name.

There is yarnjs.mk for JS repositories but it doesn't include installation of yarn as opposed to other tools in build submodule, like Go, Helm etc.