hashicorp / packer-plugin-scaffolding Goto Github PK

Quick start repository for creating a Packer plugin.

License: Mozilla Public License 2.0

Go 60.67% HCL 17.32% Makefile 4.21% Shell 17.80%

packer-plugin-scaffolding's Introduction

Packer Plugin Scaffolding

This repository is a template for a Packer multi-component plugin. It is intended as a starting point for creating Packer plugins, containing:

A builder (builder/scaffolding)
A provisioner (provisioner/scaffolding)
A post-processor (post-processor/scaffolding)
A data source (datasource/scaffolding)
Docs (docs)
A working example (example)

These folders contain boilerplate code that you will need to edit to create your own Packer multi-component plugin. A full guide to creating Packer plugins can be found at Extending Packer.

In this repository you will also find a pre-defined GitHub Action configuration for the release workflow (.goreleaser.yml and .github/workflows/release.yml). The release workflow configuration makes sure the GitHub release artifacts are created with the correct binaries and naming conventions.

Please see the GitHub template repository documentation for how to create a new repository from this template on GitHub.

Packer plugin projects

Here's a non exaustive list of Packer plugins that you can checkout:

Looking at their code will give you good examples.

Build from source

Clone this GitHub repository locally.
Run this command from the root directory:

go build -ldflags="-X github.com/hashicorp/packer-plugin-scaffolding/version.VersionPrerelease=dev" -o packer-plugin-scaffolding

After you successfully compile, the packer-plugin-scaffolding plugin binary file is in the root directory.
To install the compiled plugin, run the following command

packer plugins install --path packer-plugin-scaffolding github.com/hashicorp/scaffolding

Build on *nix systems

Unix like systems with the make, sed, and grep commands installed can use the make dev to execute the build from source steps.

Build on Windows Powershell

The preferred solution for building on Windows are steps 2-4 listed above. If you would prefer to script the building process you can use the following as a guide

$MODULE_NAME = (Get-Content go.mod | Where-Object { $_ -match "^module"  }) -replace 'module ',''
$FQN = $MODULE_NAME -replace 'packer-plugin-',''
go build -ldflags="-X $MODULE_NAME/version.VersionPrerelease=dev" -o packer-plugin-scaffolding.exe
packer plugins install --path packer-plugin-scaffolding.exe $FQN

Running Acceptance Tests

Make sure to install the plugin locally using the steps in Build from source.

Once everything needed is set up, run:

PACKER_ACC=1 go test -count 1 -v ./... -timeout=120m

This will run the acceptance tests for all plugins in this set.

Registering Plugin as Packer Integration

Partner and community plugins can be hard to find if a user doesn't know what they are looking for. To assist with plugin discovery Packer offers an integration portal at https://developer.hashicorp.com/packer/integrations to list known integrations that work with the latest release of Packer.

Registering a plugin as an integration requires metadata configuration within the plugin repository and approval by the Packer team. To initiate the process of registering your plugin as a Packer integration refer to the Developing Plugins page.

Requirements

packer-plugin-sdk >= v0.5.2
Go >= 1.20

Packer Compatibility

This scaffolding template is compatible with Packer >= v1.10.2

packer-plugin-scaffolding's People

Contributors

Stargazers

Watchers

packer-plugin-scaffolding's Issues

GitHub releases and packer usage example

For better understanding how to make a packer plugin it would be educative to have:

GitHub releases of this plugin.
A working example (e.g. in a example kind of directory) which could be used by packer init/packer build.
A github action workflow that actually uses the example to run packer to test run the plugin.

Add documentation generating examples to scaffolding

Coming back on this, we should probably add some partials to the example docs, so users that adopt it will see how they're used.

Originally posted by @lbajolet-hashicorp in #108 (comment)

expand .gitignore

I have a plugin created from this scaffolding, and ended up adding the following rules to the .gitignore:

**/packer_log*.txt
**/git_*test.pkr.hcl
crash.log
.docs
docs.zip

Reasoning:

**/packer_log*.txt and **/git_*test.pkr.hcl match the files generated by a failed acceptance test run
crash.log file is created when packer crashes
.docs folder is now created by make generate, and it's my understanding that this is just for previewing locally and doesn't need to be checked in, though I might be misunderstanding how the plugin docs gets built into the website.
docs.zip is a build artifact

IMO these should all be part of the scaffolding here.

Plugins don't have access to their own version numbers

Attempting to import "github.com/hashicorp/packer-plugin-x" from any Builder, Provisioner, PostProcessor or Datasource will fail because it results in a circular import. Thus, it is impossible for code in a plugin to obtain it's own version number, if it follows this scaffold.

There should be a way for plugin code to obtain its own version through Prepare() or some other call.

❗️ Critical vulnerabilities found by Dependabot in template...

After generating a new repo, Dependabot finds a number of CRITICAL and HIGH vulnerabilities...

goreleaser config is out of date

The goreleaser.yml file references a make ci-release-docs target that was removed from the GNUMakefile. It also references a docs.zip file that is no longer created afaict.

Somewhat relatedly, the release workflow uses the --rm-dist flag which has been deprecated since 2023-01-17 and now no longer works.

How to get packer to use "dev" version of plugin

The GNUmakefile supplied with this project helpfully provides a dev target that builds the plugin and copies it to ~/.packer.d/plugins. However I am currently unable to get packer to use that generated version.

Here is what "packer-plugin-lxdapi describe" returns for me
{"version":"0.0.4-dev","sdk_version":"0.3.1","api_version":"x5.0","builders":["builder"],"post_processors":[],"provisioners":[],"datasources":[]}

Here is the output from packer when I try to use my plugin. You can see that it finds the plugin under "/home/dman/.packer.d/plugins/packer-plugin-lxdapi". However I cant seem to find a packer invocation / version specification where I could coax packer into using this locally built packer plugin over the ones that are uploaded from github.

Could you please clarify what the local development workflow for plugins should look like?

2023/02/20 04:05:38 [INFO] Packer version: 1.8.5 [go1.18.9 linux amd64] 2023/02/20 04:05:38 Old default config directory found: /home/dman/.packer.d 2023/02/20 04:05:38 [TRACE] discovering plugins in /usr/bin 2023/02/20 04:05:38 Old default config directory found: /home/dman/.packer.d 2023/02/20 04:05:38 [TRACE] discovering plugins in /home/dman/.packer.d/plugins 2023/02/20 04:05:38 [DEBUG] Discovered plugin: lxdapi = /home/dman/.packer.d/plugins/github.com/deepankarsharma/lxdapi/packer-plugin-lxdapi_v0.0.2_x5.0_linux_amd64 2023/02/20 04:05:38 [DEBUG] Discovered plugin: lxdapi = /home/dman/.packer.d/plugins/github.com/deepankarsharma/lxdapi/packer-plugin-lxdapi_v0.0.4_x5.0_linux_amd64 2023/02/20 04:05:38 [INFO] found external [builder] builders from lxdapi plugin 2023/02/20 04:05:38 [DEBUG] Discovered plugin: lxdapi = /home/dman/.packer.d/plugins/packer-plugin-lxdapi 2023/02/20 04:05:38 [INFO] found external [builder] builders from lxdapi plugin 2023/02/20 04:05:38 [TRACE] discovering plugins in . 2023/02/20 04:05:38 [INFO] PACKER_CONFIG env var not set; checking the default config file path 2023/02/20 04:05:38 [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/dman/.packerconfig 2023/02/20 04:05:38 [WARN] Config file doesn't exist: /home/dman/.packerconfig 2023/02/20 04:05:38 Old default config directory found: /home/dman/.packer.d 2023/02/20 04:05:38 [INFO] Setting cache directory: /home/dman/.cache/packer 2023/02/20 04:05:38 Old default config directory found: /home/dman/.packer.d 2023/02/20 04:05:38 [TRACE] listing potential installations for "github.com/deepankarsharma/lxdapi" that match "0.0.4-dev". plugingetter.ListInstallationsOptions{FromFolders:[]string{"/usr/bin/packer", ".", "/home/dman/.packer.d/plugins"}, BinaryInstallationOptions:plugingetter.BinaryInstallationOptions{APIVersionMajor:"5", APIVersionMinor:"0", OS:"linux", ARCH:"amd64", Ext:"", Checksummers:[]plugingetter.Checksummer{plugingetter.Checksummer{Type:"sha256", Hash:(*sha256.digest)(0xc0001fab80)}}}} 2023/02/20 04:05:38 [TRACE] version "v0.0.2" of file "/home/dman/.packer.d/plugins/github.com/deepankarsharma/lxdapi/packer-plugin-lxdapi_v0.0.2_x5.0_linux_amd64" does not match constraint "0.0.4-dev" 2023/02/20 04:05:38 [TRACE] version "v0.0.4" of file "/home/dman/.packer.d/plugins/github.com/deepankarsharma/lxdapi/packer-plugin-lxdapi_v0.0.4_x5.0_linux_amd64" does not match constraint "0.0.4-dev"

Trying to run locally | invalid version: unknown revision

❯ goreleaser build --snapshot --rm-dist
   • building...
   • loading config file       file=.goreleaser.yml
   • loading environment variables
   • getting and validating git state
      • ignoring errors because this is a snapshot error=git doesn't contain any tags. Either add a tag or use --snapshot
      • releasing v0.0.0, commit b4ed3b9a6b83253d5db1c9d714c33d3a0a41a8ee
      • pipe skipped              error=disabled during snapshot mode
   • parsing tag
   • running before hooks
      • running go test ./...
   • setting defaults
      • snapshotting
      • github/gitlab/gitea releases
      • project name
      • loading go mod information
      • building binaries
      • creating source archive
      • archives
      • linux packages
      • snapcraft packages
      • calculating checksums
      • signing artifacts
      • docker images
      • docker manifests
      • artifactory
      • blobs
      • homebrew tap formula
      • scoop manifests
      • twitter
      • milestones
   • snapshotting
   • checking ./dist
      • --rm-dist is set, cleaning it up
   • loading go mod information
   • writing effective config file
      • writing                   config=dist/config.yaml
   • generating changelog
   • building binaries
      • building                  binary=/Users/nathanpierce/packer-plugin-scaffolding/dist/plugin-check_linux_amd64/packer-plugin-scaffolding_vv0.0.0-SNAPSHOT-b4ed3b9_linux_amd64
      • running hook              hook=go install github.com/hashicorp/packer/cmd/packer-plugins-check@latest &&
packer-plugins-check -load=packer-plugin-scaffolding_vv0.0.0-SNAPSHOT-b4ed3b9_linux_amd64

      ⨯ go: downloading github.com/hashicorp/packer v1.7.3
 cmd=[go install github.com/hashicorp/packer/cmd/packer-plugins-check@latest]
      ⨯ go install github.com/hashicorp/packer/cmd/packer-plugins-check@latest: github.com/hashicorp/packer@none updating to
	github.com/hashicorp/[email protected] requires
	github.com/hashicorp/[email protected] requires
	github.com/outscale/osc-sdk-go/[email protected]: invalid version: unknown revision f27e09c295b2
 cmd=[go install github.com/hashicorp/packer/cmd/packer-plugins-check@latest]
   ⨯ build failed after 37.01s error=post hook failed: "go: downloading github.com/hashicorp/packer v1.7.3\ngo install github.com/hashicorp/packer/cmd/packer-plugins-check@latest: github.com/hashicorp/packer@none updating to\n\tgithub.com/hashicorp/[email protected] requires\n\tgithub.com/hashicorp/[email protected] requires\n\tgithub.com/outscale/osc-sdk-go/[email protected]: invalid version: unknown revision f27e09c295b2\n": exit status 1

❯ go version
go version go1.16.5 darwin/amd64

Unable to run packer-sdc successfully

zen4:~/packer-plugin-lxdapi$ go version
go version go1.20.1 linux/amd64

zen4:~/packer-plugin-lxdapi$ packer-sdc --version
0.3.4

zen4:~/packer-plugin-lxdapi$ go generate ./builder/lxdapi
mapstructure-to-hcl2: lxdapi.[Config]: internal error: package "context" without types was imported from "packer-plugin-lxdapi/builder/lxdapi"
builder/lxdapi/builder.go:1: running "packer-sdc": exit status 1

Would appreciate pointers on how to get around this issue.