Giter VIP home page Giter VIP logo

edgex-snap-hooks's Introduction

edgex-snap-hooks

Go Reference

Snap hooks library used by EdgeX Foundry Go service snaps.
It provides utilites to implement snap hooks, including some wrappers for the snapctl commands.

Usage

Download or upgrade to the latest version:

go get github.com/canonical/edgex-snap-hooks/v3

Please refer to go get docs for details.

The jakarta branch is split from master as of v2.1.3. It would only receive patch updates but no minor or major releases. The jakarta-only tags follow v2.1.X+jakarta scheme, where X is greater than 3.

Example

package main

import (
	"fmt"
	"os"

	hooks "github.com/canonical/edgex-snap-hooks/v3"
)

func main() {
	var err error

	if err = hooks.Init(false, "edgex-device-example"); err != nil {
		fmt.Printf("initialization failure: %s", err)
		os.Exit(1)
	}

	// copy file from $SNAP to $SNAP_DATA
	if err = hooks.CopyFile(hooks.Snap+"/config.json", hooks.SnapData+"config.json"); err != nil {
		hooks.Error(err.Error())
		os.Exit(1)
	}
  
	// read env var override configuration
	cli := hooks.NewSnapCtl()
	envJSON, err := cli.Config(hooks.EnvConfig)
	if err != nil {
		hooks.Error(fmt.Sprintf("Reading config 'env' failed: %v", err))
		os.Exit(1)
	}
	hooks.Debug(fmt.Sprintf("envJSON: %s", envJSON))
}

Testing

The tests need to run in a snap environment:

Build and install:

snapcraft
sudo snap install --dangerous ./edgex-snap-hooks_test_amd64.snap

The tests files are read relative to project source inside the snap. The edgex-snap-hooks.test command runs go test -v --cover internally and accepts all other go test arguments.

Run all tests:

make test

Run top-level tests:

sudo edgex-snap-hooks.test

Run tests in one package, e.g. snapctl:

sudo edgex-snap-hooks.test ./snapctl

Run one unit test, e.g. TestGet:

sudo edgex-snap-hooks.test ./snapctl -run TestGet

Development

make try

You can now edit the files locally, copy them to prime directory, and re-run the tests without rebuilding the project. E.g.:

make sync
sudo edgex-snap-hooks.test ./snapctl

edgex-snap-hooks's People

Contributors

bnevis-i avatar farshidtz avatar monicaisher avatar renovate[bot] avatar tonyespy avatar vli11 avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar

edgex-snap-hooks's Issues

Invalid .env files created and processed by snaps

The env files created by the edgex-snap-hooks library (see here) use the .env extension which is for the dotenv format. However, they are treated as bash scripts and include export commands. The export is expected by the consumer, because the snaps source the file as a bash script (example). The dotenv format is a de-facto standard and is not bash. The exporting should happen outside when the env file is being loaded.

References:

The env files are currently bash files, e.g.:

service.env:

export X=1
export Y=2

However, it should be:

X=1
X=2

Tests should cleanup snap options

Several tests in top-level, snapctl, options packages set snap options during the tests. These should removed using snapctl unset after the tests.

Removing them using snap unset from outside is not possible because the snap has no configure hook. The current workaround is to re-install the snap or enter the shell environment and use snapctl unset to remove them.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/workflows/issues-to-jira.yml
.github/workflows/test.yml
  • actions/checkout v3
  • whywaita/setup-lxd v1
  • ubuntu 20.04
gomod
go.mod
  • github.com/stretchr/testify v1.8.4
  • go 1.18

  • Check this box to trigger a request for Renovate to run again on this repository

Upgrade snaps to use v2.2.0

Once released, the module needs to be upgraded for the following:

CHECK THE BOX WHEN MERGED

snap `unset` is doing `set` with an empty value

For unset a non-existing option, snap will set the option with an empty value instead of clearing the existing value. Here is the detail:

$ snap get edgex-device-mqtt -d
{}

$ sudo snap unset edgex-device-mqtt config.service.port
$ snap get edgex-device-mqtt -d
{
     "config": {
         "service": {}
    }
}

Setting a bad app key removes other keys from env file

$ snap set edgex-app-service-configurable config.service-startupmsg="testing injection"
$ snap set edgex-app-service-configurable apps.app-service-configurable.config.service-port=11111
$ cat /var/snap/edgex-app-service-configurable/current/config/res/app-service-configurable.env 
export SERVICE_STARTUPMSG="testing injection"
export SERVICE_PORT="11111"

$ snap set edgex-app-service-configurable apps.app-service-configurable.config.service.port=11111
error: cannot perform the following tasks:
- Run configure hook of "edgex-app-service-configurable" snap (run hook "configure": edgex-app-service-configurable.configure: could not process options: error converting config key to environment variable key: config key must not contain dots: service.port)
$ cat /var/snap/edgex-app-service-configurable/current/config/res/app-service-configurable.env 
export SERVICE_STARTUPMSG="testing injection"

Only using `unset apps` and `unset config` together will do the expected `unset apps`' job

set app, unset apps, does not work:

$ sudo snap set edgex-device-mqtt apps.device-mqtt.config.service.port=1111
$ sudo snap restart edgex-device-mqtt.device-mqtt

$ sudo snap unset edgex-device-mqtt apps.device-mqtt.config.service.port
$ sudo snap restart edgex-device-mqtt.device-mqtt

$ snap get edgex-device-mqtt -d
{
    "apps": {
        "device-mqtt": {
            "config": {
                "service": {
                    "port": 1111
                }
            }
        }
    }
}
$ journalctl -f | grep device-mqtt |grep "Web server starting"
msg="Web server starting (localhost:1111)"

set app, unset apps and config, work as expected:

$ sudo snap set edgex-device-mqtt apps.device-mqtt.config.service.port=1111
$ sudo snap restart edgex-device-mqtt.device-mqtt

$ sudo snap unset edgex-device-mqtt apps.device-mqtt.config.service.port
$ sudo snap unset edgex-device-mqtt config.service.port
$ sudo snap restart edgex-device-mqtt.device-mqtt

$ snap get edgex-device-mqtt -d
{}
$ journalctl -f | grep device-mqtt |grep "Web server starting"
msg="Web server starting (localhost:59982)" <----------------------------------------default service port

See more details in this PR

Fail to write temp env file because of missing directory

The configure hook writes the temp file in a directory that is created using the install hook when deploying config files to snap data. If the install hook doesn't perform that, the configure hook fails with error:

snap install --dangerous ./edgex-device-usb-camera_0.0.1-dev.5_amd64.snap 
error: cannot perform the following tasks:
- Run configure hook of "edgex-device-usb-camera" snap if present (run hook "configure": edgex-device-usb-camera.configure: could not process options: failed to write /var/snap/edgex-device-usb-camera/x2/config/device-usb-camera/res/device-usb-camera.env.tmp  - open /var/snap/edgex-device-usb-camera/x2/config/device-usb-camera/res/device-usb-camera.env.tmp: no such file or directory)

The configure hook should add the logic to create the necessary directory structure before attempting to write this file or any other.

Workaround using install hook:

  1. Remove the snap with --purge flag
  2. Add the directory creation in install hook
  3. Install again

snapctl command error logs not returned to the caller

Not all error logs are returned to the caller.

For example, an erroneous config set returns the following only: exit status 1

There may be more information returned in stdout and stderr by snapctl which the functions don't return.

Errors do get written to journals but those are invisible to the caller and need to be queried separately.

Redundant snap name input

The snap name must be available from SNAP_NAME environment variable:

edgex-snap-hooks/utils.go

Lines 154 to 157 in 5c9f465

SnapName = os.Getenv(snapNameEnv)
if SnapName == "" {
return errors.New("SNAP_NAME is not set")
}

Yet, another string can be passed on to the Init function as snap name:

func Init(setDebug bool, snapName string) error {

Could these two strings differ in practice?

Non-config options validation

This code skips the validation part (lines 113-116) because only the service that is passed to function is read from the map (line 136) and processed:

func processAppCustomOptions(service, key string, value configOptions) error {
switch service {
case "secrets-config":
return processSecretsConfigOptions(key, value)
default:
return fmt.Errorf("Unknown custom option %s for service %s", key, service)
}
}
// Process the "apps.<app>.<custom.option>" where <custom.option> is not "config"
func ProcessAppCustomOptions(service string) error {
var options snapOptions
// get the 'apps' json structure
jsonString, err := snapctl.Get("apps").Document().Run()
if err != nil {
return err
}
err = json.Unmarshal([]byte(jsonString), &options)
if err != nil {
return err
}
log.Debugf("Processing custom options for service: %s", service)
appOptions := options.Apps[service]
log.Debugf("Processing custom options: %v", appOptions)
if appOptions != nil {
for k, v := range appOptions {
if k != "config" {
if err := processAppCustomOptions(service, k, v); err != nil {
return err
}
}
}
}
return nil
}

There is no easy way to validate all custom options because the processing is requested selectively. Full validation can be done within the configure hook, even when options aren't being processed at that stage.

Add logging functions with string formatting

The current logging functions take string as input. It would be useful to add corresponding functions (Debugf, Errorf, Infof, Warnf) to accept string formats in a similar fashion to fmt.Printf.

Example code that can be simplified:
https://github.com/edgexfoundry/edgex-go/blob/3acce916edd26c48cc534c43f5f6a7ed858771e5/snap/local/hooks/cmd/configure/configure.go#L552

Moreover, the existing functions can be changed to accept one or more interfaces, instead of just one string. See fmt.Print.

Reject dot separator in config keys

The config keys can include dot or hyphen as separator. The use of dot allow setting complex keys with parent-child relationship. This is sometimes useful, e.g. for service.port, but confusing when mixed with hyphen-separated keys with map to the same underlying environment variable.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.