Hi, thanks for the great tool (aws-vault)!

I ran into a problem when not specifying a backend. In that case, the backend is picked from the list provided by AvailableBackends(). Unfortunately, that list's order is not deterministic. Here is what I get on consecutive runs:

[keyring] Considering backends: [keychain pass file]
...
[keyring] Considering backends: [pass keychain file]

As you can tell, I am running this on MacOS and my credentials are stored in keychain. Thus the first run succeeded, the second failed. The reason for this behavior is the range supportedBackends in AvailableBackends() which does not guarantee any order on a map.

I can work around this with the --backend command line option but an automatic selection would be great. Possible solutions, that I see:

• Define a fixed order for all supported backends, pass and file bringing up the rear.
• Define a default backend for each OS and make sure it is first in the list.
• Or fix the opener for pass to fail if it is not configured. That would remedy the current situation (and may be a bug in its on right) but would not make the behavior determnistic.
• Remove the whole selection code and make --backend mandatory.

I am happy to provide a patch for either of these solutions. I'd recommend one of the first two.

I'm experiencing a build error in a macOS 10.14 test runner with go 1.17.1 for version 1.1.6

This is the relevant logs:

2021-11-02T17:48:27.5633810Z ##[group]Run go test -v
2021-11-02T17:48:27.5634440Z �[36;1mgo test -v�[0m
2021-11-02T17:48:27.5830430Z shell: /bin/bash -e {0}
2021-11-02T17:48:27.5830960Z env:
2021-11-02T17:48:27.5831340Z   GOLANG_VERSION: 1.17.1
2021-11-02T17:48:27.5831920Z   GOROOT: /Users/runner/hostedtoolcache/go/1.17.1/x64
2021-11-02T17:48:27.5832440Z ##[endgroup]
2021-11-02T17:48:29.7078820Z go: downloading github.com/99designs/keyring v1.1.6
2021-11-02T17:48:29.7182240Z go: downloading github.com/OpenPeeDeeP/xdg v1.0.0
2021-11-02T17:48:29.7285450Z go: downloading github.com/atotto/clipboard v0.1.0
2021-11-02T17:48:29.7398930Z go: downloading github.com/boltdb/bolt v1.3.1
2021-11-02T17:48:29.7600460Z go: downloading github.com/docopt/docopt.go v0.0.0-20180111231733-ee0de3bc6815
2021-11-02T17:48:29.7713480Z go: downloading github.com/hgfischer/go-otp v1.0.0
2021-11-02T17:48:29.7789550Z go: downloading github.com/mitchellh/cli v1.1.0
2021-11-02T17:48:29.7901810Z go: downloading github.com/pkg/errors v0.9.1
2021-11-02T17:48:29.7968160Z go: downloading github.com/spf13/viper v1.9.0
2021-11-02T17:48:29.8292050Z go: downloading github.com/dvsekhvalnov/jose2go v1.5.0
2021-11-02T17:48:29.8884850Z go: downloading github.com/keybase/go-keychain v0.0.0-20201121013009-976c83ec27a6
2021-11-02T17:48:29.8987950Z go: downloading github.com/mitchellh/go-homedir v1.1.0
2021-11-02T17:48:29.9090050Z go: downloading github.com/mtibben/percent v0.2.1
2021-11-02T17:48:29.9170520Z go: downloading golang.org/x/crypto v0.0.0-20210921155107-089bfa567519
2021-11-02T17:48:29.9664840Z go: downloading github.com/armon/go-radix v1.0.0
2021-11-02T17:48:29.9764090Z go: downloading github.com/bgentry/speakeasy v0.1.0
2021-11-02T17:48:29.9865140Z go: downloading github.com/fatih/color v1.9.0
2021-11-02T17:48:29.9967210Z go: downloading github.com/mattn/go-isatty v0.0.12
2021-11-02T17:48:30.0067170Z go: downloading github.com/posener/complete v1.2.3
2021-11-02T17:48:30.0523910Z go: downloading github.com/fsnotify/fsnotify v1.5.1
2021-11-02T17:48:30.0727300Z go: downloading github.com/magiconair/properties v1.8.5
2021-11-02T17:48:30.0971020Z go: downloading github.com/mitchellh/mapstructure v1.4.2
2021-11-02T17:48:30.1130490Z go: downloading github.com/spf13/afero v1.6.0
2021-11-02T17:48:30.1533140Z go: downloading github.com/spf13/cast v1.4.1
2021-11-02T17:48:30.1673040Z go: downloading github.com/spf13/jwalterweatherman v1.1.0
2021-11-02T17:48:30.1772980Z go: downloading github.com/spf13/pflag v1.0.5
2021-11-02T17:48:30.2136750Z go: downloading github.com/subosito/gotenv v1.2.0
2021-11-02T17:48:30.2292700Z go: downloading gopkg.in/ini.v1 v1.63.2
2021-11-02T17:48:30.2455100Z go: downloading github.com/mattn/go-colorable v0.1.6
2021-11-02T17:48:30.2836740Z go: downloading golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359
2021-11-02T17:48:30.3088470Z go: downloading github.com/hashicorp/go-multierror v1.1.0
2021-11-02T17:48:30.3711810Z go: downloading golang.org/x/text v0.3.6
2021-11-02T17:48:31.0311310Z go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
2021-11-02T17:48:31.0438220Z go: downloading github.com/hashicorp/hcl v1.0.0
2021-11-02T17:48:31.1632690Z go: downloading github.com/pelletier/go-toml v1.9.4
2021-11-02T17:48:31.2856870Z go: downloading gopkg.in/yaml.v2 v2.4.0
2021-11-02T17:48:31.3107750Z go: downloading github.com/hashicorp/errwrap v1.0.0
2021-11-02T17:48:50.6767980Z # github.com/99designs/keyring
2021-11-02T17:48:50.6876420Z ##[error]../../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:136:8: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
2021-11-02T17:48:50.6975510Z ##[error]../../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:181:9: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
2021-11-02T17:48:50.7079880Z ##[error]../../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:181:21: undefined: keychain.Access
2021-11-02T17:48:50.7081830Z FAIL	github.com/endorama/2ami [build failed]
2021-11-02T17:48:50.7181870Z ##[error]../../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:187:9: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
2021-11-02T17:48:50.7282550Z ##[error]../../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:187:21: undefined: keychain.Access
2021-11-02T17:48:50.8041020Z ##[error]Process completed with exit code 2.

The reported issues are:

• kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
• undefined: keychain.Access

Referenced at:

• keyring/keychain.go

  Line 136 in 68286ee

  kcItem.SetAccess(nil)

• keyring/keychain.go

  Line 181 in 68286ee

  kcItem.SetAccess(&gokeychain.Access{

• keyring/keychain.go

  Line 187 in 68286ee

  kcItem.SetAccess(&gokeychain.Access{

Hi, I'm the (main) maintainer of aws-okta. We don't sign our Mac OS X builds, so we constantly run into the issue of users having to reauthorize aws-okta for all its Keychain items, every time the binary changes (upgrades, new builds during dev, ...).

Probably, the best solution is to figure out signing, but my workaround was to collapse all items into a single JSON item. But it made me think that maybe this is useful to keyring more generally. It's essentially a JSON-backed, unencrypted keyring in a second Keychain-backed keyring item.

First off, I'm wondering if this is actually useful to any other users.

Second, I'm wondering how this should be implemented.

One approach is to add a new type like:

type KeyringInKeyring struct {
	Parent *Keyring
	ParentItemKey string
}

That would basically proxy Keyring.* methods to the parent keyring.

I'm not sure of the general utility of this approach. Another would be to solve this specific use case with a new backend SingleItemKeychain that would embed all Items in a single Keychain item.

Thoughts? To be clear, I'm considering contributing this myself.

FYI

As you are requiring go 1.14 in your go.mod, you could use os.UserHomeDir() and remove the dependency to github.com/mitchellh/go-homedir.

Likewise, os.UserConfigDir() could be useful.

keyring throws compilation erros due to incompatibility with the latest version of go-keychain.

keybase/go-keychain#60

# github.com/99designs/keyring
428./keychain.go:151:9: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
429./keychain.go:151:21: undefined: keychain.Access
430./keychain.go:157:9: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
431./keychain.go:157:21: undefined: keychain.Access
432./keychain.go:187:9: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess

I'm getting a compile error when trying to use keyring...

github.com/99designs/keyring/keychain.go:85:16: unknown field 'SelfUntrusted' in struct literal of type keychain.Access

While attempting to update my go dependencies for aws-vault and keyring I ran across this issue.

$ go get github.com/99designs/keyring
go: finding github.com/99designs/keyring latest
# github.com/99designs/keyring
../../go/pkg/mod/github.com/99designs/[email protected]/pass.go:34:3: cannot use pass (type *passKeyring) as type Keyring in return argument:
        *passKeyring does not implement Keyring (missing GetMetadata method)

My guess is that #29 from @lox forgot to add this here and now it's a problem when loading it with go modules.

github.com/keybase/go-keychain

cgo-gcc-prolog:81:11: warning: 'SecKeychainCreate' is deprecated: first deprecated in macOS 12.0 - Custom keychain management is no longer supported [-Wdeprecated-declarations]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Security.framework/Headers/SecKeychain.h:301:10: note: 'SecKeychainCreate' has been explicitly marked deprecated here
cgo-gcc-prolog:139:11: warning: 'SecKeychainLock' is deprecated: first deprecated in macOS 12.0 - Custom keychain management is no longer supported [-Wdeprecated-declarations]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Security.framework/Headers/SecKeychain.h:357:10: note: 'SecKeychainLock' has been explicitly marked deprecated here
cgo-gcc-prolog:159:11: warning: 'SecKeychainOpen' is deprecated: first deprecated in macOS 12.0 - Custom keychain management is no longer supported [-Wdeprecated-declarations]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Security.framework/Headers/SecKeychain.h:287:10: note: 'SecKeychainOpen' has been explicitly marked deprecated here
cgo-gcc-prolog:183:11: warning: 'SecKeychainUnlock' is deprecated: first deprecated in macOS 12.0 - Custom keychain management is no longer supported [-Wdeprecated-declarations]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Security.framework/Headers/SecKeychain.h:347:10: note: 'SecKeychainUnlock' has been explicitly marked deprecated here
cgo-gcc-prolog:203:11: warning: 'SecTrustedApplicationCreateFromPath' is deprecated: first deprecated in macOS 10.15 - No longer supported [-Wdeprecated-declarations]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Security.framework/Headers/SecTrustedApplication.h:59:10: note: 'SecTrustedApplicationCreateFromPath' has been explicitly marked deprecated here

Related: #56, #59, #60

Problem

Encountering the following errors on macOS 10.15 Catalina with the latest keyring release:

cgo-gcc-prolog:203:11: warning: 'SecTrustedApplicationCreateFromPath' is deprecated: first deprecated in macOS 10.15 - No longer supported [-Wdeprecated-declarations]
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Security.framework/Headers/SecTrustedApplication.h:59:10: note: 'SecTrustedApplicationCreateFromPath' has been explicitly marked deprecated here

This is reported in keybase/go-keychain #58 and seems to have been fixed in keybase/go-keychain #60.

Proposal

Upgrade the corresponding dependency to use a newer version of keybase/go-keychain.

Hi there,

I'm trying to cross-compile from linux with OSX as target but I get the following error

building for OS darwin ARCH amd64
# github.com/keybase/go-keychain
../../../go/pkg/mod/github.com/keybase/[email protected]/corefoundation_1.10.go:9:10: fatal error: 'CoreFoundation/CoreFoundation.h' file not found
#include <CoreFoundation/CoreFoundation.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
# net
/usr/local/go/src/net/cgo_bsd.go:15:72: could not determine kind of name for C.AI_MASK
# os/user
/usr/local/go/src/os/user/getgrouplist_darwin.go:16:11: warning: implicit declaration of function 'getgrouplist' is invalid in C99 [-Wimplicit-function-declaration]

I saw the issue #23 but I don't understand how the PR #20 could fix this issue, that PR "just" add cgo build tag

What I have tried

I've tried the following build commands:

• GOOS=darwin GOARCH=amd64 go build works without building keychain
• GOOS=darwin GOARCH=amd64 CGO_ENABLED=1 go build doesn't work, fail withe the above error
• CC=clang-11 GOOS=darwin GOARCH=amd64 CGO_ENABLED=1 go build doesn't work, fails with the above error
• CC=clang-11 CXX=clang++-11 GOOS=darwin GOARCH=amd64 CGO_ENABLED=1 go build doesn't work, faisl with the above error

Build environment

go version:

go version go1.15.3 linux/amd64

keyring module version

github.com/99designs/keyring v1.1.6

clang-11 --version:

Ubuntu clang version 11.0.0-++20201008094836+176249bd673-1~exp1~20201008080315.114
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

clang++-11 --version:

Ubuntu clang version 11.0.0-++20201008094836+176249bd673-1~exp1~20201008080315.114
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

OS version (lsb_release -a):

No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.5 LTS
Release:	18.04
Codename:	bionic

Hello-

I have forked github.com/gsterjov/go-libsecret as r00t2.io/gosecret (GitHub mirror as github.com/johnnybubonic/gosecret) and rewritten it (I maintain two branches; v0 for backwards-compat and v1 and onwards for a much-improved API). Among such improvements are:

• Full documentation
• Drop-in compatibility with go-libsecret (see docs) in the v0 branch
• Gofmt'd code
• MUCH easier to use functionality
  • And full functionality, matching SecretService spec
  • With extensibility supported, as the Dbus connections/objects for each struct are now exported.
• And much more.

Please feel free to let me know if it serves your purposes better.

If you do use it, I recommend exporting the underlying structs so that a keyring may be typeswitched into the original and the methods for those structs able to be used by users of this library. 👍

I'm building in a docker ubuntu 20.04 Linux container.

Below is a snippet of the code I'm using, the the relevant part of the Docker build file that I'm using to run my server in.
My server compiles code on demand for different environments. The code shown is part of that file that is being compiled when the error is thrown.

I have it working

• Linux->Windows
• Windows->Linux
• Windows-> Darwin
• Darwin->Windows
• Darwin->Linux

The last thing I need is to be ale to compile on Linux for Darwin. Since I'm running in Docker it is a must have, the other combinations make development easy.

I can build other go files that doesn't use this library.

The code is fairly straight forward. Here is the portion that uses this library

import (
	set_var "components/payload/macos/commands/set-var"
	"components/payload/types"
	types3 "components/protocol/types"
	"github.com/99designs/keyring"
	"os/user"
	"strconv"
)

[some code redacted]

	kr, e1 := keyring.Open(keyring.Config{})
	if e1 != nil {
		response.Value = []byte("Failed to get a keyring object.  This one is not on me.")
		return
	}

	keys, e1 := kr.Keys()
	if err != nil {
		response.Value = []byte("Failed getting a list of keys from keyring\n")
		return
	}
	var item keyring.Item
	occurred := 0
	for _, k := range keys{
		if k == name {
			item, err = kr.Get(k)
			if err == nil {
				if(skip == occurred) {
					response.Value = []byte( item.Data)
				}else{
					occurred++
				}
				return
			}else{
				response.Value = []byte("I was not able to retrieve any values, I think they might have not allowed the activity.")
				return
			}
		}
	}

I have built the container with

FROM ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get -y update 
RUN apt-get -y upgrade
RUN apt-get install -yq wget git build-essential mysql-client

#required for windows xcompile
RUN apt-get install -yq mingw-w64 libc6-dev-i386 

#required for darwin xcompile
RUN apt-get install -yq libpython2.7 libpython2.7-dev
RUN apt-get install -yq llvm libicu-dev clang 

RUN wget -c https://dl.google.com/go/go1.16.5.linux-amd64.tar.gz -O - | tar -xz -C /usr/local

ENV PATH /usr/local/go/bin:$PATH
ENV GOROOT /usr/local/go
ENV GOPATH /var/go

# Now ot create the actual osx xcompiler
RUN apt-get install -yq gcc g++ zlib1g-dev libmpc-dev libmpfr-dev libgmp-dev
RUN apt-get install -yq cmake patch python libssl-dev lzma-dev libxml2-dev
RUN git clone https://github.com/tpoechtrager/osxcross.git

#Note you can get update tarbars at https://developer.apple.com/download/all/?q=xcode
# the instructions for creating the tarball are in the osxcross repo
# we are storing the current on in gitlab xcode 12.5.1

COPY ./deploy/osxcross/MacOSX11.3.sdk.tar.bz2 /osxcross/tarballs/MacOSX11.3.sdk.tar.bz2
#COPY ./deploy/osxcross/MacOSX11.sdk.tar.bz2 /osxcross/tarballs/MacOSX11.sdk.tar.bz2
#COPY ./deploy/osxcross/MacOSX10.15.sdk.tar.bz2 /osxcross/tarballs/MacOSX10.15.sdk.tar.bz2

ENV CXX clang++ 
ENV CC clang 

RUN cd /osxcross
RUN  UNATTENDED=1 SDK_VERSION=11.3 /osxcross/build.sh
ENV PATH /osxcross/target/bin:$PATH

# I don't think I need this. but I'll leave it here for now.
#RUN /osxcross/build_gcc.sh


#reset the build in environment    
RUN cd ..
ENV CXX g++ 
ENV CC gcc 

I add a bunch of modules after this.

Since the server is building this code, I log much of the environment settings in place in order to debug issues like this:

================================================================
=  Build File 
=  OS = darwin
=  ARCH = amd64
================================================================
Old PATH = /osxcross/target/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
New PATH = /osxcross/target/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/go/bin
*****************************
**** Environment : darwin - amd64
*****************************
*****************************
CGO_ENABLED=1
GO111MODULE=off
CXX=o64-clang++
GOOS=darwin
HOME=/root
GOARCH=amd64
PWD=/usr/lib/test
GOROOT=/usr/local/go
PATH=/osxcross/target/bin:/usr/local/go/bin:/usr/share/swift/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/go/bin
OLDPWD=/
GOPATH=/var/go
CC=o64-clang
CGO_CFLAGS=-I/usr/share/swift/usr/lib/swift
OSXCROSS_NO_INCLUDE_PATH_WARNINGS=1
MACOSX_DEPLOYMENT_TARGET=11.3
*****************************

Command : [go build -ldflags -extldflags "-static" -s -w -trimpath -gcflags -trimpath=/usr/lib/test/server -asmflags -trimpath=/usr/lib/test/server -o /usr/lib/test/server/L01F/darwin/amd64/test /usr/lib/test/server/L01F/test.go]

[0] go
[1] build
[2] -ldflags
[3] -extldflags "-static" -s -w
[4] -trimpath
[5] -gcflags
[6] -trimpath=/usr/lib/test/server
[7] -asmflags
[8] -trimpath=/usr/lib/test/server
[9] -o
[10] /usr/lib/test/server/L01F/darwin/amd64/test
[11] /usr/lib/test/server/L01F/test.go
------- err | # github.com/99designs/keyring
------- err | /var/go/src/github.com/99designs/keyring/keychain.go:136:8: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
------- err | /var/go/src/github.com/99designs/keyring/keychain.go:181:9: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
------- err | /var/go/src/github.com/99designs/keyring/keychain.go:181:21: undefined: keychain.Access
------- err | /var/go/src/github.com/99designs/keyring/keychain.go:187:9: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
------- err | /var/go/src/github.com/99designs/keyring/keychain.go:187:21: undefined: keychain.Access
Error darwin - amd64 : exit status 2

FIX FOR THIS ERROR
EDIT : Note to fix the above error I had found a thread that suggested adding the following line to the go.mod.

replace github.com/keybase/go-keychain => github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4

so the output of that is noted below.

In the context of exposing the keyring in a rpc endpoint (for transaction signing) it would be nice to add the ability to lock or unlock the keyring securely.

one could also specify the duration of the unlock so that only during this timeframe, access to the keyring is possible

(for ref lockAccount and unlockAccount in ethereum rpc endpoints)

Consider the following scenario:

• software X v1 comes out, using keyring as a backend to store credential.
• someone use it on a system where only the fallback encrypted file backend is supported, credentials are stored there.
• software X v2 comes out, with an updated keyring that support a real keyring for this system
• user software X, the new backend is selected and all the credentials are not available anymore :-(

What about a "cascading keyring" that would wrap all the available backend for the system at hand and cascade the reads to the next backend if the previous one doesn't have it ? That would at least fix the disappearing credentials.

Bonus point would be to have a way to move all the credentials in the same backend, either up or down the stack to consolidate the storage in a single backend. That would allow to implement automatic migrations.

I have set keyring:default:key1, keyring:default:key2 already, and can be found in windows Credential Manager.
then test following code:

kr, err := keyring.Open(keyring.Config{
    AllowedBackends: []keyring.BackendType{keyring.WinCredBackend},
})
if err != nil {
    t.Fatal(err)
}
keys, err := kr.Keys()
if err != nil {
    t.Fatal(err)
}

it would return empty keys. but if I call kr.Set() before kr.Keys(), it would return keyring:default:key1 and keyring:default:key2

kr, err := keyring.Open(keyring.Config{
    AllowedBackends: []keyring.BackendType{keyring.WinCredBackend},
})
if err != nil {
    t.Fatal(err)
}
item1 := keyring.Item{
    Key:  "test3",
    Data: []byte("loose lips sink ships"),
}
err = kr.Set(item1)
if err != nil {
    t.Fatal(err)
}
keys, err := kr.Keys()
if err != nil {
    t.Fatal(err)
}

If keyring is a standalone lib, we should remove references to aws vault

$ ag aws.?vault
keychain_test.go
155:	return filepath.Join(os.TempDir(), fmt.Sprintf("aws-vault-test-%d.keychain", time.Now().UnixNano()))

kwallet.go
15:	APPID             = "aws-vault"
16:	FOLDER            = "aws-vault"

file.go
19:	if password := os.Getenv("AWS_VAULT_FILE_PASSPHRASE"); password != "" {
53:		dir = filepath.Join(home, "/.awsvault/keys/")

README.md
4:Keyring provides utility functions for and a common interface to a range of secure credential storage services. Originally developed as part of [AWS Vault](https://github.com/99designs/aws-vault), a command line tool for securely managing AWS access from developer workstations.
8:  * [Secret Service](https://github.com/99designs/aws-vault/pull/98)
9:  * [KDE Wallet](https://github.com/99designs/aws-vault/pull/27)
10:  * [Encrypted File](https://github.com/99designs/aws-vault/pull/63)
47:  * Ideally verify that [aws-vault](https://github.com/99designs/aws-vault) works with your changes (optional)

secrets.go
12:const secretsCollection = "awsvault"
79:		return &secretsError{"The awsvault collection does not exist. Please add a key first"}
119:	// pack the secret into the aws-vault item

https://github.com/99designs/keyring/blob/master/file.go#L43

After finding the code linked above, I noticed that it doesn't account for ~accountName and will do bad things when you try

[~] $ aws-vault --backend=file --file-dir=~agargiulo/thisCanNotWorkCanIt  list
aws-vault: error: list: mkdir /Users/agargiuloagargiulo: permission denied

But I'm not sure how to go about fixing this. It's nice to support ~/ expansion but ~agargiulo is 100% not the same as /Users/agargiuloagargiulo

Does Keyring detect all available backend keys on the current OS in runtime? I am getting different results when executing binaries on macOS but built in different OS environments.

This is the output I got when running on macOS

> go run main.go show
keychain
pass
file

This is the output after executing the binary built in a Linux container, somehow the keychain is missing in the list when ran on macOS

# building a darwin/amd64 binary in Golang Docker image
> docker run -it golang:latest bash -v $PWD/.
> export GOOS=darwin
> export GOARCH=amd64
> export GO111MODULE=on
> go build  
> exit # exit out of the container
> ./bkb show
pass
file

Hello, first of all thank you for this project.

I'm trying to cross-compile this package from Linux to MacOSX but I'm having some issues.

My compile instruction is GOOS=darwin GOARCH=amd64 CGO_ENABLED=1 go build, but it fails:

$ GOOS=darwin GOARCH=amd64 CGO_ENABLED=1 go build
# os/user
.../goenv/versions/1.10/src/os/user/getgrouplist_darwin.go: In function ‘mygetgrouplist’:
.../goenv/versions/1.10/src/os/user/getgrouplist_darwin.go:14:11: warning: implicit declaration of function ‘getgrouplist’ [-Wimplicit-function-declaration]
  int rv = getgrouplist(user, (int) group, buf, ngroups);
           ^
# github.com/99designs/keyring/vendor/github.com/keybase/go-keychain
vendor/github.com/keybase/go-keychain/corefoundation_1.10.go:9:43: fatal error: CoreFoundation/CoreFoundation.h: No such file or directory
compilation terminated.
# net
.../goenv/versions/1.10/src/net/cgo_bsd.go:15:72: could not determine kind of name for C.AI_MASK

Is cross-compiling like this possible?
I tried to compile it with xgo, and this "works" ( I didn't tested the resulting binary but at least it compiles ), but I was wondering if you could provide some suggestions/instructions on how to cross-compile this package.

Thank you!

Would be amazing if you could start following semantic versioning, will help those using this project manage updates.

Thanks.

My app does not use "secret-service" backend on Linux, however each execution leaves dbus-daemon running, which eventually leads to OutOfMemory.
godbus.Conn has both Open() and Close(), however keyring does not have a Close() method, so there is no way to request Close() in godbus.Conn. I'm not 100% sure Close() will help, but it seems godbus does a lot of clean up in this method.

env
Linux 4.15.0-96-generic
go1.14
Ubuntu 18.04.4 LTS
pass v1.7.1

issue
Hi, it seems when using colon in ServiceName name like abcd:abcd, the keys is not saved in the backend.

package main

import (
  "fmt"
  "github.com/99designs/keyring"
)

func Store(sname string) {
  ring, _ := keyring.Open(keyring.Config{
    ServiceName: sname,
  })
  fmt.Println(ring)
  _ = ring.Set(keyring.Item{
  Key: "foo",
	Data: []byte("secret-bar"),
  })
}

func GetKey(sname string) {
  ring, _ := keyring.Open(keyring.Config{
    ServiceName: sname,
  })
  i, _ := ring.Get("foo")
  k, _ := ring.Keys()
  fmt.Printf("%s", i.Data)
  fmt.Println(k)
}


func main() {
  name := "abcd"
  Store(name)
  GetKey(name)

}

go run test.go
&{abcd 0xc000182040 0xc000202f40 0xc000202000}
secret-bar[foo]

ServiceName changed to abcd:abcd

go run test.go
&{abcd:abcd 0xc000182040 <nil> 0xc000182980}
[]

Thiis pattern is used in a friend's app and works under MacOS

Hi 👋 thanks for the library and for the work on aws-vault,

I'm not sure if this is even possible... reading the source it seems like it is, but I am no Go expert. I would like to implement a lastpass backend for this library. Is that something you would be amenable to receiving?

When I built my cli app, a warning occured:

# github.com/keybase/go-keychain
cgo-gcc-prolog:202:11: warning: 'SecTrustedApplicationCreateFromPath' is deprecated: first deprecated in macOS 10.15 - No longer supported [-Wdeprecated-declarations]
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Security.framework/Headers/SecTrustedApplication.h:59:10: note: 'SecTrustedApplicationCreateFromPath' has been explicitly marked deprecated here

Maybe the package should be updated?

Behaviour should be like ssh where it doesn't prompt for your passphrase if there is no password

when running this sample of code on headless linux (CentOS Linux release 8.4.2105)
``
ring, _ := keyring.Open(keyring.Config{
ServiceName: "example",
})

err = ring.Set(keyring.Item{
Key: "foo",
Data: []byte("secret-bar"),
})

if(err != nil) {

log.Fatal(err)

}

i, _ := ring.Get("foo")

fmt.Printf("%s", i.Data](url))
``
i get this error message : No such interface 'org.freedesktop.DBus.Properties' on object at path /

also when trying the below:

dbus-run-session -- sh
gnome-keyring-daemon --unlock
password CTRL-D

then run the above code i again still getting an error
No such interface 'org.freedesktop.Secret.Collection' on object at path /org/freedesktop/secrets/collection/login

any idea how to make the above code working with headless linux ?

99designs/aws-vault#131 indicates the changes in that branch need to be upstreamed to this repo in order to support TouchID. Opening an issue here.

Would it be as simple as moving the files from that branch into a PR for this repo?

The environment variable PASSWORD_STORE_DIR is used to override the password storage directory, see https://git.zx2c4.com/password-store/about/

I think there is a windows key chain lib floating around on GitHub.

Thank you for all the effort dedicated to aws-vault !

My setup is such that the pass storage location is behind a symlink

ls -lh ~/.password-store

/home/user/.password-store -> /the/real/path/to/.password-store

Keyring v1.1.4 is not able to follow the symlink to the password store directory to fetch sensitive information.

Example aws-vault with keyring v1.1.4 failing to retrieve secrets from the pass backend

aws-vault --version
v5.2.1

aws-vault --backend=pass  exec pers/aws/profiles/testground -- aws sts get-caller-identity
aws-vault: error: exec: Error getting temporary credentials: profile pers/aws/profiles/testground: credentials missing

PR to follow.

Calling keyring.Set() using the wincred backend on Windows10 and it constantly is returning error="The stub received bad data. but subsequent Get() calls returns the saved data. Hence it seems that I should be ignoring this error? Not getting any errors with the file or macOS keychain backends.

Using keyring v1.1.6

At the minute it's hard to set per-backend global options like a passphrase (the file provider), or per-Item options like "synchronize to keychain", or "only store on this device". I'd like to change the API for accessing keyring to make this possible, ideally in a typesafe way that works with the per-platform conditional compilation. These are some options I'm considering, would love feedback.

Global Options

keyring.Options struct

This would move to a structure more like the [aws.Config](https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config)]:

opts := keyring.NewOptions().
  WithDefaultKeychain().
  WithFilePassword("llamas rock").
  WithAllowedBackends(keyring.KeychainBackend, keyring.FileBackend)
  
kr, err := keyring.Open("aws-vault", opts)

Typed keyrings

The advantage of this approach is it keeps extensions within the scope of the objects. It increases complexity in that you have to have the provider implemented in a globally compiled file with the per-platform stuff conditionally compiled in a different file. Not sure how possible that would be.

kr, _ := keyring.Open("aws-vault", func(k keyring.Provider) {
  if keychain, ok := kr.(keyring.KeychainBackend); ok {
    kr.SetKeychain("login")
  }
})

kr, _ := keyring.OpenWithPreference("aws-vault", 
  []keyring.Backend{keyring.KeychainBackend, keyring.FileBackend},
  func(k keyring.Provider) bool{
    // ...
  }
})

Use a context and string based keys

You could also use the context for cancellation I guess. I'm still not sold on our context overlords.

ctx := context.Background()

for k, v := range map[string]interface{}{
  "name": "aws-vault",
  "keychain.name": "login",
  "file.passphrase": "llamas rock".
} {
  ctx = context.WithValue(ctx, k, v)
}

kr, _ := keyring.Open(ctx)

Hello. Currently on KDE Plasma5 the secret service is not implemented by kwallet and I use KeePassXC feature for that purpose (apps like vscode, skype... need it for storing credentials). My question is can I implement such a service using your library? If not do you know the right library if exists. And if it is possible with your library can you provide a link to sample usage or a code sample. Thanks in advance.

It looks like this project uses two DBUS bindings packages:

• github/aulanov/go.dbus (kwallet.go)
• github.com/godbus/dbus (libsecret.go)

Is there any reason not to use the same one for both backends?

https://github.com/jaraco/keyring
https://github.com/frankosterfeld/qtkeychain
https://github.com/tmc/keyring
https://github.com/zalando/go-keyring
https://github.com/hwchen/keyring-rs
https://pypi.python.org/pypi/keyring
https://github.com/docker/docker-credential-helpers

I have no idea. Should they be used? Maybe? Did they one time know a guy who was a password storage subsystem? Almost certainly.

That is all.

./../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:55:8: query.SetMatchSearchList undefined (type keychain.Item has no field or method SetMatchSearchList)
../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:55:28: undefined: keychain.NewWithPath
../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:115:34: undefined: keychain.Keychain
../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:124:12: queryItem.SetMatchSearchList undefined (type keychain.Item has no field or method SetMatchSearchList)
../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:136:8: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:146:9: undefined: keychain.Keychain
../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:166:9: kcItem.UseKeychain undefined (type keychain.Item has no field or method UseKeychain)
../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:181:9: kcItem.SetAccess undefined (type keychain.Item has no field or method SetAccess)
../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:181:21: undefined: keychain.Access
../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:267:36: undefined: keychain.Keychain
../../go/pkg/mod/github.com/99designs/[email protected]/keychain.go:181:21: too many errors

If you are like me and use go releaser and have started using this package, you will notice that keychain doesn't appear on macOs. This is because the keychain is disabled in this package if CGO is not enabled. However, it didn't seem trivial to set up go releaser with CGO enabled until i found this https://github.com/plentico/osxcross-target

So if you are using go releaser with gitub action, use that. It worked flawlessly for me

This is affecting aws-vault. For some reason, the calls from this are causing the keyring to freeze on future calls, but if it is the process to spawn the gnome-keyring then it also freezes for future calls as well as freezes other applications making calls to the keyring.

Anyone with this issue you can run pkill gnome-keyring and re-run the command and it works consistently.

Though installing another password manager may work too.

Current code is incompatible with the new release: https://kde.org/announcements/kde-frameworks-5.61.0.php

KWallet Framework

Move kwalletd initialization earlier (bug 410020)
Remove kde4 migration agent completely (bug 400462)

Specifically: https://bugs.kde.org/show_bug.cgi?id=40046
You can check the last comment there:

KDE Frameworks 5.61 does NOT provide the org.kde.kwalletd interface any more, at if the KDE Neon packages are used.

keyring is using the old interface and now does not work with error (in aws-vault)

❯ aws-vault --debug list                                  
2019/08/26 11:17:18 [keyring] Considering backends: [kwallet]
2019/08/26 11:17:18 [keyring] Failed backend kwallet: The name org.kde.kwalletd was not provided by any .service files
aws-vault: error: Specified keyring backend not available, try --help

This may be a user configuration issue: when pass is the backend used, the request (via aws-vault for requests errors rather than prompting for the passphrase. There are a few ways around this. pass CREDNAME invokes gpg-agent within the user session.

Ideally exception handling around this would catch this failure mode--but I could be wildly off base!

Building fails on Windows ARM64 with Go 1.17:

PS C:\Users\amits\work\github.com\99designs\keyring> $env:GOOS="windows"
PS C:\Users\amits\work\github.com\99designs\keyring> $env:GOARCH="arm64"
PS C:\Users\amits\work\github.com\99designs\keyring> go build
# golang.org/x/sys/windows
..\..\..\..\go\pkg\mod\golang.org\x\[email protected]\windows\dll_windows.go:24:6: missing function body
..\..\..\..\go\pkg\mod\golang.org\x\[email protected]\windows\dll_windows.go:25:6: missing function body
..\..\..\..\go\pkg\mod\golang.org\x\[email protected]\windows\zsyscall_windows.go:2533:38: undefined: WSAData
..\..\..\..\go\pkg\mod\golang.org\x\[email protected]\windows\zsyscall_windows.go:2773:51: undefined: Servent
..\..\..\..\go\pkg\mod\golang.org\x\[email protected]\windows\zsyscall_windows.go:2787:50: undefined: Servent

Upgrading the golang.org/x/sys package gets the build to pass:

> go get golang.org/x/sys
go get: upgraded golang.org/x/sys v0.0.0-20190712062909-fae7ac547cb7 => v0.0.0-20211216021012-1d35b9e2eb4e

> go build #works

>

Example on first-use of an application:

• The macos keychain backend will return an empty list and nil error the first time Keys() (or Get()) is called:

2019/06/01 07:11:51 [keyring] Considering backends: [keychain pass file]
2019/06/01 07:11:51 [keyring] Querying keychain for service="vault", keychain=""
2019/06/01 07:11:51 [keyring] Found 0 results
$ echo $?
0

• The libsecret backend will return a secretError with message that the keychain does not exist:

2019/06/01 07:13:25 [keyring] Considering backends: [secret-service kwallet pass file]
Error: Failed to read tokens from backend storage: The collection "vault" does not exist. Please add a key first
$ echo $?
255

The returned error is a generic type secretError and so I could handle this by wrapping keyring.List() and checking err.Error() contains the string "Please add a key first", but I am wondering what the expected behavior is for the library itself and if perhaps libsecret should be modified to not return an err if the collection does not yet exist?

I haven't checked the other backend types for how they handle first-use of Get and Keys yet.

https://github.com/99designs/keyring/blob/master/wincred.go#L78

It looks like this is a remnant of the time before this library was a separate project from aws-vault application.

Of note, based on cursory testing, it seems that the result of this is that windows credentials will always show up under aws-vault: service name. When I set ServiceName variable on the configuration myself to xyz, the credential appears in Windows Credential manager as aws-vault:xyz:<my-key>. I am new to the project, but it looks like simply removing "aws-vault:" + from this line would resolve the issue.

Also, I checked to see if this would break aws-vault project, and I don't think it will. I actually think it fixes a cosmetic bug there, but I don't use aws-vault so I can't be sure. Based on this line and my own test, I believe that currently, aws-vault creds are probably showing up in Windows Credential Manager as aws-vault:aws-vault:<some_cred>

https://github.com/99designs/aws-vault/blob/master/cli/global.go#L83

Perhaps this will be trivial to prove and fix for someone on the project.

Using OSX Catalina (10.15.7) and calling keyring.Remove(key) returns an error Keychain Error. (-25244) instead of removing the provided key from the OSX keychain. For now I'm just calling keyring.Set() and over writing the old values with zero.

you can see how I'm using the code here if you have issues reproducing: https://github.com/synfinatic/onelogin-aws-role/blob/main/cmd/keyring.go#L153

Apologies if this is known or documented already.

I ran into a head-scratcher of an issue with the secret-service backend and I was able to resolve it by changing the collection name (ServiceName in keyring.Config).

Example code:

package main

import (
	"log"

	"github.com/davecgh/go-spew/spew"
	"github.com/99designs/keyring"
)

func main() {
	//app := "vault-token-helper"
	//app := "toy"
	//app := "vault_token_helper"
	app := "vault"
	cfg := keyring.Config{
		ServiceName:              app,
		KeychainTrustApplication: true,
		KeychainSynchronizable:   false,
		// FileDir:
		// LibSecretCollectionName
		// PassDir
		// PassCmd
		// PassPrefix (our app name?)
	}

	kr, err := keyring.Open(cfg)
	if err != nil {
		log.Fatal(err)
	}

	i := keyring.Item{
		Key:                       "fookey",
		Data:                      []byte("foobartokenbaz"),
		Label:                     "Vault - fookey",
		Description:               "hashicorp Vault token",
		KeychainNotSynchronizable: true,
	}
	err = kr.Set(i)
	if err != nil {
		log.Fatal(err)
	}

	i2, err := kr.Get("fookey")
	if err != nil {
		log.Fatal(err)
	}
	spew.Dump(i2)
}

The issue is with the ServiceName: app, provided in keyring.Config. When using names such as vault-token-helper, vault_token_helper, the library would create a new collection on every run:

(screenshot from seahorse)

When I use a shorter collection name such as toy or vault I get the expected behavior - the collection is only created once and future calls to Set() update the existing item rather than creating a new collection on every run.

It's also worth noting that if you do not pass a ServiceName in the keyring.Config the library will use the default secret-service collection name which also exhibits the "create new collection on every call" behavior.

At this time I'm not sure if this is a bug in 99designs/keyring, a bug in gsterjov/go-libsecret, or just some limitation of collection naming in the DBus Secret Service API, but figured I would at least document what I've found in case someone knows the answer or runs into a similar issue.

I believe the login keychain is already open when the user is logged into their Mac, is there a way to turn off the popup when storing in login keychain after verifying once?

I found that at the moment, choosing the values for the config is fairly hard. It requires to spend a good amount of time reading the code to see what backend do with what, understand the default values, understand in what situation which backend is available ...

It would be very helpful I think if you could:

• provide a recommended configuration in the Readme
• say in Config's doc what backend use the value and what the default is
• maybe have a table in the Readme that say which backend are active depending on the system ?

The current tag 1.0.0 isn't recognized by go modules; you need a v prefix. Without v it basically just ignores the tag and we're back to SHA refs.

Related: #24

Hi, looks like if you've got a dash in ServiceName in the secret service backend, it will fail to find it next time.

Here's a test:

ad34891

ubuntu@primary:~/Home/dev/src/github.com/99designs/keyring$ go test -count 1 -run TestRoundtripPublicOnly -v ./...
=== RUN   TestRoundtripPublicOnly
    libsecret_test.go:172: The specified item could not be found in the keyring
2021/03/30 17:56:17 collection: /org/freedesktop/secrets/collection/session
2021/03/30 17:56:17 collection with _2d: /org/freedesktop/secrets/collection/keyring_2dtest
    libsecret_test.go:185: The specified item could not be found in the keyring
--- FAIL: TestRoundtripPublicOnly (2.67s)
FAIL
FAIL    github.com/99designs/keyring    2.678s
?       github.com/99designs/keyring/cmd/keyring        [no test files]
FAIL

^ running gnome-keyring-daemon in the background.

I noticed through dbus-monitor that the actual collection name is keyring_2dtest, with some sort of escaping applied to the - character. I'm not deep into DBus/Secret Service, but it looks like sort of like URI escaping, like url.PathEscape.

Needs some more investigation, but yeah, 100% busted.