Sorry to be off topic here, but I was enticed to come on this thread from its title, as I'm too, unsure how to get started with charm.

In fact, it's a bit worse, I wonder what I could use charm for. I tried to understand the possible usages from its website but was left wondering.

Maybe there is some documentation or a simple wiki page ? Or better, a video showing us what good things we can do with it ?

Originally posted by @stephaneeybert in #93 (comment)

The current help dialogue leaves much to be desired and doesn't really help new users understand how to backup and restore keys

when we run charm backup-keys also provide a seed phrase

I get why - but it might be nice to at least have an option to choose and use a pre-existing encrypted ssh key, or to encrypt the key and save the password for it in the keychain (I'm on macOS).

type successMsg bool should be a struct{} type since successMsg is literally just a success message. We're using other messages for errors.

https://github.com/charmbracelet/charm/blob/master/ui/linkgen/linkgen.go#L34
https://github.com/charmbracelet/charm/blob/master/ui/link/link.go#L41

See: charmbracelet/skate#2

Cool tool!

I was wondering why RSA is the default algorithm instead of Ed255 (which you already implemented)?

Also, if you supported P256, there could be a nice integration with TPM 2.0 (e.g. with go-tpm), to have keys that are actually bound to the machine they're generated on vs. lying around in ~/.local/share/charm.

The new /.well-known/openid-configuration endpoint recently introduced by #59 is no longer returning a valid jwks_uri:

[rubiojr@nano tavern] curl localhost:35354/.well-known/openid-configuration
{"issuer":"","authorization_endpoint":"","token_endpoint":"","jwks_uri":"http://:35354/v1/public/jwks","userinfo_endpoint":"","id_token_signing_alg_values_supported":null}

I think this is the result of #76 removing localhost as the default in the server configuration struct.

Show current host in the UI -> show the URL to the host or its nickname so it's clear where their data is hosted

(mentioned in the Slack)

When running the Docker container, it creates files with root as the owner.

jeff@library:~/charm-r/data$ ls -al
total 28
drwxr-xr-x 5 root root 4096 Feb 27 18:08 .
drwxr-xr-x 3 jeff jeff 4096 Feb 27 18:08 ..
drwx------ 2 root root 4096 Feb 27 18:08 db
drwx------ 2 root root 4096 Feb 27 18:08 files
drwx------ 2 root root 4096 Feb 27 18:08 .ssh
-rw-r--r-- 1 root root 8192 Feb 27 18:08 stats

Ideally, there should be a way to set a uid/gid for the application, like how the linuxserver.io folks do their containers that allow environment variables to be set (PUID/PGID). See https://hub.docker.com/r/linuxserver/radarr and https://github.com/linuxserver/docker-baseimage-ubuntu for examples. The base image is where all the smarts are for PUID/PGID.

I was able to create my own docker image using linuxserver/docker-baseimage-ubuntu and adapting the raddar docker config. I'm not sure if that is the approach that should be taken, but the linuxserver folks have already done the hard work.

"charm with custom docker container"
jeff@library:~/charm/data$ ls -al
total 28
drwxr-xr-x 5 jeff jeff 4096 Feb 27 17:47 .
drwxr-xr-x 3 jeff jeff 4096 Feb 27 18:19 ..
drwx------ 2 jeff jeff 4096 Feb 27 17:47 db
drwx------ 3 jeff jeff 4096 Feb 27 17:47 files
drwx------ 2 jeff jeff 4096 Feb 27 17:46 .ssh
-rw-r--r-- 1 jeff jeff 8192 Feb 27 17:47 stats

On a freshly initialized charm server, if you try to copy the first file to charm:/ it breaks the file server:

[rubiojr@nano charm] charm fs cp hello.txt charm:/
[rubiojr@nano charm] charm fs cp hello.txt charm:/bar.txt
Error: server error: 500 Internal Server Error
Usage:
  charm fs cp [charm:]PATH [charm:]PATH [flags]

Flags:
  -h, --help        help for cp
  -r, --recursive   copy directories recursively

server error: 500 Internal Server Error

Relevant logs from the server:

2021/12/18 12:47:06 Opening SQLite db: ./data/db
2021/12/18 12:47:06 Starting SSH server on :35353
2021/12/18 12:47:06 HTTP server listening on: :35354
2021/12/18 12:47:39 ssh api-auth
2021/12/18 12:47:39 JWT for user d0e256c3-e84a-4170-b4b8-98545e66b7cb
2021/12/18 12:47:39 <- POST /v1/encrypt-key [::1]
2021/12/18 12:47:39 Adding encrypted key 7ae84ea9-f714-4e3d-ae1b-6e7fb7c1eefc <nil> for user d0e256c3-e84a-4170-b4b8-98545e66b7cb
2021/12/18 12:47:39 -> 200 OK 0B 14.556183ms
2021/12/18 12:47:39 <- POST /v1/fs//?mode=420 [::1]
2021/12/18 12:47:39 -> 200 OK 0B 455.725µs
2021/12/18 12:48:05 ssh api-auth
2021/12/18 12:48:05 JWT for user d0e256c3-e84a-4170-b4b8-98545e66b7cb
2021/12/18 12:48:05 <- POST /v1/fs//6522161c3dfb9b87027e670c7d95dc8115ad8259a2d269?mode=420 [::1]
2021/12/18 12:48:05 cannot post file: open data/files/d0e256c3-e84a-4170-b4b8-98545e66b7cb/6522161c3dfb9b87027e670c7d95dc8115ad8259a2d269: not a directory
2021/12/18 12:48:05 -> 500 Internal Server Error 29B 399.826µs

It seems the server creates the user directory as a regular file instead of a directory and every other FS operation expects that root to be a directory?

Somewhat related, if you try to copy a file to the root of an already initialized server, you'll get a 500:

[rubiojr@nano charm] charm fs cp hello.txt charm:/hello.txt
[rubiojr@nano charm] charm fs cp hello.txt charm:/
Error: server error: 500 Internal Server Error
Usage:
  charm fs cp [charm:]PATH [charm:]PATH [flags]

Flags:
  -h, --help        help for cp
  -r, --recursive   copy directories recursively

server error: 500 Internal Server Error

[rubiojr@nano charm] ./charm serve
2021/12/18 12:50:40 Opening SQLite db: ./data/db
2021/12/18 12:50:40 Starting SSH server on :35353
2021/12/18 12:50:40 HTTP server listening on: :35354
2021/12/18 12:50:50 ssh api-auth
2021/12/18 12:50:50 Creating user for key 02a9f5a4adccc36187270b76b75d743b5708a417
2021/12/18 12:50:50 JWT for user ba69b17a-6db9-4791-be4a-12bf5401cfec
2021/12/18 12:50:50 <- POST /v1/encrypt-key [::1]
2021/12/18 12:50:50 Adding encrypted key e10b870b-488d-4bb0-8888-eb0d6e7b209a <nil> for user ba69b17a-6db9-4791-be4a-12bf5401cfec
2021/12/18 12:50:50 -> 200 OK 0B 14.337438ms
2021/12/18 12:50:50 <- POST /v1/fs//e9bd3e5aa7f5f458de54de6f2779d130d4d41d7da30b79e780?mode=420 [::1]
2021/12/18 12:50:50 -> 200 OK 0B 506.6µs
2021/12/18 12:50:55 ssh api-auth
2021/12/18 12:50:55 JWT for user ba69b17a-6db9-4791-be4a-12bf5401cfec
2021/12/18 12:50:55 <- POST /v1/fs//?mode=420 [::1]
2021/12/18 12:50:55 cannot post file: open data/files/ba69b17a-6db9-4791-be4a-12bf5401cfec: is a directory
2021/12/18 12:50:55 -> 500 Internal Server Error 29B 389.371µs

It would be nice if the crypto piece of charm could expand to signing data with my identity and verifying signed data.

Im specifically thinking about use cases like:

https://github.com/dominictarr/scuttlebutt
https://www.w3.org/TR/activitypub/
https://saltpack.org/
https://book.keybase.io/docs/cli#signing

where data is replicated and verified using signatures.

Requirements:

1. As a charm user, I would like to be able to generate a signature using my identity from an arbitrary set of bytes.
2. As a charm user, I would like to extract an identity and validate a set of bytes using a signature.

> ssh git.charm.sh                  

z4h: failed to upload bootstrap script

Is that a problem with your server or with my zsh?

P.S. I guess it is with my zsh, using z4h. I tried with sh and it worked. Anyway, shouldn't it work with zsh?

Currently only unencrypted SSH keys are supported, we should also provide support for password encrypted keys.

I was expecting the database file to reside inside the directory specified with --data-dir, based on:

rubiojr@nano charm] ./charm serve -h
                                                            
  Start the SSH and HTTP servers needed to power a SQLite-  
  backed Charm Cloud.

Usage:
  charm serve [flags]

Aliases:
  serve, server

Flags:
      --data-dir string   Directory to store SQLite db, SSH keys and file data

It doesn't seem to be the case currently:

[rubiojr@nano charm] mkdir /tmp/charmtest
[rubiojr@nano charm] cd /tmp/charmtest/
[rubiojr@nano charmtest] charm serve --data-dir foo
2021/12/09 14:13:50 Opening SQLite db: ./data/db
2021/12/09 14:13:52 Starting SSH server on :35353
2021/12/09 14:13:52 HTTP server listening on: :35354
^C
[rubiojr@nano charmtest] find .
.
./foo
./foo/.ssh
./foo/.ssh/charm_server_rsa.pub
./foo/.ssh/charm_server_rsa
./data
./data/stats
./data/files
./data/db
./data/db/charm_sqlite.db
[rubiojr@nano charmtest] 

Looks like that happens in

I'm using CHARM_SERVER_DATA_DIR=charmdata charm serve as a workaround that does the trick.

EDIT: tested with charm pulled from main today

It seems it's not currently possible to setup a custom location for the charm client keys (https://github.com/charmbracelet/charm/blob/main/client/env.go#L11), other than changing the "host" directory for the keys.

Would it be acceptable for DataPath to be part of the Charm client Config and default to the current way we calculate DataPath to avoid breaking changes?

Happy to PR this.

I know ya'll are working on the documentation but I was just trying to use the CLI as a proof of concept.

I was able to link two devices nicely, I can see them both with charm keys and the "Current Key" switches in a sensible way between the two machines.

kv commands work fine for a single machine, but I can't figure out how to share across linked accounts. charm kv sync doesn't appear to do anything for me. Am I misunderstanding the usage? Is this only the purview of skate? I saw this (charmbracelet/skate#31) and I thought that I could just use charm kv.

Thanks for the cool tools!

support expiration time (lifetime) of the key

add tests for linking multiple clients to the same charm account

When I backup the keys on windows, the backup tar file contains a series of blank tarballs

Hi developers,

Dependency github.com/jacobsa/crypto is no longer maintained. The latest commit was 3 years ago, and the author isn't responding to new issues and PRs. It fails to build in riscv64 environment now. Do you have any plan to replace this dependency with a well-maintained alternative? Also, I can maintain a fork of it to make it build in riscv64 and accept new changes, so that Charm can use it.

When setting a username, pre-populate the input field in the "set username" section with the current username, if one is set.

I recently tried the program Skate v0.1.2 and I think I observed a big bug with kv. A very big file (000001.vlog or 000003.vlog) is generated at the path ~/.local/share/charm/<charm server>/kv/charm.sh.skate.default/. It is 2 GiB long it is mostly filled with null bytes.

This is very problematic as this prevent me to run Skate on my Raspberry Pi which have not enough RAM to load this file.

I observed this bug by using Skate release v0.1.2 on my local machine and on the self-hosted Charm server I tried the commit f4634fe and the release v0.9.1.

Right now the charm backup-keys command adds the kv directory to the tarball it creates. KV DBs should be omitted so the backup is strictly just keys.

Right now, we don't try to operate on the path specified in the charm import-keys argument until absolutely necessary. In particular, after potentially prompting the user about overwriting. We should stat the path in the argument first so we can exit before putting throwing up a prompt in front of the user.

Apologies if I've missed something obvious here. I was looking to test charm KV and ran the following commands:

$ charm keygen

  ✔  Generated keys

$ charm id

  We were’t able to authenticate via SSH, which means
  there’s likely a problem with your key.

  You can generate SSH keys by running  charm keygen . You
  can also set the environment variable  CHARM_SSH_KEY_PATH
  to point to a specific private key, or use  -i specifify a
  location.

I expected charm id to report an identity associated with the keys I just created.

I tried this in a basic Docker container and got the same result:

$ docker run -it ubuntu:latest /bin/bash
...

# apt update && apt install -y wget
...

# wget https://github.com/charmbracelet/charm/releases/download/v0.10.3/charm_0.10.3_linux_amd64.deb
...

# dpkg -i charm_0.10.3_linux_amd64.deb
Selecting previously unselected package charm.
(Reading database ... 4470 files and directories currently installed.)
Preparing to unpack charm_0.10.3_linux_amd64.deb ...
Unpacking charm (0.10.3) ...
Setting up charm (0.10.3) ...

# charm keygen

  ✔  Generated keys

# charm id

  We were’t able to authenticate via SSH, which means
  there’s likely a problem with your key.

  You can generate SSH keys by running  charm keygen . You
  can also set the environment variable  CHARM_SSH_KEY_PATH
  to point to a specific private key, or use  -i specifify a
  location.

I also tried running a local charm server, but got the same result (output from the same Docker container):

# charm serve &
[1] 2609
2022/03/08 21:34:47 Opening SQLite db: data/db/charm_sqlite.db
2022/03/08 21:34:47 http://localhost:35354
2022/03/08 21:34:47 Starting SSH server on :35353
2022/03/08 21:34:47 Starting HTTP server on: :35354
2022/03/08 21:34:47 Starting HTTP health server on: :35356

# export CHARM_HOST=localhost

# charm id

  We were’t able to authenticate via SSH, which means
  there’s likely a problem with your key.

  You can generate SSH keys by running  charm keygen . You
  can also set the environment variable  CHARM_SSH_KEY_PATH
  to point to a specific private key, or use  -i specifify a
  location.

Is there something that I'm missing?

when importing a key, the background color seems to be displaced.

examples:

been trying to self-host charm following the tls instructions but it's pretty confusing. including some more instructions on how to do it on a fresh vps would be useful, as well as explaining what tls brings over and against the basic ssh connection.

Hi,

I try to backup my keys but after the import, I lose my files.

Steps to reproduce

1. I work with a localhost charm server export CHARM_HOST=localhost
2. Copy a file: charm fs cp delta.md charm:/
3. Check that the file is listed: charm fs tree charm:/
4. Backup: charm backup-keys
5. Remove the keys: rm -fr ~/.local/share/charm
6. Listed the (new) user content: charm fs tree charm:/
7. Import the keys to retrieve my files: charm import-keys charm-keys-backup.tar
8. My file is not listed: charm fs tree charm:/

A simple tar -czf of the keys directory work fine.
I use ubuntu.

Just wanted to let you know that after installing the RPM on Fedora 35, when I enter charm link I'm greeted with something that looks like it exits early. It shows the linking progress indicator in a frozen state and just skips to showing my prompt again.

~ on ☁️  [email protected] 
❯ charm link
                                                            
  You can link the SSH keys on another machine to your Charm
  account so both machines have access to your stuff. Keys  
  can be unlinked at any time.                              
                                                            
  ⣽  Generating link...                                     

~ on ☁️  [email protected] 
❯ charm fs

~ on ☁️  [email protected] 
❯ charm link
                                                            
  You can link the SSH keys on another machine to your Charm
  account so both machines have access to your stuff. Keys  
  can be unlinked at any time.                              
                                                            
  ⣽  Generating link...                                     

~ on ☁️  [email protected] 
❯ charm link

Also, when I run charm after installing the regular old Linux binary, I get the following error which is because my firewall has restrictions on outgoing ports:

❯ charm
                                                                                
   Charm                                                                        
                                                                                
  Uh oh, there’s been an error: dial tcp 18.214.45.34:35353: connect: connection
                                                                                

~/Downloads/charm_0.9.2_linux_x86_64 on ☁️  [email protected] took 3s

It might make sense to use NGINX to proxy 443 (or 80) to 35353 for your production server since this might be a common issue with people behind restrictive enterprise firewalls

I have a Caddy server proxying a few other servers and doing TLS termination for them, so I'd like to do that with the Charm server also, and setup a Charm server using the docker container, behind a Caddy container doing the TLS termination.

not that this helps a lot but the issues looks less boring :)

The setup is straight forward but I'm finding some resistance: if I don't specify CHARM_SERVER_HTTP_SCHEME=https server side (as documented in the README), the client tries to communicate using plain HTTP with the server eventually (presumably after being told to use http by the server), so I need to set CHARM_SERVER_HTTP_SCHEME=https. However, if I do that, the server crashes when starting up because it obviously requires key material to do the TLS termination and I'm not setting CHARM_SERVER_TLS_KEY_FILE and CHARM_SERVER_TLS_CERT_FILE:

charm          | 2021/12/09 16:02:51 HTTPS server listening on: :35354                                                                                                                        
charm          | 2021/12/09 16:02:51 Server crashed: open : no such file or directory                                                                                                         

It'd be nice if an external proxy (like Caddy in my case) could do the TLS termination while the server lets the client know it should still use https to communicate (I think this is related to the URL schema being used to send requests.

I'm currently using a small patch and seems to be working as expected, with Charm server serving plain text and Caddy doing the TLS termination, but keeping the client happy and the transport layer secure:

diff --git a/server/http.go b/server/http.go
index 642996f..7fed612 100644
--- a/server/http.go
+++ b/server/http.go
@@ -88,7 +88,7 @@ func (s *HTTPServer) Start() {
 	}()
 
 	log.Printf("%s server listening on: %s", strings.ToUpper(s.cfg.HTTPScheme), listenAddr)
-	if useTls {
+	if useTls && !s.cfg.TLSDisableTermination {
 		log.Fatalf("Server crashed: %s", server.ListenAndServeTLS(s.cfg.TLSCertFile, s.cfg.TLSKeyFile))
 	} else {
 		log.Fatalf("Server crashed: %s", server.ListenAndServe())
diff --git a/server/server.go b/server/server.go
index 0ea177c..038e840 100644
--- a/server/server.go
+++ b/server/server.go
@@ -17,21 +17,22 @@ import (
 
 // Config is the configuration for the Charm server.
 type Config struct {
-	Host        string `env:"CHARM_SERVER_HOST" default:"localhost"`
-	SSHPort     int    `env:"CHARM_SERVER_SSH_PORT" default:"35353"`
-	HTTPPort    int    `env:"CHARM_SERVER_HTTP_PORT" default:"35354"`
-	HTTPScheme  string `env:"CHARM_SERVER_HTTP_SCHEME" default:"http"`
-	StatsPort   int    `env:"CHARM_SERVER_STATS_PORT" default:"35355"`
-	HealthPort  string `env:"CHARM_SERVER_HEALTH_PORT" default:"35356"`
-	DataDir     string `env:"CHARM_SERVER_DATA_DIR" default:"./data"`
-	TLSKeyFile  string `env:"CHARM_SERVER_TLS_KEY_FILE" default:""`
-	TLSCertFile string `env:"CHARM_SERVER_TLS_CERT_FILE" default:""`
-	TLSConfig   *tls.Config
-	PublicKey   []byte
-	PrivateKey  []byte
-	DB          db.DB
-	FileStore   storage.FileStore
-	Stats       stats.Stats
+	Host                  string `env:"CHARM_SERVER_HOST" default:"localhost"`
+	SSHPort               int    `env:"CHARM_SERVER_SSH_PORT" default:"35353"`
+	HTTPPort              int    `env:"CHARM_SERVER_HTTP_PORT" default:"35354"`
+	HTTPScheme            string `env:"CHARM_SERVER_HTTP_SCHEME" default:"http"`
+	StatsPort             int    `env:"CHARM_SERVER_STATS_PORT" default:"35355"`
+	HealthPort            string `env:"CHARM_SERVER_HEALTH_PORT" default:"35356"`
+	DataDir               string `env:"CHARM_SERVER_DATA_DIR" default:"./data"`
+	TLSKeyFile            string `env:"CHARM_SERVER_TLS_KEY_FILE" default:""`
+	TLSCertFile           string `env:"CHARM_SERVER_TLS_CERT_FILE" default:""`
+	TLSConfig             *tls.Config
+	TLSDisableTermination bool `env:"CHARM_SERVER_TLS_DISABLE_TERMINATION"`
+	PublicKey             []byte
+	PrivateKey            []byte
+	DB                    db.DB
+	FileStore             storage.FileStore
+	Stats                 stats.Stats
 }
 
 // Server contains the SSH and HTTP servers required to host the Charm Cloud.

Diff here. I'm currently unaware of how many kittens per day this patch could eat.

With that patch, I can start the server with the following env variables and have Caddy in front, serving and doing the TLS termination:

CHARM_SERVER_TLS_DISABLE_TERMINATION=1 CHARM_SERVER_HTTP_SCHEME=https CHARM_SERVER_HOST=my.awesome.charm ./charm serve

Did I miss something or having external TLS termination isn't currently possible?

❤️

Issue

Listing out the keys using the the db.Keys() function gives me wrong results.

If I run a small program that creates 3 kv pairs and then use db.Keys() to list them out I get a duplicate key value.

I ran my example program to generate the keys, then used the charm cli tool to list my keys out, then run the program again which shows the difference between .Keys() and .View() functions:

➜  charm-test charm kv list @charm-kv-keys-test-db      
key1	value1
key2	value2
key3	value3
➜  charm-test go run main.go                      
Charm kv .Keys()
0: key3
1: key2
2: key3
Charm kv .View() to list keys and values
key1 - value1
key2 - value2
key3 - value3

Example program to demonstrate

package main
import (
	"github.com/charmbracelet/charm/kv"
    "github.com/dgraph-io/badger/v3"
    "fmt"
    "log"
)

func main(){
    // Open a database (or create one if it doesn’t exist)
    db, err := kv.OpenWithDefaults("charm-kv-keys-test-db")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()


    // Save some data
    if err := db.Set([]byte("key1"), []byte("value1")); err != nil {
        log.Fatal(err)
    }

    if err := db.Set([]byte("key2"), []byte("value2")); err != nil {
        log.Fatal(err)
    }

    if err := db.Set([]byte("key3"), []byte("value3")); err != nil {
        log.Fatal(err)
    }

    fmt.Println("Charm kv .Keys()")
    if keys,err := db.Keys(); err != nil{
        log.Fatal(err)
    }else{
        for i,k := range keys{
            fmt.Printf("%d: %s\n", i, string(k))
        }
    }

    fmt.Println("Charm kv .View() to list keys and values")
    // Go full-blown Badger and use transactions to list values and keys
	db.View(func(txn *badger.Txn) error {
		opts := badger.DefaultIteratorOptions
		opts.PrefetchSize = 10
		it := txn.NewIterator(opts)
		defer it.Close() //nolint:errcheck
		for it.Rewind(); it.Valid(); it.Next() {
			item := it.Item()
			k := item.Key()
			err := item.Value(func(v []byte) error {
				fmt.Printf("%s - %s\n", k, v)
				return nil
			})
			if err != nil {
				panic(err)
			}
		}
		return nil
	})
}

Possible fix?

Disclaimer: I only started looking at Go last week, so could be that I'm doing something dumb here!

However, when running a debug session to try and figure it out, I did find that I could get it to work if I changed the following line in the kv.go file from

ks = append(ks, it.Item().Key())

to

ks = append(ks, it.Item().KeyCopy(nil))

Please see charmbracelet/homebrew-tap#1

Custom SSH keys are supported as of v0.11.0 with a env var. We should also support the -i KEYFILE flag on all commands.

"charm kv" is not working right on an arm64 system (Raspberry Pi, Debian Buster).

> charm kv list
foo     bar
~ ......................................................................................................................
> charm kv set foo baz
Error: while opening memtables error: while opening fid: 3 error: while updating skiplist error: mremap size mismatch: r
Usage:
  charm kv set KEY[@DB] VALUE [flags]

Flags:
  -h, --help   help for set

Error: while opening memtables error: while opening fid: 5 error: while updating skiplist error: mremap size mismatch: requested: 72 got: 134217728

Not really sure what's going on, but it has a strong smell of being this:

https://discuss.dgraph.io/t/error-mremap-size-mismatch-on-arm64/15333

for which a similar issue has this patch:

dgraph-io/ristretto#281

Encryption example states you can do:
charm encrypt < myfile.txt > encryptedfile.txt but the command is actually charm crypt encrypt < myfile.txt > encryptedfile.txt

Also when you just run charm crypt nothing seems to happen which can be confusing to new users. Should have some kind of feedback (maybe pop open help menu since it requires commands or flags to do something)

I was looking for a way to store my secrets recently, and a common occurrence among the suggestions and solutions was to encrypt each secret using another key and store the encrypted keys in a secure place where only you have access to it (automated solutions would handle the encrypting and storing for you). but the problem with those is, the secrets are not synced across all my devices. and if I loose the access to my device or the "master" key, I loose all my secrets.

Today this idea popped in my head: If charm can securely store and sync my markdown notes across my devices, why can it not store my secrets? So the immediate solution would be to create a markdown file with all the secrets in it but that's not really a solution at all imo.

So I propose the following feature request:

charm secrets                                     # list all secrets' names (similar to `charm keys`)

charm secrets add SECRET_NAME < cat secret_file   # add a secret (similar to `glow stash`)

charm secrets get SECRET_NAME                     # get the value of the secret

charm secrets remove SECRET_NAME                  # get the value of the secret

this would allow things like

some_tool --passphrase $(charm secrets get SECRET_NAME)

command name and format can change

I recently tried to self-host charm using the docker-compose file from this repository.
I am running charm behind Nginx with SSL certificates and I am able to use it on a single client. I am able to see files generated through charm fs cp ... and also skate or charm kv generates the corresponding databases.

So I added a second client through charm link I get a success and can see the key in charm keys.

However, when I try to run charm fs ls / on the second client, I receive an error.

Error: open /: Couldn't validate the authenticity of the ciphertext and associated data.

Trying to link a third client gives the same result/error.

Also I can see in the logs that the client tries to authenticate, which returns a 200 success. And when trying to use charm fs I see the server returning 200 as well in the logs. So it seems the client is somewhat broken or having trouble decrypting the contents?

charm         | 2022/01/24 14:22:07 <- GET /v1/fs//
charm         | 2022/01/24 14:23:40 -> 200 OK 317B 1.384314ms

Using the same command on the first client returns my folder list. Any idea what I'm missing?

Setting a style width in the App style would solve this

Expected non-truncated output:

Uh oh, there’s been an error: dial tcp: address cloud.dev.charm.sh:32984712983741982037410831923987414379843870923:35353: too many colons in address

was curious what I could do with charm and was looking around. there is mention of seamless user auth. and I see jwt stuff in the godoc. but how to use it is alittle unclear. maybe add more docs to readme or add links to read more about how to use charm in this way.

the open questions I have

1. How do I establish identity on a machine?
2. Once established what do I do with that identity? Can I use it as auth on a backend system? If so, how?
3. How do I migrate that identity from one machine to another?

add subcommand like charm where to show where your charm-related keys, DBs, and other info is stored. Currently, the only way to find this path is in the Backup option of the TUI.

When importing keys, your keys in your data dir may be overriden if you accept it, it does not, though, remove the old keys from your account.

Not sure if this is the expected behavior or if we should change it so it unlinks the key as well

Update docs to cover data storage max for our servers, how to change storage limits on self-hosted server, how and when we encrypt

This affects binary output in particular. See: charmbracelet/skate#16

➜  dev go get github.com/charmbracelet/charm/cmd/charm
# github.com/charmbracelet/charm/ui/username
../go/src/github.com/charmbracelet/charm/ui/username/username.go:130:20: too many arguments in call to textinput.Blink
	have (textinput.Model)
	want ()
../go/src/github.com/charmbracelet/charm/ui/username/username.go:130:20: cannot use textinput.Blink(m.input) (type tea.Msg) as type tea.Cmd in return argument: need type assertion
../go/src/github.com/charmbracelet/charm/ui/username/username.go:194:20: undefined: textinput.Update
../go/src/github.com/charmbracelet/charm/ui/username/username.go:234:18: undefined: textinput.Update
../go/src/github.com/charmbracelet/charm/ui/username/username.go:243:7: undefined: textinput.View

➜ dev go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/stephen/.cache/go-build"
GOENV="/home/stephen/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/stephen/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/stephen/go-1.14"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/stephen/go-1.14/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build142608877=/tmp/go-build -gno-record-gcc-switches"

As I can't run both OS at the same time, I can't complete the verification process.
I've tried backing up and loading the backup on the other OS, but that didn't work either.

Hi, I tried to spin up a self hosted charm using docker, the docker image seems to store invalid created_at dates for encrypt_key. When using charm serve instead, there is no problem.

On a first glance regarding the error message:

The docker image seems to set encrypt_key.created_at to 2021-12-08 12:15:47.525768 +0100 +0100 while charm serve stores 2021-12-08 12:09:31.894222 +0100 CET. I guess that produces the error message Error fetching encrypt keys: sql: Scan error on column index 2, name "created_at": unsupported Scan, storing driver.Value type string into type *time.Time.

Versions

• charmcli/charm:v0.9.0 (tested on linux and macOS 12.0.1)
• charm v0.9.0 (on macOS brew install charmbracelet/tap/charm)
• skate v0.1.0 (bb1d9c9) (on macOS brew install charmbracelet/tap/skate)

Fails: docker

1. Run Docker image
2. skate set hello world -> Encryption key mismatch
3. skate set hello world -> failed crypt check
4. Stop docker image
5. Start charm serve
6. skate list -> failed crypt check

$ rm -rf ./data "~/Library/Application Support/charm/"
$ cat docker-compose.yml
---
version: "3.1"
services:
  soft-serve:
    image: charmcli/charm:v0.9.0
    container_name: charm
    volumes:
      - ./data:/data
    ports:
      - 35353:35353
      - 35354:35354
      - 35355:35355
      - 35356:35356
    restart: unless-stopped

$ docker-compose up                                   
Creating network "charm_default" with the default driver
Creating charm ... done
Attaching to charm
charm         | 2021/12/08 11:15:41 Opening SQLite db: /data/db
charm         | 2021/12/08 11:15:42 Starting SSH server on :35353
charm         | 2021/12/08 11:15:42 HTTP server listening on: :35354
charm         | 2021/12/08 11:15:47 ssh api-auth
charm         | 2021/12/08 11:15:47 Creating user for key 37f0505b79ab53d19e595489d7911c28f348ce6a
charm         | 2021/12/08 11:15:47 JWT for user b3ef3220-3beb-401d-8b48-e0fb9a0f30e4
charm         | 2021/12/08 11:15:47 <- POST /v1/encrypt-key 172.21.0.1
charm         | 2021/12/08 11:15:47 Adding encrypted key 539f7c93-a0fa-4d65-8279-f414e3fbf699 2021-12-08 12:15:47.525768 +0100 +0100 for user b3ef3220-3beb-401d-8b48-e0fb9a0f30e4
charm         | 2021/12/08 11:15:47 -> 200 OK 0B 33.6366ms
charm         | 2021/12/08 11:15:49 ssh api-auth
charm         | 2021/12/08 11:15:49 JWT for user b3ef3220-3beb-401d-8b48-e0fb9a0f30e4
charm         | 2021/12/08 11:15:49 Error fetching encrypt keys: sql: Scan error on column index 2, name "created_at": unsupported Scan, storing driver.Value type string into type *time.Time
^CGracefully stopping... (press Ctrl+C again to force)
Killing charm  ... done
ERROR: 2

$ charm serve                                         
2021/12/08 12:16:04 Opening SQLite db: ./data/db
2021/12/08 12:16:04 Starting SSH server on :35353
2021/12/08 12:16:04 HTTP server listening on: :35354
2021/12/08 12:16:09 ssh api-auth
2021/12/08 12:16:09 JWT for user b3ef3220-3beb-401d-8b48-e0fb9a0f30e4
2021/12/08 12:16:09 Error fetching encrypt keys: sql: Scan error on column index 2, name "created_at": unsupported Scan, storing driver.Value type string into type *time.Time

Here the commands for the client

$ export CHARM_HOST=127.0.0.1

# during docker-compose up
$ skate set hello world
Error: Encryption key mismatch
Usage:
   set KEY[@DB] VALUE [flags]

Flags:
  -h, --help   help for set

Encryption key mismatch

$ skate set hello world
Error: failed crypt check
Usage:
   set KEY[@DB] VALUE [flags]

Flags:
  -h, --help   help for set

failed crypt check

# during charm serve
$ skate list           
Error: failed crypt check
Usage:
   list [@DB] [flags]

Flags:
  -d, --delimiter string   delimiter to separate keys and values (default "\t")
  -h, --help               help for list
  -k, --keys-only          only print keys and don't fetch values from the db
  -r, --reverse            list in reverse lexicographic order
  -v, --values-only        only print values

failed crypt check

Works: charm serve

1. Run charm serve
2. skate set hello world -> works
3. Stop docker image
4. Run docker image
5. skate list -> works

$ charm --version
charm version v0.9.0 (e554127)
$ rm -rf ./data "~/Library/Application Support/charm/"
$ charm serve
2021/12/08 12:09:17 Opening SQLite db: ./data/db
2021/12/08 12:09:19 Starting SSH server on :35353
2021/12/08 12:09:19 HTTP server listening on: :35354
2021/12/08 12:09:31 ssh api-auth
2021/12/08 12:09:31 Creating user for key 37f0505b79ab53d19e595489d7911c28f348ce6a
2021/12/08 12:09:31 JWT for user 72dad0be-b338-4f45-918f-519c1cae3806
2021/12/08 12:09:31 <- POST /v1/encrypt-key 127.0.0.1
2021/12/08 12:09:31 Adding encrypted key 42f2ce6f-89c5-4c1f-baa4-5cee035cd3d7 2021-12-08 12:09:31.894222 +0100 CET for user 72dad0be-b338-4f45-918f-519c1cae3806
2021/12/08 12:09:31 -> 200 OK 0B 2.310272ms
2021/12/08 12:09:31 <- GET /v1/seq/b1f17f58c3aed7b14e097fbc55b3dc410d9f9c92f3cd974544e358cb0b9e26d82db7808901b0 127.0.0.1
2021/12/08 12:09:31 -> 200 OK 10B 2.989027ms
2021/12/08 12:09:31 <- GET /v1/fs/b1f17f58c3aed7b14e097fbc55b3dc410d9f9c92f3cd974544e358cb0b9e26d82db7808901b0 127.0.0.1
2021/12/08 12:09:31 -> 404 Not Found 29B 568.249µs
2021/12/08 12:09:31 <- POST /v1/seq/b1f17f58c3aed7b14e097fbc55b3dc410d9f9c92f3cd974544e358cb0b9e26d82db7808901b0 127.0.0.1
2021/12/08 12:09:31 -> 200 OK 10B 1.580633ms
2021/12/08 12:09:31 <- POST /v1/fs/b1f17f58c3aed7b14e097fbc55b3dc410d9f9c92f3cd974544e358cb0b9e26d82db7808901b0/825034e5c6b9fc697cb0214f873d34baa8?mode=432 127.0.0.1
2021/12/08 12:09:31 -> 200 OK 0B 1.983326ms
^C

$ cat docker-compose.yml
---
version: "3.1"
services:
  soft-serve:
    image: charmcli/charm:v0.9.0
    container_name: charm
    volumes:
      - ./data:/data
    ports:
      - 35353:35353
      - 35354:35354
      - 35355:35355
      - 35356:35356
    restart: unless-stopped

$ docker-compose up
Starting charm ... done
Attaching to charm
charm         | 2021/12/08 11:09:46 Opening SQLite db: /data/db
charm         | 2021/12/08 11:09:46 Starting SSH server on :35353
charm         | 2021/12/08 11:09:46 HTTP server listening on: :35354
charm         | 2021/12/08 11:09:50 ssh api-auth
charm         | 2021/12/08 11:09:50 JWT for user 72dad0be-b338-4f45-918f-519c1cae3806
charm         | 2021/12/08 11:09:50 <- GET /v1/fs/b1f17f58c3aed7b14e097fbc55b3dc410d9f9c92f3cd974544e358cb0b9e26d82db7808901b0 172.19.0.1
charm         | 2021/12/08 11:09:50 -> 200 OK 308B 9.3245ms

And here are the commands of the client:

$ export CHARM_HOST=127.0.0.1

# while charm serve
$ skate set hello world

# while docker-compose up
$ skate list
hello	world

Any skate set with this user will succeed with docker-compose from now on.

Steps to reproduce

• Create a test directory or file: charm fs cp -r test charm:/test
• Try to remove the recently added directory using a charm URI: charm fs rm charm:/test

Expected result

The test directory is removed from the charm server.

Observed result

The test directory is not removed from the server and the client doesn't exit with an error.

It seems the rm command does not handle charm URIs so charm: is interpreted as a path element and encrypted as part of the path to delete. The server is using os.RemoveAll, which returns no error if the path doesn't exist, so the client will get a 200 response from the server, making the user think the operation completed successfully.

I've installed charm to test the Zsh completion. The instructions say to put the generated output from charm completion zsh to a file _charm in $fpath. The generated output defines all the necessary completion functions, but does not call _charm at the end.

This means that in the first tab completion, only the function definitions are loaded, so you need to press TAB a second time to get completions.

I've seen you use https://github.com/spf13/cobra for the completion generation. Something like this would be needed to fix it, so maybe you just need to update Cobra: https://github.com/spf13/cobra/blob/52e6099aead5c377be3c46f34bf0c2d3e0c128b6/zsh_completions.go#L251-L254

Thanks!