infrahq / infra Goto Github PK

Add a helm chart for installing Infra (both server and per-cluster engine).

Helm charts will provide several benefits for the setup experience:

• Install in one line (e.g. helm install infra)
• Ability to upgrade / cleanup all resources in one go
• Runtime flags vs editing yaml files (e.g. helm install infra --set engine=true --set server=infra.example.com --set token=n8ap78n1p27dn127dn91
• Waiting & output so users know what to to do next. For example:

$ helm repo add infrahq https://helm.infrahq.com
$ helm install infra infrahq/infra

Waiting for load balanacer to be ready...

Run "infra login 35.181.19.128" to continue

after running infra user list, user expects to see who he/she is logged in as

If the admin creates a user the password they specify is preserved in their command history. We should prompt for the password separately similarly to the initial admin flow.

Proposed flow:

$ infra user create [email protected]
? Password *********
✓ User created

Another option to remove all personal info from the command history:

$ infra user create
? Email [email protected]
? Password *********
✓ User created

Currently, the post-login SSO page is a plaintext page:

From watching users login via Okta (or later, other Identity Sources) it's not clear they have successfully loggedin

Google provides a good example of what it should look like:

Tip: the success html response is currently served here: https://github.com/infrahq/infra/blob/main/internal/cmd/local_server.go#L28

For ease of install with initial customers, the Infra engine defaults to skipping TLS verification.

This isn't a secure default. Instead, it should attempt to contact the regsitry with TLS verification on, and then print an error and try again if TLS verification fails.

Specific line where TLS verification is disabled by default: https://github.com/infrahq/infra/blob/main/internal/cmd/cmd.go#L966

Group support. For IDPs that support groups, groups should be synchronized into Infra.

Currently the API only returns a single error string. It should return a series of errors for endpoints with multiple parameters.

Currently, Infra will generate certificates for any domain verified by ACME. However, this can lead to a denial of service if users continually try different domains for the same Infra registry.

Infra should support limiting certificate generation to a specific domain.

CLI Flag

infra registry --domain=infra.example.com

Tip: the domain would need to be provided to the autocert manager https://github.com/infrahq/infra/blob/main/internal/registry/registry.go#L134

Currently, infra already supports two kinds of tokens to authenticate with infra registry, and the grpc interceptor (middleware) authenticates based on the length of the token. This is poor practice and is not backwards compatible.

The two kinds of tokens currently used to authenticate with Infra Registry:

• token: a random string of length 36 (12 bytes for ID, 24 bytes for secret ID) – to be used by Infra CLI to log in and generate short-lived credentials, and later the Web Client, etc.
• apikey: a random string of length 24 – to be used by infra engine to authenticate

Specific line where this occurs: https://github.com/infrahq/infra/blob/main/internal/registry/grpc.go#L81

Instead, we should instead:

• Strictly separate API endpoints used by Infra CLI (and later the Web UI) and Infra Engine
• Adopt a backwards-compatible format for tokens and api keys (similar to GitHub)

Add integration tests to test interactions between the different components:

High-level areas to cover:

• Registry <> Engine
• CLI <> Registry
• CLI <> Engine

Currently, JWT credentials are generated and fetched for every invocation of kubectl (which in turn, calls infra creds) as per https://github.com/infrahq/infra/blob/main/internal/cmd/cmd.go#L990

Instead, infra client should cache per-destination credentials on first create, for the same duration as the expiration of the JWT token, and re-use these for future invocations of infra creds

Currently, when installing the engine on a cluster, specifying a cluster name is required. However, for many clusters this can be determined automatically.

For cases where it can't be determined, we can use a fallback that can later be edited by users.

Example of how DataDog does this: DataDog/datadog-agent#8225

Currently, users need to pass -i or --insecure to every command if using self-signed certs. Instead, we should allow them to verify and pin a certificate, similar as browsers. This is exactly what logging into a node via ssh does as well.

$ infra login infra.example.com
Cannot verify certificate (fingerprint: 18:40:a9:b8:1c:18:40:a9:b8:1c:18:40:a9:b8:1c)
Are you sure you want to continue connecting (yes/no)? yes

Infra needs structured logs with levels for easy debugging when an issue arises. There are many options but a popular one is https://github.com/sirupsen/logrus

• one early adopter ran into issue as his company uses infra namespace for other things.

When a user's session with Infra server expires, they should be automatically prompted (via kubectl or infra commands) automatically to re-log in.

We should use the standard wording of:
login
login to
logged into
logout
logged out

Rather than:
log in
log out
etc.

Expected behavior:
If the infra source create command fails the source is not added to the list of available sources.

Current behavior:
If the infra source create command returns an error on an rpc call the source is still added to the list of available sources.

Ex:

$ infra source create okta \
    --api-token AAA \
    --domain dev-123456-admin.okta.com \
    --client-id BBB \
    --client-secret CCC
Error: rpc error: code = Unknown desc = your Okta domain should not contain -admin. Current value: https://dev-123456-admin.okta.com. You can copy your domain from the Okta Developer Console. Follow these instructions to find it: https://bit.ly/finding-okta-domain

$  infra login localhost

Could not verify certificate for host localhost
? Are you sure you want to continue (yes/no)? Yes
? Choose a login method:  [Use arrows to move, type to filter]
> Okta [dev-123456.okta.com]
  Okta [example.okta.com]
  Okta [dev-123456-admin.okta.com]
  Username & password

dev-123456-admin.okta.com should not have been added.

Not sure what is causing this yet, for about every 1 in 3 Okta authentications my log in fails with an rpc error after a callback with the code.

$ infra login localhost

Could not verify certificate for host localhost
? Are you sure you want to continue (yes/no)? Yes

Welcome to Infra. Get started by creating your admin user:
? Email [email protected]
? Password *********
✓ Creating admin user...
✓ Logged in
✓ Kubeconfig updated

$ infra login localhost

Could not verify certificate for host localhost
? Are you sure you want to continue (yes/no)? Yes
? Email [email protected]
? Password *********
✓ Logging in with username & password...
✓ Logged in
Error: rpc error: code = Unauthenticated desc = unauthorized

$ infra login localhost

Could not verify certificate for host localhost
? Are you sure you want to continue (yes/no)? Yes
? Email [email protected]
? Password *********
✓ Logging in with username & password...
✓ Logged in

$ infra login localhost

Could not verify certificate for host localhost
? Are you sure you want to continue (yes/no)? Yes
? Email [email protected]
? Password *********
✓ Logging in with username & password...
✓ Logged in
Error: rpc error: code = Unauthenticated desc = unauthorized

Infra should be hidden from end-users when connecting to a cluster via Infra.

This will help users in two ways:

1. Infra will be invisible to end-users connecting via kubectl
2. It will stop users from accidentally deleting or modifying infra - creating a chicken & egg problem to reconnect

A few alternative ideas:

1. Remove infra resources from any rbac rules provided by infra – this will solve point number 2. above
2. Put infra in the kube-system namespace, where these kinds of tools usually live

Currently, published binaries and Docker images are unsigned. To help customers verify their integrity we should sign them:

• Binaries: sign with goreleaser (see this)
• Docker: sign with Docker Content Trust

Currently API objects, database models are based off of the Go struct tags. This is great for short term and rapid changes to the API. Long term we'll need backwards compatibility and easy api generation for clients (e.g. the CLI, engine). Protocol buffers are a great way to do this.

To keep using gorm tags we can inject tags via https://github.com/favadi/protoc-go-inject-tag

Upon logging out, Infra should remove all the contexts that it generated via infra login

For debugging and upgrading users should be able to check which version of infra they have

$ infra version
0.1.0

Some example Go projects with versioning:

• Stripe CLI
• HashiCorp Waypoint

A stretch goal: Infra also attempts to contact the current server and prints what version the server is running:

$ infra version
Client: 0.1.0
Server: 0.1.0

Currently, specifying permissions via -c infra.yaml will revoke existing permissions that aren't specified in the config file. Instead, it should only replace permissions previously specified in the config, and overlay this with permissions managed via the API.

Currently a kubectl rollout restart -n infra deployment/infra is required to update a config - this isn't obvious to users. Infra should watch the config file and automatically apply it on the fly.

For new installs, we should allow users to sign up as the first admin. This will help first time setups. For security, this needs to be easy to disable via a command line flag or config file.

An alternative would be to print a randomly generated password – but this would require addl' copying from the user, and some might miss it or clear their terminal, leaving Infra locked.

Installing CLI tools on Mac is best done via brew. This way, users on macOS can install the infra CLI via brew install infrahq/tap/infra

When an admin user creates a new user they specify their initial password. The admin should also be given the option of enforcing a password change the first time a user logs in. This ensures user confidentiality.

When a user sets their password (either the initial admin login or subsequent user creation) they should be required to confirm their password before the user is created/updated. If the password and confirmation do not match the user should be required to re-type both fields (repeat this state until they match).

Currently users need to set up Okta manually and provide infra four values:

• Client ID
• Client Secret
• Api token <- especially hard to generate, requires creating a user, scoping access and logging in as them
• Domain

an alternative would be to programatically create the Okta app via an Api Token:

$ infra providers create --okta --okta-api-token=xxxxxxx --okta-domain=example.okta.com

Benefits:

• No extra read-only user (can use Okta's machine to machine identity - only available via their API)
• One command

Downsides:

• "Magic" - less visibility for teams who want to configure Okta manually.

Currently users need to provide a password as an argument: infra users create EMAIL PASSWORD

If a parameter is not specified we should interactively ask for a password:

$ infra users create
? Email [email protected]
? Password *****************

Currently, all commands are shown in one big list:

Infrastructure Identity & Access Management (IAM)

Usage:
  infra [command]

Available Commands:
  list        List resources
  grant       Grant access to a resource
  revoke      Revoke access to a resource
  inspect     Inspect access for a resource or user
  users       Manage users
  providers   Manage identity providers
  login       Log in to Infra server
  logout      Log out of Infra server
  server      Start Infra server
  engine      Start Infra engine
  help        Help about any command

However this makes it difficult to discover related or advanced commands. Many CLI tools like kubectl, gh and docker properly separate commands to help discovery & organization. For example with gh:

Work seamlessly with GitHub from the command line.

USAGE
  gh <command> <subcommand> [flags]

CORE COMMANDS
  gist:       Manage gists
  issue:      Manage issues
  pr:         Manage pull requests
  release:    Manage GitHub releases
  repo:       Create, clone, fork, and view repositories

ACTIONS COMMANDS
  actions:    Learn about working with GitHub actions
  run:        View details about workflow runs
  workflow:   View details about GitHub Actions workflows

ADDITIONAL COMMANDS
  alias:      Create command shortcuts
  api:        Make an authenticated GitHub API request
  auth:       Login, logout, and refresh your authentication
  completion: Generate shell completion scripts
  config:     Manage configuration for gh
  help:       Help about any command
  secret:     Manage GitHub secrets
  ssh-key:    Manage SSH keys

FLAGS
  --help      Show help for command
  --version   Show gh version

EXAMPLES
  $ gh issue create
  $ gh repo clone cli/cli
  $ gh pr checkout 321

ENVIRONMENT VARIABLES
  See 'gh help environment' for the list of supported environment variables.

LEARN MORE
  Use 'gh <command> <subcommand> --help' for more information about a command.
  Read the manual at https://cli.github.com/manual

FEEDBACK
  Open an issue using 'gh issue create -R github.com/cli/cli'

Reproduction Steps:

1. Create an Okta account that is linked to a Google account
2. Sign out of the Okta portal
3. Run infra login $HOST
4. At the Okta login screen select Need help signing in? > Sign in with Google
5. Authenticate with Google
6. After logging in the response is not sent to the Infra client and it just waits forever

$ infra login localhost

Could not verify certificate for host localhost
? Are you sure you want to continue (yes/no)? Yes
? Choose a login method: Okta [dev-50966300.okta.com]
✓ Logging in with Okta...

Note:
I'll test this with a non-Google login flow too to make sure that is actually working as expected.

Currently Infra will always assume the user's kubeconfig is located at ~/.kube/config. Instead we should load the kubeconfig the same way as kubectl and other tools

Currently, when setting up Infra for the first time, users need to choose between:

1. Trusting self-signed certificates (skipping TLS verification)
2. Assigning a domain name to their cluster

As a free service, Infra could provision a dev domain for the customer, for example:

<Random ID>.infrahq.dev -> 35.18.29.192

This would enable them to use Infra securely until they can assign their own domain name.

Currently, Infra can only map permissions to cluster-wide roles. Customers require the ability to map users to namespace-specific roles. For example:

# example infra.yaml

permissions:
  - user: [email protected]
    role: edit
    destination: production
    namespace: frontend       # Additional namespace key

Currently there's no way to view the logs of the Infra client relay. They should be stored in a file, and only log errors by default.

The code for the client relay is under https://github.com/infrahq/infra/blob/main/internal/cmd/client.go