pulumi / kubernetes-guides Goto Github PK

Currently @pulumi/eks requires very expansive permissions:

• Create/delete permissions for IAM Roles, RolePolicyAttachments, InstanceProfile, etc.
• Create/delete permissions for SecurityGroup and SecurityGroupRule.
• Create/delete permissions for various EC2 resource types, like ASGs.

We should change the EKS package to allow us to deploy the IAM and SecurityGroup resources separately from the EKS compute resources, like ASGs. Notably, we already don't provision VPCs for the user, so there is some precedent for this.

Rationale: is that IAM and SecurityGroup are perhaps the highest blast radius resource types in the AWS API. In particular, they have among the most serious security implications when misconfigured. Admin permissions should therefore be granted very, very sparingly.

It should be simple to split out the SecurityGroups; for IAM, the industry-standard best practice is to have a three-tiered permissions model:

• [Very small] IAM admins: have administrative permissions over IAM.
• [Most/all of engineering] IAM users: are allowed to pass existing roles to other resources, but can't create their own. Users in this group can also administrate their own passwords.
• [Everything else] No IAM Permissions: no permissions whatsoever.

KtPW has a bunch of scripts that make it easy to integrate with CI systems, but no specific guidance on how to accomplish this.

Some of the KX code examples are using an old spec that takes a name and the pod spec. We've since moved to a podbuilder spec that is simplified, so we need to update the code samples to reflect the latest API

As identified in pulumi/pulumi-azuread#226, we need to ensure that the code for AzureAD is updated to match that of 5.x.x where there were breaking changes

While we are iterating on the content here, let's remove it from the docs site and move it into this repo and keep it up-to-date with the project overall.

GCP currently has distinct stacks for identity and managed infrastructure. AWS and Azure need roughly the same breakdown.

running through the lab, I am new to pulumi.

When I do pulumi up for the wordpress service, it cannot create the pvc.

Here is the diagnostics, using the default wordpress helm chart 2.1.3. Also tried with the actual latest wordpress chart 5.2.2 with similar results.

Diagnostics:
  kubernetes:extensions:Deployment (wpdev-wordpress):
    error: Plan apply failed: 3 errors occurred:
    
    * Timeout occurred for 'wpdev-wordpress'
    * Minimum number of Pods to consider the application live was not attained
    * 1 Pods failed to schedule because: [Unschedulable] persistentvolumeclaim "wpdev-wordpress" not found
 
  kubernetes:core:Service (wpdev-mariadb):
    error: Plan apply failed: 2 errors occurred:
    
    * Timeout occurred for 'wpdev-mariadb'
    * Service does not target any Pods. Selected Pods may not be ready, or field '.spec.selector' may not match labels on any Pods
 
  kubernetes:core:PersistentVolumeClaim (wpdev-wordpress):
    error: Plan apply failed: persistentvolumeclaims "wpdev-wordpress" is forbidden: Internal error occurred: 2 default StorageClasses were found
 
  kubernetes:core:Service (wpdev-wordpress):
    error: Plan apply failed: 2 errors occurred:
    
    * Timeout occurred for 'wpdev-wordpress'
    * Service does not target any Pods. Selected Pods may not be ready, or field '.spec.selector' may not match labels on any Pods

Networking and security infrastructure like (in the case of AWS) VPCs and SecurityGroups have significant security implications in their setup, and the blast radius of changes to this plane are very high. We should split these into their own stacks, instead of provisioning them along with databases and compute.

Our current plan for each of the clouds, roughly, is to have an architecture with:

• A subnet with a small set of publicly-available hosts (e.g., bastion hosts)
• A subnet with a larger set of hosts which can only be reached from the publicly-available hosts
• A subnet with the managed data services (e.g., RDS) which can only be reached from the managed compute subnet.

README.md Step 4 and config.ts Pulumi key names do not match.

README.md: Step 4, config set examples

$ pulumi config set k8s-gke-cluster:identityStackRef myUser/k8s-gcp-identity/dev-1573589109
$ pulumi config set k8s-gke-cluster:infraStackRef myUser/k8s-gcp-infra/dev-1573589378

config.ts: lines 5 & 6

const identityStackName = new pulumi.StackReference(pulumiConfig.require("identityStackName"));
const infraStackName = new pulumi.StackReference(pulumiConfig.require("infraStackName"));

There is a set of systems components that are considered industry standard in Kubernetes clusters. We should change the KtPW examples to have tunable, prod-first deployments of these things.

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

The Crosswalk for Kubernetes guide and code in this repo is written in TypeScript. I'd appreciate a .NET version also.

https://www.pulumi.com/docs/guides/crosswalk/kubernetes/

While trying to migrate the TypeScript version to C# I've discovered that there are differences in the Azure AD Identity API e.g.

1. I cannot set replyUrls or type on Pulumi.AzureAD.ApplicationArgs.
2. ServicePrincipalPassword does not have a Value or EndDate property to set the password or its expiry.
3. Assignment does not have the PrincipalId or RoleDefinitionName.
4. Group does not have a Name property.

Trying to deploy wordpress service, got an error :"provider properties must be strings", this is most likely because kubeconfig output of infrastructure stack is a JSON object?

The basic idea in our model is that the user will bootstrap infrastructure with the following series of actions:

1. Create an account on one of the major cloud providers.
2. Use the root account to provision the Pulumi identity stack.
3. Once the identity stack is provisioned, we should have a user account for CI on the identity stack. This account has IAM admin permissions -- and ideally nothing else. Set up CI (e.g., using travis CI) to use this account.
4. All groups, policies, roles, as well as many service accounts, and sometimes users, should henceforth be provisioned via PR.
5. Don't use the root account again, ever, unless you need to.

Currently this is already implemented for GCP.

What happened?

After a fresh clone of the project and following the instructions in azure/01-identity the code has many type errors and does not build.

I assume that this is because the code is written for an old version of the Pulumi Azure providers. Dependencies in package.json should be explicit rather than use 'latest' or the code should be updated.

Pulumi cli: v3.46.1
npm ls --depth 0
.../kubernetes-guides/azure/01-identity
├── @pulumi/[email protected] invalid: "latest" from the root project
├── @pulumi/[email protected] invalid: "latest" from the root project
├── @pulumi/[email protected]
├── @pulumi/[email protected] invalid: "latest" from the root project
├── @types/[email protected]
└── [email protected]

Steps to reproduce

Clone and follow instructions in azure/01-identity

Expected Behavior

It should depoly project successfully.

Actual Behavior

It fails to build the project out of the box. Additionally pulumi about shows an error running npm ls.

Output of pulumi about

CLI
Version 3.46.1
Go Version go1.19.2
Go Compiler gc

Plugins
NAME VERSION
azure 5.24.0
azuread 5.31.0
nodejs unknown
random 4.8.2

Host
OS darwin
Version 12.6
Arch arm64

This project is written in nodejs: executable='/Users/robin/.nvm/versions/node/v16.16.0/bin/node' version='v16.16.0'

Current Stack: dev

Found no resources associated with dev

Found no pending operations associated with dev

Backend
Name pulumi.com
URL https://app.pulumi.com/robinsummerhill
User robinsummerhill
Organizations robinsummerhill, emuanalytics

Pulumi locates its logs in /var/folders/ww/z6xt_hxs1fj_s7_p2y26pry00000gn/T/ by default
warning: Failed to get information about the Pulumi program's dependencies: failed to run "/Users/robin/.nvm/versions/node/v16.16.0/bin/npm ls --json --depth=0": exit status 1

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

Is it in scope for this project to also describe dev, staging, and prod workflows? For the moment, (I believe) this repo suggests only a PR -> prod workflow.

Assuming the current repo structure (under gcp/):

identity/
├── Pulumi.yaml
├── Pulumi.identity.yaml
├── index.ts
└── package.json
infrastructure/
├── Pulumi.yaml
├── Pulumi.infrastructure.yaml
├── index.ts
└── package.json

I'm puzzled about what "the Prod Way" of supporting a workflow with independent dev, staging and prod environments would look like.

Assuming a branch per environment, e.g. dev, staging, prod, then the Pulumi.<stack-name>.yaml files will (should) differ in at least secretsprovider and encryptedkey across the three branches (at least for self-managed backends) and possibly also for config/gcp:project. So there will always be merge conflicts when promoting features from one environment's branch to another, which opens the door for human error at exactly the points where you want to be reducing the potential for human error (i.e. when graduating from dev to staging or from staging to prod).

I quite liked how identity bootstrapping was described as a series of actions in #13 (comment). If we could come up with an analagous description here to at least spec out a future PR, that would be a pretty great outcome.