Giter VIP home page Giter VIP logo

terraform-aws-munki-repo's Introduction

Terraform Munki Repo

terraform-munki-repo is a Terraform module that will set up a production-ready Munki repo for you. It is designed for use with Terraform 1.0.x More specifically, it will create:

  • An S3 bucket to store your Munki repo
  • An S3 bucket to store your logs
  • A CloudFront Distribution so your clients will pull from an AWS endpoint near them
  • A Lambda@Edge function that will set up basic authentication

Why should I use this?

A Munki repo is a basic web server. But you still need to worry about setting up one or more servers, patching those servers, and scaling them around the world if you have clients in more than one country.

Amazon Web Services has crazy-high levels of uptime - more than we could ever manage ourselves. CloudFront powers some of the world's busiest websites without breaking a sweat, so it can handle your Munki repo without any trouble. It makes sense to offload the running of these services so that we can get on with our day.

Initial Terraform / AWS Setup

  1. Register for an AWS account if you don't have one.
  2. Once you have logged in and set up billing, head over to IAM and create a user with the following permissions: AWSLambdaFullAccess, IAMFullAccess, AmazonS3FullAccess, CloudFrontFullAccess
  3. Generate an access key and secret key for the user. Download the CSV (or you'll lose the secret key; it's only displayed once at initial creation).
  4. Install awscli from https://awscli.amazonaws.com/AWSCLIV2.pkg
  5. Install terraform 1.0.7 from https://releases.hashicorp.com/terraform/1.0.7/terraform_1.0.7_darwin_amd64.zip (Unzip and copy the terraform binary to /usr/local/bin/terraform)
  6. Run aws configure.
    • For "AWS Access Key ID" enter the "Access key ID" from the credentials.csv you downloaded in step 3.
    • For "AWS Secret Access Key" enter the "Secret access key" from the credentials.csv you downloaded in step 3.
    • For "Default region name" enter "us-east-1". Lambda@Edge currently requires the us-east-1 region.
    • For "Default output format" enter either "text" or "json" (or whatever you prefer)

Terraform Usage

Create a new empty directory wherever you want to store this. Inside that directory, create a file called main.tf. Put the following content in it - adjust the variables to match what you want the bucket to be called (the name must be globally unique across all of Amazon), and the username and password your Munki clients will use to access the repo)

# change the following variables

# prefix should be globally unique. Some characters seem to cause issues;
# I'd recommend sticking with lower-case-letters and underscores
# Something like org_yourorg_munki might be a good prefix.
variable "prefix" {
  default = "you-better-change-me"
}

# you'd need to change this only if you have an existing bucket named
# "munki-s3-bucket"
variable "munki_s3_bucket" {
  default = "munki-s3-bucket"
}

# the price class for your CloudFront distribution
# one of PriceClass_All, PriceClass_200, PriceClass_100
variable "price_class" {
  default = "PriceClass_100"
}

# the username your Munki clients will use for BasicAuthentication
variable "username" {
  default = "YOU_BETTER_CHANGE_ME"
}

# the password your Munki clients will use for BasicAuthentication
variable "password" {
  default = "YOU_BETTER_CHANGE_ME"
}

# the rest should be able to be left as-is unless you are an expert

# minimum amount of time (in seconds) that you want objects to stay in CloudFront caches before CloudFront queries your origin to see whether the object has been updated checks for updates
variable "default_cache_behavior_min_ttl" {
  default = 0
}

# default amount of time (in seconds) that an object is in a CloudFront cache before CloudFront forwards another request in the absence of an Cache-Control max-age or Expires header
variable "default_cache_behavior_default_ttl" {
  default = 86400
}

# maximum amount of time (in seconds) that an object is in a CloudFront cache before CloudFront forwards another request to your origin to determine whether the object has been updated
variable "default_cache_behavior_max_ttl" {
  default = 31536000
}

# minimum amount of time (in seconds) that you want catalog objects to stay in CloudFront caches before CloudFront queries your origin to see whether the object has been updated
variable "catalogs_ordered_cache_behavior_min_ttl" {
  default = 0
}

# default amount of time (in seconds) that a catalog object is in a CloudFront cache before CloudFront forwards another request in the absence of an Cache-Control max-age or Expires header
variable "catalogs_ordered_cache_behavior_default_ttl" {
  default = 30
}

# maximum amount of time (in seconds) that a catalog object is in a CloudFront cache before CloudFront forwards another request to your origin to determine whether the object has been updated
variable "catalogs_ordered_cache_behavior_max_ttl" {
  default = 60
}

# minimum amount of time (in seconds) that you want manifest objects to stay in CloudFront caches before CloudFront queries your origin to see whether the object has been updated
variable "manifests_ordered_cache_behavior_min_ttl" {
  default = 0
}

# default amount of time (in seconds) that a manifest object is in a CloudFront cache before CloudFront forwards another request in the absence of an Cache-Control max-age or Expires header
variable "manifests_ordered_cache_behavior_default_ttl" {
  default = 30
}

# maximum amount of time (in seconds) that a manifest object is in a CloudFront cache before CloudFront forwards another request to your origin to determine whether the object has been updated
variable "manifests_ordered_cache_behavior_max_ttl" {
  default = 60
}

# minimum amount of time (in seconds) that you want icon objects to stay in CloudFront caches before CloudFront queries your origin to see whether the object has been updated
variable "icons_ordered_cache_behavior_min_ttl" {
  default = 0
}

# default amount of time (in seconds) that a icon object is in a CloudFront cache before CloudFront forwards another request in the absence of an Cache-Control max-age or Expires header
variable "icons_ordered_cache_behavior_default_ttl" {
  default = 30
}

# maximum amount of time (in seconds) that a icon object is in a CloudFront cache before CloudFront forwards another request to your origin to determine whether the object has been updated
variable "icons_ordered_cache_behavior_max_ttl" {
  default = 60
}


# NOTE: currently the _only_ supported provider region is us-east-1.
provider "aws" {
  region  = "us-east-1"
}

module "munki-repo" {
  source          = "grahamgilbert/munki-repo/aws"
  version         = "0.3.0"
  munki_s3_bucket = var.munki_s3_bucket
  username        = var.username
  password        = var.password
  prefix          = var.prefix
  price_class     = var.price_class
}

# These help you get the information you'll need to do the repo sync
# and to configure your Munki clients to use your new cloud repo

output "cloudfront_url" {
  value = module.munki-repo.cloudfront_url
}

output "munki_bucket_id" {
  value = module.munki-repo.munki_bucket_id
}

output "username" {
  value = var.username
}

output "password" {
  value = var.password
}

cd into the directory containing your main.tf and run the following commands:

$ terraform init
$ terraform get
$ terraform plan

If everything goes well and terraform says it will create everything you expect, you can apply (type in yes when you are asked):

$ terraform apply

Then you can get your distribution's url:

$ terraform output cloudfront_url

Get the S3 bucket id:

$ terraform output munki_bucket_id

(Unless you've changed it from the suggested name in the main.tf above, it will be "munki-s3-bucket")

Get the username and password:

$ terraform output username
$ terraform output password

(Again these should match the ones you put into main.tf)

Getting your Munki repo into S3

Assuming your repo is in /Users/Shared/munki_repo - adjust this path for your environment.

$ aws s3 sync "/Users/Shared/munki_repo" s3://<munki_bucket_id> --exclude '*.git/*' --exclude '.DS_Store' --delete

(Be sure to substitute your actual munki_bucket_id for <munki_bucket_id> -- unless you've changed things in main.tf it will be "munki-s3-bucket")

Now it's just a matter of configuring your Munki clients to connect to your new repo. The Munki wiki covers configuring the clients to use BasicAuthentication using the username and password you've chosen. Be sure also to set Munki's SoftwareRepoURL to "https://<cloudfront_url>" where you substitute the cloudfront_url you retreived earlier.

terraform-aws-munki-repo's People

Contributors

chefaustin avatar grahamgilbert avatar gregneagle avatar gyamada619 avatar johnmikee avatar jsharpe avatar opragel avatar stilljake avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

terraform-aws-munki-repo's Issues

ARN must end with the version number error

I ran this setup and everything worked except this.

Error: Error applying plan:

1 error(s) occurred:

  • module.munki-repo.aws_cloudfront_distribution.www_distribution: 1 error(s) occurred:

  • aws_cloudfront_distribution.www_distribution: error creating CloudFront Distribution: InvalidLambdaFunctionAssociation: The function ARN must reference a specific function version. (The ARN must end with the version number.) ARN: arn:aws:lambda:us-west-1:080972764581:function:munki_basic_auth:$LATEST
    status code: 400, request id: 27138030-6c9f-11e9-815d-e1426c0813f0

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

Cloudfront Middleware replace BasicAuthentication on Clients?

Hi Graham,

Verifying that using CloudFront-Middleware replaces the need for BasicAuthentication on Munki clients?

Asking because I get the following error after running managedsoftwareupdate.
ERROR: Could not retrieve managed install primary manifest.

Also enabled FollowHTTPRedirects on the client.
FollowHTTPRedirects: 'https'

Cheers,
Ernesto

Issue defining AWS region

How do I define my desired aws region to avoid this? I can successfully run terraform plan:

$ terraform plan -out saved_plan
provider.aws.region
  The region where AWS operations will take place. Examples
  are us-east-1, us-west-2, etc.

  Default: us-east-1
  Enter a value: us-west-1

and the plan completes OK, but it fails on terraform apply:

$ terraform apply "saved_plan"

Error: Error applying plan:

1 error occurred:
	* provider.aws: Invalid AWS Region: 

Nodejs deprecation warning

I received an email from Amazon saying nodejs10 was being deprecated so I should upgrade my existing functions to 12 or 14. Looking at

runtime = "nodejs12.x"
it should be 12 but running aws lambda list-functions --function-version ALL --output text --query "Functions[?Runtime=='nodejs10.x'].FunctionArn" dumps out the basic_auth.

I can submit a pr for the version change as nodejs14 looks to be available now https://aws.amazon.com/blogs/compute/node-js-14-x-runtime-now-available-in-aws-lambda/ but im more curious why the function is being created with the old runtime. I verified the module was a clone off the current master branch and redeployed just to verify that it creates with 10.

Lambda@edge function must always be defined in us-east-1

Lambda@edge functions must be defined in us-east-1, otherwise attempting to deploy this recipe to a different region results in the error:

aws_cloudfront_distribution.www_distribution: error creating CloudFront Distribution: InvalidLambdaFunctionAssociation: The function must be in region 'us-east-1'

S3 bucket error

I followed the write up exactly as written. I initially missed the part about using the older version of Terraform. Confirmed I have all the items set up in my environment. I modified the tf file as instructed and get the following error:

1 error occurred:
* module.munki-repo.aws_s3_bucket.log_bucket: 1 error occurred:
* aws_s3_bucket.log_bucket: Error creating S3 bucket: InvalidBucketName: The specified bucket is not valid.
status code: 400, request id: , host id:

Googling around I see other users having issues and it coming down to their bucket names not conforming to the requirements by AWS. I confirmed my S3 bucket in the tf does not already exist and follows the naming guidelines.

Error: Missing required argument

When running terraform plan using grahamgilbert/munki-repo/aws version 0.1.0 I get several deprecation warnings (which I can ignore for now), and one error:

Error: Missing required argument

The argument "region" is required, but was not set.

I tried adding region: us-west-1 to main.tf, but it just triggers a different error. Here's the complete output from terraform plan:

$ terraform plan

Warning: Interpolation-only expressions are deprecated

  on .terraform/modules/munki-repo/grahamgilbert-terraform-aws-munki-repo-49bbb84/cloudfront.tf line 4, in resource "aws_cloudfront_distribution" "www_distribution":
   4:     domain_name = "${aws_s3_bucket.www.bucket_regional_domain_name}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.

(and 8 more similar warnings elsewhere)


Warning: Quoted references are deprecated

  on .terraform/modules/munki-repo/grahamgilbert-terraform-aws-munki-repo-49bbb84/lambda.tf line 42, in resource "aws_lambda_function" "basic_auth_lambda":
  42:   provider         = "aws.use1"

In this context, references are expected literally rather than in quotes.
Terraform 0.11 and earlier required quotes, but quoted references are now
deprecated and will be removed in a future version of Terraform. Remove the
quotes surrounding this reference to silence this warning.


Error: Missing required argument

The argument "region" is required, but was not set.

Move secret out of Lamba function

I put together a variation of this module that moves the secret from sitting in plain text in the lambda function to amazon secrets-manager. Since this is a decent sized change in architecture I was curious if you would be interested in a PR or not.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.