Giter VIP home page Giter VIP logo

quicklab's Introduction

QuickLab

_______       _____      ______ ______        ______
__  __ \___  ____(_)________  /____  / ______ ___  /_
_  / / /  / / /_  /_  ___/_  //_/_  /  _  __ `/_  __ \
/ /_/ // /_/ /_  / / /__ _  ,<  _  /___/ /_/ /_  /_/ /
\___\_\\__,_/ /_/  \___/ /_/|_| /_____/\__,_/ /_.___/

Create simple, monitored labs.

QuickLab includes four components:

  • a private network
  • a bastion host
  • a managed kubernetes cluster
  • monitoring

QuickLab uses Terraform to create its components in AWS, and uses Sumo Logic for monitoring.

QuickLab aims to enable builders and tinkerers by offering a ready-made playground that's "well-architected" and flexible enough for a variety of projects.

QuickLab is designed to create lab infrastructure only (and only the above-listed components). Once a QuickLab is created, users are free to deploy applications or create additional lab resources.

Use Cases

In no particular order:

  • software development
  • application and infrastructure deployment
  • telemetry collection and analysis
  • cybersecurity attack simulation (and detection) range

Bonus: Anything else you might think of! Once your network with bastion and/or cluster are created you can use them to create any number of scenarios that call for a VM or a kubernetes cluster!

Diagram

QuickLab AWS QuickLab on AWS, showing all components enabled, including the network (VPC), bastion (EC2 Instance), cluster (EKS) and monitoring (Sumo Logic)

Documentation

quicklab's People

Contributors

jeff-d avatar

Watchers

 avatar

quicklab's Issues

With sumo enabled I receive the following error "Field with the given name already exists" in the otc section.

While I can fully deploy quicklabs with montoring=none, if I enable sumo monitoring, a series of errors are reported.
I will report them one at a time in separate tickets, although they may have the same solution related to local.fields

Running the following:

terraform apply -auto-approve -var="create_network=true" -var="create_bastion=true" -var="create_cluster=false" -var="monitoring=sumo"

Code block:

# Sumo Logic Fields sent via otelcol-sumo
## RR this keeps failing
resource "sumologic_field" "otc" {
  for_each = var.create_bastion ? toset(local.fields.otc) : toset([])

  field_name = each.key
  data_type  = "String"
  state      = "Enabled"
}

Error:

│ Error: {"id":"XTXOM-K8KCR-XV1Z0","errors":[{"code":"field:already_exists","message":"Field with the given name already exists"}]}

│ with module.sumo["gewq"].sumologic_field.otc["host.group"],
│ on modules/sumo/bastion.tf line 18, in resource "sumologic_field" "otc":
│ 18: resource "sumologic_field" "otc" {

deprecated argument in terraform resource "aws_cloudwatch_event_rule"

╷
│ Warning: Argument is deprecated
│
│   with module.sumo.aws_cloudwatch_event_rule.dlq_eks,
│   on modules/sumo/cluster.tf line 231, in resource "aws_cloudwatch_event_rule" "dlq_eks":
│  231:   is_enabled          = true
│
│ Use "state" instead
│
│ (and one more similar warning elsewhere)
╵

AWS (sumo): Error creating cloudtrail S3 Bucket

Error output

╷
│ Error: creating S3 Bucket (quicklab-ck1x-ct-us-west-1-20240104165736104900000001) Notification: operation error S3: PutBucketNotificationConfiguration, https response error StatusCode: 400, RequestID: [redacted], HostID: [redacted], api error InvalidArgument: Unable to validate the following destination configurations
│
│   with module.sumo["ck1x"].aws_s3_bucket_notification.sumo,
│   on modules/sumo/cloudtrail.tf line 229, in resource "aws_s3_bucket_notification" "sumo":
│  229: resource "aws_s3_bucket_notification" "sumo" {
│
╵

Sumo monitoring problems when using free sumo account

It appears that the lookup tables are not available in the free account. Is there a work around or must I use a full account to play with quick labse?

Error: {"id":"KWB1A-ND8B2-RX15P","errors":[{"code":"lookup:invalid_request","message":"Invalid lookup table request.","detail":"Scalable Lookups is not available in your account type."}]}

│ with module.sumo["gewq"].sumologic_lookup_table.aws_admins,
│ on modules/sumo/cloudtrail.tf line 99, in resource "sumologic_lookup_table" "aws_admins":
│ 99: resource "sumologic_lookup_table" "aws_admins" {

Missing --profile on shell command?

When using STS authentication, I specify the profile to use in the aws.auto.tfvars file and it performs well except here where the --profile parameter is missing which causes an error.

It seems modules\cluster\main.tf line 75 must need to include the --profile

Code block:

provisioner "local-exec" {
   command = "aws eks --region ${data.aws_region.current.name}  update-kubeconfig --kubeconfig ~/.kube/${aws_eks_cluster.this.name} --name ${aws_eks_cluster.this.name} --alias ${aws_eks_cluster.this.name} --user-alias ${aws_eks_cluster.this.name}"
 }

Error:

module.cluster["gewq"].null_resource.kubeconfig (local-exec): Executing: ["/bin/sh" "-c" "aws eks --region us-west-2 update-kubeconfig --kubeconfig ~/.kube/quicklab-gewq-cluster --name quicklab-gewq-cluster --alias quicklab-gewq-cluster --user-alias quicklab-gewq-cluster"]
module.cluster["gewq"].local_file.astroshop_values: Creation complete after 0s [id=6b1692a79fd7c80b381f04545599c1991b5082e5]
module.cluster["gewq"].data.tls_certificate.cluster: Read complete after 1s [id=34b882443ccd2d1a84de1dbac994b28b7f371f17]
module.cluster["gewq"].data.aws_ssm_parameter.eks_ami_release_version: Read complete after 1s [id=/aws/service/eks/optimized-ami/1.28/amazon-linux-2/recommended/release_version]
module.cluster["gewq"].aws_iam_openid_connect_provider.cluster: Creating...
module.cluster["gewq"].aws_launch_template.this: Creation complete after 1s [id=lt-06b55e7f658b40c93]
module.cluster["gewq"].aws_iam_openid_connect_provider.cluster: Creation complete after 0s [id=arn:aws:iam::927411430400:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/689F29F151F9D052EE606C66A3C2B95D]
module.cluster["gewq"].data.aws_iam_policy_document.irsa_trust_policy: Reading...
module.cluster["gewq"].data.aws_iam_policy_document.irsa_trust_policy: Read complete after 0s [id=1998513131]
module.cluster["gewq"].aws_eks_node_group.this: Creating...
module.cluster["gewq"].aws_iam_role.irsa: Creating...
module.cluster["gewq"].aws_iam_role.irsa: Creation complete after 1s [id=quicklab-gewq-cluster-irsa-role]
module.cluster["gewq"].aws_iam_role_policy_attachment.addon["AmazonEKS_CNI_Policy"]: Creating...
module.cluster["gewq"].aws_iam_role_policy_attachment.addon["AmazonEBSCSIDriverPolicy"]: Creating...
module.cluster["gewq"].aws_iam_role_policy_attachment.addon["AmazonEKSWorkerNodePolicy"]: Creating...
module.cluster["gewq"].aws_iam_role_policy_attachment.lbc: Creating...
module.cluster["gewq"].aws_iam_role_policy_attachment.addon["AWSFaultInjectionSimulatorEKSAccess"]: Creating...
module.cluster["gewq"].aws_iam_role_policy_attachment.addon["AWSFaultInjectionSimulatorEKSAccess"]: Creation complete after 0s [id=quicklab-gewq-cluster-irsa-role-2023111521440083110000000f]
module.cluster["gewq"].aws_iam_role_policy_attachment.addon["AmazonEKSWorkerNodePolicy"]: Creation complete after 0s [id=quicklab-gewq-cluster-irsa-role-20231115214400966900000010]
module.cluster["gewq"].aws_iam_role_policy_attachment.addon["AmazonEBSCSIDriverPolicy"]: Creation complete after 0s [id=quicklab-gewq-cluster-irsa-role-20231115214400982300000011]
module.cluster["gewq"].aws_iam_role_policy_attachment.lbc: Creation complete after 0s [id=quicklab-gewq-cluster-irsa-role-20231115214400998700000012]
module.cluster["gewq"].aws_iam_role_policy_attachment.addon["AmazonEKS_CNI_Policy"]: Creation complete after 0s [id=quicklab-gewq-cluster-irsa-role-20231115214401005700000013]

module.cluster["gewq"].null_resource.kubeconfig (local-exec): An error occurred (UnrecognizedClientException) when calling the DescribeCluster operation: The security token included in the request is invalid.

With sumo enabled I receive the following error "Field with the given name already exists" in the system section

While I can fully deploy quicklabs with montoring=none, if I enable sumo monitoring, a series of errors are reported.
I will report them one at a time in separate tickets, although they may have the same solution related to local.fields

Running the following:

terraform apply -auto-approve -var="create_network=true" -var="create_bastion=true" -var="create_cluster=false" -var="monitoring=sumo"

Code block:

resource "sumologic_field" "system" {
  for_each = var.create_bastion ? toset(local.fields.resourcedetection.system) : toset([])

  field_name = each.key
  data_type  = "String"
  state      = "Enabled"
}

Error:

│ Error: {"id":"Y4I1F-1Q5CB-BQ2S9","errors":[{"code":"field:already_exists","message":"Field with the given name already exists"}]}

│ with module.sumo["gewq"].sumologic_field.system["host.name"],
│ on modules/sumo/bastion.tf line 25, in resource "sumologic_field" "system":
│ 25: resource "sumologic_field" "system" {



│ Error: {"id":"OOGFF-0VUBP-NS0MZ","errors":[{"code":"field:already_exists","message":"Field with the given name already exists"}]}

│ with module.sumo["gewq"].sumologic_field.system["host.id"],
│ on modules/sumo/bastion.tf line 25, in resource "sumologic_field" "system":
│ 25: resource "sumologic_field" "system" {

QuickLab cluster should not leave behind unused EBS volumes

Currently, QuickLab cluster users need to remove pvc-related EBS volumes manually in AWS (look for detached volumes with names like quicklab-gnap-cluster-dynamic-pvc-d3ab7504-7b74-4ade-962f-8f411421aaa2), even after toggling off the cluster (var.create_cluster = false) and after a terraform destroy.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.