Giter VIP home page Giter VIP logo

compose-x / ecs-files-composer Goto Github PK

View Code? Open in Web Editor NEW
15.0 2.0 1.0 1.04 MB

A simple program / side-car service which only purpose is to pull files and inject it to docker volumes

Home Page: https://docs.files-composer.compose-x.io/

License: Mozilla Public License 2.0

Dockerfile 1.04% Makefile 4.58% Python 94.39%
aws aws-ecs docker-compose aws-s3 aws-secrets-manager aws-ssm docker-volumes

ecs-files-composer's Introduction

ECS Files Composer

https://codebuild.eu-west-1.amazonaws.com/badges?uuid=eyJlbmNyeXB0ZWREYXRhIjoiRWk3VUhxUi9peEdZRGs2cGFiTk5XM0VDK1FEQTBMN2JTdHh5b091NTVVdFd3RmpoM1lpdGkrTGtTZDJzVCt5dDBCc3Zsc0dXWHI5RHJRSG82UFJNdUJzPSIsIml2UGFyYW1ldGVyU3BlYyI6InJlYXlBWStNMDVZNEoyMnQiLCJtYXRlcmlhbFNldFNlcmlhbCI6MX0%3D&branch=main

What does it do?

ECS Files Composer, although can be used in EKS and other Docker context, is a small program that will allow users to define files they need pulled out of AWS Services, such as AWS S3 or AWS SSM Parameter Store, and load the content to a given location, adding builtin capabilities to set file ownership, mode, and other handy features.

The configuration of the job to perform can be written in YAML or JSON (see examples), so long as they comply to a given schema.

Why use it?

Having your core application, when reliant on configuration files, can be tricky to start in a way that the configuration needs to be pulled first and then started. This can add un-necessary complexity and logic to the application. And some docker images you might pull off the shelves from DockerHub do not necessarily allow for configuration override from environment variables.

By using a sidecar that handles all of that logic, you delegate all of these activities to it. And with the ability to define which container to start first with success criteria, you also ensure that your application won't start without the configuration files it needs.

Hint

This app / docker image can be used in any context, locally, on-premise, with Docker, on AWS ECS / EKS or in other cloud platforms.

How to use it ?

Full official documentation

Docker

docker run public.ecr.aws/compose-x/ecs-files-composer:${VERSION:-latest} -h
docker run -v $PWD:/ /var/tmp,:/public.ecr.aws/compose-x/ecs-files-composer:${VERSION:-latest} -f files.yaml

Attention!

The default user is root to avoid running into issues when using chmod/chown and other commands. Change behaviour at your own risks.

CLI

usage: ecs_files_composer [-h] [-f FILE_PATH | -e ENV_VAR | --from-ssm SSM_CONFIG | --from-s3 S3_CONFIG] [--role-arn ROLE_ARN] [_ ...]

optional arguments:
  -h, --help            show this help message and exit
  -f FILE_PATH, --from-file FILE_PATH
                        Configuration for execution from a file
  -e ENV_VAR, --from-env-var ENV_VAR
                        Configuration for execution is in an environment variable
  --from-ssm SSM_CONFIG
                        Configuration for execution is in an SSM Parameter
  --from-s3 S3_CONFIG   Configuration for execution is in an S3
  --role-arn ROLE_ARN   The Role ARN to use for the configuration initialization

ecs-files-composer's People

Contributors

dependabot[bot] avatar johnpreston avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

kingdama

ecs-files-composer's Issues

[FR] Decode base64 prior to run the context (plain/jinja)

In some cases, such as storing template into SSM which does not support {{}} in a Sting parameter, it could be useful to encode to base64 the template before executing Jinja etc.

Therefore the decoding should be happening first and then evaluate the context.

[FR] Enable files dependencies

I want to be able to define that one file depends on the presence of another before acquiring.

files:
  /path/to/A: {}
  /path/to/B:
    depends_on:
      - /path/to/A

[FR] Create / Publish HELM chart

This sidecar could also be very helpful in a K8s context. Creating a HELM chart would allow to enable users to use this in their K8s/EKS context.

[FR] Support GIT source

I would like to be able to retrieve files from a GIT source.

  • git+ssh://
  • git+https://
  • git+http://
  • codecommit://

One must agree on:

  • When using SSH, I understand that I need to provide the private and public key for the authentication to work.
  • When authentication required, I understand that I must provide username/passwords

When using CodeCommit, the IAM role used by the task must provide access to the necessary resources on AWS.
For codecommit, will use https://github.com/aws/git-remote-codecommit as library to retrieve the URL to peform the clone.

Generate SSL Certificate

Is your feature request related to a problem? Please describe.
Some applications such as NGINX / Apache might want to use SSL even self-signed.
I would like to be able to generate a self-signed certificate that the application can then use simply to enable SSL on endpoint.

FR: Allow rendering of Jinja templated files

As a developer I might be using an application (3rd party or else) that cannot use environment variables.
So I created a template file, that should use variables that a render engine such as Jinja can use to render a final version of the configuration needed by the application

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.