Giter VIP home page Giter VIP logo

dev-charts's Introduction

THIS HAS MOVED TO THE OFFICIAL CHARTS REPO on the gh-pages branch.

dev-charts

This is an experimental project to automate system charts and adding overlays files on top of it.

Charts will have two categories:

  1. Charts that Rancher created and maintained (Rancher original).

  2. Charts that Rancher modified from upstream (Rancher modified).

Rancher original chart is created and maintained by Rancher Team, such as rancher-cis-benchmark, rancher-k3s-upgrader.

Rancher modified chart is modified from upstream chart, while there are customizations added into the upstream chart from rancher side.

For Rancher original charts, it should have the following tree structure

packages/${CHART_NAME}/
  charts/                   # regular helm chart directory
    templates/
    Chart.yaml
    values.yaml

For Rancher modified charts, it should have the following tree structure

packages/${CHART_NAME}/
  package.yaml              # metadata manifest containing upstream chart location, package version
  ${CHART_NAME}.patch       # patch file containing the diff between modified chart and upstream
  overlay/*                 # overlay files that needs to added on top of upstream, for example, questions.yaml

A regular package.yaml will have the following content:

url: https://charts.bitnami.com/bitnami/external-dns-2.20.10.tgz # url to fetch upstream chart
packageVersion: 00 # packageVersion of modified charts, producing a $version-$packageVersion chart. For example, if istio 1.4.7 is modified with changes, rancher produces a 1.4.700 chart version that includes the modification rancher made on top of upstream charts.

Here is an example of upstream chart based on git repository

url: https://github.com/open-policy-agent/gatekeeper.git  # Url to fetch upstream chart from git
subdirectory: chart/gatekeeper-operator # Sub directory for helm charts in git repo
type: git # optinal, indicate that upstream chart is from git
commit: v3.1.0-beta.8 # the revision of git repo
packageVersion: 00 # package version

Workflow

Modifying Rancher original charts is the same workflow as modifying helm charts. First make changes into charts/ and commit changes. CI will automatically upload artifacts if file contents have been changed.

Modifying Rancher modified takes extra steps, as it requires modifications to be saved into patch files so that later it can retrieve the chart based on upstream chart and patch files.

The step includes:

  1. Run make CHART={CHART_NAME} prepare

    This prepares charts with the current upstream chart and current patch.

  2. Change the version in package.yaml. If upstream chart needs to be updated, update url to point the latest chart. packageVersion also needs to updated.

  3. Make modification to your charts.

  4. Run make CHART={CHART_NAME} patch

    This will compare your current chart with upstream chart and generate the correct patch.

  5. Run make CHART={CHART_NAME} clean

    This will clean up the charts directory so that it won't committed.

This repo provides a workflow that automatically uploads patch files and tarball of charts. Commit will only need to update package/${chart-name}/charts and make sure patches are up-to-date with the latest chart. It also automatically build github pages to serve index.yaml and artifacts of charts.

Override existing Chart

By defauly CI script doesn't allow changes to be made against existing chart. In order to make changes you have to bump chart version. There is a backdoor method to make changes to your existing chart without having to bump version. You can delete the tar.gz file you want to override and commit the change. Here is an example of commit.

Helm repo index

To add this repo as a helm repo, run

helm repo add ${repo_name} https://dev-charts.rancher.io

To use a forked version of this chart repo, you can try either of these:

  1. If you just need to test chart tar.gz file, you can run make CHART=${name} charts to generate tar.gz files. It will be generated under docs/${chart_name}.

  2. You can also setup github page to serve your tar.gz files on your forked repo. Github pages usually requires you to have this set up on specific branches.

  3. You can directly add https://github.com/rancher/dev-charts into rancher catalog. In order to show all the charts you have to run make CHART=${chart_name} prepare and make sure there is chart-original folder on each chart folder if your chart relies on a upstream chart.

Makefile

make bootstrap:

Download binaries that are needed for ci scripts.

make prepare:

Prepare the chart for modification. This will apply the upstream chart with the current patch. Use CHART=${NAME} for specific chart.

make charts:

Generate tarball for each charts. Use CHART=${NAME} for specific chart.

make patch:

Compare the current chart with upstream and generate patch file. Use CHART=${NAME} for specific chart.

make validate:

Validate if patch file can be applied.

make mirror:

Run image mirroring scripts.(Experimental)

dev-charts's People

Contributors

aiyengar2 avatar brendarearden avatar cbron avatar deniseschannon avatar galal-hussein avatar ibuildthecloud avatar paynejacob avatar strongmonkey avatar vincent99 avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dev-charts's Issues

Replace the second timestamp from patch files

Example:

+++ packages/coredns/charts/templates/configmap.yaml 2020-05-19 23:46:53.326737422 +0200

The second timestamp in patch files generated from make patch introduces a ton of new changes per commit that are unrelated to the actual change that needs to be reviewed.

Issues

  • A PR with several commits such as #29 introduces a large number of changes between commits that purely come from timestamp changes (i.e. modifying 2020-07-19 21:58:02.000000000 -0700 to 2020-07-20 00:18:18.000000000 -0700). As a result, the PR becomes difficult to review for a reviewer as they need to filter through all of the timestamp changes in the patch file to find the actual code changes that have been made.
  • Rebasing patch changes within a single PR that is bumping the package version involves re-running the make patch command several times for each commit and encountering several Git conflicts related to timestamp changes. For example, if I have 3 commits and I make a change in commit 1 (i.e. rebase, pick commit 1, run make prepare, make the change necessary, and run make patch), all of the timestamps from commit 1 will change to the current time. This results in several merge conflicts when applying commit 2 onto commit 1, and similarly when applying commit 3 to commit 2.

Proposed solution

On make patch:

  • Replace the timestamp printed by the diff with <package-name>_<package-version> instead. This will introduce changes to all existing patches within the dev-charts repo.

On make prepare:

  • Replace the <package-name>_<package-version> with the current unix timestamp before applying the diff to ensure that the patch step still works as expected.

Once this change is applied, any changes to the chart that necessitate a package version bump would first need to have a commit that just updates the chart version and regenerates the patch followed by the actual changes introduced into the chart for the new version in separate commits.

Alternate solution

Remove timestamps altogether from the patches.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.