Giter VIP home page Giter VIP logo

kit's Introduction

Kit

CodeQL Go goreleaser

Why

Make the dev loop crazy fast.

What

Kit is a software development tool designed to turbo-charge the software development process, with inspiration from several tools, including Foreman, Docker Compose, Podman, Tilt, Skaffold, and Garden. It combines key features and capabilities of these tools, while also offering additional functionality and flexibility.

It is designed to work seamlessly with both local-dev and cloud-dev environments, such as Codespaces and Gitpod.

Watch the video

Key features of Kit include:

  • Local testing: Kit is designed for local testing, allowing developers to test their code on their local machines before pushing it to a test environment or production. This speeds up the testing process and helps developers catch and fix bugs more quickly.
  • Advanced DAG architecture: Kit's directed acyclic graph (DAG) structure allows for optimized parallel processing, reducing the time required for testing and speeding up the development process.
  • Probes: You can specify liveness probes for your tasks to see if they're working, automatically restarting them when they go wrong. You can also specify readiness probes for your tasks to see if they're ready.
  • Dependency management: You can specify dependencies between tasks, so when upstream tasks become successful or ready, downstream tasks are automatically started.
  • Comprehensive container management: Kit can manage both host processes and containers, providing a comprehensive view of the entire software system and quickly identifying any issues or bugs that arise.
  • Automatic rebuilding and restarting: Kit can automatically rebuild and restart applications in response to changes in the source code or configuration files, allowing developers to test their code quickly and efficiently.
  • Parameterizable Kit allows you to run tasks with different parameters.
  • Flexible integration and extensibility: Kit is designed to be highly flexible and extensible, with support for a wide range of programming languages, frameworks, and tools. It can be easily integrated with existing systems and customized to meet specific needs.
  • Terminal output: Tasks run concurrently and their status is muxed into a single terminal window, so you're not overwhelmed by pages of terminal output.
  • Log capture: Logs are captured so you can look at them anytime.

Kit was written with extensive help from AI.

Install

Like jq, kit is a tiny (8Mb) standalone binary. You can download it from the releases page.

If you're on MacOS, you can use brew:

brew tap kitproj/kit --custom-remote https://github.com/kitproj/kit
brew install kit

Otherwise, you can use curl:

curl -q https://raw.githubusercontent.com/kitproj/kit/main/install.sh | sh

We do not support go install.

Usage

Apps are described by a DAG, for example:

---
title: Example of an app
---
flowchart LR
    api(name: api\ncommand: java -jar target/api.jar\nworkingDir: ./api\nports: 8080):::host --> build-api(name: build-api\ncommand: mvn package\nworkingDir: ./api\nwatch: ./api):::host
    api --> mysql(name: mysql\n image: mysql:latest):::container
    processor(name: processor\ncommand: ./processor):::host --> build-processor(name: build-processor\ncommand: go build ./processor\nwatch: ./processor):::host
    processor --> kafka(name: kafka\nimage: ./src/images/kafka):::container
    processor --> object-storage(name: object-storage\nimage: minio:latest):::container
    processor --> api
    ui(name: ui\ncommand: yarn start\nworkingDir: ./ui\nports: 4000):::host --> build-ui(name: build-ui\ncommand: yarn install\nworkingDir: ./ui):::host
    ui --> api

Create a tasks.yaml file, e.g.:

apiVersion: kit/v1
kind: Tasks
metadata:
  name: my-proj
spec:
  tasks:
    - command: go build -v .
      name: build-bar
      watch: demo/bar/main.go
      workingDir: demo/bar
    - command: ./demo/bar/bar
      dependencies: build-bar
      env:
        - PORT=9090
      name: bar
      ports: "9090"
    - dependencies: bar
      name: up

Start:

kit up

You'll see something like this:

screenshot

Logs are stored in ./logs.

Documentation

  • Usage - how to use the various features of kit
  • Examples - examples of how to use kit, e.g. with MySQL, or Kafka
  • Reference - reference documentation for the various types in kit

kit's People

Contributors

acepie avatar alexec avatar dependabot[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

acepie geekychris

kit's Issues

Runs indefinitely when used as an `onCreateCommand` in GH Codespaces

Follow-up to argoproj/argo-workflows#11928 (comment) and argoproj/argo-workflows#11423.

[...] Codespaces running indefinitely on kit pre-up: argoproj/argo-workflows#11423. I even got a notification from GH that it used up 24 hours of compute before I even had access to a terminal (it apparently ran all night even after I closed the tab and turned off my computer).
I just looked into that yesterday and think it can be solved by switching the devcontainer's onCreateCommand to a postCreateCommand, but something weird is going on with how Codespaces is interacting with kit specifically at build-time. I saw it restart a few processes repeatedly too, not sure why.

Possibly related to #42 given the restarts I saw, but not sure

Disable watch option

I’d like the option to disable watching for file changes. The use case is, I don’t want to restart services because I’m using hot reload which is much faster.

Better output when in error state

Currently, when a task goes into error state it is hard to know what went wrong. It is not enough to look at the just the stderr, you will also need to look at stdout.

Also, when readiness is failing.

Vs Skaffold, Draft, Garden, Tilt, DevSpace, TelePresence, Docker Dev Enviroments

Skaffold handles the workflow for building, pushing and deploying your application, allowing you to focus on what matters most: writing code.

13k stars.

Are your servers running locally? In Kubernetes? Both? Tilt gives you smart rebuilds and live updates everywhere so that you can make progress.

6.2k stars.

Telepresence gives developers infinite scale development environments for Kubernetes.

5.5k stars.

DevSpace is an open-source developer tool for Kubernetes that lets you develop and deploy cloud-native software faster.

3.3k stars

Garden combines rapid development, testing, and DevOps automation in one tool. Get realistic cloud-native environments for every occasion and never worry about the difference between dev, CI and prod again.

2.7k stars.

Draft is a tool made for users who are just getting started with Kubernetes, or users who want to simplify their experience with Kubernetes.

250 stars

Dev Environments lets you create a configurable developer environment with all the code and tools you need to quickly get up and running.

Conditional tasks

It might be useful to only run tasks under certain conditions. Eg only run a build if a file does not exist.

Always run all tasks

Today you must specify the task you want to run.

What about if it just ran everything instead? How would you choose to just run some tasks?

Allows comments

Currently, Kit re-writes the tasks.yaml, loosing comments. I don't like that.

Env file support

If you want to change many environment variables at once you need to give them all. It is verbose. It might be nice to have .env files imported, e.g. “kit -e test” would import “test.env”.

Conditionally skipping/running tasks

I would be good to be able to conditionally run tasks. This would be a bit like Github actions, e.g.`

- name: foo
   command: sh script
   if: success()
- name: foo
   command: sh script
   if: failure()
- name: foo
   command: sh script
   if: always()
- name: foo
   command: sh script
   if: env.FOO == 1

Docs

  • Reference
  • Schema
  • Examples
  • #23

Memory leak?

All I know is that when I swapped it out, my fans made a lot less noise and my memory usage was considerably lower 😅 .
I did not trace it carefully, but it seemed like there might be a memory leak in the auto-restart feature or something as I noticed the memory just kept increasing when I made changes to the code and not going back down. It could be a memory leak in one of the tasks themselves too though; again, I did not trace it, that was merely an observation and not one of the initial motivators.

There aren't too many places there could be a memory leak. Perhaps it is in the watch?

Easier to debug faults

Apps tend to consist of:

  • Downstream dependencies. E.g. Kafka. These don’t generally go faulty, because faults are caused by code changes.
  • The app itself. This goes faulty a lot, because user changes the code and introduces faults.

When app goes faulty, we want to see the logs:

  1. All logs, muxed to the terminal.
  2. Just the app logs.

Currently we do (2). Should we do (1)?

What I actually do is have a second terminal open running tail -f. This is poor UX.

Should we switch out of max mode automatically?

`ARGO_LOG_LEVEL` has no effect

If I set:

- name: controller
  env:
    - ARGO_LOG_LEVEL=debug

in argo-workflows' tasks.yaml file, it seems to have no effect. (I haven't tried the other environment variables.)

Custom color and prefix support for logging

Having each service/task/background process have a custom color and prefix would be nice.

prefixes

Generally I'm fine with the default prefix of using the task name, but I think it's more commonly [task] rather than task: (the latter is what Kit currently does). Brackets are more obvious to me that it's a prefix as well, since they are not used in logs commonly (whereas colons sometimes are).

colors

Having each task log as a different color from each other is super useful for readability, IMO. A different color with good contrast by default would be great. (WCAG compliant colors are ideal, although this is a terminal, not a web interface, so WCAG does not quite apply, but the contrast semantics are still useful for accessibility).

task comparison for reference

Task lets you choose an "output mode" (group/interleaved/prefixed) and lets you customize the prefix per task. Colors can be globally configured and toggled with env vars, but doesn't seem like they can be customized per task.

Profiles

I’d like to be able to label a task with one of more profiles and the run ‘kit -p profile-1 up’ so it only runs those tasks.

"Memoization" / work avoidance / don't run task if sources haven't changed

Mentioned in #40 (comment) but didn't get an answer. I think this behavior is not currently supported.

Task supports specifying sources and generates and will perform a checksum or timestamp (checksum is the default, timestamp matches Make) check on those when specified. If the files listed in generates exist and the sources haven't changed, the task will not be run.

Make supports a similar feature by default that uses timestamps.

Would be good feature parity for Kit to support this, especially as it can result in using a lot less compute and time

Better name

Wants

  • Easy to type: 2 or 3 letters long, that flow.
  • Unique.
  • Work well with up and down sub-commands. E.g. spin up

Ideas

  • awn
  • spin
  • app
  • pup
  • pu
  • joy

demo kit

This is a suggestion for a demo that the docs can go over. Would consider this part of docs but some kind of demo that shows the marriage of docker tasks and non docker tasks. Perhaps something resembles a pseudo twitter app:

couple of docker containers:

  • postgres
  • memcache

simple app in something like node or java that provides a simple account / tweet type data model backed by posgres and leverages memcache as a cache layer.

With something like this you can show:

  • standard docker containers
  • an app under development leveraging those containers
  • some kind of liveness probe of the code under test

`yarn install` task sometimes runs indefinitely inside of Kit

Summary

I'm not really sure why this is happening, but when Node deps change, sometimes a task that does yarn install will just run indefinitely and never finish inside of Kit.

Details

The ui-deps task in Argo Workflows runs yarn install and several times I have noticed it just run for many minutes, at which point I just Ctrl+C to kill it.
Then I run yarn install manually outside of Kit, and it installs everything fine within 30s. When I run Kit again after the manual yarn install, Kit will run fine.

yarn install always runs something, but it basically no-ops when there's nothing to install.

Hypothesis

I have a feeling this may be because of the auto-updating progress indicator that Yarn has. Kit seems to not capture it properly in its logs, and so maybe it's failing to see it finish? I think this requires a TTY or something (I forget the exact terminology and shell semantics here, but similar to docker -t. See also this SO Answer on shell output buffering)

Example

For example, in the below Kit run:

■ go-deps    waiting  waiting for mutex "downloads"
■ install    waiting  
■ build-cont waiting  
■ port-forwa waiting  [9000] 
■ controller waiting  [9090] 
■ build-argo waiting  
■ server     waiting  [2746] 
■ ui-deps    running  warning " > [email protected]" has incorrect peer dependency "react-dom@>=17.0.0".
■ ui         waiting  [8080] 

ui-deps will have the same message indefinitely. Enabling debug logging with "4+Enter" shows no logs either.
That message is the last message before Yarn's installation progress indicator, which is a self-updating message (i.e. it does not output new lines, it continuously updates the same log line) of the form: ####------ 4/10 (but longer), where the percentage of hashes and the number represents progress.

Comparison to other Task Runners?

For example, task and xc are existing, quite popular task runners written in Go that largely cover the same feature set.

And other tools that have slightly different goals, such as just, mask, and Earthly (Earthly is my current go-to for a variety of reasons).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.