Giter VIP home page Giter VIP logo

stepman's Introduction

stepman

You can manage decentralized StepLib Step (script) Collections with stepman.

You can register multiple collections, audit them, manage the caching of individual Steps locally, define and handle fallback URLs for downloading the Step codes (archives), and share new Steps into public StepLib collections with stepman.

Public Beta: this repository is still under active development, frequent changes are expected, but we we don't plan to introduce breaking changes, unless really necessary. Feedback is greatly appreciated!

Part of the Bitrise Continuous Integration, Delivery and Automations Stack, with bitrise and envman.

Install

Check the latest release for instructions at: https://github.com/bitrise-io/stepman/releases

Share your own Step

Call stepman share and follow the guide it prints.

stepman's People

Contributors

birmacherakos avatar godrei avatar kdobmayer avatar lpusok avatar lszucs avatar matrangam avatar ofalvai avatar renovate[bot] avatar sujitpoudel avatar trapacska avatar vasarhelyia avatar viktorbenei avatar zsolt-marta-bitrise avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

stepman's Issues

stepman update - print which collection(s) it updates

Right now it's quite hard to tell which collection has an issue from the output. stepman should print which collection it updates before it'd attempt to update it.

Example log:

stepman update
INFO[19:27:47] [STEPMAN] - Update
INFO[19:27:47] [STEPMAN] - No step collection specified, update all
Already up-to-date.
remote: Counting objects: 12, done.
remote: Total 12 (delta 4), reused 4 (delta 4), pack-reused 8
Unpacking objects: 100% (12/12), done.
From github.com:viktorbenei/bitrise-steplib
   5f2f48f..64c78e0  master     -> origin/master
Updating 5f2f48f..64c78e0
error: Your local changes to the following files would be overwritten by merge:
    bitrise.yml
Please commit your changes or stash them before you can merge.
error: Your local changes to the following files would be overwritten by merge:
    _scripts/audit/audit_changed_steps.go
Please commit your changes or stash them before you can merge.
Aborting
2016/07/03 19:27:52  [!] Git pull failed, error (exit status 1)
FATA[19:27:52] Failed to update collection (), err: exit status 1

Retry spec download if fails

stepman should retry (with a short delay) the spec download (before falling back to a mirror/alternative source) if the first attempt fails, to improve reliability.

Include a license

I wanted to install bitrise-cli, but while I'm at it I decided to make archlinux packages for envman, stepman and bitrise-cli. However, I can't publish this one since it has no license. Both envman and bitrise are under the MIT license.

I would have opened a PR, but it didn't seem right to "push" a license to your repo!

Thanks!

Invalid (json) output if multiple collections are present

stepman step-info -f json -i script
{"step_id":"script","step_version":"1.1.0","latest_version":"1.1.0","description":"Specify the script content and (optionally) the script runner\nbinary and this step will execute it.\n\nThe *runner* can even be a command, if it's available in the PATH\nand can execute the script it will work.\n\nFor example to run a [Go](https://golang.org/) Hello World \"script\"\nyou can define the content:\n\n    package main\n    import \"fmt\"\n    func main() {\n        fmt.Println(\"hello world\")\n    }\n\nSet the *Execute with* parameter to `go run` and specify\na *script file path* because `go run` can only accept (at the moment)\nfiles with a .go extension (just specify `step.go` as the script path).\n\nIf the `go` binary can be found in the PATH your Hello World\nscript will run without any problem.","source":"https://github.com/bitrise-io/steps-script","steplib":"[email protected]:viktorbenei/bitrise-steplib.git","inputs":[{"key":"content","description":"Type your script here.\n\n**Make sure that it returns a non zero exit code in case of an error!**\nThe step will only fail if your script returns with a non zero exit code!\n","default_value":"#!/bin/bash\n\necho \"Hello Bitrise!\"","is_expand":false},{"key":"runner_bin","description":"The executor to be used for running the script.\nIf it's available in the PATH you can just\nspecify `ruby` or `python`, generally\nif you know the path of the executor you should\ndefine that, like `/bin/bash`.\n\nYou can even specify an execution command\nlike `go run` instead of just a binary.\n","default_value":"/bin/bash","is_expand":false},{"key":"is_debug","description":"If debug=yes the step will print debug infos about\nthe working dir, tmp file path, exit code, etc.\n","value_options":["no","yes"],"default_value":"no","is_expand":false},{"key":"working_dir","description":"This directory will be set as the current working\ndirectory for the script.\n","is_expand":true},{"key":"script_file_path","description":"Save the specified script content to this path\nbefore execution. The file will be removed after\nthe script finishes.\n\nGenerally you don't have to define this but\nthere might be cases where the file's name or path\nactually matters.\nFor example `go run` only accepts `.go` files.\n\nIf you specify a relative path then it will be\nrelative to the *working directory* (which you\ncan optinally specify as well).\n","is_expand":true}],"global_info":{}}
{"step_id":"script","step_version":"1.1.0","latest_version":"1.1.1","description":"Specify the script content and (optionally) the script runner\nbinary and this step will execute it.\n\nThe *runner* can even be a command, if it's available in the PATH\nand can execute the script it will work.\n\nFor example to run a [Go](https://golang.org/) Hello World \"script\"\nyou can define the content:\n\n    package main\n    import \"fmt\"\n    func main() {\n        fmt.Println(\"hello world\")\n    }\n\nSet the *Execute with* parameter to `go run` and specify\na *script file path* because `go run` can only accept (at the moment)\nfiles with a .go extension (just specify `step.go` as the script path).\n\nIf the `go` binary can be found in the PATH your Hello World\nscript will run without any problem.","source":"https://github.com/bitrise-io/steps-script","steplib":"https://github.com/bitrise-io/bitrise-steplib.git","inputs":[{"key":"content","description":"Type your script here.\n\n**Make sure that it returns a non zero exit code in case of an error!**\nThe step will only fail if your script returns with a non zero exit code!\n","default_value":"#!/bin/bash\n\necho \"Hello Bitrise!\"","is_expand":false},{"key":"runner_bin","description":"The executor to be used for running the script.\nIf it's available in the PATH you can just\nspecify `ruby` or `python`, generally\nif you know the path of the executor you should\ndefine that, like `/bin/bash`.\n\nYou can even specify an execution command\nlike `go run` instead of just a binary.\n","default_value":"/bin/bash","is_expand":false},{"key":"is_debug","description":"If debug=yes the step will print debug infos about\nthe working dir, tmp file path, exit code, etc.\n","value_options":["no","yes"],"default_value":"no","is_expand":false},{"key":"working_dir","description":"This directory will be set as the current working\ndirectory for the script.\n","is_expand":true},{"key":"script_file_path","description":"Save the specified script content to this path\nbefore execution. The file will be removed after\nthe script finishes.\n\nGenerally you don't have to define this but\nthere might be cases where the file's name or path\nactually matters.\nFor example `go run` only accepts `.go` files.\n\nIf you specify a relative path then it will be\nrelative to the *working directory* (which you\ncan optinally specify as well).\n","is_expand":true}],"global_info":{}}

Note: I had two collection - and the output includes the script step / full JSON twice, I guess one for each collection

Fallback handling in case the StepLib (spec / source) is not available

Simply download the full spec (JSON) from s3 or from another Spec git repository, if the primary source can't be accessed (e.g. GitHub rate limiting)

Solution idea

  • Include the "alternatives" / "caches" / "mirrors" list in the StepLib's git repo spec, in the steplib.yml
  • The primary StepLib git repo still have to be downloaded at least once (e.g. from GitHub), but then the cache/mirror can be used in case the primary source (GitHub) is not available
  • Cache / mirror should be either an URL (ZIP, tar) or a git (clone) url; a list should be allowed with priority - similar to how download_locations work for steps
  • we should set up a mirror on GitLab for the main Bitrise StepLib - GitLab has an automatic GitHub mirroring feature
  • maybe also set up an S3 cache, just to be sure & to test
  • the ZIP/tar should not include the .git directory - to save space (.git is pretty much the largest dir, as it includes diffs and git history)

Notes

Only download the full spec.json, and do it only if there was a change. First send a HEAD request, to see when it was last modified, and store the modified date when downloading the spec.json, so that it can be used for comparison in subsequent checks.

Short commands doesn't work

This:

stepman d -c https://github.com/bitrise-io/bitrise-steplib
FATA[19:34:06] [STEPMAN] - Missing step id

should be the same as this, but it's not:

stepman delete -c https://github.com/bitrise-io/bitrise-steplib

the second finished successfully

stepman step-info - which collection if none is specified?

I think it tries to search in all - it prints correct info from the primary, which includes [email protected], but I also have older (forked) collection, which do not include this version.

stepman step-info -f json -i script
{"step_id":"script","step_version":"1.1.0","latest_version":"1.1.0","description":"Specify the script content and (optionally) the script runner\nbinary and this step will execute it.\n\nThe *runner* can even be a command, if it's available in the PATH\nand can execute the script it will work.\n\nFor example to run a [Go](https://golang.org/) Hello World \"script\"\nyou can define the content:\n\n    package main\n    import \"fmt\"\n    func main() {\n        fmt.Println(\"hello world\")\n    }\n\nSet the *Execute with* parameter to `go run` and specify\na *script file path* because `go run` can only accept (at the moment)\nfiles with a .go extension (just specify `step.go` as the script path).\n\nIf the `go` binary can be found in the PATH your Hello World\nscript will run without any problem.","source":"https://github.com/bitrise-io/steps-script","steplib":"[email protected]:viktorbenei/bitrise-steplib.git","inputs":[{"key":"content","description":"Type your script here.\n\n**Make sure that it returns a non zero exit code in case of an error!**\nThe step will only fail if your script returns with a non zero exit code!\n","default_value":"#!/bin/bash\n\necho \"Hello Bitrise!\"","is_expand":false},{"key":"runner_bin","description":"The executor to be used for running the script.\nIf it's available in the PATH you can just\nspecify `ruby` or `python`, generally\nif you know the path of the executor you should\ndefine that, like `/bin/bash`.\n\nYou can even specify an execution command\nlike `go run` instead of just a binary.\n","default_value":"/bin/bash","is_expand":false},{"key":"is_debug","description":"If debug=yes the step will print debug infos about\nthe working dir, tmp file path, exit code, etc.\n","value_options":["no","yes"],"default_value":"no","is_expand":false},{"key":"working_dir","description":"This directory will be set as the current working\ndirectory for the script.\n","is_expand":true},{"key":"script_file_path","description":"Save the specified script content to this path\nbefore execution. The file will be removed after\nthe script finishes.\n\nGenerally you don't have to define this but\nthere might be cases where the file's name or path\nactually matters.\nFor example `go run` only accepts `.go` files.\n\nIf you specify a relative path then it will be\nrelative to the *working directory* (which you\ncan optinally specify as well).\n","is_expand":true}],"global_info":{}}
FATA[19:30:49] Collection doesn't contain step (id:script) (version:1.1.0)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.