Giter VIP home page Giter VIP logo

tic's Introduction

tic

tic CircleCI CRAN status codecov Lifecycle: stable tic status badge

The goal of tic is to enhance and simplify working with continuous integration (CI) systems.

The following providers are supported:

Provider R package Platforms Info
Circle CI {circle} Linux via Docker images from rocker
Github Actions {ghactions} Linux, macOS, Windows

In addition there is partial support for Drone CI via class DRONE_CI. This means that {tic} recognizes Drone CI as a CI platform and does not exit early or requires setting a fake env var to mimic another CI provider. However, there is neither an R client package for Drone CI currently nor {tic} templates available for bootstrapping CI configuration via tic::use_tic().

To learn more about CI, read our Getting Started vignette.

The most important improvements over existing solutions are:

  1. Deployment to a Git repository is greatly simplified. Update your repository with results from the CI build.

  2. Support for R packages and other kinds of projects (bookdown, blogdown, etc.), with predefined templates. Set up your project to deploy rendered versions of your book or blog with a single push to Git.

  3. Workflow specification in a single .R file, regardless of CI system used. Forget about .yml files or web browser configurations.

Installation

{tic} can be installed from GitHub with:

remotes::install_github("ropensci/tic")

Setup

By calling tic::use_tic() a production ready CI setup is initialized, tailored to your specific R project. The created templates will use the providers Circle CI and Github Actions.

If only the CI YAML templates from {tic} are desired, the use_<provider>_yml() functions can be used. Refer to the complete list of options.

For an R package, the following steps will be set up for the CI workflow:

  • Installation of required dependencies for the project (dependencies are scraped from the DESCRIPTION file*)
  • Satisfying build-time dependencies of steps to be run in all CI stages (by scraping pkg::fun calls in tic.R)
  • Checking of package via rcmdcheck::rcmdcheck()
  • Creation of a pkgdown site including Github deployment
  • Running a code coverage and upload to codecov.io

See the Getting Started vignette for more information and links to minimal example repositories for various R projects (package, blogdown, bookdown and more).

Quickstart

If you are a new user, run

tic::use_tic()

If you already use {tic} and want to configure a new CI provider, do one of the following (depending on your preferred CI provider)

### Circle CI ------------------------------------------------------------------

circle::use_circle_deploy() # (optional for deployment)
tic::use_circle_yml() # optional: Change `type` arg to your liking
tic::use_tic_r("package", deploy_on = "circle")
# (all of the above in one call)
# tic::use_tic(wizard = FALSE, linux = "circle", mac = "none", windows = "none",
#              matrix = "circle", deploy = "circle")
tic::use_update_tic()

### GitHub Actions -------------------------------------------------------------

tic::use_ghactions_deploy() # (optional for deployment)
tic::use_ghactions_yml() # optional: Change `type` arg to your liking
tic::use_tic_r("package", deploy_on = "ghactions")
# (all of the above in one call)
# tic::use_tic(wizard = FALSE, linux = "ghactions", mac = "ghactions",
#              windows = "ghactions", matrix = "ghactions", deploy = "ghactions")

tic::use_tic_badge("ghactions")
tic::use_update_tic()

Good to know

We would like to mention that {tic} is a choice and sits on top of existing community efforts providing R support for various CI providers. While {tic} will prevent you from dealing/learning every CIs YAML syntax, you will have to learn {tic}'s way of specifying your tasks on CI systems.

Also, there is no way around familiarizing yourself with the basics of CI systems in general. Without this knowledge, you will also have a hard way understanding {tic}.

We also recommend to take a look at the projects providing the direct R support for each CI system (which {tic} builds upon) to gain a deeper understanding of the whole concept.

Keeping CI templates up-to-date

{tic} is refreshing its template every few months. There are two ways to stay up-to-date and have a functional CI YAML definition:

  1. Manual updates: You can keep {tic} up-to-date by rerunning the use_*_() function which you have used to init {tic} the first time. If you have forgotten which settings you have chosen back then, you can have a look at the first line of your YAML file.

  2. Automatic updates via GHA: If you're using GitHub actions, you can use tic::use_update_tic(). This will initialize a new workflow which checks daily for new template updates. Important: This workflow requires a personal access token with "workflow" scopes. After creation, you can add it via tic::gha_add_secret() to your repo account. Hint: If you're operating in an organization, you can set this GHA secret organization-wide which saves you adding it to multiple single repos.

See vignette "Updating Templates" for more information.

Vignettes

Limitations

The setup functions in this package assume Git as version control system, and GitHub as platform. Automated setup works best if the project under test is located in the root of the Git repository. Multi-project repositories are not supported, see the comment by @jwijffels for guidance to work around this limitation.

The DESCRIPTION files needs to live in the project root. To simplify its creation have a look at usethis::use_package() or usethis::use_description().

tic's People

Contributors

adamhsparks avatar cderv avatar gadenbuie avatar github-actions[bot] avatar jeroen avatar kapsner avatar karthik avatar krlmlr avatar maelle avatar mikabr avatar muschellij2 avatar pat-s avatar psychelzh avatar rekyt avatar robinlovelace avatar statnmap avatar web-flow avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

tic's Issues

Support sequences of steps

get_stage("deploy") %>%
  sequence_start("pkgdown") %>%
  step_build_pkgdown() %>%
  step_push_deploy() %>%
  sequence_stop("pkgdown")

Need a better name for it.

Issue when force deploy to master

Hi there,

I am trying to deploy my pkgdown docs from the dev branch to master/docs but I keep facing conflicts with untracked git files introduced during travis building:

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	sperrorest.Rcheck/
	sperrorest_2.1.1.tar.gz
	travis_wait_162.log
nothing added to commit but untracked files present (use "git add" to track)
Dropped refs/stash@{0} (26e5d8e7bad31ed5c4798cb77225478885791803)
Script failed with status 1
failed to deploy

https://travis-ci.org/pat-s/sperrorest/jobs/311851002

Any idea what I should do here?

Dependency structure

Any package that is imported by tic can't be updated easily on Windows. This means we need to be very careful with our dependencies.

  • provide a wrapper around remotes::install_deps()?
  • run each step in its own process?
  • parse tic.R only once, so that parsed representation doesn't require R6?
  • ...?

Install Makevars

for e.g. enabling additional warnings when compiling native code.

[pkgdown] Some modified files are not deployed

Hi there,

in my last build I saw that some modified files by build_site() (e.g. docs/index.html) are not deployed to docs/.

Travis build: https://travis-ci.org/pat-s/mlr/builds/311728468
Repo docs folder: https://github.com/pat-s/mlr/tree/master/docs

You can see that Travis build 42 pushed some files to docs but docs/index.html was not pushed by Travis build 42 although it was listed as modified in the Travis build (see link above).
This caused that the hosted site still reflects the status before the deployment as the changes in docs/index.html are missing.

Any idea why this is happening? ๐Ÿค”

Color output

via crayon.

  • Show headers in magenta
  • Nicer formatting of the results of a stage

Travis integration

@jimhester: The tic package aims at providing a CI-agnostic way to specify tests and deployments. I wonder what's the best way to integrate it with language: r on Travis. Ideally, I'd like the Travis runner to call functions exported from tic, but I'm not sure how to provide a smooth transition for existing .travis.yml files.

CC @karthik.

Clarify steps

Purpose of step_hello_world

If this is just to help the user test tic, then we should state that explicitly.

Similarly with step_test_ssh, we should guide the user on when they might use it.

Support more than one deploy key

From ropensci-archive/travis#61 (comment):

This is rather difficult: GitHub doesn't allow using the same deploy key for two repositories, and we can only use one private key on Travis CI without involving an SSH agent. This means that we currently can have either drat or pkgdown/packagedocs, but not both.

Shelved for now.

Documentation

  • functions
  • classes
  • DSL
  • pkgdown
  • example repos
    • package
    • website
    • bookdown
    • blogdown
    • drat (#1)
    • packagedocs (#3)

Also set warnings_are_errors: true in .travis.yml .

Add rOpenSci as funder

Also add rOpenSci footer:

[![ropensci_footer](https://ropensci.org/public_images/ropensci_footer.png)](https://ropensci.org)

Shortcut for enabling SSH access

stage %>%
    add_step(step_install_ssh_keys()) %>%
    add_step(step_add_to_known_hosts("github.com")) %>%
    add_step(step_test_ssh())

Used all over the place.

Stage names

Stages can have arbitrary names

Clarification needed. If these are not standard stages that describe when parts of a recipe will be run, what does it mean for the user to have something else?

CI-agnostic

  • configuration
  • environment variables
  • commit message

Should at least work for AppVeyor.

Shortcut for package checking

get_stage("before_install") %>%
  add_step(step_run_code(update.packages(ask = FALSE)))

get_stage("install") %>%
  add_step(step_run_code(remotes::install_deps(dependencies = TRUE)))

get_stage("script") %>%
  add_step(step_rcmdcheck())

get_stage("after_success") %>%
  add_step(step_run_code(covr::codecov(quiet = FALSE)))

Improve DSL

  • steps class, with steps() constructor
  • as.steps.list()
  • as.steps.step()
  • as.steps.formula() to support ~{} notation

use_tic()

  • Only if repo is GitHub-enabled
    • Ask in interactive mode
  • Enable travis
  • Create .travis.yml and appveyor.yml
  • Autodetect package type, create corresponding tic.R
    • Set up deployment if necessary
  • Create PAT

Custom steps

This part of the README could use a figure of some sort to help the under understand the workflow.

Define all build stages available on Travis CI and/or AppVeyor

Travis docs: https://docs.travis-ci.com/user/customizing-the-build

AppVeyor docs: https://www.appveyor.com/docs/build-configuration/#build-pipeline

*: AppVeyor equivalents

  • before_install (* before_build)
  • install (* build_script)
  • after_install (* after_build)
  • before_script (* before_test)
  • script (* test)
  • before_cache (* no equivalent)
  • after_success (* on_success)
  • after_failure (* on_failure)
  • before_deploy (* run before on_success)
  • deploy (* deploy_script, run before on_success)
  • after_deploy (* run before on_success)
  • after_script (* on_finish)

Stages and steps need clarification for the end user

tic embraces this concept. Each stage has a name (e.g., "after_success", "deploy", ...) and has an arbitrary number of steps.

The documentation needs to address how a user makes sense of after_success and say deploy. e.g. if a user were to use this to build package docs with pkgdown, then on success, they would include the recipe to "deploy" the docs to a branch. Then where does deploy come into play since we just deployed under after_success. This type of ambiguity needs clarification since we cannot assume the end user understands these stages.

Pass on information between steps

Example: A "deploy" step requires a path created in a "build" step.

Idea: Maintain a persistent cache of results, support naming steps. Fail with duplicate names.

Build statuses

Integration of Travis and Appveyor APIs to get build statuses is beyond the scope of this package, right?

Feature request: Deploy single files

Currently, one can only deploy directories using step_push_deploy(), is that correct?

It would be great if the path argument could detect if the path given is a directory or a file and do the right thing.

Also, currently a push seems to wipe the complete repo?
While this is fine for a clean gh-pages branch or docs/ folder, it does not help for single file deployments.
I looked into the source code but adapting such a function seems to be harder than initially thought (no experience yet with R6).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.