Giter VIP home page Giter VIP logo

buildabiocworkshop's Introduction

BuildABiocWorkshop

This package is a template for building a Bioconductor workshop. The package includes Github actions to:

  1. Set up bioconductor/bioconductor_docker:devel on Github resources
  2. Install package dependencies for your package (based on the DESCRIPTION file)
  3. Run rcmdcheck::rcmdcheck
  4. Build a pkgdown website and push it to github pages
  5. Build a docker image with the installed package and dependencies and deploy to the Github Container Repository at the name ghcr.io/gihub_user/repo_name, all lowercase.

Responsibilities

Package authors are primarily responsible for:

  1. Creating a landing site of their choosing for their workshops (a website). This website should be listed in the DESCRIPTION file as the URL.
  2. Creating a docker image that will contain workshop materials and the installed packages necessary to run those materials. The name of the resulting docker image, including "tag" if desired, should be listed in a non-standard tag, DockerImage: in the DESCRIPTION file.

Both of those tasks can be accomplished using the Github actions included in this template package. The vignette accompanying this package describes how to accomplish both of these tasks.

Details

For detailed instructions, see the How to build a workshop article/vignette.

Results of successful deployment

  • A working docker image that contains the installed package and dependencies.
  • An up-to-date pkgdown website at https://YOURUSERNAME.github.io/YOURREPOSITORYNAME/
  • Docker image will be tagged with latest, sha-XXXXXX where XXXXXX is the hash of the current master commit, and master.

To use the resulting image:

docker run -e PASSWORD=<choose_a_password_for_rstudio> -p 8787:8787 YOURDOCKERIMAGENAME

Once running, navigate to http://localhost:8787/ and then login with rstudio:yourchosenpassword.

To try with this repository docker image:

docker run -e PASSWORD=abc -p 8787:8787 ghcr.io/bioconductor/buildabiocworkshop

NOTE: Running docker that uses the password in plain text like above exposes the password to others in a multi-user system (like a shared workstation or compute node). In practice, consider using an environment variable instead of plain text to pass along passwords and other secrets in docker command lines.

Whatcha get

buildabiocworkshop's People

Contributors

aedin avatar almahmoud avatar kevinrue avatar kozo2 avatar link-ny avatar lwaldron avatar seandavi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

buildabiocworkshop's Issues

docker run -e PASSWORD=<pwd>... exposes <pwd> to other users on the same machine

Drive-by comment; Be careful when passing secrets/credentials via command-line arguments as is:

docker run -e PASSWORD=<choose_a_password_for_rstudio> -p 8787:8787 YOURDOCKERIMAGENAME

If this is done on a computer to which others have a access, e.g. computer clusters, then they can see your password with a simple ps aux call.

Deploy error

I was getting this error in the deploy step - "Error: The deploy step encountered an error: The process '/usr/bin/git' failed with exit code 128".

In case anyone else encounters this error, adding write permissions to the workflow file fixed it for me. Here is the thread with the solution: JamesIves/github-pages-deploy-action#1110 (comment)

An error from `sysreqs` is causing the GitHub Actions workflow in this template to fail

Hello @seandavi and team,

I already opened an issue on the sysreqs repo about this, see: r-hub/sysreqs#24. Starting last night, the GHA workflow to build my workshop started failing even though I had only incremented the package version number.

Wondering if this is happening to anyone else? I am pretty sure it is due to an expired SSL certification for the sysreq website, see the issue for more details.

Best,
Christopher Eeles
Software Developer
Haibe-Kains Lab
PM-Research | UHN

How to format DOCKER_USERNAME in secrets?

I have an issue that is slightly related to #10 . I too had issues with my docker credentials failing after 10+ minutes of the action (although in my case I am glad it failed after making the pkgdown website, not before). I think my problem is that I am not formatting the DOCKER_USERNAME and DOCKER_PASSWORD properly and I cannot find an example anywhere. My action fails with

Pushing image [jdrnevich/buildacarpentriesworkshop:latest jdrnevich/buildacarpentriesworkshop:latest jdrnevich/buildacarpentriesworkshop:sha-38bb549]
The push refers to repository [docker.io/jdrnevich/buildacarpentriesworkshop]
60176156405e: Preparing
... # removed a bunch of Preparing and Waiting lines
30691b678e1c: Waiting
denied: requested access to the resource is denied
Error: exit status 1
Usage:
  github-actions build-push [flags]

I created a docker account and got an access token. How exactly should I enter them in the secrets? I have tried:

DOCKER_USERNAME: jdrnevich
DOCKER_PASSWORD: mytokenid

and

DOCKER_USERNAME jdrnevich
DOCKER_PASSWORD mytokenid

Both failed. Instead of trying lots of combos and waiting 10 min to see, I thought I'd ask here and also request you add the formatting to Section 6 of HOWTO_BUILD_WORKSHOP.Rmd. Thanks!

missing .nojekyll

Hi Sean, @seandavi
Thanks so much for making a template for this!

I was able to run the GH Action using the template provided but at some step
the push to gh-pages did not include a .nojekyll file which was not letting
my website be served.

Perhaps this is an issue for one of the steps?

PS. This could be an interaction that I don't fully understand with a custom CNAME
and gh-pages. I see other pages rendered fine without one:
https://github.com/PeteHaitch/BioC2020_DelayedArray_workshop/tree/gh-pages
https://PeteHaitch.github.io/BioC2020_DelayedArray_workshop

Thanks.
-Marcel

Deployment workflow out of date

Hi there,

Following the instructions in the package exactly leads to failing GH Action checks at the "Deploy" phase of r-build-and-check. This is presumably due to an error introduced within the past month causing the workflow to be unable to initialize a repository (see Issue #1110, Issue #1083). This has subsequently been fixed in later releases, but the workflow bundled in the template repository doesn't seem to work.

The fixes for this are:

  • update JamesIves/github-pages-deploy-action@releases/v3 -> JamesIves/github-pages-deploy-action@releases/v4 in basic_checks.yaml:68
  • Add an entry in the README to tell users to ensure "Read and Write permissions" are checked under Settings > Actions > General > Workflow permissions

Check docker credentials and package dependencies at the beginning of the basic_checks.yaml Action

Hi,

I had some issues getting the action to run without error--nothing too serious but I did have to wait 13 minutes to find out that I had entered the wrong docker username/password (visible here: https://github.com/cmirzayi/EpiForBioWorkshop2020/runs/774158738?check_suite_focus=true) and 19 minutes before failure because I had not added pkgdown as a dependency (https://github.com/cmirzayi/EpiForBioWorkshop2020/runs/774201957?check_suite_focus=true). If possible, it would be good if the action checked for these sorts of errors before running for 10+ minutes.

Thanks!
Chloe

Error in Install System Dependencies actions step

Hi,

I'm getting the following error in the install dependencies step:

The following packages have unmet dependencies: libgit2-dev : Depends: libcurl4-gnutls-dev but it is not going to be installed E: Unable to correct problems, you have held broken packages. ##[error]Process completed with exit code 100.

any help?
dario

ERROR in docker build-push-action action step

Hi @seandavi,

when I reach the docker step it gives me an error on the installation of the AnnotationDbi package.

Here the log error section:

Bioconductor version 3.12 (BiocManager 1.30.10), ?BiocManager::install for help

  • installing source package ‘AnnotationDbi’ ...
    ** using staged installation
    ** R
    ** inst
    ** tests
    ** byte-compile and prepare package for lazy loading
    Bioconductor version 3.12 (BiocManager 1.30.10), ?BiocManager::install for help
    Error in dyn.load(file, DLLpath = DLLpath, ...) :
    unable to load shared object '/usr/local/lib/R/site-library/Rcpp/libs/Rcpp.so':
    /usr/local/lib/R/site-library/Rcpp/libs/Rcpp.so: undefined symbol: EXTPTR_PTR
    Calls: ... asNamespace -> loadNamespace -> library.dynam -> dyn.load
    Execution halted
    ERROR: lazy loading failed for package ‘AnnotationDbi’
  • removing ‘/usr/local/lib/R/site-library/AnnotationDbi’
    Error in i.p(...) :
    (converted from warning) installation of package ‘AnnotationDbi’ had non-zero exit status
    Calls: ... with_rprofile_user -> with_envvar -> force -> force -> i.p
    Execution halted
    The command '/bin/sh -c Rscript -e "devtools::install('.', dependencies=TRUE, build_vignettes=TRUE, repos = BiocManager::repositories())"' returned a non-zero code: 1
    Error: exit status 1
    Usage:
    exit status 1

It seems that some Rcpp shared library is missing, any suggestion on how to solve it?

Thanks,
Dario

Warnings about various deprecated actions

As of 12 June 2023, we are getting warnings about Node 12 deprecation in the following:

  • r-build-and-check actions/checkout@v2, actions/cache@v1
  • docker-build-and-push actions/checkout@v2, docker/setup-buildx-action@79abd3f

We are getting warnings about save-state or set-output in docker-build-and-push. See this run, for instance.

It looks like fixes for all of these might just involve bumping the actions version, as I don't see any calls to the deprecated APIs directly in the workflow. I may take a crack at fixing this when I get a couple hours to play around.

Message from docker/build-push-action

Is this an issue? You can see this in the log for job1, under the docker/build-push-action section.

WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /github/home/.docker/config.json.
Login Succeeded
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

I don't know where /github/home/.docker/config.json is (on GitHub? On DockerHub?, in the Docker container itself?) I assume the latter, but can't actually find it.

getting pkgdown to work; problem for pre-existing github.io?

Is there a potential issue with conflict with pre-existing github.io site?

github.com/vjcitn/intmmo is an attempt to produce a workshop site
with minimal changes to the template. i already have vjcitn.github.io and
attempt to produce vjcitn.github.io/intmmo by altering DESCRIPTION, basic_checks.yaml,
and _pkgdown.yml, and having a vignette with the name of the repo.
The actions workflow succceeds, the gh-pages branch of intmmo seems properly populated, but vjcitn.github.io/intmmo generates 404 next to some pre-existing content in my gh-pages.

I will try to learn more about pkgdown but I wonder if there is a simple explanation and
workaround.

loading data inside the workshop package

Hi,

is there a way to create a dataset and include it into the workshop material?

I tried to include an .rda file into the package data folder and recalling it with the data("datasetname") function, but this is not working.

Do you know why? Is there any idea to do something like that?

Thanks!

Use binary repositories for CRAN package installation

By default, BiocManager::repositories() on the Bioconductor docker images (now) uses the RStudio package manager for fast binary installation of CRAN packages. Does this line
https://github.com/seandavi/BuildABiocWorkshop/blob/5ba55a4f6ae3df095f89d71643e05fb2f0082f2c/.github/workflows/basic_checks.yaml#L44 defeat that behavior, replacing BiocManager's repository with the CRAN source repository? Whatever the case, CRAN packages are being installed from source...

Also, setting options(Ncpus = 8) (or however many CPUs are available) installs packages in parallel, potentially quite a bit faster (even Ncpus = 2 on a single CPU machine would speed things up, I think...)

I see that removing the options() in the Dockerfile might also be appropriate? https://github.com/seandavi/BuildABiocWorkshop/blob/5ba55a4f6ae3df095f89d71643e05fb2f0082f2c/Dockerfile#L7-L9

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.