Forked repo isn't used on EC2 instances even though it is declared at deployment about tibanna HOT 3 CLOSED

I had to adapt a little the Dockerfile to generate my container but otherwise it worked like a charm. Thank you again for your help!

(what I had to change on the Dockerfile, in case someone need to do the same:

@ -38,7 +38,7 @@ RUN apt update -y && apt upgrade -y &&  apt install -y \
     nodejs
 
 RUN ln -s /usr/bin/python3.8 /usr/bin/python
-RUN ln -s /usr/bin/pip3 /usr/bin/pip
 
 WORKDIR /usr/local/bin
 
@@ -56,7 +56,7 @@ RUN wget https://golang.org/dl/go1.15.3.linux-amd64.tar.gz && \
     rm go1.15.3.linux-amd64.tar.gz
 RUN export SINGULARITY_VERSION=3.3.0 && \
     export PATH=/usr/local/bin/go/bin/:$PATH && \
-    wget https://github.com/sylabs/singularity/releases/download/v${SINGULARITY_VERSION}/singularity-${SINGULARITY_VERSION}.tar.gz && \
+    wget https://github.com/hpcng/singularity/releases/download/v${SINGULARITY_VERSION}/singularity-${SINGULARITY_VERSION}.tar.gz && \
     tar -xzf singularity-${SINGULARITY_VERSION}.tar.gz && \
     rm singularity-${SINGULARITY_VERSION}.tar.gz && \
     cd singularity && \
@@ -92,7 +92,9 @@ COPY run.sh .
 COPY cron.sh .
 RUN chmod +x run.sh cron.sh
 ARG version
-RUN pip install tibanna==$version
+RUN git clone --single-branch --branch mybranch https://github.com/my/repo tibanna-git
+RUN pip install -e ./tibanna-git

)

Cheers,
−Nils

from tibanna.

Hi @nigiord, thank you for using Tibanna. The repo name and branch name are used to retrieve the awsf3/aws_run_workflow_generic.sh code which delegates most of the tasks to the tibanna-awsf docker image. Before 1.0.0, they were used to retrieve the contents from the awsf/ directory. Now most of them are in the docker image. If you don't have to change awsf3/aws_run_workflow_generic.sh, you can still use the original repo and branch name and just build your own docker image and specify it in the config (or you can change the default image name in your forked version). Alternatively, you can revert to <1.0.0 to use the awsf/ folder in your own repo and branch. I hope this helps.

Soo

from tibanna.

Hi @SooLee and thank you for your help,

If you don't have to change awsf3/aws_run_workflow_generic.sh, you can still use the original repo and branch name and just build your own docker image and specify it in the config (or you can change the default image name in your forked version).

I see, so the idea is to make my changes in any file of awsf3/ as needed, build a new docker image, upload it to dockerhub and tell Tibanna to use it instead of the default. Is that correct?

I looked at the documentation but I didn't see immediately where I can define the awsf image when deploying a unicorn or submitting a job. I looked a little at the current code though and found that the awsf_image config argument exists but is not documented: https://github.com/4dn-dcic/tibanna/blob/master/tibanna/ec2_utils.py#L306-L307

So basically, if I use tibanna through snakemake, I guess I'll just have to use

snakemake --tibanna --tibanna-config awsf_image=myrepo/myimage:myversion

in order to use my modified code. I'll try to test that, let me know if you think I'm on the wrong path.

−Nils

from tibanna.