drillster / drone-rsync Goto Github PK

I would like to create some directory structure and upload artifacts there, but rsync can't do that and i forced to use appleboy/drone-ssh plugin earlier in pipeline just to run mkdir -p. Thats ugly.
Would be great if there were a way to execute commands before rsync upload.

like this command:

script:
      - sed -i "s|'hostname'       => .*|'hostname'       => 'mysql' ，|g" config.php

in this scrip the sed command will separate two commands by ，

sed -i "s|'hostname' => .*|'hostname' => 'mysql' && |g" config.php

how to solve it?

The -r option doesn't do anything because you're using -a which implies -r.

From the rsync man page:

-a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)

This means you can either remove the recursive option (I don't think it would be used anyway), or you'll need to use -lptgoD instead of -a.

I would like to ask for some detailed help in using drone.rsync.

I think I already read almost all posts and documentation regarding its use (including #3 and #4), but I am not able to solve my problem.

I am using drone 0:7 (server and agent), running on a dedicated drone server, accessing a target server ("a_server.a_domain.com").

Here are some lines of my pipeline (I didn't list the build part, that works fine):

deploy:
image: drillster/drone-rsync
hosts: [ "a_server.a_domain.com" ]
key: ${RSYNC_KEY}
user: ubuntu
port: 22
source: a/source
target: /a/target
exclude: [ ".git"]
delete: true
recursive: true

When committing a change to the bitbucket repository to where the drone server is integrated, at the deploy stage I get the following error: "No private key specified!"

Before committing the change, I added the secret keys with drone CLI, in the following way:

drone secret add path_to/repository rsync_key @/path/to/.ssh/id_rsa --image drillster/drone-rsync

I also run:
drone secret add path_to/repository RSYNC_KEY @/path/to/.ssh/id_rsa --image drillster/drone-rsync
in case the secret is referenced with capital letters.

I changed key: ${RSYNC_KEY} to [RSYNC_KEY] and [ RSYNC_KEY ] without any luck, because I get the following error:

rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.2]

What am I doing wrong?

Of course I established an SSH connection with a private key before doing the above (SSH access from the drone server to the target server works fine, even I set it to be sudo without prompting a password).

Any help is appreciated.

Can we have support for multiple source directories ?
Something like:

source:
 - libs/
 - bin/

Hi,
I am experiencing an issue similar to #18 .

This is my .drone.yml file:

kind: pipeline
type: docker
name: default

steps:
  - name: deploy
    image: drillster/drone-rsync
    environment:
      RSYNC_KEY:
        from_secret: ssh_key
      RSYNC_USER:
        from_secret: user
    settings:
      hosts: 
        - test.example.com
      source: ./
      target: ~/myfolder
      script:
        - touch test

This is the log of the build job:

latest: Pulling from drillster/drone-rsync
Digest: sha256:<hash>
Status: Image is up to date for drillster/drone-rsync:latest
Port not specified, using default port 22!
$ rsync -az  -e 'ssh -p 22 -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o StrictHostKeyChecking=no' ./ ******@test.example.com:~/myfolder ...
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.3]

Any idea on how to debug this?

I'm having issues utilizing the script area and using an environment variable within. I'm trying to use my TARGET variable which I have set and using for this plugin. I would also like to use it in a script after the rsync is complete.

image: drillster/drone-rsync
    environment:
      RSYNC_KEY:
        from_secret: key
      RSYNC_USER:
        from_secret: user
      TARGET:
        from_secret: target
    settings:
      hosts:
        from_secret: host
      source: artifact.tar.gz
      target:
        from_secret: target
      script:
        - cd $${TARGET}
        - ls -al

This does not change directories, but simply runs ls -al at the root. Am I using the variable incorrectly?

Hi,
when using latest ArchLinux together with the docker image for deploying something I get

Status: Image is up to date for drillster/drone-rsync:latest
$ rsync -az  -r --del -e 'ssh -p 2222 -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o StrictHostKeyChecking=no' ./_site/ user@host:~/htdocs/website/ ...
rsync: This rsync lacks old-style --compress due to its external zlib.  Try -zz.
rsync error: syntax or usage error (code 1) at main.c(1578) [server=3.1.3]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.3]

This seems to be related to ArchLinux using a newer and patched version of zlib and not the bundled one from rsync. Could you maybe think about updating the base of alpine and see if the other rsync package there fixes this compatibility error? See https://www.archlinux.org/news/rsync-compatibility/

I have the following deploy step:

- name: deploy-website
  image: drillster/drone-rsync
  settings:
    hosts:
      from_secret: plugin_hosts
    port: "2223"
    key:
      from_secret: rsync_key
    user:
      from_secret: rsync_user
    source: ./dst/*
    target:
      from_secret: plugin_target
    recursive: true
    delete: true

But when the step is run i get the errors:

rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.3] 

I tried running the command printed at the first line in hand, and it all works. It's only a problem when i try to run it from drone.

I have trawled through the discourse, and github issues and tried the proposed solutions. Nothing works.

This plugin is great, it works well with ssh key.
Will plan to support the option --password-file ?

Hello,

I'm using the last Drone version and i saw that you don't have any doc on that format.
Could you update it ?

Thx

Please release a new Version on Docker Hub. The last one is two years old! Thanks

drone log

latest: Pulling from drillster/drone-rsync
--
2 | Digest: sha256:7a784ca34d783affb8cc6d9c7f8a51066fcff4b6db50d490645fb82db23e3afd
3 | Status: Downloaded newer image for drillster/drone-rsync:latest
4 | $ rsync -az  -e 'ssh -p 2222 -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o StrictHostKeyChecking=no' --include=yuezi.tar.gz ./ [email protected]:/var/www/yuezi ...
5 | bash: rsync: command not found
6 | rsync: connection unexpectedly closed (0 bytes received so far) [sender]
7 | rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.3]

2020-04-29 17:33:27 UTC+8(China)

As the subject says, sometimes rsync is used to transfer files from one local directory to another, without having to go remote.

An example use case is from static site generators. You can just copy over your tree from public/ or whatever the built site is generated into to another local folder (which is actually a host folder mounted in the image).

There's no need to go through different hosts in this case, and most docker images used by drone don't have rsync installed.

Hello there,

It's written in the docs that RSYNC_USER secret can be used instead of specifiying the key in the .drone.yml file.

The following secret values can be set to configure the plugin:

RSYNC_USER - corresponds to user
RSYNC_KEY - corresponds to key

But when I do that and run drone exec.

I get the following error.

No private key specified!

Hmmm, these variables are confusing :/

This is with a drone build from master a few hours ago:

I add a id_rsa.pub file as you do in the docs.
I use that in my .drone.yml as (any of these):

key: ${KEY}
key: '${KEY}'
key: "${KEY}"

All of them generate this rsync command:

Port not specified, using default port 22!
$ rsync -az  -e 'ssh -p 22 -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o StrictHostKeyChecking=no' /my/build.tgz [email protected]:build/build.tgz ...
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]

And this from the server:

   Mar 30 22:07:23 machine sshd[6388]: Received disconnect from 111.222.111.222: 11:  [preauth]

Any idea how to add a private key in the latest drone? Thanks!

(I added this to a closed issue previously.)

Hello,

Again I need some help using the drillster/drone-rsync plugin.

After copying files during the deploy phase, I need to run the following shell command:

rsync -avs --delete --ignore-times --filter='-p storage/*/*' /path/to/source/ /path/to/target

If I run it from the command line of the target server, it works.

If I add it to the pipeline under script: it fails with the following error message:

unexpected end of filter rule: -p

I tried many combinations, with single and double quotes and back ticks without any success. I think that the problem lies in the filter itself having so many slashes...

Any help is appreciated!

Hi all my builds stop working like an hour ago due its get connection denied public/private key fail, I think it was related to the recently merged pull request #20

this is the log i get before the crash:

Port not specified, using default port 22!
--
2 | $ rsync -az  -r --del -e 'ssh -i /root/.ssh/id -p 22 -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o StrictHostKeyChecking=no' --exclude=.certificates --exclude=.git --exclude=.env --exclude=storage --exclude=traefik/logs --exclude=node_modules ./ *****@*********.cp,:/srv/***** ...
3 | rsync: connection unexpectedly closed (0 bytes received so far) [sender]
4 | rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.3]

Hi,
I've installed Drone on arm (Raspberry Pi) machine and can't use this plugin image. The log output shows

latest: Pulling from drillster/drone-rsync
Digest: sha256:7a784ca34d783affb8cc6d9c7f8a51066fcff4b6db50d490645fb82db23e3afd
Status: Downloaded newer image for drillster/drone-rsync:latest
standard_init_linux.go:211: exec user process caused "exec format error"

and fails.

Pleasecreate arm & arm64 targets images since Docerfie doesn't contain any magic supported only by amd64 platform.

In the secrets config section for later drone versions (>= 0.6), there is a typo in the specification of the private key file, id_rsa.pub references a public key.

I am using drone-rsync plugin to sync built files to multiple hosts, it works like a charm before we changed the ssh service port on a node to a custom port number.

I know we can specify a port number in config, but can I set ports for each node ?

something like:
10.10.10.1:22
10.10.10.2:32
10.10.10.3:22

Hello,
I couldn't find any info on that, but I'm curious.
Is there a way to transfer a file from remote to local using this plugin?