Files already exist in <location> when migrating a repo. about copybara HOT 12 OPEN

No problem at all :). We should improve the documentation.

Let me write here a short example for now:

Lets say we want to move 'public' folder from git.origin but we don't want to export 'public/SECRET.txt'

origin_files would be:

origin_files = glob(["public/**"], exclude = ["public/SECRET.txt"])

Note that we exclude public/SECRET.txt and not just SECRET.txt. This is because the include part can contain multiple folders.

Now lets say we want to move everything to the root of the destination:

transformations = [
core.move("public", "")
]

Now lets say we have in the destination a README.txt file that we don't want to delete:

destination_files= glob(["**"], exclude = ["README.txt"])

Now lets say that instead we want to keep it in the 'public' folder in the destination. You would not include any transformations (You can skip transformations = ...) and instead you would write the following destinations_files:

destination_files= glob(["public/**"], exclude = ["public/README.txt"])

Or if you don't have a README.txt inside public:

destination_files= glob(["**"])

from copybara.

The problem is that you are using a ssh key with password and Copybara doesn't have the ability (For now) to ask for a password in the middle of the process. Currently we only support either using .netrc with your password/token (works with 2FA) or ssh keys without password.

The only think I can think of as a workaround is to checkout from your remote in a wrapper script and then use copybara with the local repo. If you are using cached credentials for git, maybe you could do something like:
#!bin/bash
git ls-remote internal_repo
copybara ...

To force the first git invocation to ask for the credentials if needed.

from copybara.

Hmmm, lemme see what I can around this. I figured it might be something like that.

from copybara.

This did not resolve the issue, I switched to ssh key without password to try it out and got this:

770 jeffryhesse:IQreport (master *)$ copybara copy.bara.sky --force
Copybara source mover (Version: Unknown version)
Task: Git Origin: Initializing local repo
WARN: Cannot find last imported revision, but proceeding because of --force flag
Task: Moving 
ERROR: Files already exist in /Users/jeffryhesse/copybara/temp/workdir5707807481329282041/checkout: 

from copybara.

K,

The second part of the error:

ERROR: Files already exist in /Users/jeffryhesse/copybara/temp/workdir5707807481329282041/checkout:

is expected.

This move doesn't make sense:

core.move("", ""),

The reason for the failure is that we have a check that doesn't allow to move things like:
foo/bar/x
foo/baz/z

core.move("foo","foo/bar")

since it is normally a mistake.

from copybara.

Interesting. There isn't a ton of docs around this stuff so sorry for the question spamming via issues. What does the move become then? Is it source to destination? If you want to move things basically identically, what's the way to approach that? Basically I'm trying to move a specific folder from an internal repo into the root of a new repo.

from copybara.

Once I get all this working I'm going to submit a PR with some docs for you all.

from copybara.

That would be great. Thanks!!

from copybara.

That worked! Now I need to figure out how to maintain the previous author (one of the big reasons I'm using Copybara).

from copybara.

This all worked perfectly. I'm going to open a PR with some of this stuff for examples, etc... and then you can close this issue if you'd like.

from copybara.

Thanks!!

from copybara.

I came here by googling for that error message. My solution was to use overwrite=True. Just in case someone else comes here and is as impatient as me when reading the move reference:

transformations = [
    core.move("README-public.md", "README.md", overwrite=True)
],

from copybara.