iv-mexx / gitlab-migrator Goto Github PK

View Code? Open in Web Editor NEW

10.0 10.0 7.0 60 KB

Tool to migrate private gitlab projects

License: MIT License

Ruby 100.00%

fastlane gitlab-instance gitlab-migrator migration

gitlab-migrator's People

Contributors

Stargazers

Watchers

Forkers

chraibi msutter respinoza raiman264 anandlahoti pascalkern leandropissurno

gitlab-migrator's Issues

GitlabCreateProjectAction: Mapping users

In this line

users from source who are found in destination are mapped.

But why is this necessary here? User x wants to merge its own project from source to destination. So why to we need to map user y which has nothing to do with that project?

This Code is amazing! I could migrate my repositories from Gitlab-Server1 to Gitlab-Server2. However, the wiki-pages are not imported. I know these are yet other Git-repos, but would be nice to migrate them too.

Migrating to Gitlab.com

Hi,
just wanted to ask if you already have experience with migrating to Gitlab.com.

I just tried to migrate a dummy project. It seems that reading from Gitlab.com is ok, however creating projects hangs.

After some debugging I found that this LINE blocks

users_dst = gitlab_dst.users.auto_paginate

Makes sense somehow. Gitlab.com has >100K users. ^^

Therefore, looping over users and groups should be avoided.

Managed to migrate one project, but had to make API calls with curl to get e.g. group id on Gitlab.com and Users id, and do the user mapping manually.

Not so nice. Any thoughts?

env

Could you please explain the difference between

FL_GITLAB_ENDPOINT_SRC

and

FL_GITLAB_TOKEN_SRC

in the file env?

User contribution Summary

I was just wondering if you already found a way to setup the contribution events for users correctly after merging the projects.

I understand that the activity of a user starts with the minute the project is migrated to the new server, however I could not find in the API documentation
any hints how to modify the activity calendar.

Any thoughts? Maybe when mapping the users copy also their contribution events too?

Issues are not migrated...

At the moment issues are not migrated (which is the point for me to use gitlab-migrator).

Since this is a feature that is advertised, I wonder what I did wrong...

Difference between namespaces and groups

While preparing for the D-day, I found the following problem:

in src_server I have user joe
in dst_server I have user joe1

These are the same user. I can make sure that is the case with the variable user_mapping

Now we want to migrate joe/foo to new server. However, ensure_group() will tell us that the group joe does not exist and will create a group joe.

The fact that the group does not exist may be correct, however I think here the namespace is of kind "user" and not "group".

Therefore, the case

group = gitlab_dst.namespaces.auto_paginate.select { |g| g.id == dst_owner_id and g.kind == 'user'}.first

with

owner_id = original_project.owner.id
dst_owner_id = user_mapping[owner_id]

should be inserted in the else-case.

This worked fine for me, at least.

Migrating Snippets

Got the following error while migrating a project with snippets

/usr/local/lib/ruby/gems/2.3.0/gems/gitlab-3.7.0/lib/gitlab/request.rb:80:in `validate': [!] Server responded with code 400, 
message: 400 (Bad request) "file_name" not given. Request URI: https://gitlab.version.fz-juelich.de/api/v3/projects/251/snip$
ets (Gitlab::Error::BadRequest)
        from /usr/local/lib/ruby/gems/2.3.0/gems/gitlab-3.7.0/lib/gitlab/request.rb:49:in `post'
        from /usr/local/lib/ruby/gems/2.3.0/gems/gitlab-3.7.0/lib/gitlab/client/snippets.rb:44:in `create_snippet'
        from /Users/chraibi/Workspace/gitlab-migrator/fastlane/actions/gitlab_create_project.rb:121:in `block in migrate_sni$
pets'
        from /Users/chraibi/Workspace/gitlab-migrator/fastlane/actions/gitlab_create_project.rb:119:in `each'
        from /Users/chraibi/Workspace/gitlab-migrator/fastlane/actions/gitlab_create_project.rb:119:in `migrate_snippets'
        from /Users/chraibi/Workspace/gitlab-migrator/fastlane/actions/gitlab_create_project.rb:54:in `run'
        from /usr/local/lib/ruby/gems/2.3.0/gems/fastlane-2.0.3/fastlane/lib/fastlane/runner.rb:225:in `block (2 levels) ine$ecute_action'

the API suggests the existence of the attribute file_name.

According to the documentation when API-creating snippets file_name is required, however
snippets (created using the web-interface for instance) may have no file_name.

gitlab (LoadError)

I want to migrate a Gitlab-server to another one. Both running on different Linux-machine.

From my laptop (OS X Yosemite 10.10.5) I try to use gitlab-migrator

I edited the file env as stated in the readme.

Running

fastlane list_projects

fails with the following errors:

/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:126:in `require': cannot load such file -- gitlab (LoadError)
        from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:126:in `require'
        from /Users/chraibi/gitlab-migrator/fastlane/actions/gitlab_create_project.rb:6:in `<module:Actions>'
        from /Users/chraibi/gitlab-migrator/fastlane/actions/gitlab_create_project.rb:2:in `<module:Fastlane>'
        from /Users/chraibi/gitlab-migrator/fastlane/actions/gitlab_create_project.rb:1:in `<top (required)>'
        from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:126:in `require'
        from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:126:in `require'
        from /Library/Ruby/Gems/2.0.0/gems/fastlane-1.48.0/lib/fastlane/actions/actions_helper.rb:83:in `block in load_external_actions'
        from /Library/Ruby/Gems/2.0.0/gems/fastlane-1.48.0/lib/fastlane/actions/actions_helper.rb:82:in `each'
        from /Library/Ruby/Gems/2.0.0/gems/fastlane-1.48.0/lib/fastlane/actions/actions_helper.rb:82:in `load_external_actions'
        from /Library/Ruby/Gems/2.0.0/gems/fastlane-1.48.0/lib/fastlane.rb:29:in `<module:Fastlane>'
        from /Library/Ruby/Gems/2.0.0/gems/fastlane-1.48.0/lib/fastlane.rb:20:in `<top (required)>'
        from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:73:in `require'
        from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:73:in `require'
        from /Library/Ruby/Gems/2.0.0/gems/fastlane-1.48.0/bin/fastlane:6:in `<top (required)>'
        from /usr/bin/fastlane:23:in `load'
        from /usr/bin/fastlane:23:in `<main>'

My environment:

ruby -v; bundle -v; gem -v

ruby 2.0.0p481 (2014-05-08 revision 45883) [universal.x86_64-darwin14]
Bundler version 1.11.2
2.0.14

Before starting I run bundle install
Here is the output I get:

Rubygems 2.0.14 is not threadsafe, so your gems will be installed one at a time. Upgrade to Rubygems 2.1.0 or higher to enable parallel gem installation.
Using i18n 0.7.0
Using json 1.8.3
Using minitest 5.8.1
Using thread_safe 0.3.5
Using addressable 2.3.8
Using extlib 0.9.16
Using multi_json 1.11.2
Using babosa 1.0.2
Using colored 1.2
Using highline 1.7.8
Using security 0.1.3
Using excon 0.45.4
Using plist 3.1.0
Using rubyzip 1.1.7
Using multipart-post 2.0.0
Using terminal-table 1.4.5
Using multi_xml 0.5.5
Using coderay 1.1.0
Using method_source 0.8.2
Using slop 3.6.0
Using claide 0.9.1
Using unf_ext 0.0.7.1
Using dotenv 2.0.2
Using mini_magick 4.0.4
Using rouge 1.10.1
Using net-ssh 3.0.1
Using pbxplorer 1.0.0
Using mime-types 2.6.2
Using netrc 0.7.8
Using slack-notifier 1.4.0
Using jwt 1.5.2
Using little-plugger 1.1.4
Using memoist 0.12.0
Using retriable 1.4.1
Using terminal-notifier 1.6.3
Using bundler 1.11.2
Using tzinfo 1.2.2
Using fastimage 1.6.8
Using launchy 2.4.3
Using autoparse 0.3.3
Using commander 4.3.5
Using credentials_manager 0.10.0
Using faraday 0.9.2
Using httparty 0.13.4
Using pry 0.10.3
Using unf 0.1.4
Using xcpretty 0.2.1
Using net-sftp 2.1.2
Using logging 2.0.0
Using activesupport 4.2.4
Using sentry-raven 0.15.2
Using faraday_middleware 0.10.0
Using signet 0.6.1
Using gitlab 3.4.0 from git://github.com/NARKOZ/gitlab.git (at master@ef41dfa)
Using domain_name 0.5.25
Using xcpretty-travis-formatter 0.0.4
Using xcodeproj 0.28.2
Using fastlane_core 0.26.3
Using spaceship 0.14.2
Using krausefx-shenzhen 0.14.6
Using googleauth 0.4.2
Using http-cookie 1.0.2
Using gym 1.1.5
Using scan 0.3.1
Using snapshot 1.1.1
Using cert 1.2.3
Using deliver 1.5.1
Using pem 1.0.1
Using pilot 1.0.1
Using produce 1.0.0
Using sigh 1.1.3
Using xcode-install 1.0.1
Using google-api-client 0.8.6
Using rest-client 1.8.0
Using frameit 2.3.0
Using supply 0.2.1
Using fastlane 1.40.0
Bundle complete! 2 Gemfile dependencies, 77 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.

Any idea what went wrong here? In gitlab_create_project.rb:L6 there is a
require 'gitlab', But gitlab is already installed. See:

Using gitlab 3.4.0 from git://github.com/NARKOZ/gitlab.git (at master@ef41dfa)

Edit:
I solved the problem by explicitly installing gitlab:

sudo gem install gitlab

I thought this is already done by bundle install, is it not?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.