frej / fast-export Goto Github PK

View Code? Open in Web Editor NEW

796.0 796.0 252.0 584 KB

A mercurial to git converter using git-fast-import

Home Page: http://repo.or.cz/w/fast-export.git

Python 73.43% Shell 26.35% Makefile 0.22%

fast-export's People

Contributors

Stargazers

Watchers

Forkers

danielj7 zpao clee jjk-jacky psibre scm-tools zborboa-tesla dalager ichizok smillaedler houdatahbaz rjkroege nvdnkpr mentalpower tracywebtech otac0n ritcheyer loqu8 dynaryan sunliwen swiety sojin vmurillo bg-o wrprice daolis mopenjdk cspeckmim mranganathan17 gmcgovern nickpack alficles tanvir-ahmed-m4 apcurium danny0838 jmichaelbako hanswessels linagee jimon bayangroup saadsami119 paulcampbell planestraveler tinysun212 fbucek ekabardinsky transformersprimeabcxyz dritter slachiewicz gertjan stuarthillary rudy-tmc mwin007 vedraiyani anicolaspp gitsly hausgross digideskio cdimech nicktimko imz bimki sbraz happy-ferret vad1m synisterss pengbai wdv4758h emilianbold jlahoda synaptic-cl mp-github ralic ermshiperete cumuru atykhyy tonysrc katolaz jebcat1982 hkmshb bwits jeonghanlee jonmills gabro fjmorenorios eysner chubbymaggie rockdreamer vsajip tobiasmende budmc29 alex-aralis undeadjoe ceqi-zz okobets ruslanvs tbm timmuller shackett-ttd-zz benashz

fast-export's Issues

gets confused by missing original branch commits in a clone

I had a local clone of the orginal hg repo, which I initially used for
hg-fast-export -r ../orig_hg -M master

Then I committed my change(s) in a another local hg clone of the
original clone (a separate hg repo for my changes is a way to easily
separate them in the resulting git branches).

I imported my changes with hg-fast-export -r ../my_hg -M my

I was very surprized not to see my new change on top of the master
git branch, as a single new commit of the my git branch. (This is
what I can reproduce reliably; once there was also a strange case,
when my and master git branches diverged, although master should
be the ancestor. I think it was a more complex case of the same
problem.)

How to reproduce

in a "top-level" dir

mkdir orig_hg
cd orig_hg/

in orig_hg/

hg init
echo a >f
hg add f
hg commit -m A
hg branch t
echo b >f
hg commit -m B
hg update default
echo c >f
hg commit -m C
cd ..

in the top-level dir

hg clone -b default orig_hg my_hg
cd my_hg/

in my_hg/

echo d >f
hg commit -m D
cd ..

in the top-level dir

mkdir tmp_git
cd tmp_git/

in tmp_git/

tmp_git $ git init
tmp_git $ hg-fast-export -r ../orig_hg -M master
master: Exporting full revision 1/3 with 1/0/0 added/changed/removed files
t: Exporting simple delta revision 2/3 with 0/1/0 added/changed/removed files
master: Exporting simple delta revision 3/3 with 0/1/0 added/changed/removed files
Issued 3 commands
git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects:       5000
Total objects:            9 (         0 duplicates                  )
      blobs  :            3 (         0 duplicates          0 deltas of          0 attempts)
      trees  :            3 (         0 duplicates          0 deltas of          3 attempts)
      commits:            3 (         0 duplicates          0 deltas of          0 attempts)
      tags   :            0 (         0 duplicates          0 deltas of          0 attempts)
Total branches:           2 (         2 loads     )
      marks:           1024 (         3 unique    )
      atoms:              1
Memory total:          2344 KiB
       pools:          2110 KiB
     objects:           234 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 8589934592
pack_report: pack_used_ctr            =          6
pack_report: pack_mmap_calls          =          2
pack_report: pack_open_windows        =          1 /          1
pack_report: pack_mapped              =        614 /        614
---------------------------------------------------------------------

tmp_git $ gitk --all
tmp_git $ hg-fast-export -r ../my_hg -M my
Issued 0 commands
git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects:       5000
Total objects:            0 (         0 duplicates                  )
      blobs  :            0 (         0 duplicates          0 deltas of          0 attempts)
      trees  :            0 (         0 duplicates          0 deltas of          0 attempts)
      commits:            0 (         0 duplicates          0 deltas of          0 attempts)
      tags   :            0 (         0 duplicates          0 deltas of          0 attempts)
Total branches:           0 (         0 loads     )
      marks:           1024 (         0 unique    )
      atoms:              0
Memory total:          2282 KiB
       pools:          2048 KiB
     objects:           234 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 8589934592
pack_report: pack_used_ctr            =          0
pack_report: pack_mmap_calls          =          0
pack_report: pack_open_windows        =          0 /          0
pack_report: pack_mapped              =          0 /          0
---------------------------------------------------------------------

tmp_git $ hg-fast-export -r ../my_hg -M my
Issued 0 commands
git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects:       5000
Total objects:            0 (         0 duplicates                  )
      blobs  :            0 (         0 duplicates          0 deltas of          0 attempts)
      trees  :            0 (         0 duplicates          0 deltas of          0 attempts)
      commits:            0 (         0 duplicates          0 deltas of          0 attempts)
      tags   :            0 (         0 duplicates          0 deltas of          0 attempts)
Total branches:           0 (         0 loads     )
      marks:           1024 (         0 unique    )
      atoms:              0
Memory total:          2282 KiB
       pools:          2048 KiB
     objects:           234 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 8589934592
pack_report: pack_used_ctr            =          0
pack_report: pack_mmap_calls          =          0
pack_report: pack_open_windows        =          0 /          0
pack_report: pack_mapped              =          0 /          0
---------------------------------------------------------------------

tmp_git $ gitk --all

No ne commit in my. Bad.

A workaround:

tmp_git $ rm -rfv .git/hg*
rm: removing file `.git/hg2git-heads'
rm: removing file `.git/hg2git-mapping'
rm: removing file `.git/hg2git-marks'
rm: removing file `.git/hg2git-state'
tmp_git $ hg-fast-export -r ../my_hg -M my
my: Exporting full revision 1/3 with 1/0/0 added/changed/removed files
my: Exporting simple delta revision 2/3 with 0/1/0 added/changed/removed files
my: Exporting simple delta revision 3/3 with 0/1/0 added/changed/removed files
Issued 3 commands
git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects:       5000
Total objects:            3 (         6 duplicates                  )
      blobs  :            1 (         2 duplicates          0 deltas of          0 attempts)
      trees  :            1 (         2 duplicates          0 deltas of          1 attempts)
      commits:            1 (         2 duplicates          0 deltas of          0 attempts)
      tags   :            0 (         0 duplicates          0 deltas of          0 attempts)
Total branches:           1 (         1 loads     )
      marks:           1024 (         3 unique    )
      atoms:              1
Memory total:          2344 KiB
       pools:          2110 KiB
     objects:           234 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 8589934592
pack_report: pack_used_ctr            =          2
pack_report: pack_mmap_calls          =          1
pack_report: pack_open_windows        =          1 /          1
pack_report: pack_mapped              =        238 /        238
---------------------------------------------------------------------

tmp_git $ gitk --all

The cause

The cause is probably that orig_hg had an extra commit in branch
t, totalling 3 with the last sequential number 2. And in my_hg
with the extra branch omitted there are total 3 commits with the last
one having number 2, too.

So, it is somehow unnoticed in the result of hg-fast-export.

Tags are not preserved after "git push"

I just run fast-export against my psutil repository:
https://code.google.com/p/psutil/source/checkout

fast-export.sh completes successfully and I can see all my tags exported into the new GIT repository:

giampaolo@UX32VD:/tmp/newgit$ git tags
release-0.1.0   Avoided to call distutils.core.setup() twice which caused setup.py to generate two .exe installers when running "setup.py bdist_wininst".
release-0.1.1   Get rid of MANIFEST file.
release-0.1.2   Added distutil's "keyword" argument.
...

I then pushed the new GIT repository on a remote repository:

git remote add origin [email protected]:grodola/prjname.git
git push origin mastter

Everything looks good except that if I change directory then clone the repository I just created all the tags are gone. Any idea?

Import HG bookmarks to Git branches

HG bookmarks are effectively the same thing as Git feature branches, while HG branches are a very different beast, much less useful IMHO.

Therefore it would be useful if fast-export converted HG bookmarks → Git branches, instead of failing with an error:

Error: repository has at least one unnamed head: hg r779

fatal: Missing space after > in ident string

I am getting an error importing a repository

fatal: Missing space after > in ident string:  <unspecified>P <devnull@localhost> 1404822619 +0100
fast-import: dumping crash report to .git/fast_import_crash_5724
Traceback (most recent call last):
  File "../fast-export/hg-fast-export.py", line 504, in <module>
    notes=options.notes,encoding=encoding,fn_encoding=fn_encoding))
  File "../fast-export/hg-fast-export.py", line 404, in hg2git
    sob,brmap,hgtags,encoding,fn_encoding)
  File "../fast-export/hg-fast-export.py", line 244, in export_commit
    export_file_contents(ctx,man,changed,hgtags,fn_encoding)
  File "../fast-export/hg-fast-export.py", line 141, in export_file_contents
    wr(d)
  File "../fast-export/hg-fast-export.py", line 37, in wr
    wr_no_nl(msg)
  File "../fast-export/hg-fast-export.py", line 34, in wr_no_nl
    sys.stdout.write(msg)
IOError: [Errno 32] Broken pipe

I think this might be because the committer name is <unspecified>P which has a 2nd set of <> causing the parser to configure the user name with the host name. The relevant part of the crash report is:

committer Paul Hickman <devnull@localhost> 1404740209 +0100
  data 80
  from :3015
  M 100644 inline Scripts/Migrate-107000001-Client-Paul.sql
  data 797

  commit refs/heads/master
  mark :3017
* committer <unspecified>P <devnull@localhost> 1404822619 +0100

Active Branch LRU
-----------------
    active_branches = 1 cur, 5 max

  pos  clock name
   1)   3016 refs/heads/master

Inactive Branches
-----------------
refs/heads/master:
  status      : active loaded
  tip commit  : 3854800d894b00d6f468e99e6cb5eca640a7cf31
  old tree    : 2c9298f523dbb78a984546965e79261220de00da
  cur tree    : 2c9298f523dbb78a984546965e79261220de00da
  commit clock: 3016
  last pack   : 0


Marks
-----
  exported to /home/lfs/dest/.git/hg2git-marks.tmp

-------------------
END OF CRASH REPORT

git-sh-i18n: No such file or directory

Where do we get this dependency?

Found this script: https://github.com/git/git/blob/master/git-sh-i18n.sh

If this is it is there an expected deployment step or just add it to the PATH.

there appears to be a python mercurial dependency as well. I guessed at pip install mercurial.

hg-reset.sh: hg-reset.py: error: no such option: --mapping

$ hg-reset.sh -R DOES_NOT_MATTER
Using last hg repository "/mnt/dev/hg/cmake"
Usage: hg-reset.py [options]

hg-reset.py: error: no such option: --mapping

sanitize_name methos doesn't cover all required cases.

Sanitize name is failing if branch name has less than two symbols.
Also it filtering regexp doen't cover cases if branch name has ' = and some other symbols.

My version of this method looks like:

def sanitize_name(name,what="branch"):
        """Sanitize input roughly according to git-check-ref-format(1)"""

        def dot(name):
            if len(name) == 0:
              return name
            if name[0] == '.' and len(name) == 1:
              return '_'
            if name[0] == '.':    
              return '_'+name[1:]
            return name

        n=name
        p=re.compile('([[ ~^:?\\\\*\)\(\=\,\'#]|\.\.)')
        n=n.strip()
        n=p.sub('_', n)
        if n[-1] in ('/', '.'): n=n[:-1]+'_'
        n='/'.join(map(dot,n.split('/')))
        p=re.compile('_+')        
        n=p.sub('_', n)  
        if n!=name:
            sys.stderr.write('Warning: sanitized %s [%s] to [%s]\n' % (what,name,n))
        return n

ImportError: No module named mercurial

Mercurial is installed in system.

➜  git_repo git:(master) which hg
/usr/local/bin/hg

Export script reports fatal error when hg branch name contains ampersand

Use the default options for hg-fast-export.sh i.e. no mapping files are supplied
Try to convert a Mercurial repository that contains a branch whose name contains an ampersand character
Along with any messages warning of sanitized branch names, a message reads

fatal: Needed a single revision

hg-fast-export/git-hg seems don't respect the order of a merge

In Git, usually, the first parent of a merge is the branch which other branches merged into.

I have tried cosmin's git-hg which is a wrapper around this script and abourget's git-hg-again both on mutt's hg repo, it seems that the later respects the order of a merge well, the former is a bit random. You can see from the screenshots below.

A merge history graph of mutt imported cosmin's your git-hg:

A merge history graph of mutt imported by abourget's git-hg-again:

The actuall history graph plotted by hgk on mutt's hg repository:

As you can see from the above, the second graph by abourget's git-hg-again is very close to the original hgk graph and is actually reflecting the real workflow of the mutt.

This is also posted as an answer to this question.

File "../fast-export/hg-fast-export.py", line 143, in sanitize_name

Warning: sanitized tag [uploaded 2.0.0 for linux] to [uploaded_2.0.0_for_linux]
Exporting tag [uploaded_2.0.0_for_linux] at [hg r174] [git :175]
Traceback (most recent call last):
File "../fast-export/hg-fast-export.py", line 384, in
options.statusfile,authors=a,sob=options.sob,force=options.force))
File "../fast-export/hg-fast-export.py", line 325, in hg2git
c=export_tags(ui,repo,old_marks,mapping_cache,c,authors)
File "../fast-export/hg-fast-export.py", line 219, in export_tags
tag=sanitize_name(tag,"tag")
File "../fast-export/hg-fast-export.py", line 143, in sanitize_name
if n[-1] == '/': n=n[:-1]+'_'
IndexError: string index out of range

When re-running fast-export, error 'Branch [master] modified outside hg-fast-export' is shown.

This is the command I'm using:

hg-fast-export.sh -r C:/Migration/WorkingDir -A authors.map

Here's the actual error message:

Error: Branch [master] modified outside hg-fast-export:
e7443efd712a7dcff6296b975626f471e246c875 (repo) != None (cache)

Of course, I don't even have a 'master' branch in HG, I have a 'default' branch, which is then renamed to 'master' when exported to git.

This only happens when trying to catch up a previously exported repository.

File/Folder names with international characters get mangled

When converting from mercurial to git, file and folder names containing international characters get mangled, and somehow not committed properly. A folder name like västfolket will become v%E4stfolket, and all files that are affected will show as uncommitted changes in the repo afterwards.

Tested on OSX

fatal: Invalid raw date

hg-fast-export fails on a checkout of ssh://[email protected]/illumos-gate

master: Exporting simple delta revision 7371/14114 with 0/7/0 added/changed/removed files
fatal: Invalid raw date "<devnull@localhost> 1219151827 -0600" in ident:  <Gerald Jelinek> <devnull@localhost> 1219151827 -0600
fast-import: dumping crash report to .git/fast_import_crash_1875
Traceback (most recent call last):
  File "/usr/share/hg-fast-export/hg-fast-export.py", line 387, in <module>
    options.statusfile,authors=a,sob=options.sob,force=options.force))
  File "/usr/share/hg-fast-export/hg-fast-export.py", line 321, in hg2git
    c=export_commit(ui,repo,rev,old_marks,max,c,authors,sob,brmap)
  File "/usr/share/hg-fast-export/hg-fast-export.py", line 214, in export_commit
    export_file_contents(ctx,man,changed)
  File "/usr/share/hg-fast-export/hg-fast-export.py", line 125, in export_file_contents
    wr(d)
  File "/usr/share/hg-fast-export/hg-fast-export.py", line 26, in wr
    sys.stdout.write(msg)
IOError: [Errno 32] Broken pipe

The crash dump contains this command:

  commit refs/heads/master
  mark :7371
* committer <Gerald Jelinek> <devnull@localhost> 1219151827 -0600

to docker image

Hello,
I have forked your project and add all script to a docker image https://github.com/PengBAI/fast-export. It's also nice in a dockerization way ;)
Thax and regards,
Peng

Unexpected EOF while looking for matching `

Hi,

Me and @mriska get this while trying to convert a repo:

sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file
Error: repository has at least one unnamed head: hg r3868
git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects:       5000
Total objects:            0 (         0 duplicates                  )
      blobs  :            0 (         0 duplicates          0 deltas of          0 attempts)
      trees  :            0 (         0 duplicates          0 deltas of          0 attempts)
      commits:            0 (         0 duplicates          0 deltas of          0 attempts)
      tags   :            0 (         0 duplicates          0 deltas of          0 attempts)
Total branches:           0 (         0 loads     )
      marks:           1024 (         0 unique    )
      atoms:              0
Memory total:          2282 KiB
       pools:          2048 KiB
     objects:           234 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 8589934592
pack_report: pack_used_ctr            =          0
pack_report: pack_mmap_calls          =          0
pack_report: pack_open_windows        =          0 /          0
pack_report: pack_mapped              =          0 /          0
---------------------------------------------------------------------

This is using a very plain command line, with basically hg-fast-export.sh -r <sourcefolder>. The system is Mac OS X, with Python 2.7.

Any ideas? Unfortunately, this is speaking about a private repo, so I can not provide it to you for debugging.

The source repo is of relatively modest size, around 5000 commits. I've actually managed to convert it some months back, but we couldn't move over to Git at that time because of complexity within the project. Now it's time to do the conversion but this errors makes it a bit hard... 😄

Thanks in advance!

.

AttributeError: 'httprepository' object has no attribute 'changelog'

Here is the log

 [test@vm1 temp]$../fast-export/hg-fast-export.sh  -r http://www.freediameter.net/hg/freeDiameter
Traceback (most recent call last):
  File "../fast-export/hg-fast-export.py", line 427, in <module>
    options.statusfile,authors=a,sob=options.sob,force=options.force,hgtags=options.hgtags,notes=options.notes))
  File "../fast-export/hg-fast-export.py", line 336, in hg2git
    if not verify_heads(ui,repo,heads_cache,force):
  File "../fast-export/hg-fast-export.py", line 300, in verify_heads
    branches[bn] = branchtip(repo, heads)
  File "../fast-export/hg-fast-export.py", line 292, in branchtip
    if 'close' not in repo.changelog.read(h)[5]:
AttributeError: 'httprepository' object has no attribute 'changelog'
git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects:       5000
Total objects:            0 (         0 duplicates                  )
      blobs  :            0 (         0 duplicates          0 deltas)
      trees  :            0 (         0 duplicates          0 deltas)
      commits:            0 (         0 duplicates          0 deltas)
      tags   :            0 (         0 duplicates          0 deltas)
Total branches:           0 (         0 loads     )
      marks:           1024 (         0 unique    )
      atoms:              0
Memory total:          2282 KiB
       pools:          2048 KiB
     objects:           234 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 8589934592
pack_report: pack_used_ctr            =          0
pack_report: pack_mmap_calls          =          0
pack_report: pack_open_windows        =          0 /          0
pack_report: pack_mapped              =          0 /          0
---------------------------------------------------------------------

and here are some versions

[temp@vm1 temp]$ cat /etc/redhat-release 
CentOS release 6.5 (Final)

[test@vm1 temp]$ hg --version
Mercurial Distributed SCM (version 1.4)

Copyright (C) 2005-2009 Matt Mackall <[email protected]> and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[test@vm1 temp]$ git --version
git version 1.7.1

[test@vm1 temp]$ python --version
Python 2.6.6

Create versions

If you create versions, I can make a Homebrew formula for fast-export so that more people will use it.

Support Windows Better

It seems the shell script is lightweight and just invokes the python script. It'd be nice if there was a batch script on Windows to do this, or simply move the work done by the shell script into the python script instead.

fast-export fails if python3 is sytem-wide default

If the system default python is python3 (i.e. the symlink is /usr/bin/python -> /usr/bin/python3) then fast-export fails as it is python2 only. The error is in this case rather confusingly

veelai@zaphod ~/fast-export $ python3 hg-fast-export.py 
Traceback (most recent call last):
  File "fast-export/hg-fast-export.py", line 6, in 
    from mercurial import node
ImportError: No module named mercurial

A possible solution is to replace PYTHON=${PYTHON:-python} by PYTHON=${PYTHON:-python2} in the shell scripts and '#!/usr/bin/env python' by '#!/usr/bin/env python2' in the python scripts. This should work in most environments, with one caveat: Debian as greatest distribution under the sky decided to be extravagant and does not ship a python2 symlink, but only python2.6 and similar.

Branch history out of order after conversion

Hi. I just ran fast-export on several mercurial repos and am manually reviewing the commit and branch history. I noticed that everything seems in order on the default/master branch, but other branches have their commits out of order. This is confirmed by comparing both the graph history and the date sequences. It seems to be grouping some of the commits by author name and date ordering them. Repos that don't have branching seem to be fine. I have only run the command:

$ pwd
/path to git directory/
$ hg-fast-export.sh -r

Is there another pass or command I need to run to clean-up the branch history?

Prepend the commit message with the branch name

Add an option to prepend the commit message with the hg branch name, by specifying the switch -c.

New commit message will be of the form [branch name]: [original commit message]

Error: repository has at least one unnamed head

Hello,

when trying to convert http://www.arccore.com/hg/arc to git, I get the following error:

fatal: Needed a single revision
Error: repository has at least one unnamed head: hg r2212

I expect that in git terminology it complains about "detached head". Does anybody know how to fix it?

Illegal option -o pipefail

This fails for me on Ubuntu Precise with the following error.

/tmp/fast-export/hg-fast-export.sh: 6: set: Illegal option -o pipefail

It appears to be the patch that #16 added. Reverting to before that works fine.

tfoote@stitch:/tmp/fast-export$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.2 LTS
Release:    12.04
Codename:   precise
tfoote@stitch:/tmp/fast-export$ bash --version
GNU bash, version 4.2.25(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Empty path component found in input

We used the tool for Pypy repository(https://bitbucket.org/pypy/pypy/downloads).
By using author-map, we worked out the issue "Missing < in ident string" at revesion 53977.
But there comes another error "Empy path component found in input":
fatal: Empty path component found in input
fast-import: dumping crash report to .git/fast_import_crash_25934
Traceback (most recent call last):
File "/home/yicong.hyc/fast-export/hg-fast-export.py", line 445, in
notes=options.notes,encoding=encoding))
File "/home/yicong.hyc/fast-export/hg-fast-export.py", line 367, in hg2git
c=export_commit(ui,repo,rev,old_marks,max,c,authors,sob,brmap,hgtags,notes,encoding)
File "/home/yicong.hyc/fast-export/hg-fast-export.py", line 228, in export_commit
export_file_contents(ctx,man,added,hgtags,encoding)
File "/home/yicong.hyc/fast-export/hg-fast-export.py", line 140, in export_file_contents
wr(d)
File "/home/yicong.hyc/fast-export/hg-fast-export.py", line 37, in wr
wr_no_nl(msg)
File "/home/yicong.hyc/fast-export/hg-fast-export.py", line 34, in wr_no_nl
sys.stdout.write(msg)
IOError: [Errno 32] Broken pipe

Looking at crahed report, we observed:

Most Recent Commands Before Crash

D pypy/translator/jvm/test/test_tuple.py
D pypy/translator/jvm/test/test_unicode.py
...
D pypy/translator/unsimplify.py
M 100644 inline /doc/config/translation.lldebug.txt

data 38

store hg hashes in git commit notes

After migrating hg to git repository it would be worth to have the mapping of mercurial commit hashes into git commit hashes handy. Git commit notes could be used for this.

Resolve SVN code?

It seems to me that files

svn-archive.c
svn-fast-export.c
svn-fast-export.py
Makefile

are leftovers from a previous project. If they are, maybe delete them for clean-up.
There are two related entries in .gitignore as well:

svn-archive
svn-fast-export

Best, S

hg-fast-export bash script breaks when --force is used

When invoking:
hg-fast-export --force --quiet -r /path/to/hg/repo

The script bails out saying one should provide a '-r' option, which is provided.

I suspect the "break" statement in the bash loop collecting arguments is unintentional.

repo.branchtags() unavailable in Mercurial 2.9

Hello,

After upgrading to Mercurial 2.9, I have noticed the following error (via git-hg):

AttributeError: 'localrepository' object has no attribute 'branchtags'

It appears the branchtags method was removed in this c̶o̶m̶m̶i̶t̶ patch:

http://selenic.com/pipermail/mercurial-devel/2013-November/054759.html

Thank you!

Tags are not converted correctly

Used hg-fast-export on a Bitbucket repository, converting it to Git.
All went well, except for the tags. Now, the tags are gone, and the related commit points to a non existing changeset, that only exists in the old hg repository.
("Added tag 1.4.1 for changeset 5c688cb43dc7")

Fail on old repository

I have quite large repository. When I'm exporting to git from hg I'm receiving error:

master: Exporting thorough delta revision 2804/4207 with 0/23/1 added/changed/removed files
Traceback (most recent call last):
  File "../fast-export/hg-fast-export.py", line 432, in <module>
    options.statusfile,authors=a,sob=options.sob,force=options.force,hgtags=options.hgtags,notes=options.notes))
  File "../fast-export/hg-fast-export.py", line 362, in hg2git
    c=export_commit(ui,repo,rev,old_marks,max,c,authors,sob,brmap,hgtags,notes)
  File "../fast-export/hg-fast-export.py", line 224, in export_commit
    export_file_contents(ctx,man,changed,hgtags)
  File "../fast-export/hg-fast-export.py", line 136, in export_file_contents
    wr(d)
  File "../fast-export/hg-fast-export.py", line 37, in wr
    wr_no_nl(msg)
  File "../fast-export/hg-fast-export.py", line 34, in wr_no_nl
    sys.stdout.write(msg)
IOError: [Errno 32] Broken pipe

I'm used following options:
../fast-export/hg-fast-export.sh -r reponame/ --hgtags --hg-hash --force

Converting .hg Subrepository

Hi,

is it possible, to convert subrepos from hg to git submodules? we are using a lot of subrepos right now, we need to convert them too. we would like to keep the history.

best regards

exception on windows because of linefeed

please have a look at this thread :
http://stackoverflow.com/questions/9537454/how-to-solve-hg-fast-export-error-branch-name-doesnt-conform-to-git-standards

there is also the solution:
adding
import msvcrt
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)

for windows systems solved this for me

maybe you can add this.

Dieter

Fast export doesn't seem to do anything?

C:\Program Files\Git\mingw64\libexec\git-core\git-fast-import.exe statistics:

Alloc'd objects: 5000
Total objects: 0 ( 0 duplicates )
blobs : 0 ( 0 duplicates 0 deltas of 0 attempts)
trees : 0 ( 0 duplicates 0 deltas of 0 attempts)
commits: 0 ( 0 duplicates 0 deltas of 0 attempts)
tags : 0 ( 0 duplicates 0 deltas of 0 attempts)
Total branches: 0 ( 0 loads )
marks: 1024 ( 0 unique )
atoms: 0
Memory total: 2282 KiB
pools: 2048 KiB

objects: 234 KiB

pack_report: getpagesize() = 65536
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit = 8589934592
pack_report: pack_used_ctr = 0
pack_report: pack_mmap_calls = 0
pack_report: pack_open_windows = 0 / 0

pack_report: pack_mapped = 0 / 0

My hg repo has a few branches, and over 1000 commits.

If I run git checkout HEAD afer, I get this error: error: pathspec 'HEAD' did not match any file(s) known to git.

"ImportError: No module named mercurial" on OSX

I had to set the the PYTHONPATH explicitly: export PYTHONPATH="/usr/local/lib/python2.7/site-packages:$PYTHONPATH".

I see two options for this problem:
Either documenting this in the README, or having the script figuring out the correct path. Mercurial itself does it like this:

libdir = '../lib/python2.7/site-packages/'

if libdir != '@' 'LIBDIR' '@':
    if not os.path.isabs(libdir):
        libdir = os.path.join(os.path.dirname(os.path.realpath(__file__)),
                              libdir)
        libdir = os.path.abspath(libdir)
    sys.path.insert(0, libdir)

It detects the path to the executable, and tries to find the according libpath.

git-fast-import fails with fatal: mark :137 not declared

At some point syncing/mirroring the hg-repo mentioned in #21 (comment) stopped with issue visible below. [edited to add a space between link and "stopped" to make the link work/frej]

The system uses
git version 2.7.3
and
Mercurial Distributed SCM (version 3.5.2)

I would appreciate if you could have a look at it and maybe have an idea for a solution.

Output of the sync-script:

Rufe von https://bitbucket.org/rndblnch/osx-presentation ab
Suche nach Änderungen
Keine Änderungen gefunden
Using last hg repository "../osx-presentation"
master: Exporting simple delta revision 138/146 with 0/1/0 added/changed/removed files
fatal: mark :137 not declared
fast-import: dumping crash report to .git/fast_import_crash_10250
Traceback (most recent call last):
  File "/home/git/fast-export/hg-fast-export.py", line 483, in <module>
    notes=options.notes,encoding=encoding,fn_encoding=fn_encoding))
  File "/home/git/fast-export/hg-fast-export.py", line 383, in hg2git
    sob,brmap,hgtags,encoding,fn_encoding)
  File "/home/git/fast-export/hg-fast-export.py", line 235, in export_commit
    export_file_contents(ctx,man,changed,hgtags,fn_encoding)
  File "/home/git/fast-export/hg-fast-export.py", line 141, in export_file_contents
    wr(d)
  File "/home/git/fast-export/hg-fast-export.py", line 37, in wr
    wr_no_nl(msg)
  File "/home/git/fast-export/hg-fast-export.py", line 34, in wr_no_nl
    sys.stdout.write(msg)
IOError: [Errno 32] Broken pipe
Everything up-to-date
Everything up-to-date

Contents of the mentioned crash report:

fast-import crash report:
    fast-import process: 10250
    parent process     : 10249
    at Sat Apr 2 18:58:15 2016

fatal: mark :137 not declared

Most Recent Commands Before Crash
---------------------------------
  commit refs/heads/master
  mark :138
  author Renaud Blanch <[email protected]> 1454618373 +0100
  committer Renaud Blanch <[email protected]> 1454618373 +0100
  data 27
* from :137

Active Branch LRU
-----------------
    active_branches = 0 cur, 5 max

  pos  clock name

Inactive Branches

refs/heads/master:
status : dirty
tip commit : 0000000000000000000000000000000000000000
old tree : 0000000000000000000000000000000000000000
cur tree : 0000000000000000000000000000000000000000
commit clock: 0
last pack :

Marks

exported to /home/git/mirror/osx-presentation.git/.git/hg2git-marks.tmp

END OF CRASH REPORT

fails on https://bitbucket.org/pypy/pypy with Broken pipe

Fails on normal run:

><(((º> ~/repos/fast-export/hg-fast-export.sh -r ~/repos/pypy         
Error: repository has at least one unnamed head: hg r72634

Fails with --force:

><(((º> ~/repos/fast-export/hg-fast-export.sh -r ~/repos/pypy/ --force
Traceback (most recent call last):
  File "/home/saidler/repos/fast-export/hg-fast-export.py", line 468, in <module>
  notes=options.notes,encoding=encoding))
  File "/home/saidler/repos/fast-export/hg-fast-export.py", line 377, in hg2git
  sob,brmap,hgtags,notes,encoding)
  File "/home/saidler/repos/fast-export/hg-fast-export.py", line 235, in export_commit
  export_file_contents(ctx,man,changed,hgtags,encoding)
  File "/home/saidler/repos/fast-export/hg-fast-export.py", line 141, in export_file_contents
  wr(d)
 File "/home/saidler/repos/fast-export/hg-fast-export.py", line 37, in wr
 wr_no_nl(msg)
 File "/home/saidler/repos/fast-export/hg-fast-export.py", line 34, in wr_no_nl
 sys.stdout.write(msg)
 IOError: [Errno 32] Broken pipe

Windows support?

Running hg-fast-export.py for my repository throws an error 'No --marks option given'.
People say that hg-fast-export.sh seems to be working fine, but I'm running on Windows, so no bash support.

What are Windows dependencies for script to work?

I am trying to run the script on Windows, environment is:

Git Bash 1.9
Mercurial 2.9 x64

I got error message about python not available.

installed Python 3.4 x64

Now I am getting following error:

Traceback (most recent call last):
  File "c:/server/ext/fast-export/hg-fast-export.py", line 6, in <module>
    from mercurial import node
ImportError: No module named 'mercurial'

Which dependencies exactly and in what configuration do I need to run this on Windows?

fast-export failing on wrongly-named branched exported from mercurial

I have pretty old mercurial repository where branches named with useing spaces and different symbold. Import from such repository failed due to fail of def get_git_sha1(name,type='heads') method.

To fix it I think sanitize_name method should be moved into hg2git.py and called from get_git_sha1, by I didn't know good enouth codebase som I might be wrong.

Branch name with Quotes

We are converting our hg repositories to git. We are getting an error while converting it, the reason is that our branch name contains quotations characters. The branch name is [ Protocol "74 fixed" ]. It gives the following error.

Fatal: Unable to create 'C:/Dev/projectA/.git/refs/heads/Protocol "74 fixed "-Datum.lock': Invalid argument.

Could you please tell us how can we fix this error?

emacs autosaves

Hi,
the tool you created seems awesome, however I cannot use it properly. First because of the python3 issue that is easy solvable but then because at some in time in the repository there is a commit with an Emacs autosave file, i.e., a file that contains a hash as part of the name.

I believe the script is taking the remaining part of the file name as a comment :-( and therefore fails to find its proper attributes.

I wish I knew more python myself to fix it and provide the patch but sadly I do not even know where to start.

I hope this is an easy fix for yo guys.

Thanks and keep up the good work.

Support ignoring files/directories on import

I'm not sure if this is within the scope of this project but I ran into an issue when running the script when the Mercurial repository contains a git sub-repository.

I describe what happened here, but for reference, I'll restate it here.

Basically what (I think) happened was the history for the .git directory was imported but the .git directory itself was non-existant. So when trying to purge that directory it kept giving me file doesn't exist errors.

Any idea what the best course of action would be? Does hg-fast-export support ignoring git directories? I couldn't find anything in the source (although I didn't look for more than 10 min).

Thanks!

Commit messages became unreadable (Chinese text)

I want migrate from mercurial to git, and this tool works very well indeed, except all the commit messages(which were wrote in Chinese) became unreadable.

I checked the Chinese text inside code file, they were displayed very correct, and if I make a new commit in git, it can be displayed correct , too.

so , the only error is the historical message "exported" from mercurial, and I hope this could be fixed , thanks!

ps: below is some Chinese text, if you might use it for test...
你好,谢谢!

unable to resolve reference ... : Not a directory

We used the tool for pypy repo (https://bitbucket.org/pypy/pypy).
And after solved the issue "Empty path component found in input #48", there appeared another error:

Exporting tag [release-2.4.0] at [hg r73641] [git :73642]
...
Exporting tag [release-2.6.0] at [hg r77718] [git :77719]
Issued 79019 commands
error: unable to resolve reference refs/heads/stdlib-unification/py3k: Not a directory
error: Unable to lock refs/heads/stdlib-unification/py3k

git-fast-import statistics:

Alloc'd objects: 630000
...

The exporing process completed. But besides .git directory, there were no files exported.

Ommitted '--repo' argument reported as 'repo not found' instead of a bail().

On my environment, anyways, options.repourl is being parsed as an empty string.

It seems that these lines assume that it will be None: [https://github.com/frej/fast-export/blob/master/hg-fast-export.py#L378]

Anyways, this gave confusing error messages when troubleshooting cosmin/git-hg#38

AttributeError: 'module' object has no attribute 'CalledProcessError

[ksearle@dev:~/src/scs] ~/src/fast-export/hg-fast-export.sh -r /home/ksearle/scs
Traceback (most recent call last):
  File "/home/ksearle/src/fast-export/hg-fast-export.py", line 504, in ?
    notes=options.notes,encoding=encoding,fn_encoding=fn_encoding))
  File "/home/ksearle/src/fast-export/hg-fast-export.py", line 382, in hg2git
    if not verify_heads(ui,repo,heads_cache,force,branchesmap):
  File "/home/ksearle/src/fast-export/hg-fast-export.py", line 341, in verify_heads
    sha1=get_git_sha1(sanitized_name)
  File "/home/ksearle/src/fast-export/hg2git.py", line 114, in get_git_sha1
    except subprocess.CalledProcessError:
AttributeError: 'module' object has no attribute 'CalledProcessError'
git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects:       5000
Total objects:            0 (         0 duplicates                  )
      blobs  :            0 (         0 duplicates          0 deltas of          0 attempts)
      trees  :            0 (         0 duplicates          0 deltas of          0 attempts)
      commits:            0 (         0 duplicates          0 deltas of          0 attempts)
      tags   :            0 (         0 duplicates          0 deltas of          0 attempts)
Total branches:           0 (         0 loads     )
      marks:           1024 (         0 unique    )
      atoms:              0
Memory total:          2282 KiB
       pools:          2048 KiB
     objects:           234 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit      = 8589934592
pack_report: pack_used_ctr            =          0
pack_report: pack_mmap_calls          =          0
pack_report: pack_open_windows        =          0 /          0
pack_report: pack_mapped              =          0 /          0
---------------------------------------------------------------------

[ksearle@dev:~/src/scs] cat /etc/redhat-release
CentOS release 5.8 (Final)

[ksearle@dev:~/src/scs] hg --version
Mercurial Distributed SCM (version 2.2.2)

[ksearle@dev:~/src/scs] git --version
git version 1.8.2.1

[ksearle@dev:~/src/scs] python -V
Python 2.4.3

Conversion crashes due to empty commiter

Hi,

When I'm running a conversion, it fails due to a empty commiter on a (closed) branch.
Is there a way to ignore this/map this on a different user with (for example) an authors file?

no checks in ident string

I'm currently trying to convert a large mercurial repo to git using fast-export. After running it for some time, fast-export crashes with:

fatal: Missing > in ident string: test user <[email protected] 123456778 +1200
with the following traceback:
Traceback (most recent call last):
File "/home/cc/projs/fast-export/hg-fast-export.py", line 442, in
notes=options.notes,encoding=encoding))
File "/home/cc/projs/fast-export/hg-fast-export.py", line 364, in hg2git
c=export_commit(ui,repo,rev,old_marks,max,c,authors,sob,brmap,hgtags,notes,encoding)
File "/home/cc/projs/fast-export/hg-fast-export.py", line 226, in export_commit
export_file_contents(ctx,man,changed,hgtags,encoding)
File "/home/cc/projs/fast-export/hg-fast-export.py", line 140, in export_file_contents
wr(d)
File "/home/cc/projs/fast-export/hg-fast-export.py", line 37, in wr
wr_no_nl(msg)
File "/home/cc/projs/fast-export/hg-fast-export.py", line 34, in wr_no_nl
sys.stdout.write(msg)
IOError: [Errno 32] Broken pipe

frej / fast-export Goto Github PK

fast-export's People

Contributors

Stargazers

Watchers

Forkers

fast-export's Issues

How to reproduce

in a "top-level" dir

in orig_hg/

in the top-level dir

in my_hg/

in the top-level dir

in tmp_git/

A workaround:

The cause

Most Recent Commands Before Crash

C:\Program Files\Git\mingw64\libexec\git-core\git-fast-import.exe statistics:

objects: 234 KiB

pack_report: pack_mapped = 0 / 0

Inactive Branches

Marks

git-fast-import statistics:

Recommend Projects

Recommend Topics

Recommend Org