swingbit / jira2gitlab Goto Github PK
View Code? Open in Web Editor NEWMigrate Jira projects into a Gitlab instance
License: MIT License
Migrate Jira projects into a Gitlab instance
License: MIT License
I'm trying to import some issues of some projects into GitLab using your tool, because the default implementation of GitLab itself doesn't import comments etc. and seems pretty useless this way. The important thing to note is that I don't have JIRA on-prem, but in the Atlassian Cloud. When trying to read users I get the following error message:
root@potsdam:/opt/jira2gitlab# python jira-user-list.py
Get participants of GOERLITZ
[INFO] Loading Jira issues from project GOERLITZ ... 10
[INFO] #1/10 Looking at Jira issue GOERLITZ-1 ... Traceback (most recent call last):
File "/opt/jira2gitlab/jira-user-list.py", line 78, in <module>
project_users(jira_project)
File "/opt/jira2gitlab/jira-user-list.py", line 67, in project_users
jira_users.add(issue['fields']['assignee']['name'])
KeyError: 'name'
Invoking the query in the browser works and provides the following object for one of the issues:
"assignee": {
"self": "https://[...]/rest/api/2/user?accountId=60c[...]ac5",
"accountId": "60c[...]ac5",
"avatarUrls": {
"48x48": "https://[...]FMM-1.png",
"24x24": "https://[...]FMM-1.png",
"16x16": "https://[...]FMM-1.png",
"32x32": "https://[...]FMM-1.png"
},
"displayName": "Marek Maćkowiak",
"active": true,
"timeZone": "Europe/Berlin",
"accountType": "atlassian"
},
The objects for author
, reporter
etc. look the same. So is your tool simply not compatible with that version of the API or Atlassian Cloud in general or ...? What should name
contain, some username different from displayName
and accountId
?
I didn't find anything like such a username in the response. Though, the import dialog of GitLab seems to show displayName
only as well and provides a mapping based on that to usernames in GitLab.
With a large number of labels being created, clarity is quickly lost. It would be helpful if the label colors could be customized. For example, one color can be defined for all priorities-labels or all components-labels. This would increase the clarity - especially for cases where you work without a prefix.
Hello! I am hoping to use your code to import our old JIRA iissues into GitLab. There are a few of these out there, but yours feels like it's the most mature and feature-rich one out there.
I have the project in gitlab set up, the access token set up- with "project owner" rights, and "admin" rights within gitlab.
But something is still preventing the script from actually creating the issue(s). Wondering if you have an idea and can assist?
This is how far I'm getting (below). It's reading all the jira issues, and even finding and mapping the admin rights of the two users found in the first issue I have in JIRA. But when it actually goes to CREATE the issue, I am getting a weird access denied issue:
[root@butler jira2gitlab-main]# ./jira2gitlab.py
Migrating ITS to kent-test-group/it-support
[INFO] Loading Jira issues from project ITS ... 2311 <==== successfully reads all the jira issues
[INFO] #1/2311 Migrating Jira issue ITS-1 ... data: {'created_at': '2007-09-20T10:58:15.000-0500', 'assignee_ids': ['btaylor'], 'title': '[ITS-1] Directory creation request for rgddev and rgd servers', 'description': 'We need a directory to publish web sites for labs.rgd.mcw.edu and labsdev.rgd.mcw.edu . May I suggest the creation of the following directories on rgddev.mcw.edu and rgd.mcw.edu machines for publishing these sites to: \n /usr/local/apache2/labs_htdocs \nowned by the user apache. \nthanks \nGeorge\n\n___\n\nImported from Jira issue ITS-1\n\n', 'milestone_id': None, 'labels': 'jira-import, T::task, P::trivial, unix request'} ...
Traceback (most recent call last):
File "./jira2gitlab.py", line 569, in migrate_project
gl_issue.raise_for_status()
File "/usr/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://gitlab.ctsi.mcw.edu/api/v4/projects/10/issues
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./jira2gitlab.py", line 882, in
migrate_project(jira_project, gitlab_project)
File "./jira2gitlab.py", line 572, in migrate_project
raise Exception(f"Unable to create Gitlab issue for Jira issue {issue['key']}\n{e}")
Exception: Unable to create Gitlab issue for Jira issue ITS-1
403 Client Error: Forbidden for url: https://gitlab.ctsi.mcw.edu/api/v4/projects/10/issues
Migration failed
Resetting user privileges..
Hi!
Many thanks for this project. In order to use the project in a legally compliant manner, a licence valid for the entire project is required. Could you please add a license? If ok for you, an open license like MIT or Apache License 2.0 would be helpful.
Best regards, Mirjan
First of all, thank you for the script. We just migrated 1900 issues from Jira to GitLab.
That said, we had to make some adjustments including bug fixes to it:
# increase the number of retry connections
requests.adapters.DEFAULT_RETRIES = 10
# close redundant connections
# requests uses the urllib3 library, the default http connection is keep-alive, requests set False to close.
s = requests.session()
s.keep_alive = False
headers={'PRIVATE-TOKEN': GITLAB_TOKEN,'Sudo': resolve_login(author, "unknown", "file_info")},
Note: only add weight to data dict if
weight = None
...
# storypoints / weight
if JIRA_STORY_POINTS_FIELD in issue['fields'] and issue['fields'][JIRA_STORY_POINTS_FIELD]:
# print(f"weight: {issue['fields'][JIRA_STORY_POINTS_FIELD]}")
weight = int(issue['fields'][JIRA_STORY_POINTS_FIELD])
...
if weight is not None:
data['weight'] = weight
title = f"{issue['fields']['summary']}"
original_title = ""
if (len(title) > 255):
# add full original title as a comment later on
original_title = f"Full original title: {title}\n\n"
title = title[:252] + '...'
data={
'created_at': issue['fields']['created'],
'assignee_ids': [gl_assignee],
'title': title,
'description': original_title + gl_description,
'milestone_id': milestone_id,
'labels': ", ".join(labels),
}
if weight is not None:
data['weight'] = weight
gl_issue = requests.post(
f"{GITLAB_API}/projects/{gitlab_project_id}/issues",
headers={'PRIVATE-TOKEN': GITLAB_TOKEN,'Sudo': resolve_login(reporter, issue['key'], "reporter")},
verify=VERIFY_SSL_CERTIFICATE,
data=data
)
gl_issue.raise_for_status()
# Close "done" issues
# status-category can only be "new" (To Do) / "indeterminate" (In Progress) / "done" (Done) / "undefined" (Undefined)
if issue['fields']['status']['statusCategory']['key'] == "done":
status = requests.put(
f"{GITLAB_API}/projects/{gitlab_project_id}/issues/{gl_issue['iid']}",
headers={'PRIVATE-TOKEN': GITLAB_TOKEN,'Sudo': resolve_login(author, issue['key'], "sudo_close")},
verify=VERIFY_SSL_CERTIFICATE,
data={
'state_event': 'close',
'updated_at': issue['fields']['resolutiondate'],
}
)
status.raise_for_status()
# Only "outward" links were collected.
# I.e. we only need to process (a blocks b), as (b blocked by a) comes implicitly.
if j_type in ['relates to', 'blocks', 'causes']:
# Gitlab free only support "relates_to" links
gl_type = 'relates_to'
if GITLAB_PREMIUM and j_type in ['relates to', 'blocks']:
gl_type = j_type.replace(' ', '_')
try:
gl_link = requests.post(
f"{GITLAB_API}/projects/{gl_from['project_id']}/issues/{gl_from['iid']}/links",
headers={'PRIVATE-TOKEN': GITLAB_TOKEN},
verify=VERIFY_SSL_CERTIFICATE,
data={
'target_project_id': gl_to['project_id'],
'target_issue_iid': gl_to['iid'],
'link_type': gl_type,
}
)
gl_link.raise_for_status()
If I run the queries that are being used in the script using Postman, I receive data.
I am testing this out on a small project in Jira that has only one issue. Just to see how things map across in GitLab before going onto my more involved projects.
[INFO] #1/1 Migrating Jira issue ACT-1 ... Traceback (most recent call last):
File "/home/jared/jira2gitlab-main/./jira2gitlab.py", line 914, in <module>
migrate_project(jira_project, gitlab_project)
File "/home/jared/jira2gitlab-main/./jira2gitlab.py", line 507, in migrate_project
).json()
File "/usr/lib/python3/dist-packages/requests/models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Migration failed
Resetting user privileges..
The script as written (when allowed to) temporarily grants admin rights to users so that timestamps can be set properly. Clearly needed, and a cool way to handle this.
Unfortunately, the script does not check for what rights the user HAD prior to the import. If a user had admin rights beforehand, that user is incorrectly reset to non--admin when the script finishes.
Great work, thank you for sharing it!
Maybe I missed something but Request participants doesn't seem to be supported currently in the script.
This feature is not exactly equivalent on GitLab, but the /cc @username feature is probably the closest thing:
https://gitlab.com/gitlab-org/gitlab/-/issues/197288
Any thoughts about this? Either appending the cc's to the bottom of the description or adding as a comment might work, though I haven't looked at the traffic yet to see exactly what is happening when I CC a user.
Cheers!
if there is more than 20 results in a GET from a gitlab API you only get the 20 first results ...
you can extend the page up to 100 but after that you have to use the pages to get all results.
I get the following error when invoking jira-user-list.py
, while the same URL works when manually copied and executed in a browser.
401 Client Error: Unauthorized for url: https://riff-systemhaus.atlassian.net/rest/api/2/search?jql=project=%22GOERLITZ%22%20ORDER%20BY%20key&fields=*navigable,attachment,comment,worklog&maxResults=100&startAt=0
Tested the same with curl -u '[...]:[...]' [...]
and received the following output:
Basic authentication with passwords is deprecated. For more information, see: https://developer.atlassian.com/cloud/confluence/deprecation-notice-basic-auth/
That link contains the following sentence, perfectly fine corresponding to my first error:
For the following APIs and pages, all requests using basic authentication with a non-API token credential will return 401 (Unauthorized) after the deprecation period:
Looking at the code, basic auth with username+password seems to be sued, so the error makes sense. OTOH, the script seems to be created after the deprecation?
The deprecation period for this functionality has ended. From June 3rd, 2019, we will be progressively disabling the usage of this authentication method.
Would be great if you could have a look at this problem. Thanks!
I do not know how or even why this happened, but I have discovered some old issues we have in Jira somehow got changed to a REPORTER identifier of "Anonymous". And some of these issues were reported by user(s) that actually do still exist in our Jira environment.
When running the jira2gitlab import script and it hits such an issue (Reporter = "Anonymous"), I get the followwing error:
[INFO] #1006/2307 Migrating Jira issue ITS-1155 ... Traceback (most recent call last):
File "./jira2gitlab.py", line 884, in
migrate_project(jira_project, gitlab_project)
File "./jira2gitlab.py", line 446, in migrate_project
reporter = issue['fields']['reporter']['name']
TypeError: 'NoneType' object is not subscriptable
Migration failed
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.