Giter VIP home page Giter VIP logo

dump2polarion's Introduction

dump2polarion

Build status Coverage report Version Supported Python versions Code style: black

Capabilities of the dump2polarion library

  • generating XML files for XUnit, Testcase and Requirements Importers
  • submitting XML files to Importers
  • verifying that the import was successful
  • saving the import log files
  • reading Test Cases data from SVN repository with checked out Polarion project

The library supports all features of the Importers (even iterations). The export to XML files can be customized per project - lookup method, what results will be included in the XUnit (e.g. only PASSed tests), etc.

The library doesn't use the legacy webservices API, all the operations are performed using Polarion Importers.

All Polarion projects are internally stored in a SVN repo and since Polarion importers don't have any API for querying Polarion, reading Test Cases data from a SVN repo is one of the two methods available for getting data out of Polarion. The second method is parsing the log file produced by the specific importer (e.g. Test Cases importer) -- this method is also supported by the library.

polarion_dumper.py script

Script for importing tests results recorded in the CSV, SQLite, junit-report.xml (generated by pytest) or Ostriz JSON input file to Polarion using the XUnit Importer.

Can be also used for submit of pre-generated XUnit, Test Case or Requirement XML files to corresponding Polarion Importer.

By default the script waits until the Importer finishes the import job and then checks the success of the operation.

Usage

polarion_dumper.py -i {input_file}

By default the input data are submitted to Polarion. You can disable this behavior with -n option. In this case the XML file used for submission will be saved to disk. Default file location is the current directory (can be overridden with -o option).

When output file is specified with -o PATH, the XML file used for results submission will be saved to disk. If PATH is a directory, resulting file name will be generated - PATH/FILE_TYPE-TESTRUN_ID-TIMESTAMP.xml.

When the input file is a XML file in a format supported by one of the Polarion Importers (e.g. saved earlier with -o FILE -n), it is submitted to Polarion.

Configuration

You can specify credentials on command line with --user kerberos_username --password kerberos_password. Or you can set them in a config file.

The config file is specified on command line with -c config_file.yaml.

Another possibility for specifying credentials are environment variables (the same are used for pylarion):

export POLARION_USERNAME=kerberos_username
export POLARION_PASSWORD=kerberos_password

You can mix all these approaches, e.g. user name on command line and password in the environment variable.

Important

You need to specify URLs of the importer services and queues in the config file. See <https://mojo.redhat.com/docs/DOC-1098563#config>

Install

To install the package to your virtualenv, run

pip install dump2polarion

or install it from cloned directory

pip install -e .

Package on PyPI <https://pypi.python.org/pypi/dump2polarion>

Requirements

Requirements are listed in requirements.txt.

Customization

The library can be customized per Polarion project. Every project can have it's own behavior, like one is parametrized and the other is not, one wants to have all test results imported and the other wants to import just PASSes, etc.

Each exporter object (TestcaseExport, XunitExport, RequirementExport) accepts transform_func callable that is executed for every record. The callable can transform record data, e.g.:

def results_transform(result):
    """Export results only when comment is present; modify the comment."""
    comment = result.get("comment")
    if not comment:
        return None

    result = copy.deepcopy(result)

    result["comment"] = "Changed comment: {}".format(comment)
    return result

xunit_transform = XunitExport(testrun_id, tests_records, config, transform_func=results_transform)

CSV format for XUnit

There needs to be a row with field names - it is present by default when exported from Polarion.

Fields are ID; Title; Test Case ID (optional but recommended); Verdict; Comment (optional); Time (optional); stdout (optional); stderr (optional) + any other field you want. Order of the fields and case doesn't matter.

The "Verdict" field and any optional fields must be added manually. Valid values for "verdict" are "passed", "failed", "skipped", "waiting" or empty. It's case insensitive.

There can be any content before the row with field names and the test results.

SQLite format for XUnit

You can convert the CSV file exported out of Polarion using the csv2sqlite.py script:

csv2sqlite.py -i {input_file.csv} -o {output_file.sqlite3}

How to submit the XML file manually

polarion_dumper.py -i input.xml --user {user} --password {password}

or

curl -k -u {user}:{password} -X POST -F file=@./output.xml {importer_url}

More info

For CFME QE specific instructions see <https://mojo.redhat.com/docs/DOC-1098563>

For info about XUnit Importer see <https://mojo.redhat.com/docs/DOC-1073077>

For info about Test Case Importer see <https://mojo.redhat.com/docs/DOC-1075945>

For info about Requirements Importer see <https://mojo.redhat.com/docs/DOC-1163149>

dump2polarion's People

Contributors

mayurilahane avatar mkoura avatar mshriver avatar ronnypfannschmidt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

dump2polarion's Issues

BLOCKED testcases are not getting uploaded by polarion_dumper.py

Let say we have exported following csv and added verdict PASSED, FAILED and BLOCKED.

Exported on	2018-12-23 09:30
Query	(assignee.id:(ytale) AND title:OSP AND NOT status:inactive AND (TEST_RECORDS:("RHCF3/5_10_0_30",@null))) AND project.id:RHCF3

ID	        Title								Verdict
RHCF3-51011	OSP: Test DS and Volume availiblity in source and target	PASSED
RHCF3-51013	OSP: Test delete infra mapping					FAILED
RHCF3-51028	OSP: Test edit migration plan					BLOCKED

With command,
polarion_dumper.py -i 'workitems.csv' -c ~/.config/polarion_tools.yaml

it only upload PASSED and FAILED results and does not upload BLOCKED results.

Import logs says,

xUnit Import Log for job #362914
Download
2018-12-24 13:43:47,416 INFO ImportXUnitThread_15018 - Red Hat Importers version 2.0.21.
2018-12-24 13:43:47,416 INFO ImportXUnitThread_15018 - Processing job #362914.
2018-12-24 13:43:47,416 INFO ImportXUnitThread_15018 - Running as 'Import User'.
2018-12-24 13:43:47,416 INFO ImportXUnitThread_15018 - Starting import of XUnit results to Polarion.
2018-12-24 13:43:47,417 INFO ImportXUnitThread_15018 - Deserializing XUnit results.
2018-12-24 13:43:52,643 INFO ImportXUnitThread_15018 - Using Polarion lookup method: name
2018-12-24 13:43:52,771 INFO ImportXUnitThread_15018 - Importing results to test run '5_10_0_30'.
2018-12-24 13:43:52,771 INFO ImportXUnitThread_15018 - Processing test suite 'Import for RHCF3 - 5_10_0_30 testrun'.
2018-12-24 13:43:52,771 INFO ImportXUnitThread_15018 - Processing test case 'OSP: Test DS and Volume availiblity in source and target'.
2018-12-24 13:43:52,850 INFO ImportXUnitThread_15018 - Work item: 'OSP: Test DS and Volume availiblity in source and target' (RHCF3-51011)
2018-12-24 13:43:53,031 INFO ImportXUnitThread_15018 - Processing test case 'OSP: Test delete infra mapping'.
2018-12-24 13:43:53,107 INFO ImportXUnitThread_15018 - Work item: 'OSP: Test delete infra mapping' (RHCF3-51013)
2018-12-24 13:43:56,510 INFO ImportXUnitThread_15018 - Test suite 'Import for RHCF3 - 5_10_0_30 testrun' contains 3 test cases.
2018-12-24 13:43:59,016 INFO ImportXUnitThread_15018 - Polarion results published to: https://hostname/polarion/#/project/RHCF3/testrun?id=5_10_0_30
2018-12-24 13:43:59,016 INFO ImportXUnitThread_15018 - Ending import of XUnit results to Polarion.
2018-12-24 13:43:59,016 INFO ImportXUnitThread_15018 - Sending message to the UMB.
2018-12-24 13:43:59,623 INFO ImportXUnitThread_15018 - Sending import message:
Message Headers:
  JMSDestination: null
  JMSDeliveryMode: non-persistent
  JMSExpiration: 0
  JMSPriority: 0
  JMSMessageID: null
  JMSTimestamp: 0
  JMSCorrelationID: null
  JMSReplyTo: null
  JMSRedelivered: false
  JMSType: application/json
Message Properties:
  job-id: 362914
  type: import-results
Message Content:
{
  "testrun-url" : "https://hostname/polarion/#/project/RHCF3/testrun?id=5_10_0_30",
  "import-results" : [ {
    "suite-name" : "Import for RHCF3 - 5_10_0_30 testrun",
    "status" : "passed"
  } ],
  "status" : "passed",
  "log-url" : "https://hostname/polarion/rh-import/xunit-log?jobId=362914"
}
2018-12-24 13:43:59,740 INFO ImportXUnitThread_15018 - Message sent.

Issue found in - dump2polarion v0.33
Expected - polarion_dumper.py should upload blocked testcases.

testcase_result.params can be None

Hitting an exception where a testcase_result record has a 'params' key, and the value is None.

six.iteritems() blows up on passing NoneType.

The lookup is wrapped in a .get(), but when that returns None it raises exception.

Submit failed, invalid response received, Polarion 3.18.1.20180624-2340-be8212df

Hi, I am trying to use dump2polarion to import xUnit test results with against :

Build: 3.18.1.20180624-2340-be8212df
© 2018 Polarion AG

polarion_dumper.py -t 20180920-1334 -i test/results/2018_09_17__12_43_57/result.xml -c dump2polarion.yaml

Polarion response is 200 (OK) but dump2polarion fails with

dump2polarion.submit:ERROR:Submit to http://test-polarion.xxxxxx.com/polarion/import/xunit failed, invalid response received

xxxxx.com is our Polarion instance running on company premises.

I added a few logs that print out status code, encoding and response body. It seems that Polarion returns a web page with 'Your browser is not fully supported' warning.

I tried to add the following User-Agent to session headers and for submit: 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36' but it does not help to solve the problem.

Is this a known issue?

Please find the full response body below:

dump2polarion.configuration:INFO:Config loaded from dump2polarion.yaml
dump2polarion.submit:INFO:Submitting data to http://test-polarion.xxxxxxx.com/polarion/import/xunit
200
UTF-8

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
	  	<meta name="apple-itunes-app" content="app-id=600778564" />  
	  	<link rel="shortcut icon" href="/polarion/ria/images/favicon.ico?buildId=20180624-2340-be8212df" />
	  	<link rel="stylesheet" type="text/css" href="/polarion/ria/css/loginPages.css?buildId=20180624-2340-be8212df" />
	  	<link rel="stylesheet" type="text/css" href="/polarion/login.css?buildId=20180624-2340-be8212df" />
	  	<script src="/polarion/ria/javascript/jquery-3.0.0.min.js?buildId=20180624-2340-be8212df"></script>
	  	<script src="/polarion/ria/javascript/flash_detect.js?buildId=20180624-2340-be8212df" type="text/javascript"></script>
	  	<script src="/polarion/login.js?buildId=20180624-2340-be8212df" id="loginScript" type="text/javascript"></script>
	  	<script src="/polarion/ria/javascript/loginPages.js?buildId=20180624-2340-be8212df" type="text/javascript"></script>
	  	<title>Login</title>  
	</head>
	<body id="mainBody" onresize="hideCompanyLogo()" onload="hideCompanyLogo()" style="background-image: url(/polarion/ria/images/login/background.jpg?buildId=20180624-2340-be8212df)" class="background">
	
		<img src="/polarion/ria/images/login/siemens_logo.png" alt="Siemens" id="siemensLogo"/>
	
		<div id="companyAndContainer">
			
	  		
			<div id="container">
				<div id="warning" class="hidden">
					<ul>
						<li id="warningInfo">Your browser is not fully supported.<span id="moreinfo">More info...</span></li>
					
						
					
						
					
					</ul>
					<span id="lessinfo">Less info...</span>
				</div>
			
				
		
				<div id="box">
					<div id="header">
						<div id="companyName">Siemens PLM Software</div>
						<div id="productName">Polarion<sup style="line-height: 0;">&#174;</sup> ALM<sup style="line-height: 0;">&#8482;</sup></div>
					</div>
				
					<form method="post" id="logInForm" action="j_security_check" enctype="application/x-www-form-urlencoded" onsubmit="return login();">
						
						<input id="target" name="target" type="hidden" value=""/>
						<div id="loginRow">
							<div id="polarionLogo">
								
									<img src="/polarion/ria/images/logos/repo_logo.png" class="logo" />
									
							</div>
	
							<div id="login">							
								
				                <div id="username">
				                	<label for="j_username" class="hidden">Username</label>
				                	<input id="j_username" name="j_username" size="12" type="text" class="input" />
				                </div>
				                
				                <div id="password">
				                	<label for="j_password" class="hidden">Password</label>
				                	<input id="j_password" name="j_password" size="12" value="" type="password" class="input" />
				                </div>
							</div>
						</div>
						<div id="loginRow">
							<div id="serverName">
								
							</div>
							<div id="submit">
								<label id="submitMsg" class="hidden" for="submitButton">Logging in...</label>
	                			<input id="submitButton" name="submit" type="submit" value="Log In" />
	                			<label id="rememberme"><input name="rememberme" value="true" type="checkbox" /><span class="remember">Stay logged in</span></label>
							</div>
						</div>
					</form>
					<div id="options">
						
						
						 
						
					</div>
				
					<div id="copyright">© 2018 Polarion AG</div>
		    	</div>
			</div>
		</div>
		

		<iframe src="/polarion/ria/prefetch.jsp" id="prefetch"></iframe> 
	</body>
</html> 

dump2polarion.submit:ERROR:Submit to http://test-polarion.xxxxxxx.com/polarion/import/xunit failed, invalid response received

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.