xmlrunner / unittest-xml-reporting Goto Github PK
View Code? Open in Web Editor NEWunittest-based test runner with Ant/JUnit like XML reporting.
License: Other
unittest-based test runner with Ant/JUnit like XML reporting.
License: Other
would love to use xmlrunner, however, it seems to interfere with my custom arguments via argparse. I need those!
Hello to all!
Is is possible to generate a single report for a django project. For instance, let's suppose our django project has 2 apps: app1 and app2. When executing python manage.py test app1 app2 I would like to have all the output to a single file, instead of the two that are generated. The reason is that the sonarqube python plugin expects only one nosetests.xml file. It would be nice to have it as a setting on settings.py. For instance:
TEST_OUTPUT_TO_SINGLE_FILE = true or false
TEST_OUTPUT_FILE_NAME = nosetests.xml
There is a typo "fornat" instead of "format" in init.py -- which prevents the module from running.
I have two issues when trying to validate generated xml using the junit xsd found here.
Firstly, the xsd expects the testsuite to have a sequence with elements in order:
properties
testcase
system-out
system-err
However my output is in order:
properties
system-out
system-err
testcase
Secondly, 'properties' always seems to be written out, even if it contains no 'property' elements, also causing validation with this xsd to fail (as 'properties' has no minOccurs tag)
Is validation using an xsd is a target for xmlrunner? Don't know if this xsd file is the correct/official one to use, but it seems widely used as it appears in a lot of search results :)
Hi all,
Is there a way to generate one xml report per suite using xmlrunner?
I do not think that I am loading my test cases into the suite wrongly (I have my suite containing the associated test cases).
But I can't have one report generated for the whole test suite, xmlrunner seems to consider my test cases as test suites.
Is it simply not possible or is there a particular way for loading the test cases into the suite?
I think I tried all the different possibilities for loading the test cases into the test suite but the result was always the same.
Thanks in advance
Ilker
I am using subtests within some of my test methods, and xmlrunner doesn't fully support them.
It works just fine when all tests pass, but when one of them fails I get an error about not being able to get self.elapsed_time()
, and no xml file is generated. Would it be possible to add this feature?
I also noticed that your runner is importing "unittest" for Python 3, when I am using unittest2. In xmlrunner/init.py, in the rewrite branch:
if sys.version_info < (2, 7):
# monkey patch unittest
import unittest2 as unittest
and this in the master branch for xmlrunner/unittest.py (I think I am using this, I downloaded it through pip)
if sys.version_info < (2, 7):
# pylint: disable-msg=W0611
import unittest2 as unittest
(...)
else:
# pylint: disable-msg=W0611
import unittest
I've also been looking through the code to figure out if this is something I could do myself, and it seems that what is missing is a parent context for the subtests. They should have an index attribute, and no elapsed time, since that is only recorded for the full test (is it?). Don’t really know where to start tho, I just started using unittest very recently.
In the fix to the issue #12408 Django started explicitly ordering test cases in the test suite, so that all TransactionTestCases
are run together. Because of the way they work they can leave the database in a random state. Ordering makes sure all other test cases are run before them.
There's a function in django.test.simple called reorder_suite
which takes care of handling that, but currently djangotestrunner.XMLTestRunner
is not using it. This causes behavioral mismatch between xmlrunner and stock django test runner.
After tests start all print function are disabled and does not work. Which is confusing when you are use setUpClass and tearDownClass method to wait for external dependences.
And on jenkis verbose is buffered and displayed after all tests executions, not step by step like in unittest. Which is also confusing because you don't know that test is hangs up.
I'm getting the following error when installing on Python 3.3 on Debian:
pip3.3 install unittest-xml-reporting
[02:47:15][Step 1/1] ---> Running in 032fd69a71d5
[02:47:16][Step 1/1] Downloading/unpacking unittest-xml-reporting
[02:47:17][Step 1/1] Downloading unittest-xml-reporting-1.7.0.tar.gz
[02:47:17][Step 1/1] Running setup.py (path:/tmp/pip_build_root/unittest-xml-reporting/setup.py) egg_info for package unittest-xml-reporting
[02:47:17][Step 1/1] Traceback (most recent call last):
[02:47:17][Step 1/1] File "<string>", line 17, in <module>
[02:47:17][Step 1/1] File "/tmp/pip_build_root/unittest-xml-reporting/setup.py", line 11, in <module>
[02:47:17][Step 1/1] exec(compile(open(xmlrunner_version).read(), xmlrunner_version, 'exec'))
[02:47:17][Step 1/1] File "/usr/local/lib/python3.3/encodings/ascii.py", line 26, in decode
[02:47:17][Step 1/1] return codecs.ascii_decode(input, self.errors)[0]
[02:47:17][Step 1/1] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 192: ordinal not in range(128)
[02:47:17][Step 1/1] Complete output from command python setup.py egg_info:
[02:47:17][Step 1/1] Traceback (most recent call last):
[02:47:17][Step 1/1]
[02:47:17][Step 1/1] File "<string>", line 17, in <module>
[02:47:17][Step 1/1]
[02:47:17][Step 1/1] File "/tmp/pip_build_root/unittest-xml-reporting/setup.py", line 11, in <module>
[02:47:17][Step 1/1]
[02:47:17][Step 1/1] exec(compile(open(xmlrunner_version).read(), xmlrunner_version, 'exec'))
[02:47:17][Step 1/1]
[02:47:17][Step 1/1] File "/usr/local/lib/python3.3/encodings/ascii.py", line 26, in decode
[02:47:17][Step 1/1]
[02:47:17][Step 1/1] return codecs.ascii_decode(input, self.errors)[0]
[02:47:17][Step 1/1]
[02:47:17][Step 1/1] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 192: ordinal not in range(128)
Would unittest-xml-reporting process Python doctests? I didn't seem to see references in the documentation. Thanks
howdy, I followed the instructions for installing the xmlrunner (git clone then setup.py) and I was able to import xmlrunner in python 2 but not in python 3. Is this an issue at the setup.py?
I want to give xmlrunner a try :)
Traceback (most recent call last):
File "tests/test_autograde_runes.py", line 73, in <module>
catchbreak=False)
File "/usr/lib/python3.4/unittest/main.py", line 93, in __init__
self.runTests()
File "/usr/lib/python3.4/unittest/main.py", line 244, in runTests
self.result = testRunner.run(self.test)
File "/usr/local/lib/python3.4/dist-packages/xmlrunner/runner.py", line 111, in run
result.generate_reports(self)
File "/usr/local/lib/python3.4/dist-packages/xmlrunner/result.py", line 476, in generate_reports
test_runner.output.write(xml_content)
TypeError: must be str, not bytes
One workaround (not sure if it is correct) is to change test_runner.output.write(xml_content)
to test_runner.output.write(xml_content.decode('utf-8'))
Hi
there seems to be no tar-ball on pypi for this release
https://pypi.python.org/pypi/unittest-xml-reporting/
Would be nice, if those were released too, so that, for example, linux distributions can easily create a package.
Thanks
Arun
When I run tests using python -m unittest discover
the generation of XML reports is not triggered.
This is assuming I have setup my test module like this:
import unittest
import xmlrunner
...
class SomeTest(unittest.TestCase):
...
if __name__ == '__main__':
unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))
The reason seems to be that when unit tests are run using the discovery feature, main of module is not being run.
Is there another way of making this work?
Thanks for all your efforts.
There appears to be a bug in src/xmlrunner/__init__.py
Line 181 (_XMLTestResult.addFailure) that causes test failures (AssertionError) as errors.
_TestInfo.ERROR should be _TestInfo.FAILURE
I use unicode extensively in Python 2, so checking if an object is a str
fails when the object is unicode. To determine if an object is a string, isinstance(obj, basestring)
should be used. Example of culprit code:
https://github.com/danielfm/unittest-xml-reporting/blob/master/src/xmlrunner/__init__.py#L260
This will require enabling 2to3 however for Python 3.
In cmd:
testcase 01: enterAsGuest ... OK (1.001s)
testcase 03: signOut ... OK (2.001s)
Ran 2 tests in 3.008s
OK
Generating XML reports...
In XML report
<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="SanityCheck" tests="2" time="4.002">
<testcase classname="SanityCheck" name="testcase01_enterAsGuest" time="2.001"/>
<testcase classname="SanityCheck" name="testcase03_signOut" time="2.001"/>
<system-out>
<![CDATA[]]> </system-out>
<system-err>
<![CDATA[]]> </system-err>
</testsuite>
=================================================================
I import a global variable time_takens to record time taken by each test case.
update "_report_testcase" method in __init__.py:
testcase.setAttribute('time', '%.3f' % time_takens[index])
Could a parameter be added to set Java package name (and directory) for the XML files? Currently they seem to be placed into (root). Thanks
I tried to install xmlrunner both from Debian package and through pip and I still have an error during import.
So this is what I have tried
sudo apt-get install python-xmlrunner
# this installs version 1.5.0-1
I also tried
sudo pip install unittest-xml-reporting
# Requirement already satisfied (use --upgrade to upgrade): unittest-xml-reporting in /usr/lib/python2.7/dist-packages
And finally the test
python3
>>> import xmlrunner
ImportError: No module named 'xmlrunner'
I tried it on both Debian Jessie and elementary OS (Freya) so am I doing something wrong? btw I would prefer installing from .deb package because I don't have to install pip on machines which doesn't have it.
Hello,
there is a 1.9.2 version on pypi but no corresponding tag in github...
Following simple test snippet:
#!/usr/bin/env python
import unittest
import xmlrunner
class Foo(unittest.TestCase):
def testFoo(self):
self.assertTrue(False, ']]>')
unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))
results with unhandled exception:
ValueError: ']]>' not allowed in a CDATA section
Multiple CDATA sections can be added in a manner that will result with following final output:
<error message="]]>" type="AssertionError">
<![CDATA[Traceback (most recent call last):
File "./t.py", line 7, in testFoo
self.assertTrue(False, ']]]]><![CDATA[>')
AssertionError: ]]]]><![CDATA[>
]]> </error>
I upgraded to xmlrunner 1.4.1 this morning, and a number of tests broke, because an underlying helper routine was performing sys.stdout.flush(), which isn't handled by _DelegateIO.
A simple addition of " def flush(self): pass " in the _DelegateIO class definition fixes the problem, though.
[Win64 via easy_install].
Consider following failure:
======================================================================
FAIL [0.012s]: test_relation (nani.tests.related.NormalToTransFKTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jonas/workspace/project-nani/nani/tests/related.py", line 42, in test_relation
self.assertEqual(related.normal_trans.translated_field, ja.translated_field)
AssertionError: u'English' != u'\u4f55'
- English+ \u4f55
unittest-xml-reporting now fails with this:
Generating XML reports...
Traceback (most recent call last):
File "manage.py", line 16, in <module>
execute_manager(settings)
File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/core/management/commands/test.py", line 37, in handle
failures = test_runner.run_tests(test_labels)
File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/test/simple.py", line 363, in run_tests
result = self.run_suite(suite)
File "/home/jonas/workspace/project-nani/testproject/testrunner.py", line 16, in run_suite
).run(suite)
File "/usr/local/lib/python2.6/dist-packages/xmlrunner/__init__.py", line 351, in run
result.generate_reports(self)
File "/usr/local/lib/python2.6/dist-packages/xmlrunner/__init__.py", line 269, in generate_reports
report_file.write(xml_content)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u4f55' in position 280: ordinal not in range(128)
When a subtest ends, the method result.addSubTest
is called, even when no error is encountered. A TypeError
is then raised, because the error is None
. Here is the end of the stacktrace:
File "/opt/python-3.4.2/lib/python3.4/unittest/case.py", line 513, in _feedErrorsToResult
result.addSubTest(test.test_case, test, exc_info)
File "/home/jenkins/venv/0/lib/python3.4/site-packages/xmlrunner/result.py", line 245, in addSubTest
testinfo = _TestInfo(self, testcase, _TestInfo.ERROR, err, subTest=test)
File "/home/jenkins/venv/0/lib/python3.4/site-packages/xmlrunner/result.py", line 102, in __init__
self.err, test_method)
File "/opt/python-3.4.2/lib/python3.4/unittest/result.py", line 173, in _exc_info_to_string
exctype, value, tb = err
TypeError: 'NoneType' object is not iterable
Hence, the error must be converted to string only if it is not None
(https://hg.python.org/cpython/file/0be7c8f46378/Lib/unittest/result.py#l127).
Based on examining TextTestRunner
, it appears that after creating the result
object in XMLTestRunner.run
, there should be a result.failfast = self.failfast
.
Hi!, Im trying to get an XML report with the following structure:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite errors="0" failures="0" name="admin.admin_spots_tests.AdminSpotsTests" skipped="0" tests="1" time="2.235">
<properties>
<property name="test_create_spot" value="101" />
</properties>
<testcase classname="admin.admin_spots_tests.AdminSpotsTests" name="test_create_spot" time="2.235" />
</testsuite>
<testsuite errors="0" failures="0" name="rentals.rentals_search_tests.RentalsSearchTests" skipped="0" tests="1" time="2.198">
<properties>
<property name="test_search_by_rental_id" value="201" />
</properties>
<testcase classname="rentals.rentals_search_tests.RentalsSearchTests" name="test_search_by_rental_id" time="2.198" />
</testsuite>
</testsuites>
Where I want to set in the test suite properties, a list of IDs (for my test case management system).
I am creating a test suite (master test suite) and I add multiple test suites (child test suites) that contain one or many test cases.
The problem is that when I set the properties for a child test suite, its not showing in the XML.
If I set the properties of the master test suite, it does show but it adds the same property tags to every child test suite.
Any help is appreciated, thanks!
-Eduardo Reynoso
I'd like to print to stdout when debugging my tests, but do not see a way to do this. Is there an option I am missing?
I see you use _patch_standard_output()
and _restore_standard_ouput()
, but am not sure what to change to enable this.
XMLTestRunner
holds references to all the TestCase instances whose test methods it executes, via _XMLTestResult
and _TestInfo
.
I'm finding that problematic in a large suite of Django functional tests, where the TestCase instances typically grow to 50 MB each. The held references prevent the garbage collector from freeing their memory until the entire test suite has run. With a large enough test suite, the OS's OOM Killer kills the test process before it can finish.
I started a conversation on django-users about the problem more generally, and I wouldn't be surprised if there are related problems in the underlying unittest package.
Here's a patch that I've been using to investigate. I'm not advocating merging this as is. Just including it to make what I'm describing more concrete.
--- __init__.py~ 2013-05-02 16:45:19.000000000 -0400
+++ __init__.py 2013-07-18 21:39:13.000000000 -0400
@@ -8,6 +8,9 @@
import os
import sys
import time
+
+import psutil
+
from unittest import TestResult, _TextTestResult, TextTestRunner
try:
@@ -27,13 +30,24 @@
self.delegate = delegate
def write(self, text):
- self._captured.write(text)
+ #self._captured.write(text)
self.delegate.write(text)
def __getattr__(self, attr):
return getattr(self._captured, attr)
+def testcase_name(test_method):
+ testcase = type(test_method)
+
+ # Ignore module name if it is '__main__'
+ module = testcase.__module__ + '.'
+ if module == '__main__.':
+ module = ''
+ result = module + testcase.__name__
+ return result
+
+
class _TestInfo(object):
"""This class keeps useful information about the execution of a
test method.
@@ -44,11 +58,21 @@
def __init__(self, test_result, test_method, outcome=SUCCESS, err=None):
self.test_result = test_result
- self.test_method = test_method
self.outcome = outcome
self.elapsed_time = 0
self.err = err
+ #self.test_method = test_method
+ self.test_description = self.test_result.getDescription(test_method)
+ self.test_exception_info = (
+ '' if not self.err
+ else self.test_result._exc_info_to_string(
+ self.err, test_method)
+ )
+
+ self.test_name = testcase_name(test_method)
+ self.test_id = test_method.id()
+
def test_finished(self):
"""Save info that can only be calculated once a test has run.
"""
@@ -58,16 +82,18 @@
def get_description(self):
"""Return a text representation of the test method.
"""
- return self.test_result.getDescription(self.test_method)
+ #return self.test_result.getDescription(self.test_method)
+ return self.test_description
def get_error_info(self):
"""Return a text representation of an exception thrown by a test
method.
"""
- if not self.err:
- return ''
- return self.test_result._exc_info_to_string(self.err, \
- self.test_method)
+ #if not self.err:
+ # return ''
+ #return self.test_result._exc_info_to_string(self.err, \
+ # self.test_method)
+ return self.test_exception_info
class _XMLTestResult(_TextTestResult):
@@ -81,6 +107,8 @@
self.successes = []
self.callback = None
self.elapsed_times = elapsed_times
+ self.start_meminfo = psutil.Process(os.getpid()).get_memory_info()
+ self.stop_meminfo = psutil.Process(os.getpid()).get_memory_info()
def _prepare_callback(self, test_info, target_list, verbose_str,
short_str):
@@ -103,6 +131,10 @@
if self.showAll:
self.stream.writeln('%s (%.3fs)' % \
(verbose_str, test_info.elapsed_time))
+ self.stop_meminfo = psutil.Process(os.getpid()).get_memory_info()
+ self.delta_stop_meminfo = { 'rss': self.stop_meminfo.rss - self.start_meminfo.rss,
+ 'vms': self.stop_meminfo.vms - self.start_meminfo.vms }
+ self.stream.write('on stop: status: {0}, delta: {1}\n'.format(self.stop_meminfo, self.delta_stop_meminfo))
elif self.dots:
self.stream.write(short_str)
self.callback = callback
@@ -114,6 +146,10 @@
TestResult.startTest(self, test)
if self.showAll:
+ self.start_meminfo = psutil.Process(os.getpid()).get_memory_info()
+ self.delta_start_meminfo = { 'rss': self.start_meminfo.rss - self.stop_meminfo.rss,
+ 'vms': self.start_meminfo.vms - self.stop_meminfo.vms }
+ self.stream.write('on start: status: {0}, delta: {1}\n'.format(self.start_meminfo, self.delta_start_meminfo))
self.stream.write(' ' + self.getDescription(test))
self.stream.write(" ... ")
@@ -165,14 +201,7 @@
for tests in (self.successes, self.failures, self.errors):
for test_info in tests:
- testcase = type(test_info.test_method)
-
- # Ignore module name if it is '__main__'
- module = testcase.__module__ + '.'
- if module == '__main__.':
- module = ''
- testcase_name = module + testcase.__name__
-
+ testcase_name = test_info.test_name
if not testcase_name in tests_by_testcase:
tests_by_testcase[testcase_name] = []
tests_by_testcase[testcase_name].append(test_info)
@@ -201,10 +230,9 @@
_report_testsuite = staticmethod(_report_testsuite)
- def _test_method_name(test_method):
+ def _test_method_name(test_id):
"""Returns the test method name.
"""
- test_id = test_method.id()
return test_id.split('.')[-1]
_test_method_name = staticmethod(_test_method_name)
@@ -217,7 +245,7 @@
testcase.setAttribute('classname', suite_name)
testcase.setAttribute('name', \
- _XMLTestResult._test_method_name(test_result.test_method))
+ _XMLTestResult._test_method_name(test_result.test_id))
testcase.setAttribute('time', '%.3f' % test_result.elapsed_time)
if (test_result.outcome != _TestInfo.SUCCESS):
Hi DEV,
I used xmlrunner in unittest, I have a problem about Unicode now, error log is as follows:
Generating XML reports...
Traceback (most recent call last):
File "/Users/kingson/PycharmProjects/DDYAuto/src/TestSuites/login/TSTestLogin.py", line 69, in <module>
unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 95, in __init__
self.runTests()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 232, in runTests
self.result = testRunner.run(self.test)
File "/Library/Python/2.7/site-packages/xmlrunner/xmlrunner.py", line 466, in run
result.generate_reports(self)
File "/Library/Python/2.7/site-packages/xmlrunner/xmlrunner.py", line 341, in generate_reports
_XMLTestResult._report_testcase(suite, test, testsuite, doc)
File "/Library/Python/2.7/site-packages/xmlrunner/xmlrunner.py", line 294, in _report_testcase
error_info = unicode(test_result.get_error_info())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 179: ordinal not in range(128)
Maybe result include Chinese characters, Can you help me? Thanks.
python 2.6 does not have the same features in unittest
.
You can use unittest2
and monkey-patch unittest
but unittest2
has not been updated in a few years and is not in sync with the one available in python dev branch.
Problematic:
unittest.result
unittest.TextTestResult
(was _TextTestResult
)unittest.skip
unittest.util
When setting a pdb trace in a test and running the test with unittest-xml-reporting as the test runner, pdb output is not displayed.
Please release a new version of unittest-xml-reporting on pypi that contains 34633e2. Thank you in advance.
I'm using xml-reporting to run our standard python unittest suite. The tests themselves run perfectly fine all the way to the end but then we get an exception when it goes on to output the xml.
Any ideas on how we get around this or if there is a fix for it? We are running Python 2.7.6 on Ubuntu 14.04 if that matters.
This is the full full exception we see(snipped the html portion of the output):
Ran 2463 tests in 132.803s
OK
Generating XML reports...
..........html snipped...........
<!-- The above is a description of an error in a Python program, formatted
for a Web browser because the 'cgitb' module was enabled. In case you
are not reading this in a Web browser, here is the original traceback:
Traceback (most recent call last):
File "./full_test.py", line 48, in <module>
xmlrunner.XMLTestRunner(output=file('./test-reports/report.xml', 'w')).run(testRun.getTestSuite())
File "build/bdist.linux-x86_64/egg/xmlrunner/xmlrunner.py", line 468, in run
self._restore_standard_output()
File "build/bdist.linux-x86_64/egg/xmlrunner/xmlrunner.py", line 397, in _restore_standard_output
sys.stdout = sys.stdout.delegate
AttributeError: 'file' object has no attribute 'delegate'
-->
When you run python tests.py -h
you get a nice little help page with neat options (such as output buffering and verbosity). These work fine when using the default runner (unittest.main()
). However, if I use xmlrunner as shown in the readme file, these options don't work anymore (although the help page still displays them).
Since this behavior is very confusing (and misleading), you should either remove the options from the help page or add these option to the xmlrunner. I really like output buffering but it apparently isn't available with xmlrunner.
README says "Please note Python 2.6 end-of-life was in Oct 2013, and support will be dropped in next major version." Which version is the next major version? Can the work be done on the master branch now?
I'm facing the following exception when executing my tests:
File "ve/lib/python2.7/site-packages/django/test/runner.py", line 115, in run_suite
).run(suite)
File "ve/lib/python2.7/site-packages/xmlrunner/runner.py", line 59, in run
test(result)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in __call__
return self.run(*args, **kwds)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 108, in run
test(result)
File "ve/lib/python2.7/site-packages/django/test/testcases.py", line 184, in __call__
result.addError(self, sys.exc_info())
File "ve/lib/python2.7/site-packages/xmlrunner/result.py", line 207, in addError
self._save_output_data()
File "ve/lib/python2.7/site-packages/xmlrunner/result.py", line 164, in _save_output_data
self._stdout_data = sys.stdout.getvalue()
AttributeError: 'file' object has no attribute 'getvalue'
Running a test with the xml test runner differs from running a test with the default text runner. In the default setup after each test local objects are deleted. With the xml test runner the destructors are called after test completion. It is important to rise an assertionError in the test to get this error.
import xmlrunner
import unittest
class dummy(object):
def __init__(self, inst):
self.inst = inst
print("ctro, inst %s"%inst)
def __del__(self):
print("dtro, inst %s"%self.inst)
class test_spuCfgParser(unittest.TestCase):
def test_foo(self):
print("test_foo")
dummyInst = dummy("foo")
raise AssertionError("SomeError")
def test_foofoo(self):
print("test_foofoo")
dummyInst = dummy("foofoo")
raise AssertionError("SomeError")
if __name__ == '__main__':
#unittest.main()
unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'), failfast=False, buffer=False, catchbreak=False)
Output with unittest.main():
test_foo
ctro, inst foo
Fdtro, inst foo
test_foofoo
ctro, inst foofoo
Fdtro, inst foofoo
Output with xmlrunner:
Running tests...
----------------------------------------------------------------------
FF
======================================================================
FAIL [0.000s]: test_foo (__main__.test_spuCfgParser)
----------------------------------------------------------------------
Traceback (most recent call last):
File "debugxmlrunner.py", line 17, in test_foo
raise AssertionError("SomeError")
AssertionError: SomeError
Stdout:
test_foo
ctro, inst foo
======================================================================
FAIL [0.000s]: test_foofoo (__main__.test_spuCfgParser)
----------------------------------------------------------------------
Traceback (most recent call last):
File "debugxmlrunner.py", line 23, in test_foofoo
raise AssertionError("SomeError")
AssertionError: SomeError
Stdout:
test_foofoo
ctro, inst foofoo
----------------------------------------------------------------------
Ran 2 tests in 0.000s
FAILED (failures=2)
Generating XML reports...
dtro, inst foofoo
dtro, inst foo
I'm working on submitting a patch for this. Here is the offending line in result.py
u'[\x00-\x08\x0B\x0C\x0E-\x1F\uD800-\uDFFF\uFFFE\uFFFF]'
https://github.com/danielfm/unittest-xml-reporting/blob/master/src/xmlrunner/__init__.py#L224
It should use:
if isinstance(test_runner.output, basestring) ...
Configuration:
Python 2.7.5
unittest-xml-reporting 1.6.0
.Traceback (most recent call last):
File "TestScript.py", line 539, in
xmlrunner.XMLTestRunner().run(all_tests)
File "C:\Programme\Python27\lib\site-packages\xmlrunner__init__.py", line 395, in run
test(result)
File "C:\Programme\Python27\lib\unittest\suite.py", line 70, in call
return self.run(_args, *_kwds)
File "C:\Programme\Python27\lib\unittest\suite.py", line 108, in run
test(result)
File "C:\Programme\Python27\lib\unittest\suite.py", line 70, in call
return self.run(_args, *_kwds)
File "C:\Programme\Python27\lib\unittest\suite.py", line 108, in run
test(result)
File "C:\Programme\Python27\lib\unittest\case.py", line 395, in call
return self.run(_args, *_kwds)
File "C:\Programme\Python27\lib\unittest\case.py", line 315, in run
self.addSkip(result, skip_why)
File "C:\Programme\Python27\lib\unittest\case.py", line 291, in addSkip
addSkip(self, reason)
File "C:\Programme\Python27\lib\site-packages\xmlrunner__init.py", line 191, in addSkip
TestInfo(self, test, TestInfo.SKIP, err),
File "C:\Programme\Python27\lib\site-packages\xmlrunner__init.py", line 72, in init
self.err, test_method)
File "C:\Programme\Python27\lib\unittest\result.py", line 154, in _exc_info_to_string
exctype, value, tb = err
ValueError: too many values to unpack
When using xmlrunner with python 2.7's unittest framework, xmlrunner suppresses python errors in setUpClass() and tearDownClass() functions.
Running unittest framework with xmlrunner:
unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test_results'))
Output:
Running tests...
----------------------------------------------------------------------
======================================================================
Traceback (most recent call last):
File "./test_tpa.py", line 587, in <module>
unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test_results'))
File "/usr/lib/python2.7/unittest/main.py", line 95, in __init__
self.runTests()
File "/usr/lib/python2.7/unittest/main.py", line 229, in runTests
self.result = testRunner.run(self.test)
File "/usr/lib/python2.7/site-packages/xmlrunner/__init__.py", line 300, in run
result.printErrors()
File "/usr/lib/python2.7/unittest/runner.py", line 108, in printErrors
self.printErrorList('ERROR', self.errors)
File "/usr/lib/python2.7/site-packages/xmlrunner/__init__.py", line 122, in printErrorList
(flavour, test_info.get_elapsed_time(), \
File "/usr/lib/python2.7/site-packages/xmlrunner/__init__.py", line 33, in get_elapsed_time
return self.test_result.stop_time - self.test_result.start_time
AttributeError: '_XMLTestResult' object has no attribute 'stop_time'
Running without xmltestrunner:
unittest.main()
Output:
E
======================================================================
ERROR: setUpClass (__main__.Test_TPA)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./test_tpa.py", line 36, in setUpClass
blah.blah()
NameError: global name 'blah' is not defined
----------------------------------------------------------------------
Ran 0 tests in 0.000s
FAILED (errors=1)
Hi.
I'm trying to output the descriptions in the xml file, but they don't show up.
I just added descriptions=True
to xmlrunner.XMLTestRunner
. Is this correct?
# -*- coding: utf-8 -*-
import unittest
import sys
import xmlrunner
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from parametrizedTestCase import ParametrizedTestCase
TESTS_NAME = "Busca"
CAP_FIREFOX = {'browser': 'Firefox', 'browser_version': '21.0',
'os': 'Windows', 'os_version': '7', 'resolution': '1024x768'}
CAP_IE = {'browser': 'IE', 'browser_version': '10.0',
'os': 'Windows', 'os_version': '7', 'resolution': '1024x768'}
CAP_CHROME = {'browser': 'Chrome', 'browser_version': '18.0',
'os': 'Windows', 'os_version': '7', 'resolution': '1024x768'}
CAP_SAFARI = {'browser': 'Safari', 'browser_version': '8.0',
'os': 'OS X', 'os_version': 'Yosemite', 'resolution': '1024x768'}
# pylint: disable=R0904
class CompanyTestes(ParametrizedTestCase):
"""Classe com testes unitários para a busca de passagens do site."""
def local_setup(self):
self.driver = webdriver.Chrome("./chromedriver.exe")
# Setup no BrowserStack
def remote_setup(self):
desired_cap = self.param
desired_cap['browserstack.video'] = False
desired_cap['browserstack.debug'] = True
desired_cap['project'] = 'ProjetoCompany'
desired_cap['name'] = TESTS_NAME + " - " + unittest.TestCase.shortDescription(self)
# webdriver.Remote credentials - removed
def setUp(self):
self.local_setup()
# self.remote_setup()
def tearDown(self):
self.driver.quit()
# @unittest.skip("Pulando...")
def test_hi(self):
"""Description that is supposed to show up in the xml file."""
driver = self.driver
if __name__ == "__main__":
suite = unittest.TestSuite()
suite.addTest(ParametrizedTestCase.parametrize(CompanyTestes, param=CAP_SAFARI))
# suite.addTest(ParametrizedTestCase.parametrize(CompanyTestes, param=CAP_CHROME))
# suite.addTest(ParametrizedTestCase.parametrize(CompanyTestes, param=CAP_FIREFOX))
# suite.addTest(ParametrizedTestCase.parametrize(CompanyTestes, param=CAP_IE))
# runner = xmlrunner.XMLTestRunner(output='test-reports')
runner = xmlrunner.XMLTestRunner(output='test-reports', descriptions=True, verbosity=1)
runner.run(suite)
Also, I need this class to inject parameters to setUp
. Not sure if this is relevant.
parametrizedTestCase.py
:
import unittest
# http://eli.thegreenplace.net/2011/08/02/python-unit-testing-parametrized-test-cases/
class ParametrizedTestCase(unittest.TestCase):
""" TestCase classes that want to be parametrized should
inherit from this class.
"""
def __init__(self, methodName='runTest', param=None):
super(ParametrizedTestCase, self).__init__(methodName)
self.param = param
@staticmethod
def parametrize(testcase_klass, param=None):
""" Create a suite containing all tests taken from the given
subclass, passing them the parameter 'param'.
"""
testloader = unittest.TestLoader()
testnames = testloader.getTestCaseNames(testcase_klass)
suite = unittest.TestSuite()
for name in testnames:
suite.addTest(testcase_klass(name, param=param))
return suite
Output:
<?xml version="1.0" encoding="UTF-8"?>
<testsuite errors="0" failures="0" name="CompanyTestes-20161014105524" skipped="0" tests="1" time="7.086">
<testcase classname="CompanyTestes" name="test_hi" time="7.086"/>
<system-out>
<![CDATA[]]> </system-out>
<system-err>
<![CDATA[]]> </system-err>
</testsuite>
Thanks.
What about to add a param like --xmloutput dir
to the module __main__.py
so as it may be run as:
python -m xmlrunner --xmloutput test-reports test_module
I may submit a PR if you agree with this change.
In the readme, requirements ask for Python 2.7+ but it is currently successfully tested on python2.6.
The tox.ini should also specify python 3.4
I couldn't get any current version of this to work, either from this repo, or the fork by daspecter. Here's what I did to make it work, with some background detail:
Hello,
the Django TestRunner has changed in django 1.6: https://docs.djangoproject.com/en/1.6/topics/testing/overview/#running-tests
With django 1.6 beta, the XMLTestRunner looks for tests in all INSTALLED_APPS, but not in 1.6.
Is there a plan to support both < 1.6 and > 1.6 ?
Hi!
Thanks a lot for a useful runner!
I have a problem, though.
I have a big test suite that contains a number of test cases, each of those contains several tests. When I use your test runner, I get as much XML files as I have test cases and each of those files contain the results of the tests, that pertain to this test case (perfect!)
However, in each of those XML files the stdout / stderr of the whole test suite is recorded. As far as I understand the specification, the stderr / stdout tags should contain only output related to the tests in this XML file, not all tests that were ever run.
Note, that this is not the same as #1. I don't want stdout / stderr per individual test, which is against the specification, but rather the XML to contain only the output that was generated by the tests that it ran.
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.