xmlrunner / unittest-xml-reporting Goto Github PK

View Code? Open in Web Editor NEW

295.0 13.0 136.0 381 KB

unittest-based test runner with Ant/JUnit like XML reporting.

License: Other

Python 99.31% Makefile 0.69%

unittest xml-report test-runner xunit xunit-tests junit-xml jenkins pytest nose python

unittest-xml-reporting's People

Contributors

Stargazers

Watchers

Forkers

dmeister mart1n daspecster aquarion languagelab jakenbuhler davidkeijser romuald vertis-infotech adityagodbole garsue chrisklaiber d1ffuz0r cfoote czyzykowski srafi duncm josselin-c boldie open4job andymgit jollychang matthewlmcclure sneutralino pombredanne zuljinsbk pycontribs davidszotten agturcz duedil-ltd coreygriggs dnozay martync csiluszyk tomazk jaakkosipari nightblues xavfernandez theiviaxx clintmod madisona chywoo philipgarnero reedcourty arturican arcivanov sthirugn tkanemoto buster3 timgraham lmtwga blakeqd macro32-xx arunenigma rrzaripov munkyu desbma paulo-g dbrobins kirthiss coursemology yangshuang1 jerryhebert skywind0218 bdh1011 jiangbingo peniblec craigemery rubendura nnonexistent teeberg nimbals1 brettcannon jameslinus undo1 lupino liupeiyu hugovk 7geese fotto pegler ikedam danchr mikethegreat xonqnopp bbartholomew-om1 ikarhu28 gaikwad411 durgasai2308 adnanwaheed nablaa spire-inc markusj1201 mithro hip929 bbatliner-ocient leture labapart princessinconsuela httpsgithu

unittest-xml-reporting's Issues

Argparse compatibility?

would love to use xmlrunner, however, it seems to interfere with my custom arguments via argparse. I need those!

Create single report for Django

Hello to all!

Is is possible to generate a single report for a django project. For instance, let's suppose our django project has 2 apps: app1 and app2. When executing python manage.py test app1 app2 I would like to have all the output to a single file, instead of the two that are generated. The reason is that the sonarqube python plugin expects only one nosetests.xml file. It would be nice to have it as a setting on settings.py. For instance:
TEST_OUTPUT_TO_SINGLE_FILE = true or false
TEST_OUTPUT_FILE_NAME = nosetests.xml

typo "fornat" instead of "format" in init.py

There is a typo "fornat" instead of "format" in init.py -- which prevents the module from running.

xml validation

I have two issues when trying to validate generated xml using the junit xsd found here.

Firstly, the xsd expects the testsuite to have a sequence with elements in order:

properties
testcase
system-out
system-err

However my output is in order:

properties
system-out
system-err
testcase

Secondly, 'properties' always seems to be written out, even if it contains no 'property' elements, also causing validation with this xsd to fail (as 'properties' has no minOccurs tag)

Is validation using an xsd is a target for xmlrunner? Don't know if this xsd file is the correct/official one to use, but it seems widely used as it appears in a lot of search results :)

Generate a single xml report

Hi all,

Is there a way to generate one xml report per suite using xmlrunner?

I do not think that I am loading my test cases into the suite wrongly (I have my suite containing the associated test cases).
But I can't have one report generated for the whole test suite, xmlrunner seems to consider my test cases as test suites.

Is it simply not possible or is there a particular way for loading the test cases into the suite?

I think I tried all the different possibilities for loading the test cases into the test suite but the result was always the same.

Thanks in advance

Ilker

unittest.TestCase.subTest support (python 3.4+)

I am using subtests within some of my test methods, and xmlrunner doesn't fully support them.
It works just fine when all tests pass, but when one of them fails I get an error about not being able to get self.elapsed_time(), and no xml file is generated. Would it be possible to add this feature?

I also noticed that your runner is importing "unittest" for Python 3, when I am using unittest2. In xmlrunner/init.py, in the rewrite branch:

if sys.version_info < (2, 7):
    # monkey patch unittest
    import unittest2 as unittest

and this in the master branch for xmlrunner/unittest.py (I think I am using this, I downloaded it through pip)

if sys.version_info < (2, 7):
    # pylint: disable-msg=W0611
    import unittest2 as unittest

(...)
else:
# pylint: disable-msg=W0611
import unittest

I've also been looking through the code to figure out if this is something I could do myself, and it seems that what is missing is a parent context for the subtests. They should have an index attribute, and no elapsed time, since that is only recorded for the full test (is it?). Don’t really know where to start tho, I just started using unittest very recently.

Test cases in the test suite are not in the same order as run by Django

In the fix to the issue #12408 Django started explicitly ordering test cases in the test suite, so that all TransactionTestCases are run together. Because of the way they work they can leave the database in a random state. Ordering makes sure all other test cases are run before them.

There's a function in django.test.simple called reorder_suite which takes care of handling that, but currently djangotestrunner.XMLTestRunner is not using it. This causes behavioral mismatch between xmlrunner and stock django test runner.

1.7.0 prevents garbage collection by holding references to TestCase instances

I believe commit 31a49f4 from #33 regressed the change in #30.

Print from tests and buffered output on jenkins.

After tests start all print function are disabled and does not work. Which is confusing when you are use setUpClass and tearDownClass method to wait for external dependences.
And on jenkis verbose is buffered and displayed after all tests executions, not step by step like in unittest. Which is also confusing because you don't know that test is hangs up.

UnicodeDecodeError on install (Debian, Python3.3)

I'm getting the following error when installing on Python 3.3 on Debian:

pip3.3 install unittest-xml-reporting
[02:47:15][Step 1/1]  ---> Running in 032fd69a71d5
[02:47:16][Step 1/1] Downloading/unpacking unittest-xml-reporting
[02:47:17][Step 1/1]   Downloading unittest-xml-reporting-1.7.0.tar.gz
[02:47:17][Step 1/1]   Running setup.py (path:/tmp/pip_build_root/unittest-xml-reporting/setup.py) egg_info for package unittest-xml-reporting
[02:47:17][Step 1/1]     Traceback (most recent call last):
[02:47:17][Step 1/1]       File "<string>", line 17, in <module>
[02:47:17][Step 1/1]       File "/tmp/pip_build_root/unittest-xml-reporting/setup.py", line 11, in <module>
[02:47:17][Step 1/1]         exec(compile(open(xmlrunner_version).read(), xmlrunner_version, 'exec'))
[02:47:17][Step 1/1]       File "/usr/local/lib/python3.3/encodings/ascii.py", line 26, in decode
[02:47:17][Step 1/1]         return codecs.ascii_decode(input, self.errors)[0]
[02:47:17][Step 1/1]     UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 192: ordinal not in range(128)
[02:47:17][Step 1/1]     Complete output from command python setup.py egg_info:
[02:47:17][Step 1/1]     Traceback (most recent call last):
[02:47:17][Step 1/1] 
[02:47:17][Step 1/1]   File "<string>", line 17, in <module>
[02:47:17][Step 1/1] 
[02:47:17][Step 1/1]   File "/tmp/pip_build_root/unittest-xml-reporting/setup.py", line 11, in <module>
[02:47:17][Step 1/1] 
[02:47:17][Step 1/1]     exec(compile(open(xmlrunner_version).read(), xmlrunner_version, 'exec'))
[02:47:17][Step 1/1] 
[02:47:17][Step 1/1]   File "/usr/local/lib/python3.3/encodings/ascii.py", line 26, in decode
[02:47:17][Step 1/1] 
[02:47:17][Step 1/1]     return codecs.ascii_decode(input, self.errors)[0]
[02:47:17][Step 1/1] 
[02:47:17][Step 1/1] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 192: ordinal not in range(128)

processing Python doctests?

Would unittest-xml-reporting process Python doctests? I didn't seem to see references in the documentation. Thanks

import xmlrunner at python 3

howdy, I followed the instructions for installing the xmlrunner (git clone then setup.py) and I was able to import xmlrunner in python 2 but not in python 3. Is this an issue at the setup.py?

I want to give xmlrunner a try :)

Passing stream to test runner gives TypeError

Traceback (most recent call last):
  File "tests/test_autograde_runes.py", line 73, in <module>
    catchbreak=False)
  File "/usr/lib/python3.4/unittest/main.py", line 93, in __init__
    self.runTests()
  File "/usr/lib/python3.4/unittest/main.py", line 244, in runTests
    self.result = testRunner.run(self.test)
  File "/usr/local/lib/python3.4/dist-packages/xmlrunner/runner.py", line 111, in run
    result.generate_reports(self)
  File "/usr/local/lib/python3.4/dist-packages/xmlrunner/result.py", line 476, in generate_reports
    test_runner.output.write(xml_content)
TypeError: must be str, not bytes

One workaround (not sure if it is correct) is to change test_runner.output.write(xml_content) to test_runner.output.write(xml_content.decode('utf-8'))

no tar-ball on pypi

there seems to be no tar-ball on pypi for this release

https://pypi.python.org/pypi/unittest-xml-reporting/

Would be nice, if those were released too, so that, for example, linux distributions can easily create a package.

Thanks

Arun

XML reports not generated when running tests using unittest discovery

When I run tests using python -m unittest discover the generation of XML reports is not triggered.

This is assuming I have setup my test module like this:

import unittest
import xmlrunner
...

class SomeTest(unittest.TestCase):
...


if __name__ == '__main__':
    unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))

The reason seems to be that when unit tests are run using the discovery feature, main of module is not being run.

Is there another way of making this work?

Thanks for all your efforts.

Failures recorded as errors in XML

There appears to be a bug in src/xmlrunner/__init__.py Line 181 (_XMLTestResult.addFailure) that causes test failures (AssertionError) as errors.

_TestInfo.ERROR should be _TestInfo.FAILURE

Doing isinstance(foo, str) is not nice for Python 2.

I use unicode extensively in Python 2, so checking if an object is a str fails when the object is unicode. To determine if an object is a string, isinstance(obj, basestring) should be used. Example of culprit code:

https://github.com/danielfm/unittest-xml-reporting/blob/master/src/xmlrunner/__init__.py#L260

This will require enabling 2to3 however for Python 3.

In the xml report, time is the same with the time taken by last test case

In cmd:

Running tests...

testcase 01: enterAsGuest ... OK (1.001s)
testcase 03: signOut ... OK (2.001s)

Ran 2 tests in 3.008s

Generating XML reports...

In XML report
<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="SanityCheck" tests="2" time="4.002">
    <testcase classname="SanityCheck" name="testcase01_enterAsGuest" time="2.001"/>
    <testcase classname="SanityCheck" name="testcase03_signOut" time="2.001"/>
    <system-out>
<![CDATA[]]>    </system-out>
    <system-err>
<![CDATA[]]>    </system-err>
</testsuite>

=================================================================
I import a global variable time_takens to record  time taken by each test case.
update "_report_testcase" method in __init__.py:
testcase.setAttribute('time', '%.3f' % time_takens[index])

set Java package name for results

Could a parameter be added to set Java package name (and directory) for the XML files? Currently they seem to be placed into (root). Thanks

xmlrunner module is not installed for Python 3

I tried to install xmlrunner both from Debian package and through pip and I still have an error during import.

So this is what I have tried

sudo apt-get install python-xmlrunner
# this installs version 1.5.0-1

I also tried

sudo pip install unittest-xml-reporting
# Requirement already satisfied (use --upgrade to upgrade): unittest-xml-reporting in /usr/lib/python2.7/dist-packages

And finally the test

 python3
 >>> import xmlrunner
ImportError: No module named 'xmlrunner'

I tried it on both Debian Jessie and elementary OS (Freya) so am I doing something wrong? btw I would prefer installing from .deb package because I don't have to install pip on machines which doesn't have it.

Version 1.9.2 not tagged

Hello,
there is a 1.9.2 version on pypi but no corresponding tag in github...

When ]]> appears in failure/error/skipped message or on stdout/stderr, the formatter crashes

Following simple test snippet:

#!/usr/bin/env python
import unittest
import xmlrunner

class Foo(unittest.TestCase):
    def testFoo(self):
        self.assertTrue(False, ']]>')

unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))

results with unhandled exception:

ValueError: ']]>' not allowed in a CDATA section

Multiple CDATA sections can be added in a manner that will result with following final output:

        <error message="]]&gt;" type="AssertionError">
<![CDATA[Traceback (most recent call last):
  File "./t.py", line 7, in testFoo
    self.assertTrue(False, ']]]]><![CDATA[>')
AssertionError: ]]]]><![CDATA[>
]]>     </error>

DelegateIO is cool, but flush() is missing

I upgraded to xmlrunner 1.4.1 this morning, and a number of tests broke, because an underlying helper routine was performing sys.stdout.flush(), which isn't handled by _DelegateIO.

A simple addition of " def flush(self): pass " in the _DelegateIO class definition fixes the problem, though.

[Win64 via easy_install].

UnicodeEncodeError with non-ascii output in a test failure

Consider following failure:

======================================================================
FAIL [0.012s]: test_relation (nani.tests.related.NormalToTransFKTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jonas/workspace/project-nani/nani/tests/related.py", line 42, in test_relation
    self.assertEqual(related.normal_trans.translated_field, ja.translated_field)
AssertionError: u'English' != u'\u4f55'
- English+ \u4f55

unittest-xml-reporting now fails with this:

Generating XML reports...
Traceback (most recent call last):
  File "manage.py", line 16, in <module>
    execute_manager(settings)
  File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/core/management/commands/test.py", line 37, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/jonas/workspace/project-nani/testproject/.testenv13/lib/python2.6/site-packages/django/test/simple.py", line 363, in run_tests
    result = self.run_suite(suite)
  File "/home/jonas/workspace/project-nani/testproject/testrunner.py", line 16, in run_suite
    ).run(suite)
  File "/usr/local/lib/python2.6/dist-packages/xmlrunner/__init__.py", line 351, in run
    result.generate_reports(self)
  File "/usr/local/lib/python2.6/dist-packages/xmlrunner/__init__.py", line 269, in generate_reports
    report_file.write(xml_content)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u4f55' in position 280: ordinal not in range(128)

TypeError raised when a subtest ends successfully

When a subtest ends, the method result.addSubTest is called, even when no error is encountered. A TypeError is then raised, because the error is None. Here is the end of the stacktrace:

File "/opt/python-3.4.2/lib/python3.4/unittest/case.py", line 513, in _feedErrorsToResult
    result.addSubTest(test.test_case, test, exc_info)
File "/home/jenkins/venv/0/lib/python3.4/site-packages/xmlrunner/result.py", line 245, in addSubTest
    testinfo = _TestInfo(self, testcase, _TestInfo.ERROR, err, subTest=test)
File "/home/jenkins/venv/0/lib/python3.4/site-packages/xmlrunner/result.py", line 102, in __init__
    self.err, test_method)
File "/opt/python-3.4.2/lib/python3.4/unittest/result.py", line 173, in _exc_info_to_string
    exctype, value, tb = err
TypeError: 'NoneType' object is not iterable

Hence, the error must be converted to string only if it is not None (https://hg.python.org/cpython/file/0be7c8f46378/Lib/unittest/result.py#l127).

XMLTestRunner not honoring failfast

Based on examining TextTestRunner, it appears that after creating the result object in XMLTestRunner.run, there should be a result.failfast = self.failfast.

Setting properties for multiple test suites

Hi!, Im trying to get an XML report with the following structure:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
   <testsuite errors="0" failures="0" name="admin.admin_spots_tests.AdminSpotsTests" skipped="0" tests="1" time="2.235">
      <properties>
         <property name="test_create_spot" value="101" />
      </properties>
      <testcase classname="admin.admin_spots_tests.AdminSpotsTests" name="test_create_spot" time="2.235" />
   </testsuite>
   <testsuite errors="0" failures="0" name="rentals.rentals_search_tests.RentalsSearchTests" skipped="0" tests="1" time="2.198">
      <properties>
         <property name="test_search_by_rental_id" value="201" />
      </properties>
      <testcase classname="rentals.rentals_search_tests.RentalsSearchTests" name="test_search_by_rental_id" time="2.198" />
   </testsuite>
</testsuites>

Where I want to set in the test suite properties, a list of IDs (for my test case management system).
I am creating a test suite (master test suite) and I add multiple test suites (child test suites) that contain one or many test cases.

The problem is that when I set the properties for a child test suite, its not showing in the XML.
If I set the properties of the master test suite, it does show but it adds the same property tags to every child test suite.
Any help is appreciated, thanks!

-Eduardo Reynoso

print to stdout AND capture to junit XML file

I'd like to print to stdout when debugging my tests, but do not see a way to do this. Is there an option I am missing?
I see you use _patch_standard_output() and _restore_standard_ouput(), but am not sure what to change to enable this.

`XMLTestRunner` prevents garbage collection by holding references to all the TestCase instances whose test methods it executes, via `_XMLTestResult` and `_TestInfo`.

XMLTestRunner holds references to all the TestCase instances whose test methods it executes, via _XMLTestResult and _TestInfo.

I'm finding that problematic in a large suite of Django functional tests, where the TestCase instances typically grow to 50 MB each. The held references prevent the garbage collector from freeing their memory until the entire test suite has run. With a large enough test suite, the OS's OOM Killer kills the test process before it can finish.

I started a conversation on django-users about the problem more generally, and I wouldn't be surprised if there are related problems in the underlying unittest package.

Here's a patch that I've been using to investigate. I'm not advocating merging this as is. Just including it to make what I'm describing more concrete.

--- __init__.py~    2013-05-02 16:45:19.000000000 -0400
+++ __init__.py 2013-07-18 21:39:13.000000000 -0400
@@ -8,6 +8,9 @@
 import os
 import sys
 import time
+
+import psutil
+
 from unittest import TestResult, _TextTestResult, TextTestRunner

 try:
@@ -27,13 +30,24 @@
         self.delegate = delegate

     def write(self, text):
-        self._captured.write(text)
+        #self._captured.write(text)
         self.delegate.write(text)

     def __getattr__(self, attr):
         return getattr(self._captured, attr)


+def testcase_name(test_method):
+    testcase = type(test_method)
+
+    # Ignore module name if it is '__main__'
+    module = testcase.__module__ + '.'
+    if module == '__main__.':
+        module = ''
+    result = module + testcase.__name__
+    return result
+
+
 class _TestInfo(object):
     """This class keeps useful information about the execution of a
     test method.
@@ -44,11 +58,21 @@

     def __init__(self, test_result, test_method, outcome=SUCCESS, err=None):
         self.test_result = test_result
-        self.test_method = test_method
         self.outcome = outcome
         self.elapsed_time = 0
         self.err = err

+        #self.test_method = test_method
+        self.test_description = self.test_result.getDescription(test_method)
+        self.test_exception_info = (
+            '' if not self.err
+            else self.test_result._exc_info_to_string(
+                    self.err, test_method)
+        )
+
+        self.test_name = testcase_name(test_method)
+        self.test_id = test_method.id()
+
     def test_finished(self):
         """Save info that can only be calculated once a test has run.
         """
@@ -58,16 +82,18 @@
     def get_description(self):
         """Return a text representation of the test method.
         """
-        return self.test_result.getDescription(self.test_method)
+        #return self.test_result.getDescription(self.test_method)
+        return self.test_description

     def get_error_info(self):
         """Return a text representation of an exception thrown by a test
         method.
         """
-        if not self.err:
-            return ''
-        return self.test_result._exc_info_to_string(self.err, \
-            self.test_method)
+        #if not self.err:
+        #    return ''
+        #return self.test_result._exc_info_to_string(self.err, \
+        #    self.test_method)
+        return self.test_exception_info


 class _XMLTestResult(_TextTestResult):
@@ -81,6 +107,8 @@
         self.successes = []
         self.callback = None
         self.elapsed_times = elapsed_times
+        self.start_meminfo = psutil.Process(os.getpid()).get_memory_info()
+        self.stop_meminfo = psutil.Process(os.getpid()).get_memory_info()

     def _prepare_callback(self, test_info, target_list, verbose_str,
         short_str):
@@ -103,6 +131,10 @@
             if self.showAll:
                 self.stream.writeln('%s (%.3fs)' % \
                     (verbose_str, test_info.elapsed_time))
+                self.stop_meminfo = psutil.Process(os.getpid()).get_memory_info()
+                self.delta_stop_meminfo = { 'rss': self.stop_meminfo.rss - self.start_meminfo.rss,
+                                            'vms': self.stop_meminfo.vms - self.start_meminfo.vms }
+                self.stream.write('on stop: status: {0}, delta: {1}\n'.format(self.stop_meminfo, self.delta_stop_meminfo))
             elif self.dots:
                 self.stream.write(short_str)
         self.callback = callback
@@ -114,6 +146,10 @@
         TestResult.startTest(self, test)

         if self.showAll:
+            self.start_meminfo = psutil.Process(os.getpid()).get_memory_info()
+            self.delta_start_meminfo = { 'rss': self.start_meminfo.rss - self.stop_meminfo.rss,
+                                         'vms': self.start_meminfo.vms - self.stop_meminfo.vms }
+            self.stream.write('on start: status: {0}, delta: {1}\n'.format(self.start_meminfo, self.delta_start_meminfo))
             self.stream.write('  ' + self.getDescription(test))
             self.stream.write(" ... ")

@@ -165,14 +201,7 @@

         for tests in (self.successes, self.failures, self.errors):
             for test_info in tests:
-                testcase = type(test_info.test_method)
-
-                # Ignore module name if it is '__main__'
-                module = testcase.__module__ + '.'
-                if module == '__main__.':
-                    module = ''
-                testcase_name = module + testcase.__name__
-
+                testcase_name = test_info.test_name
                 if not testcase_name in tests_by_testcase:
                     tests_by_testcase[testcase_name] = []
                 tests_by_testcase[testcase_name].append(test_info)
@@ -201,10 +230,9 @@

     _report_testsuite = staticmethod(_report_testsuite)

-    def _test_method_name(test_method):
+    def _test_method_name(test_id):
         """Returns the test method name.
         """
-        test_id = test_method.id()
         return test_id.split('.')[-1]

     _test_method_name = staticmethod(_test_method_name)
@@ -217,7 +245,7 @@

         testcase.setAttribute('classname', suite_name)
         testcase.setAttribute('name', \
-            _XMLTestResult._test_method_name(test_result.test_method))
+            _XMLTestResult._test_method_name(test_result.test_id))
         testcase.setAttribute('time', '%.3f' % test_result.elapsed_time)

         if (test_result.outcome != _TestInfo.SUCCESS):

UnicodeDecodeError(Mac OS X 10.9.4/Python 2.7.5)

Hi DEV,
I used xmlrunner in unittest, I have a problem about Unicode now, error log is as follows:

Generating XML reports...
Traceback (most recent call last):
  File "/Users/kingson/PycharmProjects/DDYAuto/src/TestSuites/login/TSTestLogin.py", line 69, in <module>
    unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 95, in __init__
    self.runTests()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 232, in runTests
    self.result = testRunner.run(self.test)
  File "/Library/Python/2.7/site-packages/xmlrunner/xmlrunner.py", line 466, in run
    result.generate_reports(self)
  File "/Library/Python/2.7/site-packages/xmlrunner/xmlrunner.py", line 341, in generate_reports
    _XMLTestResult._report_testcase(suite, test, testsuite, doc)
  File "/Library/Python/2.7/site-packages/xmlrunner/xmlrunner.py", line 294, in _report_testcase
    error_info = unicode(test_result.get_error_info())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 179: ordinal not in range(128)

Maybe result include Chinese characters, Can you help me? Thanks.

`rewrite` branch does not work with python 2.6

python 2.6 does not have the same features in unittest.
You can use unittest2 and monkey-patch unittest but unittest2 has not been updated in a few years and is not in sync with the one available in python dev branch.

Problematic:

unittest.result
unittest.TextTestResult (was _TextTestResult)
unittest.skip
unittest.util
...

pdb doesn't output under test runner

When setting a pdb trace in a test and running the test with unittest-xml-reporting as the test runner, pdb output is not displayed.

Please release a new version of unittest-xml-reporting on pypi that contains 34633e2d221ecec4faacc8fdac7403c50fbc1ea1

Please release a new version of unittest-xml-reporting on pypi that contains 34633e2. Thank you in advance.

sys.stdout has no attribute delegate

I'm using xml-reporting to run our standard python unittest suite. The tests themselves run perfectly fine all the way to the end but then we get an exception when it goes on to output the xml.

Any ideas on how we get around this or if there is a fix for it? We are running Python 2.7.6 on Ubuntu 14.04 if that matters.

This is the full full exception we see(snipped the html portion of the output):

Ran 2463 tests in 132.803s

OK

Generating XML reports...
..........html snipped...........

<!-- The above is a description of an error in a Python program, formatted
     for a Web browser because the 'cgitb' module was enabled.  In case you
     are not reading this in a Web browser, here is the original traceback:

Traceback (most recent call last):
  File "./full_test.py", line 48, in &lt;module&gt;
    xmlrunner.XMLTestRunner(output=file('./test-reports/report.xml', 'w')).run(testRun.getTestSuite())
  File "build/bdist.linux-x86_64/egg/xmlrunner/xmlrunner.py", line 468, in run
    self._restore_standard_output()
  File "build/bdist.linux-x86_64/egg/xmlrunner/xmlrunner.py", line 397, in _restore_standard_output
    sys.stdout = sys.stdout.delegate
AttributeError: 'file' object has no attribute 'delegate'

-->

unittest options don't work

When you run python tests.py -h you get a nice little help page with neat options (such as output buffering and verbosity). These work fine when using the default runner (unittest.main()). However, if I use xmlrunner as shown in the readme file, these options don't work anymore (although the help page still displays them).

Since this behavior is very confusing (and misleading), you should either remove the options from the help page or add these option to the xmlrunner. I really like output buffering but it apparently isn't available with xmlrunner.

Drop Python 2.6 support

README says "Please note Python 2.6 end-of-life was in Oct 2013, and support will be dropped in next major version." Which version is the next major version? Can the work be done on the master branch now?

AttributeError in result._save_output_data

I'm facing the following exception when executing my tests:

  File "ve/lib/python2.7/site-packages/django/test/runner.py", line 115, in run_suite
    ).run(suite)
  File "ve/lib/python2.7/site-packages/xmlrunner/runner.py", line 59, in run
    test(result)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 70, in __call__
    return self.run(*args, **kwds)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/suite.py", line 108, in run
    test(result)
  File "ve/lib/python2.7/site-packages/django/test/testcases.py", line 184, in __call__
    result.addError(self, sys.exc_info())
  File "ve/lib/python2.7/site-packages/xmlrunner/result.py", line 207, in addError
    self._save_output_data()
  File "ve/lib/python2.7/site-packages/xmlrunner/result.py", line 164, in _save_output_data
    self._stdout_data = sys.stdout.getvalue()
AttributeError: 'file' object has no attribute 'getvalue'

backwards compatibility broken with Django1.2

see my fork for the fix:
https://github.com/markotibold/unittest-xml-reporting/commit/fe1ce6b0e6700d164cd476208759fd5ad3bd483a

Destructor not executed after execution of a single test

Running a test with the xml test runner differs from running a test with the default text runner. In the default setup after each test local objects are deleted. With the xml test runner the destructors are called after test completion. It is important to rise an assertionError in the test to get this error.

import xmlrunner
import unittest

class dummy(object):
    def __init__(self, inst):
        self.inst = inst
        print("ctro, inst %s"%inst)

    def __del__(self):
        print("dtro, inst %s"%self.inst)

class test_spuCfgParser(unittest.TestCase):

    def test_foo(self):
        print("test_foo")
        dummyInst = dummy("foo")
        raise AssertionError("SomeError")


    def test_foofoo(self):
        print("test_foofoo")
        dummyInst = dummy("foofoo")
        raise AssertionError("SomeError")


if __name__ == '__main__':
    #unittest.main()

    unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'), failfast=False, buffer=False, catchbreak=False)

Output with unittest.main():

test_foo
ctro, inst foo
Fdtro, inst foo
test_foofoo
ctro, inst foofoo
Fdtro, inst foofoo

Output with xmlrunner:

Running tests...
----------------------------------------------------------------------
FF
======================================================================
FAIL [0.000s]: test_foo (__main__.test_spuCfgParser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "debugxmlrunner.py", line 17, in test_foo
    raise AssertionError("SomeError")
AssertionError: SomeError

Stdout:
test_foo
ctro, inst foo

======================================================================
FAIL [0.000s]: test_foofoo (__main__.test_spuCfgParser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "debugxmlrunner.py", line 23, in test_foofoo
    raise AssertionError("SomeError")
AssertionError: SomeError

Stdout:
test_foofoo
ctro, inst foofoo

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (failures=2)

Generating XML reports...
dtro, inst foofoo
dtro, inst foo

When running in Jython I get the following error UnicodeDecodeError ( 'unicodeescape' codec can't decode bytes in position 29-35: illegal Unicode character )

I'm working on submitting a patch for this. Here is the offending line in result.py
u'[\x00-\x08\x0B\x0C\x0E-\x1F\uD800-\uDFFF\uFFFE\uFFFF]'

Incorrectly checking type of 'XMLTestRunner.output'

https://github.com/danielfm/unittest-xml-reporting/blob/master/src/xmlrunner/__init__.py#L224

It should use:

if isinstance(test_runner.output, basestring) ...

@unittest.skip("CPUxxx does not support any SD-Card") breaks test scripts

Configuration:
Python 2.7.5
unittest-xml-reporting 1.6.0

.Traceback (most recent call last):
File "TestScript.py", line 539, in
xmlrunner.XMLTestRunner().run(all_tests)
File "C:\Programme\Python27\lib\site-packages\xmlrunner__init__.py", line 395, in run
test(result)
File "C:\Programme\Python27\lib\unittest\suite.py", line 70, in call
return self.run(_args, *_kwds)
File "C:\Programme\Python27\lib\unittest\suite.py", line 108, in run
test(result)
File "C:\Programme\Python27\lib\unittest\suite.py", line 70, in call
return self.run(_args, *_kwds)
File "C:\Programme\Python27\lib\unittest\suite.py", line 108, in run
test(result)
File "C:\Programme\Python27\lib\unittest\case.py", line 395, in call
return self.run(_args, *_kwds)
File "C:\Programme\Python27\lib\unittest\case.py", line 315, in run
self.addSkip(result, skip_why)
File "C:\Programme\Python27\lib\unittest\case.py", line 291, in addSkip
addSkip(self, reason)
File "C:\Programme\Python27\lib\site-packages\xmlrunner__init.py", line 191, in addSkip
TestInfo(self, test, TestInfo.SKIP, err),
File "C:\Programme\Python27\lib\site-packages\xmlrunner__init.py", line 72, in init
self.err, test_method)
File "C:\Programme\Python27\lib\unittest\result.py", line 154, in _exc_info_to_string
exctype, value, tb = err
ValueError: too many values to unpack

Doesn't handle 2.7 setUpClass()/tearDownClass()

When using xmlrunner with python 2.7's unittest framework, xmlrunner suppresses python errors in setUpClass() and tearDownClass() functions.
Running unittest framework with xmlrunner:

unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test_results'))

Output:

Running tests...
----------------------------------------------------------------------

======================================================================
Traceback (most recent call last):
  File "./test_tpa.py", line 587, in <module>
    unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test_results'))
  File "/usr/lib/python2.7/unittest/main.py", line 95, in __init__
    self.runTests()
  File "/usr/lib/python2.7/unittest/main.py", line 229, in runTests
    self.result = testRunner.run(self.test)
  File "/usr/lib/python2.7/site-packages/xmlrunner/__init__.py", line 300, in run
    result.printErrors()
  File "/usr/lib/python2.7/unittest/runner.py", line 108, in printErrors
    self.printErrorList('ERROR', self.errors)
  File "/usr/lib/python2.7/site-packages/xmlrunner/__init__.py", line 122, in printErrorList
    (flavour, test_info.get_elapsed_time(), \
  File "/usr/lib/python2.7/site-packages/xmlrunner/__init__.py", line 33, in get_elapsed_time
    return self.test_result.stop_time - self.test_result.start_time
AttributeError: '_XMLTestResult' object has no attribute 'stop_time'

Running without xmltestrunner:

unittest.main()

Output:

E
======================================================================
ERROR: setUpClass (__main__.Test_TPA)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test_tpa.py", line 36, in setUpClass
    blah.blah()
NameError: global name 'blah' is not defined

----------------------------------------------------------------------
Ran 0 tests in 0.000s

FAILED (errors=1)

Descriptions are missing

Hi.

I'm trying to output the descriptions in the xml file, but they don't show up.

I just added descriptions=True to xmlrunner.XMLTestRunner. Is this correct?

# -*- coding: utf-8 -*-

import unittest
import sys
import xmlrunner
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from parametrizedTestCase import ParametrizedTestCase

TESTS_NAME = "Busca"

CAP_FIREFOX = {'browser': 'Firefox', 'browser_version': '21.0',
               'os': 'Windows', 'os_version': '7', 'resolution': '1024x768'}
CAP_IE = {'browser': 'IE', 'browser_version': '10.0',
          'os': 'Windows', 'os_version': '7', 'resolution': '1024x768'}
CAP_CHROME = {'browser': 'Chrome', 'browser_version': '18.0',
              'os': 'Windows', 'os_version': '7', 'resolution': '1024x768'}
CAP_SAFARI = {'browser': 'Safari', 'browser_version': '8.0',
              'os': 'OS X', 'os_version': 'Yosemite', 'resolution': '1024x768'}

# pylint: disable=R0904
class CompanyTestes(ParametrizedTestCase):
    """Classe com testes unitários para a busca de passagens do site."""

    def local_setup(self):
        self.driver = webdriver.Chrome("./chromedriver.exe")

    # Setup no BrowserStack
    def remote_setup(self):
        desired_cap = self.param
        desired_cap['browserstack.video'] = False
        desired_cap['browserstack.debug'] = True
        desired_cap['project'] = 'ProjetoCompany'
        desired_cap['name'] = TESTS_NAME + " - " + unittest.TestCase.shortDescription(self)

        # webdriver.Remote credentials - removed

    def setUp(self):
        self.local_setup()
        # self.remote_setup()

    def tearDown(self):
        self.driver.quit()

    # @unittest.skip("Pulando...")
    def test_hi(self):
        """Description that is supposed to show up in the xml file."""
        driver = self.driver

if __name__ == "__main__":
    suite = unittest.TestSuite()
    suite.addTest(ParametrizedTestCase.parametrize(CompanyTestes, param=CAP_SAFARI))
    # suite.addTest(ParametrizedTestCase.parametrize(CompanyTestes, param=CAP_CHROME))
    # suite.addTest(ParametrizedTestCase.parametrize(CompanyTestes, param=CAP_FIREFOX))
    # suite.addTest(ParametrizedTestCase.parametrize(CompanyTestes, param=CAP_IE))
    # runner = xmlrunner.XMLTestRunner(output='test-reports')
    runner = xmlrunner.XMLTestRunner(output='test-reports', descriptions=True, verbosity=1)
    runner.run(suite)

Also, I need this class to inject parameters to setUp. Not sure if this is relevant.

parametrizedTestCase.py:

import unittest

# http://eli.thegreenplace.net/2011/08/02/python-unit-testing-parametrized-test-cases/
class ParametrizedTestCase(unittest.TestCase):
    """ TestCase classes that want to be parametrized should
        inherit from this class.
    """
    def __init__(self, methodName='runTest', param=None):
        super(ParametrizedTestCase, self).__init__(methodName)
        self.param = param

    @staticmethod
    def parametrize(testcase_klass, param=None):
        """ Create a suite containing all tests taken from the given
            subclass, passing them the parameter 'param'.
        """
        testloader = unittest.TestLoader()
        testnames = testloader.getTestCaseNames(testcase_klass)
        suite = unittest.TestSuite()
        for name in testnames:
            suite.addTest(testcase_klass(name, param=param))
        return suite

Output:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite errors="0" failures="0" name="CompanyTestes-20161014105524" skipped="0" tests="1" time="7.086">
    <testcase classname="CompanyTestes" name="test_hi" time="7.086"/>
    <system-out>
<![CDATA[]]>    </system-out>
    <system-err>
<![CDATA[]]>    </system-err>
</testsuite>

Thanks.

Add param to the module to set the output directory

What about to add a param like --xmloutput dir to the module __main__.py so as it may be run as:

python -m xmlrunner  --xmloutput test-reports test_module

I may submit a PR if you agree with this change.

Python versions

In the readme, requirements ask for Python 2.7+ but it is currently successfully tested on python2.6.

The tox.ini should also specify python 3.4

Version conflicts with Django 1.1.1 and python 2.6.3.

I couldn't get any current version of this to work, either from this repo, or the fork by daspecter. Here's what I did to make it work, with some background detail:

http://opensourcemissions.wordpress.com/2011/03/04/resolving-xmlrunner-conflicts-with-django-1-1-1-and-python-2-6-3/

xml stylesheets

I am viewing unittests reports in Circle CI.

It is not pretty, because there are no stylesheets.

Is there a way to add a stylesheet to the reports?

Django 1.6 compatibility

Hello,

the Django TestRunner has changed in django 1.6: https://docs.djangoproject.com/en/1.6/topics/testing/overview/#running-tests

With django 1.6 beta, the XMLTestRunner looks for tests in all INSTALLED_APPS, but not in 1.6.
Is there a plan to support both < 1.6 and > 1.6 ?

stdout / stderr tags contain output of the whole testsuite, instead of testcase only

Hi!

Thanks a lot for a useful runner!

I have a problem, though.

I have a big test suite that contains a number of test cases, each of those contains several tests. When I use your test runner, I get as much XML files as I have test cases and each of those files contain the results of the tests, that pertain to this test case (perfect!)

However, in each of those XML files the stdout / stderr of the whole test suite is recorded. As far as I understand the specification, the stderr / stdout tags should contain only output related to the tests in this XML file, not all tests that were ever run.

Note, that this is not the same as #1. I don't want stdout / stderr per individual test, which is against the specification, but rather the XML to contain only the output that was generated by the tests that it ran.

Thanks!

xmlrunner / unittest-xml-reporting Goto Github PK

unittest-xml-reporting's People

Contributors

Stargazers

Watchers

Forkers

unittest-xml-reporting's Issues

Running tests...

Recommend Projects

Recommend Topics

Recommend Org