Now infection tries to change visibility modifiers for all unit tests. It looks like this

--- Original
+++ New
@@ @@
-    public function test0()
+    protected function test0()

After that infection marks this mutant as escaped so it pollutes log because it is false positive.

If I listen to this advice and change visibility modifiers, I will disable this test because phpunit can't call it.

See #102

Make Humbug's config compatible with Infection to transition users without any efforts (probably just by renaming the config)

<phpunit>
   <testsuites>
       <testsuite name="app">
           <directory>app/Tests</directory>
       </testsuite>
   </testsuites>
</phpunit>

Creating mutated files and processes: 0/4PHP Warning:  Creating default object from empty value in /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/src/TestFramework/PhpUnit/Config/Builder/MutationConfigBuilder.php on line 106
PHP Stack trace:
PHP   1. {main}() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/bin/infection:0
PHP   2. Symfony\Component\Console\Application->run() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/bin/infection:34
PHP   3. Symfony\Component\Console\Application->doRun() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/symfony/console/Application.php:122
PHP   4. Symfony\Component\Console\Application->doRunCommand() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/symfony/console/Application.php:216
PHP   5. Symfony\Component\Console\Command\Command->run() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/symfony/console/Application.php:858
PHP   6. Infection\Command\InfectionCommand->execute() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/symfony/console/Command/Command.php:240
PHP   7. Infection\InfectionApplication->run() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/src/Command/InfectionCommand.php:135
PHP   8. Infection\Process\Runner\MutationTestingRunner->run() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/src/InfectionApplication.php:109
PHP   9. array_map() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/src/Process/Runner/MutationTestingRunner.php:73
PHP  10. Infection\Process\Runner\MutationTestingRunner->Infection\Process\Runner\{closure}() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/src/Process/Runner/MutationTestingRunner.php:73
PHP  11. Infection\Process\Builder\ProcessBuilder->getProcessForMutant() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/src/Process/Runner/MutationTestingRunner.php:67
PHP  12. Infection\TestFramework\AbstractTestFrameworkAdapter->buildMutationConfigFile() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/src/Process/Builder/ProcessBuilder.php:62
PHP  13. Infection\TestFramework\PhpUnit\Config\Builder\MutationConfigBuilder->build() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/src/TestFramework/AbstractTestFrameworkAdapter.php:90
PHP  14. Infection\TestFramework\PhpUnit\Config\Builder\MutationConfigBuilder->setCustomAutoLoaderPath() /Users/massimo.chiarillo/Projects/symfonycon/infection/vendor/infection/infection/src/TestFramework/PhpUnit/Config/Builder/MutationConfigBuilder.php:63

Repro Steps

• git clone [email protected]:ChiarilloMassimo/infection-test.git
• Remove bootstrap="vendor/autoload.php" from phpunit.xml
• ./vendor/infection/infection/bin/infection run

Fix 🤔

I do not know if this could help massimo-me@f850943 I tried it fast in SymfonyCon Cluj 2017 🇷🇴

Running initial test suite...

Phpunit version: 6.4.4

   7 [============================]  1 sec

Generate mutants...

Processing source code files: 1/1
Creating mutated files and processes: 4/4
.: killed, M: escaped, S: uncovered, E: fatal error, T: timed out

....                                                 (4 / 4)

4 mutations were generated:
      4 mutants were killed
      0 mutants were not covered by tests
      0 covered mutants were not detected
      0 time outs were encountered

Metrics:
        Mutation Score Indicator (MSI): 100%
        Mutation Code Coverage: 100%
        Covered Code MSI: 100%

Running initial test suite...

Phpunit version: 6.4.4

   6 [============================] < 1 sec

Generate mutants...

Processing source code files: 1/1
Creating mutated files and processes: 4/4
.: killed, M: escaped, S: uncovered, E: fatal error, T: timed out

MMMM                                                 (4 / 4)

4 mutations were generated:
      0 mutants were killed
      0 mutants were not covered by tests
      4 covered mutants were not detected
      0 time outs were encountered

Metrics:
        Mutation Score Indicator (MSI): 0%
        Mutation Code Coverage: 100%
        Covered Code MSI: 0%

Please note that some mutants will inevitably be harmless (i.e. false positives).

One of the greatest point of mutation testing is that it provides a reliable way to know which part of your application is absolutely tested. A bad point is that it's horribly slow as it's a brute force technique. An easy way to get better performances is to simply play with the number of mutators applied. Indeed applying only 10 mutators is gonna result in a much faster execution time than 100 mutators.

IMO there is a lot of mutators that may not be useful depending of the context in which Infection is used. For example I could care less about the visibility mutator in my application and find it much more useful in my library.

Point is, I think it would be a good idea to provide a way for the user to define which mutators can be applied in the application and maybe providing different default sets. For example we could call that parameter level and depending of the value, we apply more or less mutators like for example PhpStan does with their analysis rules.

I'm really liking this project so far, and it should be a big help moving forward with tests for me. However one of the issues i find is that the following diff:

12) Infection\Mutator\Boolean\FalseValue
/Users/Me/Projects/project/classes/Domain/Model/User.php
exec /usr/local/Cellar/php71/7.1.11_22/bin/php /Users/Me/Projects/project/vendor/phpunit/phpunit/phpunit --configuration /var/folders/5y/g0yc55n12hjcc4zb0k3z2x_m0000gn/T/infection/phpunitConfiguration.dc85538a77301d5b6fb62b5c77d33083.infection.xml --stop-on-failure 
--- Original
+++ New
@@ @@
-                return false;
+                return true;

PHPUnit 6.4.4 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.1.11 with Xdebug 2.5.5
Configuration: /private/var/folders/5y/g0yc55n12hjcc4zb0k3z2x_m0000gn/T/infection/phpunitConfiguration.dc85538a77301d5b6fb62b5c77d33083.infection.xml

...................                                               19 / 19 (100%)

Time: 4.54 seconds, Memory: 110.00MB

OK (19 tests, 39 assertions)

Doesn't give me too much information about what change didn't get caught by tests. Of course it tells me in what file the change happened, but these can have a lot of return return false; statements.

On mutants that aren't escaped these get displayed normally

I'm not sure I should expand too much on why having too many dev dependency is an issue, if that's the case please tell me I'll update that part accordingly. To tack this issue I would like to do the two following suggestions:

A. Use PhpScoper for the PHAR.
B. Split this package in two

A) The code bundled in a PHAR is not isolated (more here) which may be an issue. PhpScoper solves that.

B) There is two cases here.

The first one is I think installing Infection as a composer dependency (I mean at a project level not globally) is a perfectly valid case. Like PhpUnit, it executes code. As a result you need to make sure you PhpUnit/Infection dependencies are compatible with your application dependencies, which is what Composer is about. Having a completely isolated PHAR is IMO the only viable alternative.

Providing an extension point. Let me extend on that. Even if we manage to isolate the PHAR, a feature I would love to see is providing an extension point to let people define their own mutants. To do so, we have two choice: require the user to use Infection as a dev dependency which IMO may easily be an issue due to the number of dependencies Infection has, especially ones like PhpParser that are often source of conflicts that takes ages to fix. The second choice, much cleaner IMO, is to adopt the same strategy as Composer with its plugins: expose an API which would be super lightweight, it just provides the necessary code to allow one to define its own mutant. Then the PHAR when executing can locate those custom mutants.

Related article: managing your PHP dependencies

First of all, thank you for this great testing framework! It really helps to find weaknesses in unit tests very quickly! 🎉

When writing tests, I often find myself disabling one code line after another to see if it is tested properly and not only executed without testing the outcome.

Do you think it would make sense to add a mutator that removes whole lines of code (or statements) to see if their effect is tested? This would find all untested statements that don’t fall into a category of the other mutators.

Running vendor/bin/infection crashes if one uses a custom printer.
In the linked repo we have printerClass="Codedungeon\PHPUnitPrettyResultPrinter\Printer" in our phpunit.xml.dist
The current workaround is to add --test-framework-options="--printer PHPUnit\\\TextUI\\\ResultPrinter". But it would be nice if infection could detect that on its own

Unfortunately I can't make it works:

D:\Projects\php\generic-storage>php D:\Tools\infection.phar

    0 [>---------------------------] < 1 secRunning initial test suite...


    1 [->--------------------------] < 1 sec

Generate mutants...

Processing source code files:  0/14
( .... 1-13 skipped ...)
Processing source code files: 14/14Creating mutated files and processes:   0/225

Warning: file_get_contents(C:\Users\oz\AppData\Local\Temp/infection/coverage-xml/index.xml): failed to open stream: No such file or directory in phar://D:/Tools/infection.phar/src/TestFramework/Coverage/CodeCoverageData.php on line 147

Call Stack:
    0.0035     681144   1. {main}() D:\Tools\infection.phar:0
    0.0045     687160   2. require('phar://D:/Tools/infection.phar/bin/infection') D:\Tools\infection.phar:10
    0.0160    1847160   3. Symfony\Component\Console\Application->run() phar://D:/Tools/infection.phar/bin/infection:33
    0.0525    2007496   4. Symfony\Component\Console\Application->doRun() phar://D:/Tools/infection.phar/vendor/symfony/console/Application.php:127
    0.0530    2073032   5. Symfony\Component\Console\Application->doRunCommand() phar://D:/Tools/infection.phar/vendor/symfony/console/Application.php:220
    0.0530    2073032   6. Symfony\Component\Console\Command\Command->run() phar://D:/Tools/infection.phar/vendor/symfony/console/Application.php:866
    0.0535    2078560   7. Infection\Command\InfectionCommand->execute() phar://D:/Tools/infection.phar/vendor/symfony/console/Command/Command.php:261
    0.0540    2131368   8. Infection\InfectionApplication->run() phar://D:/Tools/infection.phar/src/Command/InfectionCommand.php:115
    2.0333   10345128   9. Infection\Process\Runner\MutationTestingRunner->run() phar://D:/Tools/infection.phar/src/InfectionApplication.php:100
    2.0338   10347840  10. array_map() phar://D:/Tools/infection.phar/src/Process/Runner/MutationTestingRunner.php:73
    2.0338   10347896  11. Infection\Process\Runner\MutationTestingRunner->Infection\Process\Runner\{closure}() phar://D:/Tools/infection.phar/src/Process/Runner/MutationTestingRunner.php:73
    2.0338   10347896  12. Infection\Mutant\MutantCreator->create() phar://D:/Tools/infection.phar/src/Process/Runner/MutationTestingRunner.php:65
    2.0413   11533384  13. Infection\Mutant\MutantCreator->isCoveredByTest() phar://D:/Tools/infection.phar/src/Mutant/MutantCreator.php:66
    2.0413   11533384  14. Infection\TestFramework\Coverage\CodeCoverageData->hasExecutedMethodOnLine() phar://D:/Tools/infection.phar/src/Mutant/MutantCreator.php:88
    2.0413   11533384  15. Infection\TestFramework\Coverage\CodeCoverageData->getCoverage() phar://D:/Tools/infection.phar/src/TestFramework/Coverage/CodeCoverageData.php:83
    2.0413   11533480  16. file_get_contents() phar://D:/Tools/infection.phar/src/TestFramework/Coverage/CodeCoverageData.php:147


Fatal error: Uncaught TypeError: Argument 1 passed to Infection\TestFramework\PhpUnit\Coverage\CoverageXmlParser::parse() must be of the type string, boolean given, called in phar://D:/Tools/infection.phar/src/TestFramework/Coverage/CodeCoverageData.php on line 149 and defined in phar://D:/Tools/infection.phar/src/TestFramework/PhpUnit/Coverage/CoverageXmlParser.php on line 28

TypeError: Argument 1 passed to Infection\TestFramework\PhpUnit\Coverage\CoverageXmlParser::parse() must be of the type string, boolean given, called in phar://D:/Tools/infection.phar/src/TestFramework/Coverage/CodeCoverageData.php on line 149 in phar://D:/Tools/infection.phar/src/TestFramework/PhpUnit/Coverage/CoverageXmlParser.php on line 28

Call Stack:
    0.0035     681144   1. {main}() D:\Tools\infection.phar:0
    0.0045     687160   2. require('phar://D:/Tools/infection.phar/bin/infection') D:\Tools\infection.phar:10
    0.0160    1847160   3. Symfony\Component\Console\Application->run() phar://D:/Tools/infection.phar/bin/infection:33

It seems like it's trying to read code coverage results but cant find them.

Dir C:\Users\oz\AppData\Local\Temp\infection contains only 3 files:

mutant.c386525a8f162700e34fd3cb5d419d64.infection.php
phpunit.junit.xml
phpunitConfiguration.initial.infection.xml

There is no ./coverage-xml/ subdir.

Here is my infection.json (generated):

{
    "timeout": 10,
    "source": {
        "directories": [
            "src"
        ]
    },
    "logs": {
        "text": "infection-log.txt"
    }
}

And phpunit.xml:

<phpunit bootstrap="bootstrap.php">
    <testsuites>
        <testsuite name="Unit Tests">
            <directory suffix="Test.php" phpVersion="7.1.0" phpVersionOperator=">=">tests</directory>
        </testsuite>
    </testsuites>
    <filter>
        <whitelist>
            <directory>src</directory>
        </whitelist>
    </filter>
</phpunit>

Should I add/fix some parameters? Or should I run phpunit code coverage by myself with specific options?

I'd like to propose a simple way to support other testing tools.

Allow additional commands to be ran, if a mutant is not killed/timed out. If these commands terminate with a non 0 exit code, it means the mutant was killed.

For example given the follwing infection.json

{
    "timeout": 20,
    "source": {
        "directories": [
            "classes"
        ]
    },
    "logs": {
        "text": "infection-log.txt"
    },
    "phpUnit": {
        "configDir": "tests"
    }
}

It could be changed to something like this:

{
    "timeout": 20,
    "source": {
        "directories": [
            "classes"
        ]
    },
    "logs": {
        "text": "infection-log.txt"
    },
    "phpUnit": {
        "configDir": "tests"
    },
    "fallback" : [
        "vendor/bin/other-php-tester",
        "./tester.sh"
    ]
}

This would slow down the tests of course and users are responsible for adding --stop-on-failure like flags, but it would give the users more security in the results of infection.

I've just installed and try to use the infection command, and I get "Misuse of shell builtins" error.

> infection --threads=4
Running initial test suite...

Phpunit version: 6.2.4

 2871 [============================] 2 minsTests do not pass. Error code 2. "Misuse of shell builtins". STDERR: 

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/6.2/phpunit.xsd"
        backupGlobals="false"
        colors="true"
        bootstrap="autoload.php"
        beStrictAboutTestsThatDoNotTestAnything="true"
        beStrictAboutOutputDuringTests="true"
        beStrictAboutTestSize="true"
        beStrictAboutChangesToGlobalState="true"
>
   <testsuites>
       <testsuite name="Project Test Suite">
           <directory>../src/*/*Bundle/Tests</directory>
           <directory>../src/*/Bundle/*Bundle/Tests</directory>
           <directory>../src/*Bundle/Tests</directory>
           <directory>../src/Application/Tests</directory>
           <directory>../src/Domain/Tests</directory>
       </testsuite>
   </testsuites>

   <php>
       <ini name="memory_limit" value="512m"/>
   </php>

   <filter>
       <whitelist>
           <directory>../src</directory>
           <exclude>
               <directory>../src/*Bundle/Resources</directory>
               <directory>../src/*Bundle/Tests</directory>
               <directory>../src/*/*Bundle/Resources</directory>
               <directory>../src/*/*Bundle/Tests</directory>
               <directory>../src/*/Bundle/*Bundle/Resources</directory>
               <directory>../src/*/Bundle/*Bundle/Tests</directory>
           </exclude>
       </whitelist>
   </filter>

   <listeners>
       <listener class="AppBundle\Test\Listener\IntegrationTestsFixturesLoaderListener" />
   </listeners>
</phpunit>

I tried infection on a little project of mine wich has 100% coverage and where I am currently using humbug for mutation testing.

Steps to reproduce:

cd /tmp 
git clone https://github.com/bcremer/LineReader.git
cd LineReader 
composer install
wget https://github.com/infection/infection/releases/download/0.2.1/infection.phar
wget https://github.com/infection/infection/releases/download/0.2.1/infection.phar.pubkey
php infection.phar

Output:

   15 [============================]  1 sec
Generate mutants...

PHP Fatal error:  Uncaught Error: Call to a member function appendChild() on null in phar:///tmp/LineReader/infection.phar/src/TestFramework/PhpUnit/Config/MutationXmlConfiguration.php:107
Stack trace:
#0 phar:///tmp/LineReader/infection.phar/src/TestFramework/PhpUnit/Config/MutationXmlConfiguration.php(68): Infection\TestFramework\PhpUnit\Config\MutationXmlConfiguration->addTestSuiteWIthFilteredTestFiles(Object(DOMDocument), Object(DOMXPath))
#1 phar:///tmp/LineReader/infection.phar/src/TestFramework/PhpUnit/Config/MutationXmlConfiguration.php(52): Infection\TestFramework\PhpUnit\Config\MutationXmlConfiguration->setFilteredTestsToRun(Object(DOMDocument), Object(DOMXPath))
#2 phar:///tmp/LineReader/infection.phar/src/TestFramework/PhpUnit/Config/Builder/MutationConfigBuilder.php(63): Infection\TestFramework\PhpUnit\Config\MutationXmlConfiguration->getXml()
#3 phar:///tmp/LineReader/infection.phar/src/TestFramework/AbstractTestFrameworkAdapter.php(74): Infection\TestFramework\PhpUnit\Config\Builder\MutationConfigBuilder->build( in phar:///tmp/LineReader/infection.phar/src/TestFramework/PhpUnit/Config/MutationXmlConfiguration.php on line 107

PHP Version

$ php -v
PHP 7.1.7-1+ubuntu17.04.1+deb.sury.org+1 (cli) (built: Jul  7 2017 09:43:21) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.1.7-1+ubuntu17.04.1+deb.sury.org+1, Copyright (c) 1999-2017, by Zend Technologies
    with Xdebug v2.5.5, Copyright (c) 2002-2017, by Derick Rethans

Hi,

I am currently moving an existing project from humbug to infection and have an issue related to the current implementation of the timeout property.

My existing project has a few exceptional tests that take a lot more time then 10 seconds, but most tests are < 1 second of course. Now in order to get infection to execute the test suite initially and generate the coverage I have to set a high value as timeout. This does only extend the time infection then takes to detect infinite loops in the actual mutation run.

My current solution is to exclude these tests through a custom config. Nevertheless I think there is room for some improvement in infection itself.

My Suggestion:

• Remove the timeout from the initial run alltogether and trust the users here.
• Refactor the timeout into a percentage value. Mark tests that take X% longer then when they were executed initially as timeout.

There is an issue with either XDebug or PHPUnit Coverage component that leads to uncovered mutant with 100% code coverage.

--- Original
+++ New
@@ @@
     {
-        switch (true) {
+        switch (false) {
             case $data === null:
                 return new NullResource();
             case is_array($data):
             case $data instanceof \Traversable:
                 return new Collection($data, $this->transformer, $resourceKey);
             default:

I've created an issue in the PHPUnit_Coverage repository to track it: sebastianbergmann/php-code-coverage#552

Temporary solution is to rewrite this code with ifs

In order to be as efficient as possible, I would suggest to have different running strategies in order to meet different needs:

• When code coverage is provided, make use of it to know which tests to execute and in what order (already done to a certain extend I believe?)
• Special case with PhpSpec: unless doing something weird, PhpSpec is intended to have a one-one relationship between a spec file and its source file and match a certain naming convention. As such which test to execute is trivial to decide.
• No coverage: we could either bail out or execute all the tests. IMO it is perfectly viable to not have a coverage of any kind so whilst horribly slow, running all the tests for it would be an option.
• Apply multiple mutants (1)

(1) I'm expending this point separately as IMO this is a bit more complex and tricky. Right now we apply mutants one by one and for each we run them in a separate process (parallelised or not). There is a big overhead here as we need to create as many processes as mutants, and creating a process is expansive. An alternative way of doing it would be to apply several unrelated mutants (and we can deduce that from the coverage data) in the same process.

Another variant of that mode, more aggressive, would be to push it to apply multiple mutants even if they might overlap each other (e.g. multiple mutants in the same method). If the tests pass (the mutant escaped), it's a win in perf. If the mutants died, we would need to re-iterate the process with only one mutant as you can't possibly know which mutant caused the failure and that might result in a time loss. I'm not sure which balance, maybe adding some kind of conditions on what mutants can be applied together would provide a reliable way to make it faster in most cases, but that's an idea.

Description

On a fresh install infection fails after processing source code files is complete, just as soon as mutation begins. This happens with Composer and PHAR installation of 0.5.2. In both cases, infection was run without options.

I'm still looking into the source of the problem but maybe someone knows the answer.

<?xml version="1.0" encoding="UTF-8"?>
<phpunit backupGlobals="false"
        backupStaticAttributes="false"
        bootstrap="bootstrap/autoload.php"
        colors="true"
        convertErrorsToExceptions="true"
        convertNoticesToExceptions="true"
        convertWarningsToExceptions="true"
        processIsolation="false"
        stopOnFailure="false">
   <testsuites>
       <testsuite name="Unit Tests">
           <directory suffix="Test.php">./tests</directory>
       </testsuite>
   </testsuites>
   <filter>
       <whitelist processUncoveredFilesFromWhitelist="true">
           <directory suffix=".php">./app</directory>
       </whitelist>
   </filter>
   <php>
       <env name="APP_ENV" value="testing"/>
       <env name="CACHE_DRIVER" value="array"/>
       <env name="SESSION_DRIVER" value="array"/>
       <env name="QUEUE_DRIVER" value="sync"/>
   </php>
</phpunit>

Runtime

Failure occurs 1-2 seconds after infection is run. The output is the same whether Composer or PHAR installation was used (excluding the expected infection path differences in the stack trace).

Invoking with:

php infection.phar

Yields this output:

1 [============================]  1 sec

Final state

After the output in the details expander above, the directory where index.xml is expected has two files:

• mutant.b4c682a2ff4ad6d0c73c07072e36dd87.infection.php (appears to be mutated code under test)
• phpunitConfiguration.initial.infection.xml (PHPUnit configuration with paths expanded)

However some expected artifacts don't exist:

• coverage-xml does not exist
• phpunit.junit.xml does not exist

I want to have to require infection as dev dependency in my project. After installation I try to run and...:

Warning: require(/var/www/html/auth/vendor/infection/infection/app/../vendor/autoload.php): failed to open stream: No such file or directory in /var/www/html/auth/vendor/infection/infection/app/bootstrap.php on line 7

Call Stack:
    0.0001     364912   1. {main}() /var/www/html/auth/vendor/infection/infection/bin/infection:0
    0.0003     365832   2. require('/var/www/html/auth/vendor/infection/infection/app/bootstrap.php') /var/www/html/auth/vendor/infection/infection/bin/infection:8


Fatal error: require(): Failed opening required '/var/www/html/auth/vendor/infection/infection/app/../vendor/autoload.php' (include_path='.:/usr/local/lib/php') in /var/www/html/auth/vendor/infection/infection/app/bootstrap.php on line 7

Call Stack:
    0.0001     364912   1. {main}() /var/www/html/auth/vendor/infection/infection/bin/infection:0
    0.0003     365832   2. require('/var/www/html/auth/vendor/infection/infection/app/bootstrap.php') /var/www/html/auth/vendor/infection/infection/bin/infection:8

The output always contains colors. cmd.exe on Windows 10 does not support them (example bellow) . Can I disable colors?

←[1m4←[22m mutations were generated:
←[1m       0←[22m mutants were killed
←[1m       4←[22m mutants were not covered by tests
←[1m       0←[22m covered mutants were not detected
←[1m       0←[22m errors were encountered
←[1m       0←[22m time outs were encountered

As a developer, I want to be able to configure the location of the test framework.

I don't have PHPUnit installed globally on my local system as I need to use different versions for different projects. So I have a shared directory with the different versions of PHPUnit in it. It seems that Infections looks for a phpunit.phar either installed globally or in the current directory. It would be nice if I could define the location explicitely.

Hi. Isn't license file copy-pasted from humbug project ?

This part especially:

 Neither the name of Pádraic Brady, Humbug

as it points to creator of Humbug and Humbug itself

If current folder contains file 'infection-log.txt', infection will show error. After deleting the file, the problem disappears.

$  ls
composer.json  composer.lock  infection.json.dist  phpunit.xml.dist  Ranges.php  tests  vendor
$  infection 
   ____      ____          __  _
  /  _/___  / __/__  _____/ /_(_)___  ____ 
  / // __ \/ /_/ _ \/ ___/ __/ / __ \/ __ \
_/ // / / / __/  __/ /__/ /_/ / /_/ / / / /
/___/_/ /_/_/  \___/\___/\__/_/\____/_/ /_/

   0 [>---------------------------] < 1 secRunning initial test suite...

Phpunit version: 6.5.5

  20 [============================] 45 secs

Generate mutants...

Processing source code files: 6/6
Creating mutated files and processes: 68/68
.: killed, M: escaped, S: uncovered, E: fatal error, T: timed out

...............M....M...........M..M...M..M..M..M.   (50 / 68)
.M..M...M..M...M.M                                   (68 / 68)

68 mutations were generated:
     54 mutants were killed
      0 mutants were not covered by tests
     14 covered mutants were not detected
      0 errors were encountered
      0 time outs were encountered

Metrics:
        Mutation Score Indicator (MSI): 79%
        Mutation Code Coverage: 100%
        Covered Code MSI: 79%

Please note that some mutants will inevitably be harmless (i.e. false positives).
$  ls
composer.json  composer.lock  infection.json.dist  infection-log.txt  phpunit.xml.dist  Ranges.php  tests  vendor
$  infection 
   ____      ____          __  _
  /  _/___  / __/__  _____/ /_(_)___  ____ 
  / // __ \/ /_/ _ \/ ___/ __/ / __ \/ __ \
_/ // / / / __/  __/ /__/ /_/ / /_/ / / / /
/___/_/ /_/_/  \___/\___/\__/_/\____/_/ /_/

   0 [>---------------------------] < 1 secRunning initial test suite...

Phpunit version: 6.5.5

  20 [============================] 39 secs

Generate mutants...

Processing source code files: 6/7
In ParserAbstract.php line 293:
                                                                             
 Syntax error, unexpected '-', expecting T_FUNCTION or T_CONST on line 1168  
                                                                             

run [--test-framework TEST-FRAMEWORK] [--test-framework-options TEST-FRAMEWORK-OPTIONS] [--threads THREADS] [--only-covered] [-s|--show-mutations] [-c|--configuration CONFIGURATION] [--mutators MUTATORS] [--filter FILTER] [--formatter FORMATTER] [--min-msi MIN-MSI] [--min-covered-msi MIN-COVERED-MSI] [--log-verbosity [LOG-VERBOSITY]]

$  rm infection-log.txt && infection 
   ____      ____          __  _
  /  _/___  / __/__  _____/ /_(_)___  ____ 
  / // __ \/ /_/ _ \/ ___/ __/ / __ \/ __ \
_/ // / / / __/  __/ /__/ /_/ / /_/ / / / /
/___/_/ /_/_/  \___/\___/\__/_/\____/_/ /_/

   0 [>---------------------------] < 1 secRunning initial test suite...

Phpunit version: 6.5.5

  20 [============================] 46 secs

Generate mutants...

Processing source code files: 6/6
Creating mutated files and processes: 68/68
.: killed, M: escaped, S: uncovered, E: fatal error, T: timed out

...............M....M...........M..M...M..M..M..M.   (50 / 68)
.M..M...M..M...M.M                                   (68 / 68)

68 mutations were generated:
     54 mutants were killed
      0 mutants were not covered by tests
     14 covered mutants were not detected
      0 errors were encountered
      0 time outs were encountered

Metrics:
        Mutation Score Indicator (MSI): 79%
        Mutation Code Coverage: 100%
        Covered Code MSI: 79%

Please note that some mutants will inevitably be harmless (i.e. false positives).
$ 

I try to use infection in a reusable Symfony bundle. Thereby live the source files not under the src folder, but in the root of the project. Apparently this is not working with infection.

I've created a dummy project with both the working and not working version.

Feel free to comment or even PR the solution.

Thanks!

$ bin/infection --test-framework=phpspec

Running initial test suite...

Phpspec version: 4.1.0

   4 [============================] < 1 sec

Generate mutants...

Processing source code files: 2/2
Creating mutated files and processes: 3/3
.: killed, M: escaped, S: uncovered, E: fatal error, T: timed out

SSS                                                  (3 / 3)

3 mutations were generated:
      0 mutants were killed
      3 mutants were not covered by tests
      0 covered mutants were not detected
      0 time outs were encountered

Metrics:
        Mutation Score Indicator (MSI): 0%
        Mutation Code Coverage: 0%
        Covered Code MSI: 0%

Please note that some mutants will inevitably be harmless (i.e. false positives).

Running initial test suite...

Phpspec version: 4.1.0

   4 [============================]  1 sec

Generate mutants...

Processing source code files: 1/1
Creating mutated files and processes: 1/1
.: killed, M: escaped, S: uncovered, E: fatal error, T: timed out

.                                                    (1 / 1)

1 mutations were generated:
      1 mutants were killed
      0 mutants were not covered by tests
      0 covered mutants were not detected
      0 time outs were encountered

Metrics:
        Mutation Score Indicator (MSI): 100%
        Mutation Code Coverage: 100%
        Covered Code MSI: 100%

Please note that some mutants will inevitably be harmless (i.e. false positives).

Unless I missed it, right now are running the tests which requires xdebug or phpdbg to get the coverage data to then be able to do our own stuff.

I think a typical user would already have tests with coverage somehow, so IMO it would make much more sense to provide the data coverage files which when done so would avoid us to have to run the tests once.

Trying to run first time, some troubles with SourceDirGuesser:

D:\Projects\php\generic-storage>D:\Tools\php7\php.exe D:\Tools\infection.phar

Welcome to the Infection config generator

We did not find configuration file. The following questions will help us to generate it for you.


PHP Fatal error:  Uncaught TypeError: Argument 1 passed to Infection\Config\Guesser\SourceDirGuesser::Infection\Config\Guesser\{closure}() must be of the type string, array given in phar://D:/Tools/infection.phar/src/Config/Guesser/SourceDirGuesser.php:42
Stack trace:
#0 [internal function]: Infection\Config\Guesser\SourceDirGuesser->Infection\Config\Guesser\{closure}(Array)
#1 phar://D:/Tools/infection.phar/src/Config/Guesser/SourceDirGuesser.php(45): array_map(Object(Closure), Array)
#2 phar://D:/Tools/infection.phar/src/Config/Guesser/SourceDirGuesser.php(33): Infection\Config\Guesser\SourceDirGuesser->getValues('psr-0')
#3 phar://D:/Tools/infection.phar/src/Config/ValueProvider/SourceDirsProvider.php(46): Infection\Config\Guesser\SourceDirGuesser->guess()
#4 phar://D:/Tools/infection.phar/src/Command/ConfigureCommand.php(50): Infection\Config\ValueProvider\SourceDirsProvider->get(Object(Symfony\Component\Console\Input\ArrayInput), Object(Symfony\Component\Console\Output\ConsoleOutput), Array)
#5 phar://D:/Tools/infection.p in phar://D:/Tools/infection.phar/src/Config/Guesser/SourceDirGuesser.php on line 42

Hi!

Wanted to try out Infection, but could not install it in my environment:

composer require infection/infection --dev

I know that recommended installation is through PHAR or global composer installation, but I would like to track my dependencies, especially if I'm adding infection.json.dist to my project.

The problem is with the sebastian/diff package, I got conflicts:

- Installation request for infection/infection ^0.5.1 -> satisfiable by infection/infection[0.5.1].
    - Conclusion: remove sebastian/diff 2.0.1
    - Conclusion: don't install sebastian/diff 2.0.1
    - infection/infection 0.5.1 requires sebastian/diff ^1.4 -> satisfiable by sebastian/diff[1.4.0, 1.4.1, 1.4.2, 1.4.3].
    - Can only install one of: sebastian/diff[1.4.0, 2.0.1].
    - Can only install one of: sebastian/diff[1.4.1, 2.0.1].
    - Can only install one of: sebastian/diff[1.4.2, 2.0.1].
    - Can only install one of: sebastian/diff[1.4.3, 2.0.1].
    - Installation request for sebastian/diff (locked at 2.0.1) -> satisfiable by sebastian/diff[2.0.1].

composer why sebastian/diff hints me that my PHPUnit version (6.3.0) requires sebastian/diff ^2.0.

Let's think together what can be done, cause simply updating to ^2.0 might break older PHPUnit version projects. It's weird how a 3rd package dependency makes your project dependent on specific PHPUnit version.

Currently, when displaying the difference between original and mutated code, the pretty-formatted representation of the AST is used:

Depending on how the original code is formatted, it may lead to the inconsistency between the diff and the original file contents. Additionally, given that sebastian/diff 2.0 currently doesn't display common lines of code (sebastianbergmann/diff#69), in some cases, the information about how exactly the original code was mutated may be hard to understand.

PHP-Parser 4 introduces the ability to use the formatting-preserving pretty printing. Using this kind of formatting together with displaying line numbers may eliminate the need for displaying common lines in the diff.

I'd like to suggest some mutations.

(Stolen from Stryker)

PublicVisibility mutator seen as escaped mutant for a class implementing an interface.

When adding the interface, the tests are escaped. When I remove the interface, it's killed.

See git log in de provided demo project.

Escaped mutants:
================


1) /Users/l.vanderstraeten/Desktop/demo/Foo/Bar/Foobar.php:9    [M] PublicVisibility
exec /usr/local/Cellar/php71/7.1.8_20/bin/php /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/bin/phpspec run --config=/var/folders/p0/jbqrfh8j09s2z81g0c6d9m3h0000gn/T/infection/phpspecConfiguration.9943b2d5d6bc4a055bdc4ef4ff512fac.infection.yml --no-ansi --format=tap --stop-on-failure 
--- Original
+++ New
@@ @@
-    public function foobar() : string
+    protected function foobar() : string

TAP version 13

Fatal error: Access level to Foo\Bar\Foobar::foobar() must be public (as in class Foo\Bar\FoobarInterface) in /Users/l.vanderstraeten/Desktop/demo/Foo/Bar/Foobar.php on line 6

Call Stack:
   0.0002     358456   1. {main}() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/bin/phpspec:0
   0.0003     358776   2. {closure:/Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/bin/phpspec:4-27}() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/bin/phpspec:27
   0.0050    1129768   3. PhpSpec\Console\Application->run() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/bin/phpspec:25
   0.0263    1656736   4. PhpSpec\Console\Application->doRun() /Users/l.vanderstraeten/Desktop/demo/vendor/symfony/console/Application.php:125
   0.0356    2768976   5. PhpSpec\Console\Application->doRun() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Console/Application.php:89
   0.0357    2768976   6. PhpSpec\Console\Application->doRunCommand() /Users/l.vanderstraeten/Desktop/demo/vendor/symfony/console/Application.php:224
   0.0368    2798424   7. PhpSpec\Console\Command\RunCommand->run() /Users/l.vanderstraeten/Desktop/demo/vendor/symfony/console/Application.php:906
   0.0372    2799632   8. PhpSpec\Console\Command\RunCommand->execute() /Users/l.vanderstraeten/Desktop/demo/vendor/symfony/console/Command/Command.php:262
   0.0602    4609272   9. PhpSpec\Runner\SuiteRunner->run() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Console/Command/RunCommand.php:178
   0.0609    4661584  10. PhpSpec\Runner\SpecificationRunner->run() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Runner/SuiteRunner.php:56
   0.0613    4676440  11. PhpSpec\Runner\ExampleRunner->run() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Runner/SpecificationRunner.php:55
   0.0613    4676896  12. PhpSpec\Runner\ExampleRunner->executeExample() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Runner/ExampleRunner.php:83
   0.0678    5331480  13. ReflectionMethod->invokeArgs() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Runner/ExampleRunner.php:150
   0.0678    5331488  14. spec\Foo\Bar\FoobarSpec->it_is_initializable() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Runner/ExampleRunner.php:150
   0.0678    5331488  15. spec\Foo\Bar\FoobarSpec->shouldHaveType() /Users/l.vanderstraeten/Desktop/demo/spec/Foo/Bar/FoobarSpec.php:12
   0.0678    5331864  16. spec\Foo\Bar\FoobarSpec->__call() /Users/l.vanderstraeten/Desktop/demo/spec/Foo/Bar/FoobarSpec.php:12
   0.0678    5331864  17. PhpSpec\Wrapper\Subject->shouldHaveType() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/ObjectBehavior.php:155
   0.0678    5332240  18. PhpSpec\Wrapper\Subject->__call() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/ObjectBehavior.php:155
   0.0678    5332240  19. PhpSpec\Wrapper\Subject->callExpectation() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Wrapper/Subject.php:283
   0.0678    5332240  20. PhpSpec\Wrapper\Subject->makeSureWeHaveASubject() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Wrapper/Subject.php:344
   0.0679    5334936  21. PhpSpec\Util\Instantiator->instantiate() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Wrapper/Subject.php:363
   0.0679    5334936  22. class_exists() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Util/Instantiator.php:27
   0.0679    5334976  23. spl_autoload_call() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Util/Instantiator.php:27
   0.0679    5335016  24. Composer\Autoload\ClassLoader->loadClass() /Users/l.vanderstraeten/Desktop/demo/vendor/phpspec/phpspec/src/PhpSpec/Util/Instantiator.php:27
   0.0679    5335016  25. Composer\Autoload\includeFile() /Users/l.vanderstraeten/Desktop/demo/vendor/composer/ClassLoader.php:322
   0.0680    5336624  26. include('/Users/l.vanderstraeten/Desktop/demo/Foo/Bar/Foobar.php') /Users/l.vanderstraeten/Desktop/demo/vendor/composer/ClassLoader.php:444

Not quite sure how we can make that work, but basically the thread option is useless as soon as you have integration tests which rely on a DB or something. As of now a way to deal with it is simply exclude those tests from the execution which is a bit extreme. If you really want to execute Infection on them still, you basically have no choice but to execute Infection two times, one time threaded with the excluded functional tests and another time not threaded with the functional tests only.

A better way IMO would be to find a way to allow a user to say that some tests are more sensitive than other and require to be run in a single thread. To illustrate, if you have the following tests:

• FunctionalTest1
• FunctionalTest2
• UnitTest1
• UnitTest2
• UnitTest3

with three threads, we have a pool of mutants to apply, we make sure all the ones related to a functional test are executed in the same thread while we can parallelise the other. And if by chance we finish with the mutants related to the functional tests first, then the thread used is now free for the other mutants.

New mutator to implement:

- $a <=> $b
+ $b <=> $a

Why?

To determine whether the sorting function is correctly covered by tests and parameters are not mixed up.

What needs to be done?

1. First, a new mutation operator (mutator) should be created here
2. It should be registered in the configuration file.
3. Unit test should be added to proof it works as expected.

I would like to involve more contributors for the upcoming Hacktoberfest. Feel free to submit your first PR and get a T-Shirt!

Hi.
Are there any chances to support phpdbg, not only xDebug ?

I'm using codeception as my main testing framework and may be interested in providing the integration. Has anybody else already started working on codeception integration or have there been any blockers I don't know about yet?

Nikic has recently published a small article about possible performance optimizations for projects that use PHP-Parser.

I would like to post them here, discuss and step by step try all of them for Infection.

• Object reuse - implemented
• Garbage Collection - didn't give any performance boost.
• Reuse created mutants to avoid traversing/prettyprinting - implemented
• Partially disabling XDebug - implemented. We can disable xDebug for each Symfony Process when e.g. PHPUnit is run against mutated code. Actually, we can disable it even for the main Infection process, and only enable when the initial test suite is executed to generate Code Coverage (this is also a separate Symfony Process in the code).
• Partially disable phpdbg

Completely new to Infection, so likely a user error, but when running infection -s against by repo, I'm not able to see what the "mutants not covered by tests" were. My logs/infection-text.txt is empty.

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/6.4/phpunit.xsd"
        bootstrap="tests/bootstrap.php"
        backupGlobals="false"
        beStrictAboutCoversAnnotation="true"
        beStrictAboutOutputDuringTests="true"
        beStrictAboutTestsThatDoNotTestAnything="true"
        beStrictAboutTodoAnnotatedTests="true"
        colors="true"
   	 defaultTestSuite="unit"
        verbose="true">
   <testsuites>
   	<testsuite name="unit">
   		<directory suffix="Test.php">tests/Unit</directory>
   	</testsuite>
   	<testsuite name="integration">
   		<directory suffix="Test.php">tests/Integration</directory>
   	</testsuite>
   </testsuites>

   <filter>
       <whitelist processUncoveredFilesFromWhitelist="true">
           <directory suffix=".php">src</directory>
       </whitelist>
   </filter>

   <logging>
       <log type="coverage-text" target="php://stdout" showUncoveredFiles="false"/>
       <!--<log type="coverage-html" target="coverage"/>-->
   </logging>
</phpunit>

{
   "timeout": 10,
   "source": {
       "directories": [
           "src"
       ]
   },
   "logs": {
       "text": "infection-log.txt"
   }
}

Support only excludes key in infection.json.dist that was added in #120 to make Humbug's config compatible with Infection

• Remove code and test
• Update documentaion here

There is no need to mutate public to protected for methods from the implemented interface.

interface Doable
{
    public function do(): void;
}

class Foo implements Doable
{
    public function do(): void {}
}

Incorrect mutation:

class Foo implements Doable
{
-    public function do(): void {}
+    protected function do(): void {}
}

Expected result: no mutation, because it always leads to Fatal error: Access level to Foo::do() must be public (as in class Doable)...

At this point, we don't have information about parent classes/interfaces. It requires implementing a new smart feature with native reflection or BetterReflection.

As a developer, I want to have Infection Phar Distribution
And be able to self-update it

$ bin/phpspec run

All tests passed

$ infection --test-framework=phpspec

Running initial test suite...

Phpspec version: 3.4.2

    1 [============================] < 1 secTests do not pass. Error code 255. "Unknown error". STDERR:

infection.json.dist

{
    "timeout": 25,
    "source": {
        "directories": [
            "."
        ],
        "exclude": [
            "vendor"
        ]
    },
    "logs": {
        "text": "infection-log.txt"
    }
}

Because Infection generates its own PHP config file and enables the DOM extension there explicitly (extension=dom.so), execution will fail with the below error if the extension is also globally enabled in the mods-available/dom.ini file.
Unfortunately, the problem is kind of hidden as the regular execution of the testcases uses the regular php.ini (with the extension not enabled twice).

To avoid this issue, Infection should check whether the DOM extension is already enabled and if so, not include it again in its separate PHP config.

Notice: Undefined property: DOMDocument::$documentElement in phpunit/src/Util/Configuration.php on line 557
Fatal error: Uncaught Error: Call to a member function hasAttribute() on null in phpunit/src/Util/Configuration.php on line 559

First, thanks for creating infection! It's great to see an AST-based mutation testing framework. I use "the other" framework and its lack of support for PHPUnit 6 has been a concern.

In the course of setting up to try infection I'm experiencing the same output as #24 but the root cause is not the same. Though we're all well-accustomed to PHP errors, it would be best for the developer to receive an intelligible error message when a problem occurs. It's not possible to say if there was a problem finding a dependency to start infection, generating the coverage, or writing the coverage file. It simply says the file isn't there when it was later requested.

An example fix might be output like "PHPUnit version too low" or "couldn't find PHPUnit" or whatever the problem is (I still haven't diagnosed mine). Providing this output as early as possible will also make it easier for users to fix problems themselves or at least to identify them accurately when requesting support.

This is not a request for help with my particular problem. This is a request for meaningful feedback upon failure, especially the problems most likely encountered during the first run for a user or environment.

For WordPress, callback methods hooked into the actions or filters need to be public.

The Function Signature mutator naturally tries to change the callback from public to protected, and flags it as a "not covered mutant".

Is there any way (to reduce known false positives) to specify that a method really should stay public, either by adding a line comment (e.g. // infection:disable FunctionSignature) or by adding an exclusion (file and line number? class and method name?) to infection.json.dist?

It seems like running Infection using phpdbg (0.5.0) is broken in 0.7.0 and later.

I first tested it on an internal project using PHP 7.2.1, PHPUnit 6.5.5 and Infection 0.7.0 without success (and a colleague verified it with PHP 7.0.22), and then on the Infection codebase itself (69551ba) using PHP 7.2.1 and PHPUnit 6.1.0. In both cases we get the same error message.

I've checked out 0.6.2 of Infection and that version does work. Running Infection on Infection commit by commit since 0.6.2 it seems like 47c8350 is the version that breaks phpdbg, because that commit and every commit after that has the same issue.

I'll do some more digging later and maybe I can put together a failing test case for it.

$ phpdbg -qrr bin/infection
   ____      ____          __  _
  /  _/___  / __/__  _____/ /_(_)___  ____
  / // __ \/ /_/ _ \/ ___/ __/ / __ \/ __ \
_/ // / / / __/  __/ /__/ /_/ / /_/ / / / /
/___/_/ /_/_/  \___/\___/\__/_/\____/_/ /_/

Running initial test suite...

Phpunit version: 6.1.0

   2 [============================] < 1 sec

Generate mutants...

Processing source code files: 166/166
Creating mutated files and processes:    0/1008

[ERROR] Code Coverage does not exist. File
        /var/folders/1_/j6pgbhz546g4t92sckw7215c0000gn/T/infection/coverage-xml/index.xml is not found. Check phpunit
        version Infection was run with and generated config files inside
        /var/folders/1_/j6pgbhz546g4t92sckw7215c0000gn/T/infection.

Im not seeing minimum version compatibility in the docs or anywhere.
I suppose its not compatible as it's not running as expected. (I don't know what its happening) .

<phpunit bootstrap="bootstrap.php"
        backupGlobals="false"
        colors="true"
        convertErrorsToExceptions="true"
        convertNoticesToExceptions="true"
        convertWarningsToExceptions="true"
        beStrictAboutCoversAnnotation="true"
        beStrictAboutOutputDuringTests="false"
        beStrictAboutChangesToGlobalState="true"
        beStrictAboutTestsThatDoNotTestAnything="true"
        stopOnFailure="false">
   <filter>
       <whitelist>
           <directory suffix=".php">../src/</directory>
           <exclude>
               <directory suffix=".php">../vendor/</directory>
           </exclude>
       </whitelist>
   </filter>
</phpunit>

[root@test protected]# ./vendor/bin/infection
   ____      ____          __  _
  /  _/___  / __/__  _____/ /_(_)___  ____
  / // __ \/ /_/ _ \/ ___/ __/ / __ \/ __ \
_/ // / / / __/  __/ /__/ /_/ / /_/ / / / /
/___/_/ /_/_/  \___/\___/\__/_/\____/_/ /_/

Running initial test suite...

Phpunit version: 5.7.20

   1 [============================] < 1 sec

[ERROR] Project tests must be in a passing state before running Infection.

        Phpunit reported an exit code of 2.

        Refer to the phpunit's output below:

        STDOUT:

        PHPUnit 5.7.20 by Sebastian Bergmann and contributors.

        Usage: phpunit [options] UnitTest [UnitTest.php]
               phpunit [options] <directory>

        Code Coverage Options:

          --coverage-clover <file>  Generate code coverage report in Clover XML format.
          --coverage-crap4j <file>  Generate code coverage report in Crap4J XML format.
          --coverage-html <dir>     Generate code coverage report in HTML format.
          --coverage-php <file>     Export PHP_CodeCoverage object to file.
          --coverage-text=<file>    Generate code coverage report in text format.
                                    Default: Standard output.
          --coverage-xml <dir>      Generate code coverage report in PHPUnit XML format.
          --whitelist <dir>         Whitelist <dir> for code coverage analysis.
          --disable-coverage-ignore Disable annotations for ignoring code coverage.

        Logging Options:

          --log-junit <file>        Log test execution in JUnit XML format to file.
          --log-teamcity <file>     Log test execution in TeamCity format to file.
          --testdox-html <file>     Write agile documentation in HTML format to file.
          --testdox-text <file>     Write agile documentation in Text format to file.
          --testdox-xml <file>      Write agile documentation in XML format to file.
          --reverse-list            Print defects in reverse order

        Test Selection Options:

          --filter <pattern>        Filter which tests to run.
          --testsuite <name>        Filter which testsuite to run.
          --group ...               Only runs tests from the specified group(s).
          --exclude-group ...       Exclude tests from the specified group(s).
          --list-groups             List available test groups.
          --list-suites             List available test suites.
          --test-suffix ...         Only search for test in files with specified
                                    suffix(es). Default: Test.php,.phpt

        Test Execution Options:

          --report-useless-tests    Be strict about tests that do not test anything.
          --strict-coverage         Be strict about @covers annotation usage.
          --strict-global-state     Be strict about changes to global state
          --disallow-test-output    Be strict about output during tests.
          --disallow-resource-usage Be strict about resource usage during small tests.
          --enforce-time-limit      Enforce time limit based on test size.
          --disallow-todo-tests     Disallow @todo-annotated tests.

          --process-isolation       Run each test in a separate PHP process.
          --no-globals-backup       Do not backup and restore $GLOBALS for each test.
          --static-backup           Backup and restore static attributes for each test.

          --colors=<flag>           Use colors in output ("never", "auto" or "always").
          --columns <n>             Number of columns to use for progress output.
          --columns max             Use maximum number of columns for progress output.
          --stderr                  Write to STDERR instead of STDOUT.
          --stop-on-error           Stop execution upon first error.
          --stop-on-failure         Stop execution upon first error or failure.
          --stop-on-warning         Stop execution upon first warning.
          --stop-on-risky           Stop execution upon first risky test.
          --stop-on-skipped         Stop execution upon first skipped test.
          --stop-on-incomplete      Stop execution upon first incomplete test.
          --fail-on-warning         Treat tests with warnings as failures.
          --fail-on-risky           Treat risky tests as failures.
          -v|--verbose              Output more verbose information.
          --debug                   Display debugging information during test execution.

          --loader <loader>         TestSuiteLoader implementation to use.
          --repeat <times>          Runs the test(s) repeatedly.
          --teamcity                Report test execution progress in TeamCity format.
          --testdox                 Report test execution progress in TestDox format.
          --testdox-group           Only include tests from the specified group(s).
          --testdox-exclude-group   Exclude tests from the specified group(s).
          --printer <printer>       TestListener implementation to use.

        Configuration Options:

          --bootstrap <file>        A "bootstrap" PHP file that is run before the tests.
          -c|--configuration <file> Read configuration from XML file.
          --no-configuration        Ignore default configuration file (phpunit.xml).
          --no-coverage             Ignore code coverage configuration.
          --no-extensions           Do not load PHPUnit extensions.
          --include-path <path(s)>  Prepend PHP's include_path with given path(s).
          -d key[=value]            Sets a php.ini value.
          --generate-configuration  Generate configuration file with suggested settings.

        Miscellaneous Options:

          -h|--help                 Prints this usage information.
          --version                 Prints the version and exits.
          --atleast-version <min>   Checks that version is greater than min and exits.

To prevent issues like #24, version of PHPUnit/PhpSpec needs to be displayed to the console in order to determine problems with any other installed in the PATH version of test framework

Taking advantage of #88 and #89, there is one more point that I think would be awesome in terms of usability. The idea is that you don't have the same level of exigence for every part of your application. As such, I think it would be really good to be able to define scoped configuration, i.e. define options that would be applied to only a part of the code.

For example, assuming #89 is implemented, I could define a high level of strictness for a specific part of my application which is very important and a lower level of strictness for the rest.

I may have misread this but from what I see:

• we get the source files
• we generate the mutants for those files
• for each mutants, we parse the file, apply the mutation and run it

If that's what is happening, there I think we could optimize it as in the case where multiple mutants are applied to the same file, we will be parsing that file multiple times unless PhpParser has some kind of caching which I don't think it has.

This library is nothing short of amazing. Thank you!

Is there an option to hide the killed mutants in the log? When I run it, I'm only interested in everything but killed mutants and I find the killed mutants list to be a bit noisy when scrolling to find what methods need attention.

Thanks again!

When I run Infection the command quits with an exception when executing slow tests.

Environment:
Cygwin64 under Window 10

Command run:
/cygdrive/c/php/php-7.1.1/php7.1.exe ../infection.phar -v

Error:
[Symfony\Component\Process\Exception\ProcessTimedOutException] The process "C:\php\php-7.1.1\php7.1.exe C:\Users\dmecke\Dropbox\PhpstormProjects\FatefulEncountersApi\phpunit.phar --configuration C:\cygwin64\tmp/infection/phpunitConfiguration.initial.infection.xml --stop-on-failure" exceeded the timeout of 60 seconds.

Trace:
() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/vendor/symfony/process/Process.php:1262 Symfony\Component\Process\Process->checkTimeout() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/vendor/symfony/process/Process.php:385 Symfony\Component\Process\Process->wait() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/vendor/symfony/process/Process.php:206 Symfony\Component\Process\Process->run() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/src/Process/Runner/InitialTestsRunner.php:59 Infection\Process\Runner\InitialTestsRunner->run() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/src/InfectionApplication.php:73 Infection\InfectionApplication->run() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/src/Command/InfectionCommand.php:98 Infection\Command\InfectionCommand->execute() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/vendor/symfony/console/Command/Command.php:261 Symfony\Component\Console\Command\Command->run() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/vendor/symfony/console/Application.php:866 Symfony\Component\Console\Application->doRunCommand() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/vendor/symfony/console/Application.php:220 Symfony\Component\Console\Application->doRun() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/vendor/symfony/console/Application.php:127 Symfony\Component\Console\Application->run() at phar://C:/Users/dmecke/Dropbox/PhpstormProjects/infection.phar/bin/infection:33 require() at C:\Users\dmecke\Dropbox\PhpstormProjects\infection.phar:10

My max_execution_time in the php.ini is set to 30 seconds and the timeout in the infection.json.dist to 10. So this seems to be a timeout somewhat related to PHPUnit maybe?
When I reduce my testsuite in the phpunit.xml to only the plain unit tests, it works fine. It seems to be related to the slower integration tests. When I execute the complete test suite with phpunit alone, there is no exception.