stryker-mutator / stryker-mutator.github.io Goto Github PK

• Command line
• Integrate with npm
• integrate with grunt
• TBA: Integrate with gulp

In lines 8-9:

 With a rich set of mutators specifically for Scala you will be able to test bugs in ways you never knew possible!
 Currently Stryker4s is still in heavy early development, this means it's propably not ready for large projects just yet.

i would make some changes:

test bugs in ways you never knew possible! -> I don't even get what author mean by never knew possible! (it doesn't sound like english). Maybe test bugs in ways you possibly have never known?

Currently Stryker4s is still in heavy early development, this means it's propably not ready for large projects just yet. -> Currently Stryker4s is still in heavy early development, which means it's propably not ready for large projects just yet. or Currently Stryker4s is still in heavy early development. This means it's propably not ready for large projects just yet.

Add documentation for the mutator API. The goal is that anyone interested in creating mutators can go right ahead and implement them.

We might want to support the reposting of blog articles with canonicalization.

https://moz.com/learn/seo/canonicalization

On our 'redesign' branch, we now have this text:

Meet Stryker

With Stryker, we bring mutation testing to the masses.
It takes the best practices of existing mutation testing frameworks like PIT and Infection,
but with the open mentality of nodejs and the web. This is why Stryker is not a test runner or test framework.
Indeed, why would it? There are already awesome test runners and test frameworks out there.
Instead, Stryker empowers you to use what you like and focusses on being the best Mutation testing framework it can be.

This is sort of a mix from the old text with "JavaScript" removed. We should find something better.

@simondel @hugo-vrijswijk @richardwerkman @kmdrGroch @philippw @ArpadK any ideas?

The current home page is specific for stryker js. This should be changed to include stryker4s and stryker.net. The JS specific text should be moved.

Do we want to enrich our blogs with a "comments" section? I don't want to build a new comments backend from scratch, maybe we can use a serverless comment engine? More research needs to be done on this part.

Not sure if we need comments, we first need to decide on that. @stryker-mutator/contributors ?

Also link stryker-readme back to this page

C# -> stryker-net
Scala -> stryker4s
Stryker -> stryker

Hi guys,

I noticed there are some dead links on the main page.

The dead links are:

Extendable -> Stryker API leads to a 301 perm moved github page
clever reports -> clever reportsI leads to a 301 perm moved github page
Mutations -> more than 30 supported mutations leads to a github page that referece back to stryker-mutator.github.io/mutators.html

Will make a pull request shortly 👍

Whenever we change the pug files, the website should generate itself

Since we are still using grunt, it needs to be done like this. Without grunt, we will just use --prod or --dev in npm commands
Sorry @nicojs that i made an issue not PR, but removing fork, forking again etc, takes too much time, and i want to go sleep right now. It should work - it was tested by me. But check again if you like it.

webpack.dev.config.js

const path = require('path');

module.exports = {
  entry: ['./src/js/all.js', './src/scss/all.scss'],
  output: {
    path: path.resolve(__dirname, 'generated-root'),
    filename: 'js/all.bundle.js'
  },
  mode: 'development',
  module: {
    rules: [{
      test: /\.scss$/,
      use: [
        { 
          loader: 'file-loader',
          options: { 
            name: 'css/[name].css' 
          }
        }, // Extract css into seperate file
        'extract-loader',
        'css-loader?-minimize',
        'sass-loader' // compiles Sass to CSS, using Node Sass
      ]
    }
  ]
  }
};

webpack.prod.config.js

const path = require('path');

module.exports = {
  entry: ['./src/js/all.js', './src/scss/all.scss'],
  output: {
    path: path.resolve(__dirname, 'generated-root'),
    filename: 'js/all.bundle.js'
  },
  mode: 'production',
  module: {
    rules: [{
      test: /\.scss$/,
      use: [
        { 
          loader: 'file-loader',
          options: { 
            name: 'css/[name].css' 
          }
        }, // Extract css into seperate file
        'postcss-loader',
        'extract-loader',
        'css-loader?-minimize',
        'sass-loader' // compiles Sass to CSS, using Node Sass
      ]
    }
  ]
  }
};

gruntfile.js

const md = require('jstransformer')(require('jstransformer-markdown-it'));
const webpackConfigDev = require('./webpack.dev.config');
const webpackConfigProd = require('./webpack.prod.config');

module.exports = function (grunt) {

  grunt.loadNpmTasks('grunt-contrib-pug');
  grunt.loadNpmTasks('grunt-contrib-watch');
  grunt.loadNpmTasks('grunt-contrib-connect');
  grunt.loadNpmTasks('grunt-contrib-clean');
  grunt.loadNpmTasks('grunt-webpack');
  grunt.loadNpmTasks('grunt-npm');

  const watchConfig = {
    options: {
      livereload: true
    },
    pug: {
      files: ['src/**/*.pug', 'src/**/*.js', 'src/**/*.json'],
      tasks: ['pug']
    },
    webpack: {
      files: ['src/js/**/*.js', 'src/scss/**/*.scss', 'webpack.config.js'],
      tasks: ['webpack:dev']
    }
  };

  const config = {
    clean: {
      html: ['generated-root']
    },
    watch: watchConfig,
    webpack: {
      options: {
        stats: !process.env.NODE_ENV || process.env.NODE_ENV === 'development'
      },
      prod: webpackConfigProd,
      dev: webpackConfigDev
    },

    connect: {
      server: {
        options: {
          open: true,
          base: ['root', 'generated-root'],
          livereload: true,
          host: 'localhost',
          hostname: 'localhost',
          middleware: (_, _1, middlewares) => {
            middlewares.unshift((req, _, next) => {
              if (!req.url.endsWith('/') && req.url.indexOf('.') === -1) {
                req.url += '.html';
              }
              next();
            });
            return middlewares;
          }
        }
      }
    },

    pug: {
      compile: {
        options: {
          data: require('./src/fillViewModel.js'),
          filters: { // Add markdown-it-named-headers as plugin
            'markdown-it': function (text) {
              return md.render(text, {
                plugins: ['markdown-it-named-headers'],
                html: true
              }).body;
            }

          }
        },
        pretty: true,

        cwd: 'src',
        src: ['**/*.pug', '!layout/*.pug'],
        dest: 'generated-root',
        ext: '.html',
        expand: true
      }
    },

    'npm-contributors': {
      options: {
        commitMessage: 'chore: update contributors'
      }
    }
  };


  grunt.initConfig(config);
  grunt.registerTask('serve', ['clean', 'webpack:dev', 'pug', 'connect', 'watch']);
  grunt.registerTask('build', ['clean', 'webpack:prod', 'pug', 'npm-contributors']);
};

We should enrich our "blogs" section with authors and labels. For example, we could create tabs for "all", "Stryker", "Stryker.NET" and "Stryker4s". We could also add an "About the author" part with a small text about the author with a picture.

My browser is not rendering some special character correctly. Likely it is an emoji for external links. It could juist be a fault of my computer, but just wanted to inform. See this screenshot from the latest blog post at https://stryker-mutator.io/blog/2019-12-27/stryker-dashboard-host-your-mutation-testing-report.

So one of my colleagues just implemented Stryker and I was looking at what it was and how it worked and I had a few comments that I wanted to share.

For starters, the whole mutants, mutators and killed part isn't really selling me on this piece of code. I find it confusing and unclear. I needed to look at the documentation what was actually the good bit and what was the bad bit. I would prefer if there was a way to make these more descriptive or even disable the whole Mutants vibe. It also doesn't look professional.

Then when I look at the results, there's not much telling me what it did and why it went good or bad. When I look at something like this https://stryker-mutator.io/robobar-example/reports/mutation/html/index.html#success-component/success-component.js it doesn't really tell me much. I see the numbers but what do they say and why is the number important? And when clicking them, there's some code inserted but there's very little to go on with what it did and what was expected. I'm guessing you mean that it replaced the test with that and checked whether it failed or succeeded. But the whole survived bit makes it sound like it was good instead of bad. And it doesn't explain why it inserted what it did, it seems quite random and I would imagine it tries more than just 1 change in order to test it is fine but it only shows 1 modification.

The example also doesn't show what the original tests were and what they should be, this makes it easier to understand what it does. I also expected to see more visual examples in the documentation. There's a few code bits, but it never shows you what it will look like when you run it, what the results are and how you should interpret that. And with the whole mutants killing and suriviving, it doesn't make it easier to understand. For a tool that is focussing on improving tests, it doesn't do a good job showing you why, just that its bad. What I mean by this is that on pages like these: https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ I expect to find more about what I need to do to fix this, rather than just "it does x and it breaks" because that doesn't help me write better code and better tests.

I also saw that you need a stryker.conf.json file, adding yet another file to my repository. I wonder if its possible to include the config in other files as well, because I don't really see why this can't be part of something else.

And lastly I expected to find something about integrating with Jenkins or Bitbucket. I do see Travis and Azure/Gitlab, but I don't see these other 2. Whats also strange is that Azure, Gitlab and thus the ones I mentioned aren't supported, which surprised be a lot. It makes stryker unusable for many projects.

I hope this makes clear what my initial experience with this project was and what I think can be improved. Do with it what you will.

We should be able to add blogs to the repo and publish them on the website.

We added two new mutators: The array mutator and the boolean substitution mutators

@avassem85 Already documented the boolean substitution mutator. Now we just have to document the array mutator

You should really think about changing protocol to safe one.
maybe it should work? https://help.github.com/articles/securing-your-github-pages-site-with-https/

@nicojs you should remove line 11 from footer.pug, in head.pug change

 link(rel="stylesheet", href="https://fonts.googleapis.com/css?family=Roboto")
link(rel="stylesheet", href="https://fonts.googleapis.com/css?family=Lato")

to

link(rel="stylesheet", href="https://fonts.googleapis.com/css?family=Lato|Roboto")

and in layout.pug add lang="en" to html tag

@nicojs, @simondel do you want to remove highlight.js or it will stay?

The example (about building an online casino) could be made interactive by using something like Glitch to embed a runnable example. This would basically allow the user to view and edit code/tests files and to run Stryker to see what happens - pretty cool in my opinion 😎

Example here: https://glitch.com/edit/#!/night-lens
(open the embedded console by pressing 'Tools' -> 'Logs', then select the 'Console' tab)

Let's say I want to link to a specific section of a page, maybe the example on the landing page or some specific section of a blog post. It would be conventient to quickly get the direct link to that section, like for instance http://stryker-mutator.io/#an-example.

I like how this works on GitHub where a small clickable icon appears when hovering a heading. Clicking that icon updates the url to point to that section. Looks like this:

Hi, I wrote some code and its unit tests with JavaScript and Jest. I made mutation test on these unit tests.
Stryker lead me to write one extra test. I think this test is meaningless.

Striker lead me to write this extra test

test('throws error for "5" ', () => {
      expect(() => isAge('5'))
        .toThrowError(new Error('age must be a whole number'));
    });

for this mutation

//mutation
-     if (!(typeof age == 'number' && Math.trunc(age) == age))
+     if (!(true && Math.trunc(age) == age))

But, I have already write this test to test isAge function with string type argument.

test('throws error for "5.5" ', () => {
    expect(() => isAge('5.5'))
      .toThrowError(new Error('age must be a whole number'));
  });

This test should be enough. Because with this test I have already tested falsy_value && falsy_value situation.
But Stryker don not think like that. Stryker wants me to write test to check falsy_value && truthy_value situation.
If first operand of Shortcut And Operator falsy, it is not matter what is the value of second operand(is it truthy or falsy), because result is allays falsy.

reproduction steps

Environment Info

Operation System: Ubuntu 18.04.5 LTS
[email protected]
node: v15.1.0
[email protected]

//is-age.js
function isAge(age) {
  if (!(typeof age == 'number' && Math.trunc(age) == age))
    throw new Error('age must be a whole number');
  return true;
}
module.exports = isAge;

//is-age.test.js
const isAge = require('./is-age');
describe('the isAgeFunction', () => {
  test('returns true for 5', () => {
    expect(isAge(5)).toBe(true);
  });

  // Stryker wants this extra test
    /*test('throws error for "5" ', () => {
      expect(() => isAge('5'))
        .toThrowError(new Error('age must be a whole number'));
    });*/

  test('throws error for 5.5 ', () => {
    expect(() => isAge(5.5))
      .toThrowError(new Error('age must be a whole number'));
  });

  // I have already wrote this test to test string argument
  test('throws error for "5.5" ', () => {
    expect(() => isAge('5.5'))
      .toThrowError(new Error('age must be a whole number'));
  });
});

$ npm init -y
$ git init
$ touch .gitignore
$ echo "node_modules/" >> .gitignore
$ stryker init

1. None/other
2. jest
3. html
4. npm
5. JSON

$ stryker run --mutate is-age.js

While I was adding the Stryker4s roadmap to the website I came up with an idea that might be cool.

What if we added a timeline for Stryker? This way you can see big milestones like the first creation, releases etc. Then this could also be applied for roadmaps as well.

I've created a very fast small mockup too show what I was thinking.
@simondel @nicojs @richardwerkman @hugo-vrijswijk what do you guys think of this idea?
I would like your feedback first before I start sinking time into this.

Maybe I'm doing something wrong but these instructions don't seem to work:

Output:

[2017-03-15 14:29:32.963] [WARN] TestFrameworkOrchestrator - Could not find test framework `mocha`. Did you forget to load a plugin? Known test frameworks: [].
[2017-03-15 14:29:32.964] [FATAL] Stryker - Configured coverage analysis "perTest" requires there to be a testFramework configured. Either configure a testFramework or set coverageAnalysis to "all" or "off".

It seems to work with mocha-runner installed. The quick start instructions should probably include stryker-mocha-runner as well as stryker-karma-runner.

I'd say this line is responsible: https://github.com/stryker-mutator/stryker-mutator.github.io/blob/master/javascripts/main.js#L50

While working out a bug in the code I discovered that there isn't much documentation available on how to setup your development environment for stryker.

I think it would be nice to have a few things documented:

• How to clone and install dependencies
• How to run a build
• How to run tests
• How to (temporarily) install packages based on the sources

I will make a pull request to fix this :-)

Remove all generated files from the repo to reduce confusion

We need a calendar for our blog posts in early 2017

Was reading some open source guidelines and found this an excellent idea.

Any interest in this? If yes, should we document this on the website or per repo?

I've noticed there is for Javascript in Navbar. But it is also for typescript, angular, react (which is javascript at all but you know).

Maybe instead of for javascript we should write for Web or sth like this.
What is your opinion?

See changelog

Consider adding service worker for your site, allows it to better cache, show page offline etc.
https://developers.google.com/web/tools/lighthouse/audits/registered-service-worker
https://developers.google.com/web/fundamentals/primers/service-workers/

The description meta tag is broken since our redesign:

Github tries to help us by suggesting to merge back to master every time.

We should never ever ever ever allow that to happen. Sources should be merged to develop. In order to make sure this can never go wrong, I want to recreate "develop" as an orphan branch. Orphan branches basically allows multiple git repos to exist in a single git repo. Develop can then never be merged to master, as they don't share history.

Doing this can be confusing for people that already have a local develop branch tracking the remote. That's why we should do this at a good moment in time, preferably when there are no open PRs

I've created a branch pug-webpack of the work in progress.

The current issue is building the viewmodel. The grunt plugin has a configuration for the data object to give a function, building a viewmodel. As far as I can tell, the pug-html-loader just expects a JSON object.

"Old" example is in Gruntfile.js line 62. "New" example is in (this branch) webpack.config.js line 36.

Css compilation in webpack doens't seem to be minified. (still on redesign branch for now)

I think this was a great post about how stryker can be integrated into an Azure DevOps pipeline.

Yet as there is this know issue, of running dotnet tools right after instaling it on azure, that I personally have been experiencing way too often and Microsoft doesn't seem to be fixing it anytime soon, I think the post should be updated to discuss this and propose a work-around.

If it is possible, I would like all PR connected to "text" to be resolved, then we should all check through all files, if there is proper grammar and no misspells. If we do it until the end of this week, I should be able to show it to native speaker too.

At Stryker.NET we are strugling to find a good place to document our installation instructions, mutators and reporters. In the current situation we have 3 places available:

• stryker-net github
• stryker handbook github
• stryker-mutator.io

None of the locations seem ideal. In the handbook we can keep all stryker versions in one place. But it requires seperate pull requests for every feature. One implementation and one documentation.

On the stryker-net github seems ideal since we can request each PR to update the documentation as well. This makes keeping the implementation and documentation in sync more easy.

Proposal

We would load the markdown found in specific files and place it on the website every night. This gives us one place where all stryker documentation comes together and still gives us the best case of 1 place with implementation and documentation together.

We could even try to give each stryker version a /docs folder in the repo, and create a page for each *.md file found.

Benefits

Placing all documentation on the website creates a great place to discover all about stryker. In the current situation the documentation is partly scatered over different places.

Docusaurus is a way to generate static websites from markdown. It is different from pug (what we're currently using), in that it uses convention over configuration. See https://github.com/facebook/Docusaurus

It would be great if we could generate the website from the different docs directories in the stryker, stryker-net, stryker4s, and mutation-testing-elements repositories. We should move the docs we currently have in the stryker-handbook to one of these 4 (and add a "401 - moved" message on the stryker-handbook once this new website is online).

We should also have some way of authomcatically publishing the latest docs whenever a new version is released (new git tag is created?). That way we can keep the docs up-to-date, but unreleased features won't be visible on the website yet.

TODO:

• Update Docusaurus to latest alpha
• Add FAQ
• Fix docs to Stryker
• Fix docs to handbook. Do after merging?
• Add blog articles and fix blog redirects
• Fix other redirects
• Move stryker handbook docs to website
• Set up mutation-testing-elements docs (and decide on display name)
• Set up new homepage
• Fix contributing guide (min git 2.25 and use git bash on Windows)
• Check all redirect links for the last time

We should document our code and design choices on stryker-mutator.github.io.