[RFC] Case based / BDD organization strategy for tests about greenwood HOT 4 CLOSED

Yeah, I think what you're aiming and what I'm decribing are essentially having some sort of low level shared "smoke tesys" we can easily run across multiple describe / files. Like something in addition to TestSetup that could be pulled into each test. (it could also maybe include the repeated before / after we use too)?

I think as long as each test case clearly own its own specific assertions and the setup / configuration is still in the hands of each test, and we keep this utility fairly low level and a convenience for just easily sanity testing a suite of tests before moving onto case specific it, then that seems like a good balance to work towards. 👍

ex.

const TestSetup = require('./setup');
const TestUtils = require('./utils');

describe('building greenwood with custom userWorkspace directory', () => {
  before(async () => {
    // custom userland config
     const config = {
       userWorkspace: 'www'
     };

     // get a custom configured context
     CONTEXT = await setup.init(config);

     // setup a testing environment
     setup = new TestSetup(CONTEXT);
     setup.setupWorkspace();  // fs.mkdirSync, etc
    
     // run the tool
     await setup.run(['./packages/cli/index.js', 'build']);
  });

  it('should pass all smoke tests', async () => {
    TestUtils.runSmokeTest(CONTEXT);
  });

  it('should test some other things...', async () => {
    // maybe it would be good to update our CONTRIBUTING.md to add best practices, tips for writing unit tests for the project?
  });

  after(async() => {
    setup.teardownWorkspace();  // fs.remove, etc
  });

});

IDK. My main concern is really limiting what happens in this line

TestUtils.runSmokeTest(CONTEXT);

The value of testing is recreating the users experience using our tool, too much abstraction could keep us too far removed from that and miss bugs / make assumptions not specific to how a user would actually be doing it.

from greenwood.

I agree, but the way I see it there are 2 types of tests that have to be done:

1. tests on the default template
2. tests on the user workspace

each of these cases, may need to test 1 or both those types of tests.

the user workspace test should include nested directories. So for example, if we're testing a config, I want to test all userworkspace related things, including nested directories.

I'm definitely in favor of splitting it up but there's a number of shared tests.

from greenwood.

Pretty good watch on general testing strategies
https://www.youtube.com/watch?v=cB5WZgMwdpE

from greenwood.

PRs to watch for coverage here

• #66
• try and get testing of HTML output lost in #58

from greenwood.