Giter VIP home page Giter VIP logo

cola-tests's Introduction

cola-tests's People

Contributors

bmsantos avatar dcendents avatar dependabot[bot] avatar doyleyoung avatar gitter-badger avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cola-tests's Issues

Should filter test classes

Currently, instrumentation does not have a way to filter out or select the classes of interest to be instrumented. As such, a undesired class might be loaded by the class loader twice, once during instrumentation and a second time during test execution.

Adding a way to select the test classes will prevent such cases.

Should follow maven filter properties

Scenario: Should follow maven filter properties 
Given a maven filter property (-Dtest or -Dit.test)
When the instrumentation filter is processed (-DcolaTests)
Then the processing will respect maven filters

Introduce grouping tag

Feature: Execute classified test groups
Scenario: Should classify scenario
Given a Gherkin scenario tagged with @group
When the tests are executed with -Dcola.group="regex pattern"
Then the matching scenarios will be executed

Should skip Scenario

Given a gherkin Scenario
When tagged with @skip
Then  it should be skipped from execution

Introduce assigned/named parameters

Scenario: Should allow for assigned/named parameters
Given a story step
When an annotation has assigned parameters
Then the value should be injected in the step handling method

Remove default IDE class and method

With the introduction of the IdeEnabler annotation, the need for a default class and method name is no longer required.

  • Remove default from cola-tests
  • Remove default from cola-maven-plugin
  • Remove default from cola-gradle-plugin
  • Remove default from cola-idea-plugin

Introduce pre and post steps

Sometimes is useful to call pre and/or post steps without having to explicitly include them in the scenario. This complements "scenario backgrounds" and allows for extra flexibility. E.g.:

@PreSteps({ 
  "Given the details of multiple accounts",
  "When the accounts are created" }
@Given("a user")
public fun findAvailableUser() {
    user = db.loadFirstUser()
}

Should clearly identify failed step

Feature: Log failures
  Scenario Outline: Should identify failed scenario step
    Given a scenario with a problematic <step>
    When the scenario is executed
    Then the failed <step> will be displayed in error message

Examples:
| step |
| Given |
| When |
| Then |

Should report compile time error

Currently users have to use the IDE error log (e.g. Eclipse Error Log view) in order to visualize the compile time error.

It would be much nicer to have the compile time errors reported to the console when the junit test is executed.

Consider injecting a test method that will always fail but used to simply report the compile time issues.

Introduce reporting capabilities

Scenario: Should report results
Given a cola test with a report setup
When the test is executed
Then the result will be reported

Notify when examples are not provided

No error or notification is provided when there's a malformed Examples:

Feature: Malformed examples example
  Scenario Outline: Malformed examples
    Given A
    When B is <foo>
    Then C

  Examples:
  | foo
  baa
  boo

A correct version would be:

Feature: Examples example
  Scenario Outline: Examples
    Given A
    When B is <foo>
    Then C

  Examples:
  | foo |
  | baa |
  | boo |

Introduce IDE enabler annotation

Feature: Introduce IDE enabler annotation
  Scenario: Should remove method
    Given a JUnit test method annotated with @IdeEnabler
    When the test is run
    Then the method will not be executed

Introduce Filter Mechanism

Introduce a filtering API to process filters with a common interface to both Tags and Reports.

The implementation must take into consideration the @Skip and @ignore tag and prevent further filtering when present.

Should allow specialized JUnit constructors

Scenario: Should allow specialized JUnit constructors
Given a JUnit test with a specialized constructor
And a @Feature annotated field
When the test is COLA Tests compiled
Then test methods should be injected

This error can occur when using JUnit Runners like Parameterized that use a specialized constructor to inject parameter values.

Two possible solutions:

  1. Keep using reflection and provide basic values to constructor (improve current solution)
  2. Extract value from bytecode (might not work due to current bug in ASM)

Workarounds for current issue:

  1. Use @Features class annotation
  2. Use default stories field

Would be nice to have a better IDE integration

Hi,

I know this is a big one, and you probably want to focus more on the core functionalities first. But I think it would be very nice to have a better IDE integration (I personally use eclipse). Something basic, like right-clicking on a step in the feature file and being able to jump to the implementation.

I saw there are some plugins in eclipse for other BDD frameworks, maybe it's possible to extend them to add support for Cola annotations?

Anyway I thought i'd open a feature request. I understand if you don't tackle it in the near future.

Cheers

Introduce group exclusions

Feature: Exclude tagged scenarios

Scenario: Should exclude tagged scenarios
Given a tagged scenario
When excluded from execution
Then the scenario will not be executed
But all other scenarios will

Scenario: Should exclude tagged feature
Given a tagged feature
When excluded from execution
Then the feature will not be executed
But all other features will

Should skip Feature

Given a gherkin feature
When tagged with @skip
Then  it should be skipped from execution

Ignore scenario (warns when skipped)

Scenario: Should ignore scenario
Given a gherkin Scenario
When tagged with @ignore
Then it should be skipped from execution
But warn users when skipped

Should fail when the feature file cannot be loaded

Hi,

I'm starting a new project at work which requires ant to build. So I'm using instrumentation to run the JUnit tests, both in ant and in eclipse, which works fine.

However I was surprised to see that the following test did pass given that I don't have the feature file in the project. I wanted to see it fail to make sure the instrumentation was working correctly.

@Features("blabla")
public class BlablaTest extends AbstractBaseColaTest {

    @Test
    public void something() {

    }

    @Given("a <beverage>")
    public void chooseBeverage(@Projection("beverage") final String beverage) {
    }

    @When("a google search is performed")
    public void performGoogleSearch() {
    }

    @Then("the <beverage> wikipedia entry will be present")
    public void verifyBeverageIsPresentInWikipedia(@Projection("beverage") final String beverage) {
        fail("Failed!");
    }
}

In a strict BDD test class, I suppose there would not be @test annotations, so it would fail with a message: java.lang.Exception: No runnable methods

But given a test class with a Features annotation, I think the tests should fail if any of the feature file(s) listed cannot be loaded.

On a side-note, this is the first time I'll use it at work, it's a small project where I am the only developer so the context is perfect. I'll do a presentation later this year on BDD to my colleagues and I hope I'll convince more people to use it.

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.