The build is failing on JRuby, but passing on the other Rubies despite having two failing specs.

It seems that the rake task exits with 0 even when there are scenarios failing. @tooky can you take a look at this?

This should be simple now that #18 is done.

Help others contribute to the code base by explaining the core execution model

The Table class we brought over from Cucmber 1.0 has too many responsibilities, many of which don't belong in the core. We should pare it right down to the bare minimum we need, then add features back in as we find they're needed.

This will allow us to show the duration of a step, test case or test suite.

See cucumber-attic/gherkin#18

See https://github.com/cucumber/cucumber/wiki/Hooks

@mattwynne do we need to support these?

https://github.com/cucumber/cucumber/wiki/Hooks#step-hooks

Documentation for devs

Currently there is no support for pending steps.

Currently what happens in cucumber when a step of the form

Given(/^pending$/) { pending }

is executed it raises a Cucumber::Pending error. In the test runners for cucumber-core this error is not rescued and so pending steps and scenarios are not reported. It also skips After hooks.

I think we need to move Cucumber::Pending to core and change the test runners to handle pending steps.

Hello,

I see a difference between running a Scenario outline and a Scenario. If I run a scenario and error is raised in Before hook, then scenario is marked as failed and steps are skipped.

With a Scenario outline though, the steps are executed but at the end scenario is marked as failing. No backtrace is printed out. This can be rather confusing.

Right now the Reports just implement an API made of these methods:

before_test_case
before_test_step
after_test_step
after_test_case

@tooky and I have decided this should be whittled down to a single method, register_runtime_events passing in a class that can collect blocks to be called back on the report. This gives the reports more control about which events they're interested in, so that the runner doesn't need to waste time calling events that the reports don't care about.

They'll need tagging out of the main build because they'll be slow.

As @e2 noted (cucumber/gherkin#334), the ast class for Background has an attr_accessor for :feature, but since the #feature is never set on the background object, #feature is always nil. Therefore things like background.feature.name in for instance #before_background in formatter will not work.

As described in cucumber-attic/gherkin#6, Gherkin3 will provide an Execution Tree (ET) which is intended to make it simpler for Cucumber to execute and report results.

Since Gherkin3 is several months from being useable it makes sends that this project uses Gherkin2 to build an ET similar to what Gherkin3 will do in the future.

I think it would be great if the temporary builder/compiler/AST/ET lives in a separate module that can be easily swapped out when Gherkin3 is ready.

I created this ticket primarily for discussion to make sure everyone understands the plan. Please weigh in with questions and comments.

Also blank feature names

Right now we use the result classes, but saying that a Test::Case is Passed when it hasn't finished running is inaccurate. We can use the Result classes to store the result, but we should have an intermediate set of Status classes (something like Unknown, Passing, Failed, Passed) that we use during execution, then only talk about a Result when we've finished.

Thanks to @aslakhellesoy for this suggestion.

Since the mapper is just a special case filter anyway, maybe we don't need to have it here. If we leave it in the core, it will just become harder to change as other people will build things around it.

I wonder if this would help resolve my issues with it having an awkward API.

@tooky WDYT?

It would be nice if after a successful build we could kick off a build of the cucumber/cucumber tests integrating the changes.

As pointed out by @pdswan.

The current version of cucumber raises a TagExcess before executing the test suite. Cucumber core will only raise a TagExcess after it has executed the test suite.

This issue needs to be addressed for cucumber 2.0 to be compatible with the current version of cucumber.

This relates to cucumber/common#574.

This might be needed as an interim step before #20 is implemented. In implementing #13, I realised why the current Cucumber passes an array of tag expressions to Gherkin::TagExpression. It's because Cucumber's CLI currently uses multiple expressions to communicate AND logic.

Unfortunately (for both Test::Case.match_tags? and for the filter: option on Core#compile and Core#execute) I chose to accept only a single expression.

This won't integrate with the current front-end until we can pass multiple expressions.

And upgrade whatever it finds.

The name of a test case for an example scenario from an Scenario Outline is composed as:

"<outline name>, <example table name> (row <number>)"

The word "row" is hard-coded in English (in case.rb). "Row" will look a bit odd when using a language other the English in the feature file (especially when using a language not using the Latin alphabet).

Would it be better to use "#" instead of "row", to make the row designation more language neutral?

"<outline name>, <example table name> (#<number>)"

We can use bool to do the tag expressions.

The Compiler needs to be extended to compile Before / After hooks into TestSteps which the SuiteRunner can then run.

Issue: I'm updating a gem that reports on scenario success/fail plus scenario source to an external reporting tool via a REST api. It doesn't work in Cucumber 2.0 so I'm trying to learn how to do it in the Cuke 2.0 world and can't find enough documentation to get started.

I've read through the example of using a reporting class and getting it to work but I'm mystified by two things;-

1. How do I plug in a reporting class into real world Cucumber? I can't believe I'd have to write my own runner and plug the reporting class into that. How do I register my reporting class with real world Cucumber in the features/support/env.rb file ?
2. Having an api to send messages to a report class that responds to certain methods is cool, but Cucumber::Formatter::Html does loads more. Does Cucumber::Formatter::Html behave as a report class, or is it a special case that has a different api?

Is there any documentation or online email conversation I can be pointed to so I can learn how to catch the messages that happen as Cucumber runs and then send information to an external reporting tool?

Thanks

John Small

I was running the tests e noticed that Cucumber::Core::Test::DebugFilter has no test coverage.
Does cucumber/cucumber-core uses it? or can I remove?

Criteria:

1. immutable
2. equality comparison works (by location?)

See 2e85fcb#commitcomment-10458359

Still need to decide the best way to do this, but the Test::SuiteRunner is going to need to yield to the hooks as it runs each Test::Case so that we can allow for Around hooks.

Right now there's no way to pass in the path of a file when you parse it (we just default it to UNKNOWN), which means the location on the AST isn't useful when you want to print it (e.g. to show where a failed scenario is located).

We should change the way the Core#parse method works so you can (or have to) pass in a path with the gherkin doc.

I'm investigating why the wire protocol scenarios on the front-end are leaving wire servers lying around. The reason is that when the scenario fails, the after hook that cleans up and kills the wire server isn't being run.

We need to change the logic here. I think it may be as simple as changing the behaviour of HookStep#skip to still execute.

The build is broken for jruby since I introduced https://github.com/cucumber/cucumber-ruby-core/blob/master/spec/readme_spec.rb

We just need to switch to a different markdown parser that works on jruby, I think.

I noticed this from the work I was doing to get the html formatter from cucumber working.

The Line number for a Ast::DataTable::Cells instance is thrown away after parsing a DataTable.

Consider this code:

# lib/cucumber/core/ast/multiline_argument.rb
def from(argument)
  #...

  case(rubify(argument))
  when String
    #...
  when Array
    Ast::DataTable.new(argument.map{|row| row.cells})
  else
    #...
  end
end

In the Array case the argument looks like this...

[<Gherkin::Formatter::Model::DataTableRow:... @comments=[], @cells=["name", "surname"], @line=5>,
<Gherkin::Formatter::Model::DataTableRow:... @comments=[], @cells=["rob", "westgeest"], @line=6>]

So when the below is done...

argument.map{|row| row.cells} #=> [["name", "surname"], ["rob", "westgeest"]] 

the line numbers are lost.

Ast::DataTable copes with this by rescuing and returning -1 when creating it's cell matrix on initialization.

#lib/cucumber/core/ast/data_table.rb:260
def create_cell_matrix(raw) #:nodoc:
  @cell_matrix = raw.map do |raw_row|
    line = raw_row.line rescue -1
    raw_row.map do |raw_cell|
      new_cell(raw_cell, line)
    end
  end
end

This impacts the dom ids that the cucumber html formatter produces for datatable rows... i.e. all of them will be #row_-1.

This looks like a bug. Does this need to be addressed?

I heard RubyRogues No. 123 recently and it sounds as though there are some little gems int he ROM ecosystem we could re-use here.

For example, https://github.com/mbj/concord and/or https://github.com/dkubb/equalizer could be used to replace Cucumber.initializer.

Also, we could use https://github.com/dkubb/adamantium for making our value objects immutable.

It was a bad design decision to make tables mutable, and it has caused many bugs. I suggest that in the rewrite, mutators are only added for backwards compatibility, and that they issue a BIG warning that non-mutating methods should be used instead (returning a new table).

If we run in --dry-run mode, we should report all undefined steps.

Even if a step is skipped, we should report its undefined-ness.

Steps arguments are not created equal.
For instance, 'normal' step arguments that are created as capturing group
in the regular expression of a StepDefinition can be subjected to a Transform step.
Oddly enough, step arguments that come from an AST::Table cannot be Transformed.
This is annoying when, like my team mates, Transforms are used to do text substitutions
and the expectations are that the substitutions are performed for every step data value.
As a workaround to that weird limitation, I wrote long time ago a method for the World object that
invoked the Transform upon the AST::Table elements.

This worked fine until recently we got the warning:
[warning] map_column! will be deprecated

Then I realize that the idea of making Data Table immutable will only break the workaround
but will make the Transform steps completely ineffective for AST::Table.

What one needs is an implementation that follows the Principle of Least Surprise:
-One should be able to perform the same operations/transformations on step arguments
regardless the syntax used to create them (i.e. via a regexp capturing group, a multiline text or a data table).

See #73 (comment)

User would like to be able to debug / pry a failed step, then resume the scenario. I think this should be possible.

When a Test::Step fails, the Test::SuiteRunner needs to skip the remaining steps, ideally marking the undefined ones as undefined.

Gherkin3 is a brand new implementation and has some major differences to the current parser.

cucumber-core's compiler will eventually move into Gherkin3 and we will only need to deal with test cases. We want to modify the compiler here, in-place, before we extract to gherkin3.

ref cucumber/common#830
ref cucumber-attic/gherkin#12

This code was copied and pasted in from Cucumber 1.x's master branch and contains a lot of code that we don't need anymore. The #accept methods can all go, anything to do with handling results can go. Those classes can be immutable now.

See discussion in #13.

This will require a translation layer (we can put this in the front-end) to translate between bool's expressions and the ones used in Cucumber.

The tests generate a ton of warnings right now. We should make the build fail if there are warnings - and get rid of them.

Easier to understand for people.

Easier to understand for YARD.

No need for this shorthand now the code has stabilised.

I'm finding it confusing with these terms like Mapper and Mapping. I think that the verb map is appropriate, but I don't like the name Mapping for the thing that results. I think Action is a better name:

A step is mapped to an action

I'd like to rename Cucumber::Core::Test::Mapping to Action

Budded off from #15.