#assert_all support about minitest HOT 4 CLOSED

assert_any and assert_all have been discussed before and the general consensus was that it wasn't that compelling. (can't find the discussion, so maybe it was in irc or twitter or something?) It can certainly be ported over to work, but I'm not sure it fits in minitest itself.

from minitest.

Before filing an issue, I searched a lot about "assert_all minitest" but didn't find anything relevant. I don't see any reason got it not to fit into minitest. To me it seems same as it fits in test/unit.

The way it is implemented in test/unit, it can neatly show which values from collection failed. It is also much faster than using a loop with individual assertions. Also it doesn't screw assertion number statistics. And the reporting format is much more useful.

As an extreme example, I have the following test to make sure randomly generated entries stay within expected boundaries:

  test "entries doesn't lean towards extremes" do
    fake_entry = sample_entry(close: 10_000)
    assert_all(100_000.times) do
      fake_entry = fake_entry.succ
      (5_000..15_000).member? fake_entry.close
    end
  end

So at the end it is a single assertion reported (instead of 100k) and the report of a failure looks like this:

<#<Enumerator: 100000:times>> was expected to be all true values with the given block but was
<{0=>true,
 1=>true,
 2=>true,
 3=>true,
 4=>true,
 5=>true,
 6=>true,
 7=>true,
 8=>true,
 9=>true,
 10=>true,
 11=>true,
 12=>true,
 13=>true,
 14=>true,
 15=>true,
 16=>true,
 17=>true,
 18=>true,
 19=>true,
 20=>true,
 21=>true,
 22=>true,
 23=>true,
 24=>true,
 25=>true,
 26=>true,
 27=>true,
 28=>true,
 29=>true,
 30=>true,
 31=>true,
 32=>true,
 33=>true,
 34=>true,
 35=>true,
 36=>true,
 37=>true,
 38=>true,
 39=>true,
 40=>true,
 41=>true,
 42=>true,
 43=>true,
 44=>true,
 45=>true,
 46=>true,
 47=>true,
 48=>true,
 49=>false,
 50=>false,
 51=>false,
 52=>false,
 53=>true,
 54=>true,
 55=>true,
 56=>true,
 57=>true,
 58=>true,
 59=>true,
 60=>true,
 61=>true,
 62=>true,
 63=>true,
 64=>true,
 65=>true,
 66=>true,
 67=>true,
 68=>true,
 69=>true,
 70=>true,
 71=>true,
 72=>true,
 73=>true,
 74=>true,
 75=>true,
 76=>false,
 77=>true,
 78=>true,
 79=>false,
 80=>false,
 81=>false,
 82=><...>930=>true,
 99931=>true,
 99932=>true,
 99933=>true,
 99934=>true,
 99935=>true,
 99936=>true,
 99937=>true,
 99938=>true,
 99939=>true,
 99940=>false,
 99941=>false,
 99942=>false,
 99943=>false,
 99944=>false,
 99945=>true,
 99946=>true,
 99947=>false,
 99948=>false,
 99949=>true,
 99950=>true,
 99951=>true,
 99952=>false,
 99953=>false,
 99954=>false,
 99955=>false,
 99956=>true,
 99957=>true,
 99958=>true,
 99959=>true,
 99960=>true,
 99961=>true,
 99962=>true,
 99963=>true,
 99964=>true,
 99965=>true,
 99966=>true,
 99967=>true,
 99968=>true,
 99969=>true,
 99970=>true,
 99971=>true,
 99972=>true,
 99973=>true,
 99974=>true,
 99975=>true,
 99976=>true,
 99977=>true,
 99978=>true,
 99979=>true,
 99980=>true,
 99981=>true,
 99982=>true,
 99983=>true,
 99984=>true,
 99985=>false,
 99986=>false,
 99987=>false,
 99988=>false,
 99989=>false,
 99990=>false,
 99991=>false,
 99992=>false,
 99993=>false,
 99994=>false,
 99995=>false,
 99996=>false,
 99997=>true,
 99998=>true,
 99999=>true}>
/home/user/ws/coin/trade/test/price_entry_helper_test.rb:8:in `block in <class:PriceEntryHelperTest>'
    (eval):12:in `run'
    /home/user/.local/share/JetBrains/IntelliJIdea2023.1/ruby/rb/testing/patch/testunit/test/unit/ui/teamcity/testrunner.rb:93:in `start_mediator'
    /home/user/.local/share/JetBrains/IntelliJIdea2023.1/ruby/rb/testing/patch/testunit/test/unit/ui/teamcity/testrunner.rb:81:in `start'
...
116 tests, 316 assertions, 1 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications

Note that the shortening in the middle was automatically done, it was not me snipping that part.

To me it makes it much more understandable. And this is probably there are better examples. Another example would be where it read examples from a file and verifies calculated value matches what was in the text file record:

  test "table of illumination" do
    assert_all(time_percent_pairs) do |time, percent|
      percent == (LunarPhase.illumination(time) * 100).round
    end
  end

While in the second example the number is for example 20 or 30, this will not skew so much the statistics about assertions count, it is much more useful to see outright all or most values that failed, than later figuring that out in another way. I mean when one sees a summary of all that failed and passed, it is often possible to immediately deduce what was the problem.

Yes, for sure one can use other ways to achieve similar effect. But this assert_all just works in a really nice way, reports in a really nice way and keeps statistics realistic. So for me it is a net win for a unit (or any) testing framework.

P.S. Also the ability to define setup/teardown and test with class methods instead of defining the methods is very neat IMO. Especially the setup/teardown that are more naturally inherited between sub-classes. If you know any prior discussion about that, please let me know to check it out. Using ActiveSupport to get that or winding up my own implementation may not be too hard. But why not have it out of the box?

from minitest.

Minitest and test/unit aren't the same; they don't have the same goals or design. Your output above is NOT a compelling argument for adding this to minitest. It's literally 80% noise. As I said before, it's been discussed. General consensus was that it wasn't compelling enough to add. You're welcome to port them over and publish them as a gem if you want.

You can get setup/teardown/test via methods if you use minitest/spec instead of minitest/test... but then you lose inheritability of the test methods themselves.

from minitest.

Well, sounds strange to me. I didn't read or find anywhere something to show me what exactly different goals make it not compelling for minitest.

It's literally 80% noise.

Your statements are just information about decision taken in the past without any of the rationale so it doesn't open any opportunity for me to provide more useful input by addressing particular points. Or to accept these points and agree that it is not compelling. Basically a dead end for a discussion and a dead end for a project developent imho.

I just tried to be helpful. Feel free to close the issue if you're sure that community is not interested in this.

from minitest.