codewars / ttester-codewars Goto Github PK

Sometimes some test assertions require custom failure messages. Is it supported by the test framework? There are of course always the clunky myjinxin way:

1. Set one variable to string A if condition passes, else set it to string B
2. Write a test assertion that compares said variable should equal string A
3. If test fail, the message would say something along Expected "condition fulfilled", got "did not fulfill condition"

But something cleaner and easier to you would be better.

A picture worth 1000 words:

Some commands like see and discode does not work properly on the runner. see works fine for colon-defined functions but fails for built-in operators (after printing something like Code +):

in file included from *OS command line*:-1
tests.4th:4: No such file or directory
see >>>+<<<
Backtrace:
$7F4CA0458C40 throw 
$0 
$7F4CA0472298 slurp-fid 
$7F4CA0472558 gdb-addr-sep-char 
$A 
$7F4CA046D6A8 discode 
$7F4CA046E060 seecode 
$7F4CA046E5F0 xt-see 
$55C57843C1C8 
$7F4CA046E848 name-see

Which is related to how discode would fail:

in file included from *OS command line*:-1
tests.4th:4: No such file or directory
' + 8 >>>discode<<<
Backtrace:
$7F0FBE510C40 throw 
$0 
$7F0FBE52A298 slurp-fid 
$7F0FBE52A558 gdb-addr-sep-char 
$A

(I think there are a few more commands that don't work but I forgot which)

It's not Undefined word, so GForth recognizes discode, but failed due to some other reasons.

But according to official documentation at least, shouldn't discode fall back to dump if no disassembler is available? dump works fine on the runner though. Did something pretend there are disassemblers and only fail when actually trying to call them (due to something like file permission)?

If I want to test a function that pop/push variable number of items on the stack depending on input (think of registers, different opcodes consume/produce different number of items from the stack and touches different amount of registers), is there a good way to generate random tests for it without resorting to complicated workarounds like custom validation function while relying on #12? Test assertion code seems to be static so I cannot issue dynamic/conditional test assertions at runtime. Or can I use something else to generate the test code?

Let's say I have the simplest function ever:

: cat ;

s" example" describe#{
  s" returns what is passed in" it#{
    <{ 1 cat -> 1 }>
  }#
}#

The integer stack actually has 4 values below the input argument:

Dropping past them will cause problems. Besides #1, it also causes the display to become something like Wrong number of results, expected -3 , got -4.

If user code produces more than required values on the FP stack, WRONG NUMBER OF FP RESULTS: run-tests is displayed.

1. This message is displayed after "Test passed/failed" message.

2. Shouldn't this cause test to fail? If the normal stack has incorrect number of integers the test will fail with Wrong number of results, expected 1 , got x after all, so FP stack probably should have the same behaviour.

Okay, this is related to #6 but is (probably) a separate issue of itself because it means depth will return the wrong result and doing varargs operations will require the size appended at the end, which is not desirable for various reasons.

e.g:

: sum 0 begin depth 1 > while + repeat ;

s" example" describe#{
  s" returns sum" it#{
    <{ sum -> 0 }>
    <{ 5 sum -> 5 }>
    <{ 4 5 sum -> 9 }>
    <{ 1 2 3 4 5 sum -> 15 }>
  }#
}#

On the runner depth 1 > needs to be changed to depth 5 > because the original stack already has some values on them.

Shouldn't those initial 4 values (presumably test time information) be stored on the return stack (temporarily) instead? Or at least be isolated to the user so they aren't in the open and become a nuisance to user code's operations.

About a function "buddy" that returns two numbers on the stack.
In each case input is: 23 4669. Correct answer is: 48 75.

test: <{ 23 4669 buddy -> 48 75 }> (expected is correct)
function returns: 48 75 (correct result)
test answer: test passed

test : <{ 23 4669 buddy -> 48 750 }> (expected is wrong)
function returns: 48 75 (correct result)
test answer: Expected 750 48 , got 75 48 (numbers are swapped)

test: <{ 23 4669 buddy -> 75 48 }> (expected is wrong)
function returns: 48 75 (correct result)
test answer: Expected 48 75 , got 75 48 (expected and actual are swapped)

test: test : <{ 23 4669 buddy -> 48 75 }> (expected is correct)
function returns: 75 48 (wrong result)
test answer: Expected 75 48 , got 48 75 (expected and actual are swapped)

As it currently stands a trivial r> can bypass any it block, which is probably too trivial to do. Doing it more sometimes terminates the tests immediately depending on the number of r>s.