lfex / ltest Goto Github PK
View Code? Open in Web Editor NEWA Testing Framework for LFE (successor to lfeunit)
License: Other
A Testing Framework for LFE (successor to lfeunit)
License: Other
Consider (extend-module (export ...) ...)
.
Currently, system, integration, and unit tests call ltest-runner:run/1
and then ltest-runner:run-beams/1
, but they don't pass the test type (e.g., system, integration unit). This set of functions needs to be updated to pass ltest-type
as an option so it can be recorded in the test record when test state is initiated (in ltest-listener:init/1
).
Maybe something that does any standard boilerplate? At the very least probably:
(defwebtest ...)
macro that wraps (deftest ...)
There is an Erlang webdriver here:
Blog post describing it:
Selenium webdriver docs:
Is the LFE test runner usage encouraged instead of the current EUnit one that rebar3
ships with?
It looks like it is meant to be used with lfetool
which is now deprecated in flavor to rebar3
.
===> Failed to fetch and copy dep: {git,"https://github.com/lfex/lutil.git",
{ref,
"0ee02e01d643caba7a0b8365c477aea8d92bf309"}}
For large projects with lots of integration tests, for example, it would be nice to just compile unit tests if only unit tests are going to be run.
#37 (comment) ๐
Tagged by the deftestskip
macro, that is.
This popped up in the lutil project, but it looks like it's because of ltest. It might be a problem with how beams are looked up -- I believe the code was originally designed to use the _build/default
directory, but tests are now being run with the test
profile (and thus the _build/test
directory).
Admittedly the system tests are pretty useless right now, but hey, they should still pass.
I'd like to be able to configure where ltest-runner:all/0
searches for test beams. At the moment, I don't think it can be changed from .eunit/
.
One way to implement this would be to add ltest-runner:all/1
where its argument is an options proplist (or dict?):
(defun all () (all '()))
(defun all (opts)
(let ((dirs (proplists:get_value opts 'dirs '(".eunit"))))
...))
(all '(#(dirs ("_build/test/dir" ".eunit"))))
Though, other ltest-runner
functions would also have to change.
It is a beautiful runner :)
See discussion on lfex/kla#2.
Depends on issue #18
Per TODO on #37.
In order to test lfex/clj
one needs to specify ltest
as a dependency, but currently ltest
depends on clj
. Is there a way to skip this check? I'm using rebar3.
Depends on issue #18
With the latest changes to the rebar.config
, the ltest project has removed the conditions under which cyclic dependencies arise, in particular, with the latest LFE plugin for rebar3
.
In the process of this, though, some cursory examination of project code revealed that much of the work various utility functions are doing have their history in a pre-mature rebar3 solution, during the time of lfetool
.
Tasks:
ebin
dir?: rebar_app_info:ebin_dir
If you're running LFE and ltest in Erlang 17.4 you will find that the unit tests no longer compile. I haven't isolated the exact culprit, but I'm 99% certain this is due to the EUnit fixes that were added to the latest release of Erlang.
I suspect that something along the lines of changing the test macros to use underscores in module names will fix this, but I haven't tried it yet.
Successful use in Erlang 15, 16, and 17.3:
$ make check-unit
Removing EUnit test files ...
removed ./.eunit/*.beam
Cleaning ebin dir ...
Compiling only project code ...
==> my-test-lib (compile)
Compiled src/my-test-lib-util.lfe
Compiled src/my-test-lib.lfe
Removing old tests ...
rm -rf ./.eunit
mkdir: created directory ./.eunit
Compiling tests ...
Successfully compiled test modules.
------------------
Running unit tests ...
------------------
======================== EUnit ========================
module 'unit-my-test-lib-tests'
my-adder ............................. [0.002 s] [ok]
my-sum ......................................... [ok]
my-sum-zeros ................................... [ok]
Total module test time: 0.012 s
=======================================================
All 3 tests passed.
Unsuccessful use in Erlang 17.4:
$ make check-unit
Removing EUnit test files ...
removed ./.eunit/*.beam
Cleaning ebin dir ...
Compiling only project code ...
==> my-test-lib (compile)
Compiled src/my-test-lib-util.lfe
Compiled src/my-test-lib.lfe
Removing old tests ...
rm -rf ./.eunit
mkdir: created directory ./.eunit
Compiling tests ...
Successfully compiled test modules.
------------------
Running unit tests ...
------------------
======================== EUnit ========================
undefined
=ERROR REPORT==== 3-Jan-2015::14:04:40 ===
Loading of /tmp/my-test-lib/.eunit/unit-my-test-lib-tests.beam failed: badfile
*** test module not found ***
**'.eunit/unit-my-test-lib-tests'
=ERROR REPORT==== 3-Jan-2015::14:04:40 ===
beam/beam-load.c(1250): Error loading module '.eunit/unit-my-test-lib-tests':
module name in object code is unit-my-test-lib-tests
=======================================================
Failed: 0. Skipped: 0. Passed: 0.
One or more tests were cancelled.
make: *** [check-unit-only] Error 127
The color output is making it hard to read results in Jenkins jobs.
Proper description to come later.
Check out LFE 1.2.0 to 1.2.1 diff.
This would provide a convenience for the following:
OK
sOK
s lower-caseOK
s with dots_test
in the module nameThis is part of the work that needs to be done as outlined in the comments of lfetool ticket lfe-deprecated/lfetool#111
Note: Also created this ticket in the test runner project: lfe-rebar3/rebar3_lfe_ltest#2
When running the unit tests for the kanin project (see https://github.com/lfex/kanin/blob/master/test/unit-kanin-uri-tests.lfe), an error arises (unspecified ... looks like an undef of some sort). When run with rebar3, this error does not arise.
$ rebar3 as test lfe test -t unit
================================ ltest =================================
------------------------------ Unit Tests ------------------------------
module: unit-kanin-chan-tests
function_checks ................................................... [ok]
export_count ...................................................... [ok]
time: 24ms
module: unit-kanin-conn-tests
function_checks ................................................... [ok]
export_count ...................................................... [ok]
time: 23ms
module: unit-kanin-uri-tests
function_checks ................................................... [ok]
export_count ...................................................... [ok]
parse_net ....................................................... [fail]
Assertion failure:
undef
parse_direct .................................................... [fail]
Assertion failure:
undef
time: 92ms
summary:
Tests: 8 Passed: 6 Skipped: 0 Failed: 2 Erred: 0
Total time: 139ms
========================================================================
$ rebar3 as test eunit
======================== EUnit ========================
file "kanin.app"
application 'kanin'
module 'kanin-chan'
module 'kanin-conn'
module 'kanin-uri'
module 'kanin-util'
module 'kanin'
module 'unit-kanin-chan-tests'
unit-kanin-chan-tests: function_checks_test...ok
unit-kanin-chan-tests: export_count_test...[0.011 s] ok
[done in 0.017 s]
module 'unit-kanin-conn-tests'
unit-kanin-conn-tests: function_checks_test...ok
unit-kanin-conn-tests: export_count_test...ok
[done in 0.006 s]
module 'unit-kanin-uri-tests'
unit-kanin-uri-tests: function_checks_test...ok
unit-kanin-uri-tests: export_count_test...ok
unit-kanin-uri-tests: parse_net_test...[0.004 s] ok
unit-kanin-uri-tests: parse_direct_test...ok
[done in 0.016 s]
[done in 0.208 s]
[done in 0.250 s]
=======================================================
All 8 tests passed.
The new lfeunit docs for LFE.
That was done as part of lfe/lfe#64.
One of the classic problems with putting code in documentation is that it, sooner or later, begins to bit rot. If code in documentation can be run as part of a test suite (e.g., prior to commits or as a gate in a continuous integration platform), then this can be checked (and hopefully prevented).
The Python programming language pioneered this with doctests. Though sometimes hailed as a great abuse of testing (which it can be), when used appropriately (e.g., ensuring that code in documentation continues to be checked as working) it can be a great boon to software projects which strive to provide good example usage of APIs, etc., in documentation.
I suspect this will actually be comprised of a sizable number of other tickets. The following tasks will likely need to be completed to make this a reality:
(defdoctest ...)
macroRight now, each run of eunit only knows about its own state. What we want is a mechanism for tracking state of all eunit runs, not just the one that's running and will get reported on at the end of its run.
We want a report that will display after all eunit runs have been done, summarizing them.
Per TODO on #37.
There is some functionality recently added to lfetool that really belongs here in lunit. The following commits are related to this work:
In particular, the code from lfe-deprecated/lfetool@46e35eb that wasn't moved to lutil in lfe-deprecated/lfetool@23a1ed6 (and lfex/lutil@5776c9b) needs to be moved to lunit for this ticket. It is specific to lunit and can be called from other projects (especially lfetool's test runner) to get lunit skip-test info.
Depends on issue #18
I'm using these in xlfe right now, but it'd be nice to bake something into ltest.
(defmacro is-equal* (x y)
`(_assertEqual ,x ,y))
(defmacro is-error* (x y)
`(_assertError ,x ,y))
is-{,not-}exception{,*}
is-{,not-}error{,*}
is-{,not-}exit{,*}
is-{,not-}throw{,*}
Add convenient unary clauses that pass _
as the first argument to the binary versions.
Sadly, EUnit states are undocumented. The source code has to be examined when attempting to create a 100% EUnit compatible test listener.
The following files need to be compared carefully:
With the latter getting updates for the missing (or commented-out) bits. Note that this set of missing functionality dates from the earlier days of ltest (formerly lunit, formerly lfeunit).
Tasks:
start/2
(not really state management, but we can hit this while we're bringing the rest of ltest_listener
into accord with eunit_listener
)handle_begin
add another clause like for the undefined
description, but for an empty stringhandle_end
add another clause like for the undefined
description, but for an empty stringThere are places in ltest that have to do some very ugly destructuring:
Line 146 in ff9d44e
This is due to Erlang having some pretty ugly (also probably quite old) error-creation for assertions, e.g.:
Let's create some records (just the ones that are necessary) to at least keep our code clean.
These are going to need LOTS OF TESTS.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.