luajit / luajit-test-cleanup Goto Github PK
View Code? Open in Web Editor NEWCleanup Workspace for LuaJIT tests
Cleanup Workspace for LuaJIT tests
I suggest we use a declarative data-driven format to organize the test cases in each individual test file so that it is independent of the actual test runner and framework. A good test specification syntax I've been using myself for many years is TestML:
http://testml.org/specification/language//index.html
This way we can free ourselves in using what way or what combinations of ways to run the tests without touching the test files themselves.
This also makes it easy when we add unit tests for LuaJIT's internal pipelines (like individual JIT compiler's optimization passes and intermediate IR form) in the future.
I hope we can draw a clean line between test case representation and test running strategies.
ubuntu@arm:~$ ./sandbox/orig/LuaJIT/src/luajit LuaJIT-test-cleanup/test/test.lua 220
[220/508] lib/string/format/num.lua --- ExploringBinary.com/print-precision-of-dyadic-fractions-varies-by-language/
LuaJIT-test-cleanup/test/lib/string/format/num.lua:13: expected string.format("%.99e", "0") == "4.940656458412465441765687928682213723650598026143247644255856825006755072702087518652998363616359924e-324", but got "0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e+00"
stack traceback:
[C]: in function 'error'
LuaJIT-test-cleanup/test/lib/string/format/num.lua:13: in function 'check'
LuaJIT-test-cleanup/test/lib/string/format/num.lua:163: in function <LuaJIT-test-cleanup/test/lib/string/format/num.lua:160>
[C]: in function 'xpcall'
LuaJIT-test-cleanup/test/test.lua:378: in function 'execute_plan'
LuaJIT-test-cleanup/test/test.lua:413: in main chunk
[C]: at 0x00469c01
0 passed, 1 failed
ARM system, luajit revision: 1d7b5029c5ba36870d25c67524034d452b761d27
Devise a logical naming scheme and hierarchy / directory structure for the tests.
This issue will track progress for adding the test suites from Lua (http://www.lua.org/tests/) to this repository. Modifications will be made to the test suites where necessary to make them work with LuaJIT.
It maybe possible to backport additional tests from 5.2.x and 5.3.x series where the language & libraries are compatible.
I will do the work on a branch initially and once working will merge it to the master.
License: The Lua Test suites are under MIT license.
Anyone who wants to help with the cleanup effort and feels qualified, please apply here for commit access to this repo.
Note: this does not automatically grant you commit access to the LuaJIT main repo, which is a tad more sensitive.
Test 366 is failing for me with the v2.1 branch (commit LuaJIT/LuaJIT@d3e36e7):
$ ~/git/luajit/src/luajit test.lua 366
[366/508] lib/contents.lua --- pre-5.2 table
lib/contents.lua:80: got: "concat:foreach:foreachi:getn:insert:maxn:move:remove:sort"
expected: "concat:foreach:foreachi:getn:insert:maxn:remove:sort"
stack traceback:
[C]: in function 'error'
lib/contents.lua:17: in function 'check'
lib/contents.lua:80: in function <lib/contents.lua:79>
[C]: in function 'xpcall'
test.lua:378: in function 'execute_plan'
test.lua:413: in main chunk
[C]: at 0x00404a70
0 passed, 1 failed
Any tips on how to resolve this?
Howdy! I am a first-time contributor with PR #9 and I have some questions:
Hello.
$ luajit test.lua 219
[219/492] lib/string/format/num.lua --- ExploringBinary.com/print-precision-of-dyadic-fractions-varies-by-language/
lib/string/format/num.lua:13: expected string.format("%.99e", "0") == "4.940656458412465441765687928682213723650598026143247644255856825006755072702087518652998363616359924e-324", but got "0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e+00"
stack traceback:
[C]: in function 'error'
lib/string/format/num.lua:13: in function 'check'
lib/string/format/num.lua:163: in function <lib/string/format/num.lua:160>
[C]: in function 'xpcall'
test.lua:378: in function 'execute_plan'
test.lua:413: in main chunk
[C]: at 0x55fc2171a080
0 passed, 1 failed
x86_64 system, luajit revision: e296f56b825c688c3530a981dc6b495d972f3d01
This repo is cool! I am really happy to have a test suite. This seems great for people who want to maintain their own branches and keep track of how they compare with everybody else's. Like, have I broken something? Have my optimizations worked? Has somebody else made some optimizations that I should merge? etc. Just now I would like to maintain a branch called lowlevel
to soak up things like intrinsics and DynASM Lua-mode so this is right on target for me.
I whipped up a Continuous Integration job to help. The CI downloads the latest code for some well-known branches, runs the benchmark suite 100 times for each branch, and reports the results. This updates automatically when any of the branches change (including the benchmark definitions).
The reason I run the benchmarks 100 times is to support tests that use randomness to exercise non-determinism in the JIT, like roulette
(#9). Repeated tests mean that we can quantify how consistent the benchmark results are between runs, and once we have a metric for consistency then it is more straightforward to optimize (see LuaJIT/LuaJIT#218).
The branches I am testing now are master
, v2.1
, agentzh-v2.1
, corsix/x64
, and lukego/lowlevel
. If anybody would like a branch added (or removed) just drop me a comment here. Currently the benchmark definitions are coming from my fork because I wanted to include roulette
to check that variation is measured correctly.
Screenshot of the first graph (click to zoom):
and links:
Hope somebody else finds this useful, too! Feedback & pull requests welcome. I plan to keep this operational.
Split up the tests in to many small tests, and convert them into the format required by the testing framework where applicable. Reorganize them into the new directory structure.
Agree on a testing library and a test runner with minimal dependencies, preferably Lua/shell only.
Test cases should still be easy to run without any (big) dependencies.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.