maclobdell / mbed-test-runner Goto Github PK
View Code? Open in Web Editor NEWSimple Mbed OS test runner for running automated tests on multiple boards and toolchains
Simple Mbed OS test runner for running automated tests on multiple boards and toolchains
Few things:
It complains about test_results not being defined.
I think this line requires extra tab.
https://github.com/maclobdell/simple_mbed_test_runner/blob/create_scorecards/generate_scorecards.py#L96
I've specified a folder to analyze files using option "-f .\test_output\master"
However, it looks only one folder is analyzed.
It's hard to get the overall picture of what happened after the test.
Could we also generate a master json object that can later be exported using json2html module?
@maclobdell I've tried version 28374ec
The current version is using sudo
. This obviously doesn't work on Windows, but we should also consider removing it to run on Linux with user context.
take advantage of efficiency of building while tests are running
Launching tests as shown here:
python ..\simple_mbed_test_runner\run_all_tests.py
Error:
+------+--------+-----------+------------+-------------+
| name | target | toolchain | static_ram | total_flash |
+------+--------+-----------+------------+-------------+
+------+--------+-----------+------------+-------------+
Traceback (most recent call last):
File "C:\mbed\mbed-os-multi\tools\test.py", line 288, in <module>
status = print_report_exporter.report(build_report)
File "C:\mbed\mbed-os-multi\tools\test_exporters.py", line 92, in report
return self.exporter_print(test_summary_ext, print_log_for_failures=print_log_for_failures)
File "C:\mbed\mbed-os-multi\tools\test_exporters.py", line 345, in exporter_print
raise Exception("'test_run' did not have a 'result' value")
Exception: 'test_run' did not have a 'result' value
[ERROR] 'test_run' did not have a 'result' value
[mbed] ERROR: "c:\python27\python.exe" returned error.
Code: 1
Path: "C:\mbed\mbed-os-multi"
Command: "c:\python27\python.exe -u C:\mbed\mbed-os-multi\tools\test.py -t iar -m KL82Z --source ..\..
"
Tip: You could retry the last command with "-v" flag for verbose output
Exception raised here: https://github.com/ARMmbed/mbed-os/blob/master/tools/test_exporters.py#L345
I need to investigate how to fix this.
[Breaking up #6 into its three component problems. This is 2]
"I've specified a folder to analyze files using option "-f .\test_output\master"
However, it looks only one folder is analyzed."
before running tests, generate a file for each connected target that lists what features are enabled in targets.json for that target.
Using master version:
https://github.com/maclobdell/simple_mbed_test_runner/blob/master/run_all_tests.py
Modified build options to use -DMBED_TICKELESS
I see (1) a greentea error and (2) runner error: "local variable 'test_results' referenced before assignment"
...
mbedgt: test on hardware with target id: 0240000030514e45004620057bbd002083a1000097969900
mbedgt: test suite 'features-filesystem-littlefs-tests-filesystem_recovery-wear_leveling' ............ OK in 247.49 sec
test case: 'test wear leveling' .............................................................. OK in 231.68 sec
mbedgt: test case summary: 1 pass, 0 failures
mbedgt: mbed-host-test-runner: started
Exception in thread Thread-41:
Traceback (most recent call last):
File "c:\python27\lib\threading.py", line 801, in __bootstrap_inner
self.run()
File "c:\python27\lib\threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "c:\python27\lib\site-packages\mbed_greentea\mbed_greentea_cli.py", line 489, in run_test_thread
verbose=verbose)
File "c:\python27\lib\site-packages\mbed_greentea\mbed_test_api.py", line 318, in run_host_test
returncode, htrun_output = run_htrun(cmd, verbose)
File "c:\python27\lib\site-packages\mbed_greentea\mbed_test_api.py", line 134, in run_htrun
htrun_output += line.decode('utf-8')
File "c:\python27\lib\encodings\utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xfe in position 27: invalid start byte
mbedgt: could not generate test report
mbedgt: completed in 283.75 sec
mbedgt: exited with code -1000
[mbed] ERROR: "mbedgt" returned error code -1000.
[mbed] ERROR: Command "mbedgt --test-spec C:\mbed\mbed-os-multi\BUILD\tests\K64F\IAR\test_spec.json --report-json C:\mbed\mbed-os-multi\test_output\master\K64F/K64F_iar_74135_results.json" in "C:\mbed
\mbed-os-multi"
---
None
Traceback (most recent call last):
File "C:\mbed\simple_mbed_test_runner\run_all_tests.py", line 178, in <module>
main()
File "C:\mbed\simple_mbed_test_runner\run_all_tests.py", line 122, in main
generate_scorecard(output_folder_path, target, mbed_ver, other_args)
File "C:\mbed\simple_mbed_test_runner\run_all_tests.py", line 168, in generate_scorecard
scorecard_data[toolchain] = test_results
UnboundLocalError: local variable 'test_results' referenced before assignment
needs exploration. perhaps it would be more reliable and stable to to import greentea python module and run tests that way.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.