hushenwei2000 / rvv-atg Goto Github PK
View Code? Open in Web Editor NEWRISC-V Vector (RVV) Automatic Tests Generator with full instructions coverage, including self-checking test and signature test (RISC-V Compliance test).
rvv-atg's Issues
riscv_isac coverage: Error: No such option: -v
I am learning how to use this project and try to generate testcases with command python run.py -i vadd -t i and it failed with error below
$ python run.py -i vadd -t i
Tue, 07 Nov 2023 10:27:08 INFO Creating output directory: 11-07-vadd-vlen128-vsew32-lmul1.0-masked
Tue, 07 Nov 2023 10:27:08 INFO Finding CGF for: vadd in /home/hachi/sail/rvv-atg/cgfs/i/vadd.yaml.
Tue, 07 Nov 2023 10:27:08 INFO lmul: 1.0.
vta, vma False False
new_vtavma: tu, mu
Tue, 07 Nov 2023 10:27:08 INFO RVV-ATG: instr: vadd, vlen: 128, vsew: 32, lmul: 1.000000
Tue, 07 Nov 2023 10:27:08 INFO Creating empty test for vadd
cp: '11-07-vadd-vlen128-vsew32-lmul1.0-masked/vadd_empty.S' and '11-07-vadd-vlen128-vsew32-lmul1.0-masked/vadd_empty.S' are the same file
Tue, 07 Nov 2023 10:27:08 INFO Creating empty test for vadd: finish in 11-07-vadd-vlen128-vsew32-lmul1.0-masked/vadd_empty.S!
Tue, 07 Nov 2023 10:27:08 INFO Running riscof coverage: vadd.empty
Tue, 07 Nov 2023 10:27:08 INFO Running riscof coverage: vadd.empty, stage: Compiling...
riscv64-linux-gnu-gcc -march=rv64gv -w -static -mcmodel=medany -fvisibility=hidden -nostdlib -nostartfiles -T /home/hachi/sail/rvv-atg/riscof_files/spike/env/link.ld -I /home/hachi/sail/rvv-atg/env/macros/vsew32_lmul1 -I /home/hachi/sail/rvv-atg/env/p -I /home/hachi/sail/rvv-atg/env -I /home/hachi/sail/rvv-atg/env/spike -I /home/hachi/sail/rvv-atg/env/sail_cSim -mabi=lp64 vadd_empty.S -o ref_empty.elf -DTEST_CASE_1=True -DXLEN=64 -DFLEN=32;
Tue, 07 Nov 2023 10:27:08 INFO Running riscof coverage: vadd.empty, stage: ObjDumping...
Tue, 07 Nov 2023 10:27:08 INFO Running riscof coverage: vadd.empty, stage: Spike Running...
riscv_isac --verbose info coverage -d -t spike_vadd_empty.log --parser-name spike -o coverage_empty.rpt --sig-label begin_signature end_signature --test-label rvtest_code_begin cleanup_epilogs -e ref_empty.elf -c /home/hachi/sail/rvv-atg/cgfs/dataset.yaml -c /home/hachi/sail/rvv-atg/cgfs/i/vadd.yaml -x64 -v128 --vsew 32 --lmul 1 --flen 32 -l vadd > isac_log_empty 2>&1;
Tue, 07 Nov 2023 10:27:08 INFO Running riscof coverage: vadd.empty, stage: RISCV-ISAC Running...
Tue, 07 Nov 2023 10:27:08 INFO Running riscof coverage vadd finish!
Tue, 07 Nov 2023 10:27:08 INFO Creating first test for vadd
Traceback (most recent call last):
File "/home/hachi/sail/rvv-atg/run.py", line 183, in <module>
main()
File "/home/hachi/sail/rvv-atg/run.py", line 170, in main
run_integer(cwd, args, cgf, output_dir)
File "/home/hachi/sail/rvv-atg/run.py", line 106, in run_integer
first_test = create_first_test(
^^^^^^^^^^^^^^^^^^
File "/home/hachi/sail/rvv-atg/scripts/lib.py", line 118, in create_first_test
return eval(func_str)
^^^^^^^^^^^^^^
File "<string>", line 1, in <module>
File "/home/hachi/sail/rvv-atg/scripts/create_test_integer/create_test_vadd.py", line 42, in create_first_test_vadd
rs1_val, rs2_val = extract_operands(f, rpt_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hachi/sail/rvv-atg/scripts/test_common_info.py", line 193, in extract_operands
f = open(rpt_path)
^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/hachi/sail/rvv-atg/11-07-vadd-vlen128-vsew32-lmul1.0-masked/coverage_empty.rpt'
It seems that the riscv_isac command failed with error below
$ cat /home/hachi/sail/rvv-atg/11-07-vadd-vlen128-vsew32-lmul1.0-masked/isac_log_empty
INFO | ****** RISC-V ISA Coverage 0.18.0 *******
INFO | Copyright (c) 2020, InCore Semiconductors Pvt. Ltd.
INFO | All Rights Reserved.
Usage: riscv_isac coverage [OPTIONS]
Try 'riscv_isac coverage --help' for help.
Error: No such option: -v
The riscv-isac is the latest version from https://github.com/riscv-software-src/riscv-isac . Thanks in advance.
Generation of valid tests for unsupported VLEN target
It is documented in the README that the generator supports VLEN up to 1024. However it is observed that the test generator can generate tests up to VLEN of 8192. Running the following command gives generated tests.
python3 run.py -t i -i vadd --vlen 8192 --vsew 64 --lmul 1
Even python3 run.py -t i -i vadd --vlen 4096 --vsew 64 --lmul 1 and python3 run.py -t i -i vadd --vlen 2048 --vsew 64 --lmul 1 works as well. I even ran the tests with VLEN == 4096 on an RTL as well. It did not create tests for VLEN == 16384 though.
THE questions I want to ask is .. does the README need to be updated? or is there any problem that I haven't noticed?
ERROR in generation of all tests (using generate_all.py code)
After setting up all that is required, I am getting following error:
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
Traceback (most recent call last):
File "run.py", line 3, in <module>
File "run.py", line 3, in <module>
from ast import arg
from ast import arg
ImportErrorImportError: : cannot import name argcannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
Traceback (most recent call last):
File "run.py", line 3, in <module>
from ast import arg
ImportError: cannot import name arg
grep: 11-04-%s-vlen512-vsew32-lmul1.0/spike_%s_final.log: No such file or directory
cp: cannot stat '11-04-%s-vlen512-vsew32-lmul1.0/ref_final.elf': No such file or directory
A quick google lookup states something that points out a problem in calling out wrong imports (in multiple places) as in
The Python "ImportError: cannot import name" occurs when we have circular imports (importing members between the same files). To solve the error, move the objects to a third file and import them from a central location in other files, or nest one of the imports in a function.
Modifications required in generate_all.py script
The generate_all.py script is there to generate tests for all instructions. However the script is not thorough enough to create all tests. Two points needs to be addressed regarding this:
- All the instructions should be covered along with all its legal combinations. Therefore, a regressive test generation script is desired.
- All tests covering all instructions with all its variants and configuration should be placed in a directory which have a structure consistent with riscv-tests. This includes all the test placed in one folder (
rv64uvin this case) with amakfraglisting all those tests that can be included in test environment (riscv-test-env) later. Header files (whether scalar or vector) should be included in a single directory (rv64uv/macros) and no changes should have to be made in the environment itself (or at the worst, least changes are desired) that uses those tests after compiling according to any target, spike or DUT etc.
I'll try to submit a PR very soon to address this problem. I'd prefer if this generation script doesn't alter the current generate_all.py script, and can be used as a standalone script to address just this problem. I'd appreciate your comments on it.
Regards,
Quswar Abid
Spec Support ?
I can't find this mentioned anywhere in the README file. So, I wanted to know that what spec does this generates tests for? is it Riscv Vector Extension 1.0?
Ease of use for setting paths of gcc, objdump, and riscof
Opening this issue along with a PR (#4) to provide ease of use to set paths from one single location. This will make it easier for users to set up this test generator.
Invalid test generation for vaadd instruction with vlen = 512, LMUL = 8, SEW = 8 configuration
I am generating a test for vaadd fixed point instruction with the following configuration:
vlen = 512, LMUL = 8, SEW = 8
I used following command to generate the test:
python3 run.py -t x -i vaadd --vlen 512 --lmul 8 --vsew 8 --xlen 64 --flen 64
-
My understanding is that if it is generating a test for a specific instruction (in this case vaadd), there would be at least one execution of
vaaddbut I don't see any execution of this instruction for this specific configuration. -
While for the remaining configurations it is working as expected i.e. it has executed the instruction which is being tested:
core 0: 0x0000000080000260 (0x25042c57) vaadd.vv v24, v16, v8, v0.t
core 0: 0 0x0000000080000260 (0x25042c57) c8_vstart 0x0000000000000000 e8 m1 l64 v24 0x00811ecf1c452e0ae82d552b8b445536307d97442d004d0374da352ed0d72d04b49ae55d9ed4e107c4d9b11b112e3f0367f9ab10fd2ce801f68219ef66da64fe
Is there any known bugs related to this?
Note: I have checked only vaadd instruction, may be there are more cases like this.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.