chipsalliance / dromajo Goto Github PK
View Code? Open in Web Editor NEWRISC-V RV64GC emulator designed for RTL co-simulation
License: Apache License 2.0
RISC-V RV64GC emulator designed for RTL co-simulation
License: Apache License 2.0
dromajo/include/dromajo_template.h
Lines 1173 to 1175 in fae7b3a
dromajo/include/dromajo_template.h
Lines 1636 to 1638 in fae7b3a
On a linux machine, I am able to complete the following steps:
Get a trivial buildroot (~ 23 min)
Get the Linux kernel up and running (~ 3 min)
However the following step and those after it are not working:
openSBI (~ ? min)
I think
1) the "opensbi.dromajo.tar.gz" tarball is unavailable
and/or
2) the following path is missing from the opensbi-0.5 repository.
opensbi-0.5/build/platform/dromajo/firmware/fw_payload.bin
TinyEMU had some support for networking/mounting devices to share data.
Is this possible to do with Dromajo? If so any guides/advice on how to do it?
This is forked off @ss2783's pull request #15 where extending the CLINT to handle sub-word accesses ran into trouble.
The fundamental problem is that the device model is currently called upon the load/stores directly and it's up to the handler to deal sub-word handling.
I think there's a model that works better:
This significantly reduces the amount of code that needs to be written to support most devices and removes the sub-word access handling from devices (unless they really want to handle this in a special way).
Hi,
While I was using dromajo, I noticed that mie has a MIE_MCIP field.
https://github.com/chipsalliance/dromajo/blob/master/src/riscv_cpu.cpp#L1215
I could not see this field in SPEC (it is reserved). Can you please let me know if this field is intentionally left there for any reason?
Thanks
I think it will be a good idea to include some regression test to this project to prevent performance degradations.
Are there any benchmarks people commonly use for testing the code?
0x80000000 represents -2147483648 when treated as 32-bit signed integer (w), but when converting to single precision floating point number, the dromajo generates the wrong result.
It can be tested with the following code:
li t0, 0x80000000
fcvt.s.w f0, t0
fmv.x.w t1, f0
The expected result is 0xCF000000 (-2147483648.0) in t1, but dromajo generates 0x80000000 (-0.0).
Create a simple travis unit test (run some code) to make sure that the code is not broken for pull requests.
When more open cores use dromajo, add some simple cosim to the fast regression.
The specification (Volume II: RISC-V Privileged Architectures V20211203 Page 37) said: mepc is an MXLEN-bit read/write register formatted as shown in Figure 3.21. The low bit of mepc (mepc[0]) is always zero. On implementations that support only IALIGN=32, the two low bits (mepc[1:0]) are always zero.
csr write function has this mask operation:
Lines 1284 to 1287 in fae7b3a
Lines 1575 to 1577 in fae7b3a
I am working to emulate/simulate RISC-V systems (using multiple tools) with more than 8 cores and booting SMP Linux with the OpenSBI bootloader. I currently have a setup that works with both Dromajo and QEMU (v6.1.0) to boot OpenSBI v0.9 with Linux Kernel v5.9 and a simple Buildroot/Busybox filesystem. This system successfully boots in Dromajo with up to 8 cores, but fails at higher core counts due to the hard-coded core count limit of 8 in Dromajo (
dromajo/include/riscv_machine.h
Line 51 in 3675ba0
I attempted to modify the source code and increase this limit, say to 32, but running with anything more than 8 cores results in either an error (e.g, with 9 cores: riscv_cpu_read_memory: invalid physical address 0x0000000000011000) or a segmentation fault of Dromajo itself. One potential conclusion is that the Dromajo generated device tree is invalid for core counts > 8 and it isn't allocating a large enough memory range for some device.
What is the motivation for limiting Dromajo to simulating 8 cores maximum and what are the constraints or requirements for increasing the maximum core count?
Are there any short-term workarounds for running with > 8 cores (e.g., if this is indeed a device tree issue and it is possible to provide a custom device tree)?
master at 3675ba0
Note that unmodified QEMU v6.1.0 has a similar hard-coded core limit for the riscv64-virt machine, but increasing this limit and rebuilding QEMU seems to cause no problems.
Currently, Dromajo does not support checkpoint generation when run in a multicore settings.
The comments in the codebase mention of this issue. Filing it here for the documentation purposes.
The specification (Volume II: RISC-V Privileged Architectures V20211203 Page 85) said: If Svnapot is not implemented, bit 63 remains reserved and must be zeroed by software for forward compatibility, or else a page fault exception is raised.
Dromajo trace:
0 3 0x000000008000028c (0x0105051b) x10 0x0000000040201010
0 3 0x0000000080000290 (0x02a00593) x11 0x000000000000002a
0 3 0x0000000080000294 (0x00b52023)
0 3 0x0000000080000298 (0x000205b7) x11 0x0000000000020000
0 3 0x000000008000029c (0x3005b073)
Rocket/spike trace:
core 0: 0x000000008000028c (0x0105051b) addiw a0, a0, 16
core 0: 0x0000000080000290 (0x02a00593) li a1, 42
core 0: 0x0000000080000294 (0x00b52023) sw a1, 0(a0)
core 0: exception trap_store_page_fault, epc 0x0000000080000294
core 0: tval 0x0000000040201010
core 0: 0x0000000080000004 (0x34202f73) csrr t5, mcause
core 0: 0x0000000080000008 (0x00800f93) li t6, 8
Dromajo ignores the PTE N bit and calculates the address as a normal PTE.
For this and all other numeric options, if given as a string, we allow hexadecimal constants.
Building a bootrom and/or DTB should be a option (disabled by default?)
Could be related to this issue
When executing the following test 1406, PC= 0x8000_02dc
, instruction = sc.d s11, sp, (sp)
Spike (and Rocket, and BlackParrot) both raise a store access fault, but Dromajo doesn't seem to be raising this exception and continues to execute subsequent instructions from which point onwards, cosimulation fails (as an exception handler jump doesn't happen). Attached are the test binary, elf, spike simulation output, dromajo output. The DUT in the bottom right in figure is BlackParrot. (But also verified compliant behavior in Rocket). Dromajo is the only output that's off out of the other 4.
A screenshot to make life easier (Caption in each of the 4 command lines):
Seems to be doing this by design:
https://github.com/chipsalliance/dromajo/blob/master/src/dromajo_cosim.cpp#L251
This line executes the first instruction of the exception handler, thus it no longer shows up later when exception handling starts in the DUT.
I was luckly/ unlucky enough to actually reproduce this while running Linux:
hartid=0 priv: 0 exception user_ecall, epc 0x0000003fa17a4938
hartid=0 tval 0x0000000000000000
0 0x0000003fa17a4938 (0x00000073) ecall
[DEBUG] DUT raised exception 8
[DEBUG] DUT raised interrupt 7
[DEBUG] DUT also raised exception 8
[DEBUG] Interrupt: MIP <- 7: Now MIP = 80
get_irq_mask: mip=0x80 mie=0x2aa mideleg=0x222
raise_interrupt: irq=7 priv=1 pc=ffffffff80002ed0 hartid=0
hartid=0: exception interrupt #7, epc 0xffffffff80002ed0
3 0x0000000080000004 (0x34011173) x2 0x000000008000bec0 csrrw sp, mscratch, sp
3 0x0000000080000008 (0x1a010863) beqz sp, pc + 432
3 0x000000008000000c (0x04a13823) sd a0, 80(sp)
3 0x0000000080000010 (0x04b13c23) sd a1, 88(sp)
3 0x0000000080000014 (0x342025f3) x11 0x8000000000000007 csrr a1, mcause
3 0x0000000080000018 (0x0805d263) bgez a1, pc + 132
3 0x000000008000001c (0x00159593) x11 0x000000000000000e slli a1, a1, 1
3 0x0000000080000020 (0x00e00513) x10 0x000000000000000e li a0, 14
3 0x0000000080000024 (0x02b51263) bne a0, a1, pc + 36
3 0x0000000080000028 (0x08000513) x10 0x0000000000000080 li a0, 128
3 0x000000008000002c (0x30453073) csrc mie, a0
3 0x0000000080000030 (0x02000513) x10 0x0000000000000020 li a0, 32
3 0x0000000080000034 (0x34452073) csrs mip, a0
3 0x0000000080000038 (0x05013503) x10 0x0000002ad48fe000 ld a0, 80(sp)
3 0x000000008000003c (0x05813583) x11 0x0000003fa180a930 ld a1, 88(sp)
3 0x0000000080000040 (0x34011173) x2 0x0000003fef66ae50 csrrw sp, mscratch, sp
3 0x0000000080000044 (0x30200073) mret
1 0xffffffff80002ed0 (0x00021f63) bnez tp, pc + 30
[error] EMU PC ffffffff80002ed0, DUT PC ffffffff80002ecc
[error] EMU INSN 00021f63, DUT INSN 14021273
[error] EMU MSTATUS 8000000a000060a0, DUT MSTATUS 00000000
[error] DUT pending exception -1 pending interrupt -1
The DUT tried to start execute the 1st instruction of the exception handler after mret, which is at 80002ecc. However the dromajo has already at the 2nd instruction into the handler, at 80002ed0. This matches the observation in the code where the 1st instruction of the exception handler is already executed.
Another fall-out of Pull Request #15: the handling of interrupt embeds too much policy into Dromajo. For co-sim, we need to be able to adapt to the DUTs decision on when to take pending interrupts. The way current way is hacked and the better way would be to simply factor out of the instruction execution all handling of interrupts. Instead we leave it to the client to example pending interrupts and raise them as needed.
In the setup.md, boot.cfg
is mentioned multiple times. But I have no idea what that is and it seems to be undocumented.
Hi,
When I was try to build dromajo with simpoint flag as following, the build fails. can you please help with this issue? I'm using the latest commit: 3e95064
(HEAD -> master, origin/master, origin/HEAD)
Author: Tommy Thorn [email protected]
Date: Thu Mar 2 12:02:11 2023 -0800
dromajo/build_simpoint - [master] $ cmake -DSIMPOINT=On ../
-- The C compiler identification is GNU 8.5.0
-- The CXX compiler identification is GNU 8.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: dromajo/build_simpoint
dromajo/build_simpoint - [master] $ make
[ 85%] Building CXX object CMakeFiles/dromajo.dir/src/dromajo.cpp.o
[ 90%] Linking CXX executable dromajo
[ 90%] Built target dromajo
[ 95%] Building CXX object CMakeFiles/dromajo_cosim_test.dir/src/dromajo_cosim_test.cpp.o
[100%] Linking CXX executable dromajo_cosim_test
libdromajo_cosim.a(riscv_cpu.cpp.o): In function `csr_write(RISCVCPUState*, unsigned int, unsigned int, unsigned long)':
dromajo/src/riscv_cpu.cpp:1878: undefined reference to simpoint_roi' dromajo/src/riscv_cpu.cpp:1880: undefined reference to
simpoint_roi'
dromajo/src/riscv_cpu.cpp:1882: undefined reference to simpoint_roi' dromajo/src/riscv_cpu.cpp:1883: undefined reference to
simpoint_roi'
dromajo/src/riscv_cpu.cpp:1887: undefined reference to `simpoint_roi'
Thanks.
Dromajo does not respond to maxinsns variable when ncpus > 1
To reproduce:
./dromajo --trace 0 --maxinsns 100000 --ncpus 2 [riscv-bin]
As mentioned in the #35 , I think there are some ways to format the codes from different contributors.
I was trying the Dromajo setup on my Mac. I got the riscv baremetal stuff to work but the linux build is not working.
In particular I am trying this part:
Linux with buildroot
Get a trivial buildroot (~ 23 min)
I get an error. See screenshot. I installed gcc on my Mac using "brew install gcc" but it didn't help.
@abejgonzalez, this change
commit d868962
Author: abejgonzalez [email protected]
Date: Mon Nov 18 16:53:43 2019 -0800Update for Cosimulation (Parameter Overrides + Misc. Cleanup
broke the Linux booting example in run.
Prior it booted the OpenSBI and kernel as expected, after the change we get
riscv_cpu_write_memory: invalid physical address 0x0000000010000004
sbi_trap_error: hart0: page/access fault handler failed (error -2)
sbi_trap_error: hart0: mcause=0x0000000000000007 mtval=0x0000000010000004
sbi_trap_error: hart0: mepc=0x0000000080006f18 mstatus=0x8000000a00007800
sbi_trap_error: hart0: ra=0x00000000800031d4 sp=0x0000000080014d60
sbi_trap_error: hart0: gp=0x0000000000000000 tp=0x0000000080014e00
sbi_trap_error: hart0: s0=0x0000000080014d70 s1=0x0000000000000000
sbi_trap_error: hart0: a0=0x0000000010000004 a1=0x0000000000000035
...
The method of booting the system from multiple boot ROM files, as implemented in PR #76, is not the most elegant solution.
For instance, when the system only has a single boot memory, which is usually the case, it becomes challenging to place multiple generated files into that single memory space without resorting to messy workarounds.
The proposed solution is to create a single boot code file that will be divided into N sections, where N is the number of cores. Each core will generate its own recovery code and write it to its designated section. In addition to the recovery code, each core will initiate with a preamble code that reads the hart_id and calculates the PC based on that ID.
I started implementing this, please let me know if you have any suggestions or concerns.
I haven't yet looked into why, but keyboard input takes seconds to arrive which seems far slower than the rest of the execution speed. This looks like a bug.
By default the V extension is enabled in dromajo (and reported in the misa register). There doesn't seem to be any runtime option to disable it. In the code I noticed a switch in riscv_cpu.h:
/* Uncomment the next line to DISABLE Vector Simulation "V-extension" */
//#define VLEN 0
However that doesn't help because a few lines later it would be re-enabled because now VLEN is less than ELEN:
#if (VLEN_MAX < VLEN || VLEN < ELEN || !IS_PO2(VLEN))
#undef VLEN
#define VLEN VLEN_DEFAULT
#endif
Defining ELEN to 0 as well also doesn't help as that would cause ELEN to be less than ELEN_MIN:
#if (ELEN < ELEN_MIN || VLEN < ELEN || !IS_PO2(ELEN))
#undef ELEN
#define ELEN ELEN_DEFAULT
#endif
If I patch these all out I am getting some compilation errors:
error: ‘RISCVCPUState’ {aka ‘struct RISCVCPUState’} has no member named ‘most_recently_written_vregs’; did you mean ‘most_recently_written_reg’?
175 | if (cpu->most_recently_written_vregs[i]) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
| most_recently_written_reg
I ended up just patching the code inside riscv_cpu.cpp to ask it not report MCPUID_V in misa to avoid divergence from DUT. But I am feeling like this should be something supported without modifying the code.
I ran the attached assembly program with dromajo and it gives an exception on a nop instruction. I ran the same program on Spike and it ignored nop instruction and proceeded as expected.
I have attached a zip file that contains the compiled binary file (test.elf), assembly file (test.S), and disassembly of the binary (test.txt).
The following figure shows the diff between Spike trace and dromajo trace. nop instruction is located after the highlighted mret instruction.
It's clear that adding more command-line options aren't the path forward. What I suggest instead is that everything (or almost everything?) should be possible to specify as a parameter in a config file. All current command line options will internally translate to a hidden config file. Furthermore, one should be able to load multiple config files, to enable refactoring. Finally, it most be possible to give an inline config files as command line option. Example:
dromajo base.cfg -c '{ "resetvector" = "0x1000" }'
The config files are loaded in-order, such that later ones can override earlier ones. Finally, inline configs are loaded last.
dromajo lr/sc does not fully respect the riscv manual ? ==== The SC must fail if a store to the reservation set from another hart can be observed to occur between the LR and SC. write printf to include/dromajo_template.h ==== #define OP_A(size) \ .. case 2: /* lr.w */ \ printf("lr.w: addr=%x\n", addr); \ .. case 3: /* sc.w */ \ .. if (s->load_res == addr) { \ .. printf("sc.w success memval=%x hartid=%x\n", \ read_reg(rs2), s->mhartid); \ val = 0; \ s->load_res = ~0; \ } else { \ printf("sc.w failed hartid=%x\n", s->mhartid); \ val = 1; \ } \ crt.s from dromajo/run vs laur.s ==== $ diff crt.S laur.S 127,129c127,129 < # for now, assume only 1 core < li a1, 1 < 1:bgeu a0, a1, 1b --- > # laur: run on all cores > # li a1, 1 > #1:bgeu a0, a1, 1b 139c139,148 < j _init --- > # j _init > > # laur > # have each core add 1 to tohost > la a1, tohost > 1: lr.w a4, (a1) > addi a4, a4, 1 > sc.w a4, a4, (a1) > bnez a4, 1b > label1:j label1 compile and run ==== laur@laurPC-100:~/lucru/cn/riscv/64/dromajo/run$ riscv64-unknown-elf-gcc -march=rv64g -mabi=lp64 -static -mcmodel=medany -nostdlib -nostartfiles laur.S -lgcc -T test.ld laur@laurPC-100:~/lucru/cn/riscv/64/dromajo/run$ ../build/dromajo --ncpus=4 ./a.out lr.w: addr=80001000 lr.w: addr=80001000 lr.w: addr=80001000 lr.w: addr=80001000 sc.w success memval=1 hartid=0 sc.w success memval=1 hartid=1 sc.w success memval=1 hartid=2 sc.w success memval=1 hartid=3 Simulation speed: 0.29 MIPS (single-core) Power off.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.