Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed in RTL Simulation #314

Open
Tanishqgithub opened this issue Jun 26, 2024 · 25 comments
Open

Failed in RTL Simulation #314

Tanishqgithub opened this issue Jun 26, 2024 · 25 comments

Comments

@Tanishqgithub
Copy link

Respected sir

I am working on Oracle virtual box with Ubuntu 22.04 LTS version

  1. I ran the prerequisite dependency commands which gave some 404 error but was overcome using the --fix-missing

  2. cloned the repository along with the submodules

  3. make toolchain-llvm [abnormally there were some "failed" for tests when we run this]
    a. Is that ok ?
    b. i sense something wrong in my clang installation [please see "clang.png" , the path is not printed]
    image

  4. make riscv-isa-sim [no problem running it]

  5. make verilator [no problem running it after running sudo apt-get install libstdc++-12-dev ]

  6. tried running other "apps" other than hello_world like fmatmul and got o/p as in "fmatmul.txt"
    a. Why is the number of cycles 0 and performance is also not computed ?
    b. In this step is the application running on the vector processor designed only? because not integrated with CVA6 till now ..
    fmatmul.txt

  7. make riscv_tests :
    a. solved after running make toolchain-gcc

  8. from RTL Simulation of README :
    a. tried running make checkout but got the error message in "make_checkout.png"
    b. please guide from this step
    image

@mp-17
Copy link
Collaborator

mp-17 commented Jun 26, 2024

Hello @Tanishqgithub,

Provided that you have already installed Verilator and run make -C hardware apply-patches once, please try to run the following commands and let me know what happens:

# Compile fmatmul for Ara
make -C apps bin/fmatmul
# Verilate the design
make -C hardware verilate
# Run fmatmul on Ara through Verilator
make -C hardware simv app=fmatmul

@mp-17
Copy link
Collaborator

mp-17 commented Jun 26, 2024

It seems that make checkout worked correctly.

  1. Which shell are you using? Bash?
  2. Can you please format the previous message? It's very hard to read. Please put the code within ``` delimiters.
    Example:
code
code
code

@Tanishqgithub
Copy link
Author

  1. Note the prerequists command i ran which is there in [1. ] in previous message is :
sudo apt update
sudo apt-get install -y build-essential git doxygen python3-pip libsdl2-dev curl cmake gtkwave libsndfile1-dev rsync autoconf automake texinfo libtool pkg-config libsdl2-ttf-dev ninja-build
  1. After running the make -C hardware apply-patches i got this o/p :
    image

  2. After running the command mentioned above i,e :

# Compile fmatmul for Ara
make -C app bin/fmatmul
# Verilate the design
make -C hardware verilate
# Run fmatmul on Ara through Verilator
make -C hardware simv app=fmatmul

i got the following o/p :
error.txt

  1. Note from the previous message i couldn't run the make checkout command which is the starting of RTL Simulation section of README .
    image

If my command is working fine please tell me what are the next final commands after this step of command and why is there in finding the QuestaSim Version and the deps : No such file or directory

  1. Yes i am using the bash shell
    image

@mp-17
Copy link
Collaborator

mp-17 commented Jun 26, 2024

make checkout is working correctly. The messages about QuestaSim and find are acceptable.
The patches have also been applied correctly.

For the compilation, there was a typo in my command, sorry. You should first give:

make -C apps bin/fmatmul

Then, it seems you also have a problem with Verilator.

sh: 1: exec: /home/tanishq/Documents/ARAnew/ara/install/verilator/share/verilator/verilator_bin: not found
%Error: Command Failed ulimit -s unlimited 2>/dev/null; exec /home/tanishq/Documents/ARAnew/ara/install/verilator/share/verilator/verilator_bin -f

I am not sure why you get this. You can first try to re-install a fresh version of Verilator and check that you have not got any errors while installing it. Before installing it, be sure that VERILATOR_ROOT is clean with unset VERILATOR_ROOT.

If the issue is not solved, you can ask for support in the Verilator repository.

Otherwise, if you can access QuestaSim, you can simulate with it by replacing simv with simc when you issue the make command.

@Tanishqgithub
Copy link
Author

As per your suggestion i tried out to re-install verilator using the command make -B verilator after cleaning the path VERILATOR_ROOT
I got this log file :
verilator.txt
As per the log the verilator is successfully installed

but after i run

# Compile fmatmul for Ara
make -C apps bin/fmatmul
# Verilate the design
make -C hardware verilate
# Run fmatmul on Ara through Verilator
make -C hardware simv app=fmatmul

i got the following o/p which is same as previous situation :

tanishq@Tanishq:~/Documents/ARAnew/ara$ # Compile fmatmul for Ara

make -C apps bin/fmatmul

# Verilate the design

make -C hardware verilate

# Run fmatmul on Ara through Verilator

make -C hardware simv app=fmatmul

make: Entering directory '/home/tanishq/Documents/ARAnew/ara/apps'

cd fmatmul && if [ -d script ]; then python3 script/gen_data.py 128 128 128 > data.S ; else touch data.S; fi

/home/tanishq/Documents/ARAnew/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARAnew/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/data.S -o fmatmul/data.S.o

chmod +x /home/tanishq/Documents/ARAnew/ara/apps/common/script/align_sections.sh

rm -f /home/tanishq/Documents/ARAnew/ara/apps/common/link.ld && cp /home/tanishq/Documents/ARAnew/ara/apps/common/arch.link.ld /home/tanishq/Documents/ARAnew/ara/apps/common/link.ld

/home/tanishq/Documents/ARAnew/ara/apps/common/script/align_sections.sh 4 /home/tanishq/Documents/ARAnew/ara/apps/common/link.ld

/home/tanishq/Documents/ARAnew/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARAnew/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/main.c -o fmatmul/main.c.o

/home/tanishq/Documents/ARAnew/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARAnew/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/crt0.S -o common/crt0-llvm.S.o

/home/tanishq/Documents/ARAnew/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARAnew/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/printf.c -o common/printf-llvm.c.o

/home/tanishq/Documents/ARAnew/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARAnew/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/string.c -o common/string-llvm.c.o

/home/tanishq/Documents/ARAnew/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARAnew/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/serial.c -o common/serial-llvm.c.o

/home/tanishq/Documents/ARAnew/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARAnew/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/util.c -o common/util-llvm.c.o

mkdir -p bin/

/home/tanishq/Documents/ARAnew/ara/install/riscv-llvm/bin/clang -Iinclude -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARAnew/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -o bin/fmatmul fmatmul/kernel/fmatmul.c.o fmatmul/main.c.o fmatmul/data.S.o common/crt0-llvm.S.o common/printf-llvm.c.o common/string-llvm.c.o common/serial-llvm.c.o common/util-llvm.c.o -static -nostartfiles -lm -Wl,--gc-sections -T/home/tanishq/Documents/ARAnew/ara/apps/common/link.ld

ld.lld: warning: ignoring memory region assignment for non-allocatable section '.comment'

/home/tanishq/Documents/ARAnew/ara/install/riscv-llvm/bin/llvm-objdump --mattr=v -D bin/fmatmul > bin/fmatmul.dump

/home/tanishq/Documents/ARAnew/ara/install/riscv-llvm/bin/llvm-strip bin/fmatmul -S --strip-unneeded

rm common/serial-llvm.c.o common/printf-llvm.c.o common/crt0-llvm.S.o common/util-llvm.c.o common/string-llvm.c.o fmatmul/main.c.o

make: Leaving directory '/home/tanishq/Documents/ARAnew/ara/apps'

make: Entering directory '/home/tanishq/Documents/ARAnew/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

rm -rf build/verilator; mkdir -p build/verilator

/home/tanishq/Documents/ARAnew/ara/hardware/../hardware/bender script verilator -t rtl -t cv64a6_imafdcv_sv39 -t tech_cells_generic_include_tc_sram -t tech_cells_generic_include_tc_clk -t ara_test -t cva6_test -t verilator --define NR_LANES=4 --define VLEN=4096 --define ARIANE_ACCELERATOR_PORT=1 --define COMMON_CELLS_ASSERTS_OFF > build/verilator/bender_script_default

/home/tanishq/Documents/ARAnew/ara/install/verilator/bin/verilator -f build/verilator/bender_script_default           \

  -GNrLanes=4                                                         \

  -O3                                                                           \

  -Wno-fatal                                                                    \

  -Wno-PINCONNECTEMPTY                                                          \

  -Wno-BLKANDNBLK                                                               \

  -Wno-CASEINCOMPLETE                                                           \

  -Wno-CMPCONST                                                                 \

  -Wno-LATCH                                                                    \

  -Wno-LITENDIAN                                                                \

  -Wno-UNOPTFLAT                                                                \

  -Wno-UNPACKED                                                                 \

  -Wno-UNSIGNED                                                                 \

  -Wno-WIDTH                                                                    \

  -Wno-WIDTHCONCAT                                                              \

  -Wno-ENUMVALUE                                                                \

  -Wno-COMBDLY                                                                  \

  --hierarchical                                                                \

  tb/verilator/waiver.vlt                                                       \

  --Mdir build/verilator                                                       \

  -Itb/dpi                                                                      \

  --compiler clang                                                              \

  -CFLAGS "-DTOPLEVEL_NAME=ara_tb_verilator"                                        \

  -CFLAGS "-DNR_LANES=4"                                              \

  -CFLAGS -I/home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp       \

  -CFLAGS -I/home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp \

  -CFLAGS -I/home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp \

  ""                                                             \

  -LDFLAGS "-lelf"                                                              \

  ""                                                              \

  --exe                                                                         \

  /home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/*.cc            \

  /home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/*.cc      \

  /home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/*.cc      \

  /home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/ara_tb.cpp                                           \

  --cc                                                                          \

                                       \

  --top-module ara_tb_verilator &&                                                  \

cd build/verilator && OBJCACHE='' make -j4 -f Vara_tb_verilator.mk

make[1]: Entering directory '/home/tanishq/Documents/ARAnew/ara/hardware/build/verilator'

make[2]: Entering directory '/home/tanishq/Documents/ARAnew/ara/hardware'

/home/tanishq/Documents/ARAnew/ara/install/verilator/bin/verilator -f build/verilator/Vlane_e_hierMkArgs.f

sh: 1: exec: /home/tanishq/Documents/ARAnew/ara/install/verilator/share/verilator/verilator_bin: not found

%Error: Command Failed ulimit -s unlimited 2>/dev/null; exec /home/tanishq/Documents/ARAnew/ara/install/verilator/share/verilator/verilator_bin -f build/verilator/Vlane_e_hierMkArgs.f

make[2]: *** [build/verilator/Vara_tb_verilator_hier.mk:332: hier_launch_verilator] Error 127

make[2]: Leaving directory '/home/tanishq/Documents/ARAnew/ara/hardware'

make[1]: *** [Vara_tb_verilator_hier.mk:341: Vlane_e/lane_e.sv] Error 2

make[1]: Leaving directory '/home/tanishq/Documents/ARAnew/ara/hardware/build/verilator'

%Error: make -C build/verilator -f Vara_tb_verilator_hier.mk  -j 1  hier_verilation exited with 2

%Error: Command Failed ulimit -s unlimited 2>/dev/null; exec /home/tanishq/Documents/ARAnew/ara/install/verilator/bin/verilator_bin -f build/verilator/bender_script_default -GNrLanes=4 -O3 -Wno-fatal -Wno-PINCONNECTEMPTY -Wno-BLKANDNBLK -Wno-CASEINCOMPLETE -Wno-CMPCONST -Wno-LATCH -Wno-LITENDIAN -Wno-UNOPTFLAT -Wno-UNPACKED -Wno-UNSIGNED -Wno-WIDTH -Wno-WIDTHCONCAT -Wno-ENUMVALUE -Wno-COMBDLY --hierarchical tb/verilator/waiver.vlt --Mdir build/verilator -Itb/dpi --compiler clang -CFLAGS -DTOPLEVEL_NAME=ara_tb_verilator -CFLAGS -DNR_LANES=4 -CFLAGS -I/home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp -CFLAGS -I/home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp -CFLAGS -I/home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp  -LDFLAGS -lelf  --exe /home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/dpi_memutil.cc /home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/sv_scoped.cc /home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/verilator_memutil.cc /home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/verilated_toplevel.cc /home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/verilator_sim_ctrl.cc /home/tanishq/Documents/ARAnew/ara/hardware/tb/verilator/ara_tb.cpp --cc --top-module ara_tb_verilator

make: *** [Makefile:188: build/verilator/Vara_tb_verilator] Error 2

make: Leaving directory '/home/tanishq/Documents/ARAnew/ara/hardware'

make: Entering directory '/home/tanishq/Documents/ARAnew/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

build/verilator/Vara_tb_verilator  -l ram,/home/tanishq/Documents/ARAnew/ara/apps/bin/fmatmul,elf

bash: line 1: build/verilator/Vara_tb_verilator: No such file or directory

make: *** [Makefile:232: simv] Error 127

make: Leaving directory '/home/tanishq/Documents/ARAnew/ara/hardware'

Can you please help me solving this issue and this time i didn't run make -C hardware apply-patches because the last time i used ARA i had executed it , is that correct ?

@mp-17
Copy link
Collaborator

mp-17 commented Jun 28, 2024

Yes, you can skip make -C hardware apply-patches if it has already been executed.

For Verilator, this seems the "interesting" error:

sh: 1: exec: /home/tanishq/Documents/ARAnew/ara/install/verilator/share/verilator/verilator_bin: not found

There is something problematic with your verilator setup.

Try attaching the .log from when you installed verilator with make verilator.
Also, what is the output of: echo $VERILATOR_ROOT?

Ideally, VERILATOR_ROOT should be unset (i.e., unset VERILATOR_ROOT).

@Tanishqgithub
Copy link
Author

  1. when i run echo VERILATOR_ROOT i got the following output :
    image

  2. the config.log file is attached here :
    config.log

@mp-17
Copy link
Collaborator

mp-17 commented Jun 29, 2024

@Tanishqgithub,

My typo. The command should be: echo $VERILATOR_ROOT.

Do you have the full log of the make command when you issue make verilator to install Verilator?

@Tanishqgithub
Copy link
Author

Tanishqgithub commented Jun 30, 2024

  1. Oh yeah i also didn't notice that
    The output is attached below
    image
    There is something definitely problem with the installation

  2. I got the similar blank output for clang also so is this causing the problem ?
    image

  3. regarding the .log file , the log file which is attached in the previous message is the only log file which is present in directory
    /home/tanishq/Documents/ARAnew/ara/toolchain/verilator/ which is config.log above
    If that is not the complete log file , please tell me where i can find the complete log file for make verilator

@mp-17
Copy link
Collaborator

mp-17 commented Jun 30, 2024

That seems okay.

This is also okay. If you want to access the clang you installed with the make toolchains command, you can go into the ara directory and execute:

./install/riscv-llvm/bin/clang --version

When you install verilator from scratch, you can do:

make verilator |& tee verilator_install.log

Or, if you don't have tee:

make verilator &> verilator_install.log

@Tanishqgithub
Copy link
Author

Tanishqgithub commented Jul 1, 2024

Respected sir

I re-clonned the entire setup and tried to start from first :

  1. after running make toolchain-llvm &> toolchain_llvm_install.log i got this log file :
    toolchain_llvm_install.txt

  2. after running make riscv-isa-sim &> riscv_isa_sim_install.log i got this log file :
    riscv_isa_sim_install.txt

  3. after running make verilator &> verilator_install.log i got this log file :
    verilator_install.txt

  4. Before running this :

cd apps
make riscv_tests

i ran this command make toolchain-gcc &> toolchain_gcc_install.log and got this log file :
toolchain_gcc_install.txt

  1. But i got the same o/p which is shown above in previous messages after running :
# Compile fmatmul for Ara
make -C apps bin/fmatmul
# Verilate the design
make -C hardware verilate
# Run fmatmul on Ara through Verilator
make -C hardware simv app=fmatmul

O/p :

tanishq@Tanishq:~/Documents/ARA_new/ara$ # Compile fmatmul for Ara

make -C apps bin/fmatmul

# Verilate the design

make -C hardware verilate

# Run fmatmul on Ara through Verilator

make -C hardware simv app=fmatmul

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/apps'

cd fmatmul && if [ -d script ]; then python3 script/gen_data.py 128 128 128 > data.S ; else touch data.S; fi

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/data.S -o fmatmul/data.S.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/kernel/fmatmul.c -o fmatmul/kernel/fmatmul.c.o

chmod +x /home/tanishq/Documents/ARA_new/ara/apps/common/script/align_sections.sh

rm -f /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld && cp /home/tanishq/Documents/ARA_new/ara/apps/common/arch.link.ld /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

/home/tanishq/Documents/ARA_new/ara/apps/common/script/align_sections.sh 4 /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/main.c -o fmatmul/main.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/crt0.S -o common/crt0-llvm.S.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/printf.c -o common/printf-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/string.c -o common/string-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/serial.c -o common/serial-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/util.c -o common/util-llvm.c.o

mkdir -p bin/

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -Iinclude -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -o bin/fmatmul fmatmul/kernel/fmatmul.c.o fmatmul/main.c.o fmatmul/data.S.o common/crt0-llvm.S.o common/printf-llvm.c.o common/string-llvm.c.o common/serial-llvm.c.o common/util-llvm.c.o -static -nostartfiles -lm -Wl,--gc-sections -T/home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

ld.lld: warning: ignoring memory region assignment for non-allocatable section '.comment'

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/llvm-objdump --mattr=v -D bin/fmatmul > bin/fmatmul.dump

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/llvm-strip bin/fmatmul -S --strip-unneeded

rm common/serial-llvm.c.o common/printf-llvm.c.o common/crt0-llvm.S.o common/util-llvm.c.o common/string-llvm.c.o fmatmul/main.c.o

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/apps'

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

mkdir -p build

rm -rf build/verilator; mkdir -p build/verilator

/home/tanishq/Documents/ARA_new/ara/hardware/../hardware/bender script verilator -t rtl -t cv64a6_imafdcv_sv39 -t tech_cells_generic_include_tc_sram -t tech_cells_generic_include_tc_clk -t ara_test -t cva6_test -t verilator --define NR_LANES=4 --define VLEN=4096 --define ARIANE_ACCELERATOR_PORT=1 --define COMMON_CELLS_ASSERTS_OFF > build/verilator/bender_script_default

/home/tanishq/Documents/ARA_new/ara/install/verilator/bin/verilator -f build/verilator/bender_script_default           \

  -GNrLanes=4                                                         \

  -O3                                                                           \

  -Wno-fatal                                                                    \

  -Wno-PINCONNECTEMPTY                                                          \

  -Wno-BLKANDNBLK                                                               \

  -Wno-CASEINCOMPLETE                                                           \

  -Wno-CMPCONST                                                                 \

  -Wno-LATCH                                                                    \

  -Wno-LITENDIAN                                                                \

  -Wno-UNOPTFLAT                                                                \

  -Wno-UNPACKED                                                                 \

  -Wno-UNSIGNED                                                                 \

  -Wno-WIDTH                                                                    \

  -Wno-WIDTHCONCAT                                                              \

  -Wno-ENUMVALUE                                                                \

  -Wno-COMBDLY                                                                  \

  --hierarchical                                                                \

  tb/verilator/waiver.vlt                                                       \

  --Mdir build/verilator                                                       \

  -Itb/dpi                                                                      \

  --compiler clang                                                              \

  -CFLAGS "-DTOPLEVEL_NAME=ara_tb_verilator"                                        \

  -CFLAGS "-DNR_LANES=4"                                              \

  -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp       \

  -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp \

  -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp \

  ""                                                             \

  -LDFLAGS "-lelf"                                                              \

  ""                                                              \

  --exe                                                                         \

  /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/*.cc            \

  /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/*.cc      \

  /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/*.cc      \

  /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/ara_tb.cpp                                           \

  --cc                                                                          \

                                       \

  --top-module ara_tb_verilator &&                                                  \

cd build/verilator && OBJCACHE='' make -j4 -f Vara_tb_verilator.mk

make[1]: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware/build/verilator'

make[2]: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

/home/tanishq/Documents/ARA_new/ara/install/verilator/bin/verilator -f build/verilator/Vlane_e_hierMkArgs.f

sh: 1: exec: /home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/verilator_bin: not found

%Error: Command Failed ulimit -s unlimited 2>/dev/null; exec /home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/verilator_bin -f build/verilator/Vlane_e_hierMkArgs.f

make[2]: *** [build/verilator/Vara_tb_verilator_hier.mk:332: hier_launch_verilator] Error 127

make[2]: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

make[1]: *** [Vara_tb_verilator_hier.mk:341: Vlane_e/lane_e.sv] Error 2

make[1]: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware/build/verilator'

%Error: make -C build/verilator -f Vara_tb_verilator_hier.mk  -j 1  hier_verilation exited with 2

%Error: Command Failed ulimit -s unlimited 2>/dev/null; exec /home/tanishq/Documents/ARA_new/ara/install/verilator/bin/verilator_bin -f build/verilator/bender_script_default -GNrLanes=4 -O3 -Wno-fatal -Wno-PINCONNECTEMPTY -Wno-BLKANDNBLK -Wno-CASEINCOMPLETE -Wno-CMPCONST -Wno-LATCH -Wno-LITENDIAN -Wno-UNOPTFLAT -Wno-UNPACKED -Wno-UNSIGNED -Wno-WIDTH -Wno-WIDTHCONCAT -Wno-ENUMVALUE -Wno-COMBDLY --hierarchical tb/verilator/waiver.vlt --Mdir build/verilator -Itb/dpi --compiler clang -CFLAGS -DTOPLEVEL_NAME=ara_tb_verilator -CFLAGS -DNR_LANES=4 -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp  -LDFLAGS -lelf  --exe /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/dpi_memutil.cc /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/sv_scoped.cc /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/verilator_memutil.cc /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/verilated_toplevel.cc /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/verilator_sim_ctrl.cc /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/ara_tb.cpp --cc --top-module ara_tb_verilator

make: *** [Makefile:188: build/verilator/Vara_tb_verilator] Error 2

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

build/verilator/Vara_tb_verilator  -l ram,/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul,elf

bash: line 1: build/verilator/Vara_tb_verilator: No such file or directory

make: *** [Makefile:232: simv] Error 127

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

tanishq@Tanishq:~/Documents/ARA_new/ara$ 
  1. is the procedure of commands given correct regarding setup of toolchain ? or is there any other commands or
    prerequisite needed ?

  2. And i created Issue in Verilator_github and they replied saying that there is no issue with verilator rather issue with verilator installation [which is quite true]

  3. And there is another method where we can simulate using QuestaSim , can you please share some insights in installing it and then commands to simulate using this .

@mp-17
Copy link
Collaborator

mp-17 commented Jul 1, 2024

Does the following command work?
/home/tanishq/Documents/ARAnew/ara/install/verilator/bin/verilator --version

The problem remains the Verilator installation, but I don't see errors in the log and unluckily I don't know why the executable is searched in:
/home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/verilator_bin

You can try to patch the reference this way, but I don't think it will work:

mkdir -p /home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator
ln -s /home/tanishq/Documents/ARA_new/ara/install/verilator/bin/verilator_bin /home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/verilator_bin

You need a license to use QuestaSim; if you have not one, you need to use Verilator

@Tanishqgithub
Copy link
Author

  1. Yes that command works :
    image

  2. The command you gave to patch was an excellent idea and actually solved the problem but gave to another error :

tanishq@Tanishq:~/Documents/ARA_new/ara$ # Compile fmatmul for Ara

make -C apps bin/fmatmul

# Verilate the design

make -C hardware verilate

# Run fmatmul on Ara through Verilator

make -C hardware simv app=fmatmul

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/apps'

cd fmatmul && if [ -d script ]; then python3 script/gen_data.py 128 128 128 > data.S ; else touch data.S; fi

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/data.S -o fmatmul/data.S.o

chmod +x /home/tanishq/Documents/ARA_new/ara/apps/common/script/align_sections.sh

rm -f /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld && cp /home/tanishq/Documents/ARA_new/ara/apps/common/arch.link.ld /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

/home/tanishq/Documents/ARA_new/ara/apps/common/script/align_sections.sh 4 /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/main.c -o fmatmul/main.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/crt0.S -o common/crt0-llvm.S.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/printf.c -o common/printf-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/string.c -o common/string-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/serial.c -o common/serial-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/util.c -o common/util-llvm.c.o

mkdir -p bin/

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -Iinclude -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -o bin/fmatmul fmatmul/kernel/fmatmul.c.o fmatmul/main.c.o fmatmul/data.S.o common/crt0-llvm.S.o common/printf-llvm.c.o common/string-llvm.c.o common/serial-llvm.c.o common/util-llvm.c.o -static -nostartfiles -lm -Wl,--gc-sections -T/home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

ld.lld: warning: ignoring memory region assignment for non-allocatable section '.comment'

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/llvm-objdump --mattr=v -D bin/fmatmul > bin/fmatmul.dump

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/llvm-strip bin/fmatmul -S --strip-unneeded

rm common/serial-llvm.c.o common/printf-llvm.c.o common/crt0-llvm.S.o common/util-llvm.c.o common/string-llvm.c.o fmatmul/main.c.o

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/apps'

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

rm -rf build/verilator; mkdir -p build/verilator

/home/tanishq/Documents/ARA_new/ara/hardware/../hardware/bender script verilator -t rtl -t cv64a6_imafdcv_sv39 -t tech_cells_generic_include_tc_sram -t tech_cells_generic_include_tc_clk -t ara_test -t cva6_test -t verilator --define NR_LANES=4 --define VLEN=4096 --define ARIANE_ACCELERATOR_PORT=1 --define COMMON_CELLS_ASSERTS_OFF > build/verilator/bender_script_default

/home/tanishq/Documents/ARA_new/ara/install/verilator/bin/verilator -f build/verilator/bender_script_default           \

 -GNrLanes=4                                                         \

 -O3                                                                           \

 -Wno-fatal                                                                    \

 -Wno-PINCONNECTEMPTY                                                          \

 -Wno-BLKANDNBLK                                                               \

 -Wno-CASEINCOMPLETE                                                           \

 -Wno-CMPCONST                                                                 \

 -Wno-LATCH                                                                    \

 -Wno-LITENDIAN                                                                \

 -Wno-UNOPTFLAT                                                                \

 -Wno-UNPACKED                                                                 \

 -Wno-UNSIGNED                                                                 \

 -Wno-WIDTH                                                                    \

 -Wno-WIDTHCONCAT                                                              \

 -Wno-ENUMVALUE                                                                \

 -Wno-COMBDLY                                                                  \

 --hierarchical                                                                \

 tb/verilator/waiver.vlt                                                       \

 --Mdir build/verilator                                                       \

 -Itb/dpi                                                                      \

 --compiler clang                                                              \

 -CFLAGS "-DTOPLEVEL_NAME=ara_tb_verilator"                                        \

 -CFLAGS "-DNR_LANES=4"                                              \

 -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp       \

 -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp \

 -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp \

 ""                                                             \

 -LDFLAGS "-lelf"                                                              \

 ""                                                              \

 --exe                                                                         \

 /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/*.cc            \

 /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/*.cc      \

 /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/*.cc      \

 /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/ara_tb.cpp                                           \

 --cc                                                                          \

                                      \

 --top-module ara_tb_verilator &&                                                  \

cd build/verilator && OBJCACHE='' make -j4 -f Vara_tb_verilator.mk

make[1]: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware/build/verilator'

make[2]: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

/home/tanishq/Documents/ARA_new/ara/install/verilator/bin/verilator -f build/verilator/Vlane_e_hierMkArgs.f

make[2]: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

make[2]: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

/home/tanishq/Documents/ARA_new/ara/install/verilator/bin/verilator -f build/verilator/Vara_tb_verilator_hierMkArgs.f

make[2]: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

make[1]: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware/build/verilator'

make[1]: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware/build/verilator'

clang++  -I.  -MMD -I/home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/include -I/home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=0 -DVM_TRACE_FST=0 -DVM_TRACE_VCD=0 -faligned-new -fbracket-depth=4096 -fcf-protection=none -Qunused-arguments -Wno-bool-operation -Wno-tautological-bitwise-compare -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -DTOPLEVEL_NAME=ara_tb_verilator -DNR_LANES=4 -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp   -Os -c -o ara_tb.o /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/ara_tb.cpp

clang++  -I.  -MMD -I/home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/include -I/home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=0 -DVM_TRACE_FST=0 -DVM_TRACE_VCD=0 -faligned-new -fbracket-depth=4096 -fcf-protection=none -Qunused-arguments -Wno-bool-operation -Wno-tautological-bitwise-compare -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -DTOPLEVEL_NAME=ara_tb_verilator -DNR_LANES=4 -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp   -Os -c -o dpi_memutil.o /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/dpi_memutil.cc

clang++  -I.  -MMD -I/home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/include -I/home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=0 -DVM_TRACE_FST=0 -DVM_TRACE_VCD=0 -faligned-new -fbracket-depth=4096 -fcf-protection=none -Qunused-arguments -Wno-bool-operation -Wno-tautological-bitwise-compare -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -DTOPLEVEL_NAME=ara_tb_verilator -DNR_LANES=4 -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp   -Os -c -o sv_scoped.o /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/sv_scoped.cc

clang++  -I.  -MMD -I/home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/include -I/home/tanishq/Documents/ARA_new/ara/install/verilator/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=0 -DVM_TRACE_FST=0 -DVM_TRACE_VCD=0 -faligned-new -fbracket-depth=4096 -fcf-protection=none -Qunused-arguments -Wno-bool-operation -Wno-tautological-bitwise-compare -Wno-parentheses-equality -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -DTOPLEVEL_NAME=ara_tb_verilator -DNR_LANES=4 -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp   -Os -c -o verilator_memutil.o /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/verilator_memutil.cc

/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/dpi_memutil.cc:11:10: fatal error: 'libelf.h' file not found

#include <libelf.h>

        ^~~~~~~~~~

1 error generated.

make[1]: *** [Vara_tb_verilator.mk:79: dpi_memutil.o] Error 1

make[1]: *** Waiting for unfinished jobs....

make[1]: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware/build/verilator'

make: *** [Makefile:188: build/verilator/Vara_tb_verilator] Error 2

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

build/verilator/Vara_tb_verilator  -l ram,/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul,elf

bash: line 1: build/verilator/Vara_tb_verilator: No such file or directory

make: *** [Makefile:232: simv] Error 127

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

tanishq@Tanishq:~/Documents/ARA_new/ara$ 

@mp-17
Copy link
Collaborator

mp-17 commented Jul 1, 2024

Do you have libelf installed?

If not:
apt-get install libelf-dev

If it still cannot find the header after installing libelf, we need to explicitly tell clang++ where to find it.

@Tanishqgithub
Copy link
Author

Tanishqgithub commented Jul 1, 2024

  1. That worked
Simulation of Ara

=================





Simulation running, end by pressing CTRL-c.



=============

  FMATMUL  =

============







------------------------------------------------------------

Calculating a (4 x 4) x (4 x 4) matrix multiplication...

-----------------------------------------------------------



alculating fmatmul...

The execution took 745 cycles.

he performance is 0.171812 FLOP/cycle (2.147651% utilization).



------------------------------------------------------------

Calculating a (8 x 8) x (8 x 8) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 932 cycles.

The performance is 1.098712 FLOP/cycle (13.733906% utilization).



------------------------------------------------------------

Calculating a (16 x 16) x (16 x 16) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 2632 cycles.

The performance is 3.112462 FLOP/cycle (38.905777% utilization).



------------------------------------------------------------

Calculating a (32 x 32) x (32 x 32) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 11455 cycles.

The performance is 5.721170 FLOP/cycle (71.514626% utilization).



------------------------------------------------------------

Calculating a (64 x 64) x (64 x 64) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 70176 cycles.

The performance is 7.471044 FLOP/cycle (93.388054% utilization).



------------------------------------------------------------

Calculating a (128 x 128) x (128 x 128) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 534460 cycles.

The performance is 7.847742 FLOP/cycle (98.096771% utilization).

Verifying result...

Passed.

[hw-cycles]:           0

[2385856] -Info: ara_tb_verilator.sv:49: TOP.ara_tb_verilator: Core Test *** SUCCESS *** (tohost = 0)

- /home/tanishq/Documents/ARA_new/ara/hardware/tb/ara_tb_verilator.sv:52: Verilog $finish

Received $finish() from Verilog, shutting down simulation.



Simulation statistics

=====================

Executed cycles:  1233e0

Wallclock time:   711.175 s

Simulation speed: 1677.4 cycles/s (1.6774 kHz)

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

 
  1. I want to see the vcd trace file so i modified the command as specified in README :
# Compile fmatmul for Ara                                                                      
make -C apps bin/fmatmul vcd_dump=1
# Verilate the design
make -C hardware verilate 
# Run fmatmul on Ara through Verilator
make -C hardware simv app=fmatmul vcd_dump=1 vcd_path=$/home/tanishq/Documents/ARA_new/ara/

and got the following o/p :

tanishq@Tanishq:~/Documents/ARA_new/ara$ # Compile fmatmul for Ara                                                                      

make -C apps bin/fmatmul vcd_dump=1

# Verilate the design

make -C hardware verilate 

# Run fmatmul on Ara through Verilator

make -C hardware simv app=fmatmul vcd_dump=1 vcd_path=$/home/tanishq/Documents/ARA_new/ara/

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/apps'

cd fmatmul && if [ -d script ]; then python3 script/gen_data.py 128 128 128 > data.S ; else touch data.S; fi

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf -DVCD_DUMP=1 -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/data.S -o fmatmul/data.S.o

chmod +x /home/tanishq/Documents/ARA_new/ara/apps/common/script/align_sections.sh

rm -f /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld && cp /home/tanishq/Documents/ARA_new/ara/apps/common/arch.link.ld /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

/home/tanishq/Documents/ARA_new/ara/apps/common/script/align_sections.sh 4 /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf -DVCD_DUMP=1 -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/main.c -o fmatmul/main.c.o

In file included from fmatmul/main.c:26:

/home/tanishq/Documents/ARA_new/ara/apps/common/util.h:29:9: warning: VCD_DUMP successfully initialized [-W#pragma-messages]

#pragma message("VCD_DUMP successfully initialized")

        ^

In file included from fmatmul/main.c:29:

/home/tanishq/Documents/ARA_new/ara/apps/common/printf.h:59:1: warning: declaration does not declare anything [-Wmissing-declarations]

int printf_(const char *format, ...);

^~~

fmatmul/main.c:90:11: warning: unused variable 'utilization' [-Wunused-variable]

    float utilization = 100 * performance / (2.0 * NR_LANES);

          ^

3 warnings generated.

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf -DVCD_DUMP=1 -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/crt0.S -o common/crt0-llvm.S.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf -DVCD_DUMP=1 -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/printf.c -o common/printf-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf -DVCD_DUMP=1 -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/string.c -o common/string-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf -DVCD_DUMP=1 -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/serial.c -o common/serial-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf -DVCD_DUMP=1 -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/util.c -o common/util-llvm.c.o

In file included from common/util.c:21:

/home/tanishq/Documents/ARA_new/ara/apps/common/util.h:29:9: warning: VCD_DUMP successfully initialized [-W#pragma-messages]

#pragma message("VCD_DUMP successfully initialized")

        ^

1 warning generated.

mkdir -p bin/

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -Iinclude -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf -DVCD_DUMP=1 -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -o bin/fmatmul fmatmul/kernel/fmatmul.c.o fmatmul/main.c.o fmatmul/data.S.o common/crt0-llvm.S.o common/printf-llvm.c.o common/string-llvm.c.o common/serial-llvm.c.o common/util-llvm.c.o -static -nostartfiles -lm -Wl,--gc-sections -T/home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

ld.lld: warning: ignoring memory region assignment for non-allocatable section '.comment'

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/llvm-objdump --mattr=v -D bin/fmatmul > bin/fmatmul.dump

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/llvm-strip bin/fmatmul -S --strip-unneeded

rm common/serial-llvm.c.o common/printf-llvm.c.o common/crt0-llvm.S.o common/util-llvm.c.o common/string-llvm.c.o fmatmul/main.c.o

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/apps'

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

make: Nothing to be done for 'verilate'.

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2019.3) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

build/verilator/Vara_tb_verilator  -l ram,/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul,elf

Program header number 0 in `/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul' low is 80000000

Program header number 0 in `/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul' high is 80000a33

Program header number 1 in `/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul' high is 80080a77

Program header number 2 in `/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul' is not of type PT_LOAD; ignoring.

Set `ram TOP.ara_tb_verilator.dut.i_ara_soc.i_dram 10 0x80000000 0x100000 write with offset: 0x0 write with size: 0x80a78

Simulation of Ara

=================





Simulation running, end by pressing CTRL-c.

[hw-cycles]:           0

[2071108] -Info: ara_tb_verilator.sv:49: TOP.ara_tb_verilator: Core Test *** SUCCESS *** (tohost = 0)

- /home/tanishq/Documents/ARA_new/ara/hardware/tb/ara_tb_verilator.sv:52: Verilog $finish

Received $finish() from Verilog, shutting down simulation.



Simulation statistics

=====================

Executed cycles:  fcd22

Wallclock time:   712.685 s

Simulation speed: 1453.03 cycles/s (1.45303 kHz)

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'



But i couldn't see the .vcd file in location /home/tanishq/Documents/ARA_new/ara/

  1. To get the .fst file i stated the command :
# Compile fmatmul for Ara                                                                      
make -C apps bin/fmatmul        
# Verilate the design
make -C hardware verilate trace=1 
# Run fmatmul on Ara through Verilator
make -C hardware simv app=fmatmul trace=1 

and i got the following error :

tanishq@Tanishq:~/Documents/ARA_new/ara$ # Compile fmatmul for Ara                                                                      

make -C apps bin/fmatmul        

# Verilate the design

make -C hardware verilate trace=1 

# Run fmatmul on Ara through Verilator

make -C hardware simv app=fmatmul trace=1

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/apps'

cd fmatmul && if [ -d script ]; then python3 script/gen_data.py 128 128 128 > data.S ; else touch data.S; fi

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/data.S -o fmatmul/data.S.o

chmod +x /home/tanishq/Documents/ARA_new/ara/apps/common/script/align_sections.sh

rm -f /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld && cp /home/tanishq/Documents/ARA_new/ara/apps/common/arch.link.ld /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

/home/tanishq/Documents/ARA_new/ara/apps/common/script/align_sections.sh 4 /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/main.c -o fmatmul/main.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/crt0.S -o common/crt0-llvm.S.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/printf.c -o common/printf-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/string.c -o common/string-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/serial.c -o common/serial-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/util.c -o common/util-llvm.c.o

mkdir -p bin/

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -Iinclude -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -o bin/fmatmul fmatmul/kernel/fmatmul.c.o fmatmul/main.c.o fmatmul/data.S.o common/crt0-llvm.S.o common/printf-llvm.c.o common/string-llvm.c.o common/serial-llvm.c.o common/util-llvm.c.o -static -nostartfiles -lm -Wl,--gc-sections -T/home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

ld.lld: warning: ignoring memory region assignment for non-allocatable section '.comment'

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/llvm-objdump --mattr=v -D bin/fmatmul > bin/fmatmul.dump

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/llvm-strip bin/fmatmul -S --strip-unneeded

rm common/serial-llvm.c.o common/printf-llvm.c.o common/crt0-llvm.S.o common/util-llvm.c.o common/string-llvm.c.o fmatmul/main.c.o

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/apps'

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

make: Nothing to be done for 'verilate'.

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

build/verilator/Vara_tb_verilator -t -l ram,/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul,elf

ERROR: Tracing has not been enabled at compile time.

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

tanishq@Tanishq:~/Documents/ARA_new/ara$ 

Can you please tell me what step i am going wrong to generate .vcd file and .fst file

@Tanishqgithub
Copy link
Author

Tanishqgithub commented Jul 2, 2024 via email

@mp-17
Copy link
Collaborator

mp-17 commented Jul 2, 2024

I have never tried to bump .vcd files with verilator. However, you can try with the full path to a file, not a directory:

make -C hardware simv app=fmatmul vcd_dump=1 vcd_path=$/home/tanishq/Documents/ARA_new/ara/fmatmul.vcd

Regarding the verilator trace, you should verilate the design again:

make -C hardware verilate trace=1 -B

@Tanishqgithub
Copy link
Author

Tanishqgithub commented Jul 2, 2024

@mp-17

  1. vcd File 
    

There is no output fmatmul.vcd obtained in the path /home/tanishq/Documents/ARA_new/ara/fmatmul.vcd found
But the terminal o/p shows SUCESS same like o/p in previous message
The command i ran is :

# Compile fmatmul for Ara                                                                      
make -C apps bin/fmatmul vcd_dump=1
# Verilate the design
make -C hardware verilate 
# Run fmatmul on Ara through Verilator
make -C hardware simv app=fmatmul vcd_dump=1 vcd_path=$/home/tanishq/Documents/ARA_new/ara/fmatmul.vcd
  1. fst file   
    

For the command :

# Compile fmatmul for Ara                                                                      
make -C apps bin/fmatmul        
# Verilate the design
make -C hardware verilate trace=1 -B 
# Run fmatmul on Ara through Verilator
make -C hardware simv app=fmatmul trace=1 

i got this error :

build/verilator/Vara_tb_verilator -t -l ram,/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul,elf

Program header number 0 in `/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul' low is 80000000

Program header number 0 in `/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul' high is 80001a8b

Program header number 1 in `/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul' high is 80081aaf

Program header number 2 in `/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul' high is 80081fb7

Program header number 3 in `/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul' high is 80081ff7

Program header number 4 in `/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul' is not of type PT_LOAD; ignoring.

Set `ram TOP.ara_tb_verilator.dut.i_ara_soc.i_dram 10 0x80000000 0x100000 write with offset: 0x0 write with size: 0x81ff8

Simulation of Ara

=================



Tracing can be toggled by sending SIGUSR1 to this process:

$ kill -USR1 89b87

make: *** [Makefile:232: simv] Segmentation fault (core dumped)

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'


@mp-17
Copy link
Collaborator

mp-17 commented Jul 2, 2024

For the fst, you can try to apply this commit before verilating.

For the vcd file I will need more time to check

@Tanishqgithub
Copy link
Author

@mp-17
As u said for fst file i did add the commit and then i got this o/p :

tanishq@Tanishq:~/Documents/ARA_new/ara$ # Compile fmatmul for Ara                                                                      

make -C apps bin/fmatmul        

# Verilate the design

make -C hardware verilate trace=1 -B 

# Run fmatmul on Ara through Verilator

make -C hardware simv app=fmatmul trace=1 

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/apps'

cd fmatmul && if [ -d script ]; then python3 script/gen_data.py 128 128 128 > data.S ; else touch data.S; fi

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/data.S -o fmatmul/data.S.o

chmod +x /home/tanishq/Documents/ARA_new/ara/apps/common/script/align_sections.sh

rm -f /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld && cp /home/tanishq/Documents/ARA_new/ara/apps/common/arch.link.ld /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

/home/tanishq/Documents/ARA_new/ara/apps/common/script/align_sections.sh 4 /home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c fmatmul/main.c -o fmatmul/main.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/crt0.S -o common/crt0-llvm.S.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/printf.c -o common/printf-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/string.c -o common/string-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/serial.c -o common/serial-llvm.c.o

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -c common/util.c -o common/util-llvm.c.o

mkdir -p bin/

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/clang -Iinclude -march=rv64gcv_zfh_zvfh0p1 -menable-experimental-extensions -mabi=lp64d -mno-relax -fuse-ld=lld -fno-vectorize -mllvm -scalable-vectorization=off -mllvm -riscv-v-vector-bits-min=0 -Xclang -target-feature -Xclang +no-optimized-zero-stride-load -mcmodel=medany -I/home/tanishq/Documents/ARA_new/ara/apps/common -std=gnu99 -O3 -ffast-math -fno-common -fno-builtin-printf  -DNR_LANES=4 -DVLEN=4096 -Wunused-variable -Wall -Wextra -Wno-unused-command-line-argument  -ffunction-sections -fdata-sections -o bin/fmatmul fmatmul/kernel/fmatmul.c.o fmatmul/main.c.o fmatmul/data.S.o common/crt0-llvm.S.o common/printf-llvm.c.o common/string-llvm.c.o common/serial-llvm.c.o common/util-llvm.c.o -static -nostartfiles -lm -Wl,--gc-sections -T/home/tanishq/Documents/ARA_new/ara/apps/common/link.ld

ld.lld: warning: ignoring memory region assignment for non-allocatable section '.comment'

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/llvm-objdump --mattr=v -D bin/fmatmul > bin/fmatmul.dump

/home/tanishq/Documents/ARA_new/ara/install/riscv-llvm/bin/llvm-strip bin/fmatmul -S --strip-unneeded

rm common/serial-llvm.c.o common/printf-llvm.c.o common/crt0-llvm.S.o common/util-llvm.c.o common/string-llvm.c.o fmatmul/main.c.o

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/apps'

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

mkdir -p build

Bender already exists.

bender 0.27.3 available.

rm -rf build/verilator; mkdir -p build/verilator

/home/tanishq/Documents/ARA_new/ara/hardware/../hardware/bender script verilator -t rtl -t cv64a6_imafdcv_sv39 -t tech_cells_generic_include_tc_sram -t tech_cells_generic_include_tc_clk -t ara_test -t cva6_test -t verilator --define NR_LANES=4 --define VLEN=4096 --define ARIANE_ACCELERATOR_PORT=1 --define COMMON_CELLS_ASSERTS_OFF > build/verilator/bender_script_default

/home/tanishq/Documents/ARA_new/ara/install/verilator/bin/verilator -f build/verilator/bender_script_default           \

  -GNrLanes=4                                                         \

  -O3                                                                           \

  -Wno-fatal                                                                    \

  -Wno-PINCONNECTEMPTY                                                          \

  -Wno-BLKANDNBLK                                                               \

  -Wno-CASEINCOMPLETE                                                           \

  -Wno-CMPCONST                                                                 \

  -Wno-LATCH                                                                    \

  -Wno-LITENDIAN                                                                \

  -Wno-UNOPTFLAT                                                                \

  -Wno-UNPACKED                                                                 \

  -Wno-UNSIGNED                                                                 \

  -Wno-WIDTH                                                                    \

  -Wno-WIDTHCONCAT                                                              \

  -Wno-ENUMVALUE                                                                \

  -Wno-COMBDLY                                                                  \

  tb/verilator/waiver.vlt                                                       \

  --Mdir build/verilator                                                       \

  -Itb/dpi                                                                      \

  --compiler clang                                                              \

  -CFLAGS "-DTOPLEVEL_NAME=ara_tb_verilator"                                        \

  -CFLAGS "-DNR_LANES=4"                                              \

  -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp       \

  -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp \

  -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp \

  ""                                                             \

  -LDFLAGS "-lelf"                                                              \

  ""                                                              \

  --exe                                                                         \

  /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/*.cc            \

  /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/*.cc      \

  /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/*.cc      \

  /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/ara_tb.cpp                                           \

  --cc                                                                          \

  --trace-fst -Wno-INSECURE                                     \

  --top-module ara_tb_verilator &&                                                  \

cd build/verilator && OBJCACHE='' make -j4 -f Vara_tb_verilator.mk

%Error: Verilator threw signal 9. Suggest trying --debug --gdbbt

%Error: Command Failed ulimit -s unlimited 2>/dev/null; exec /home/tanishq/Documents/ARA_new/ara/install/verilator/bin/verilator_bin -f build/verilator/bender_script_default -GNrLanes=4 -O3 -Wno-fatal -Wno-PINCONNECTEMPTY -Wno-BLKANDNBLK -Wno-CASEINCOMPLETE -Wno-CMPCONST -Wno-LATCH -Wno-LITENDIAN -Wno-UNOPTFLAT -Wno-UNPACKED -Wno-UNSIGNED -Wno-WIDTH -Wno-WIDTHCONCAT -Wno-ENUMVALUE -Wno-COMBDLY tb/verilator/waiver.vlt --Mdir build/verilator -Itb/dpi --compiler clang -CFLAGS -DTOPLEVEL_NAME=ara_tb_verilator -CFLAGS -DNR_LANES=4 -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp -CFLAGS -I/home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp  -LDFLAGS -lelf  --exe /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/dpi_memutil.cc /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_dpi/cpp/sv_scoped.cc /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_memutil_verilator/cpp/verilator_memutil.cc /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/verilated_toplevel.cc /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/lowrisc_dv_verilator_simutil_verilator/cpp/verilator_sim_ctrl.cc /home/tanishq/Documents/ARA_new/ara/hardware/tb/verilator/ara_tb.cpp --cc --trace-fst -Wno-INSECURE --top-module ara_tb_verilator

make: *** [Makefile:188: build/verilator/Vara_tb_verilator] Error 255

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

make: Entering directory '/home/tanishq/Documents/ARA_new/ara/hardware'

Makefile:83: "Specified QuestaSim version (questa-2021.2) not found in PATH /home/tanishq/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin"

build/verilator/Vara_tb_verilator -t -l ram,/home/tanishq/Documents/ARA_new/ara/apps/bin/fmatmul,elf

bash: line 1: build/verilator/Vara_tb_verilator: No such file or directory

make: *** [Makefile:231: simv] Error 127

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

@mp-17
Copy link
Collaborator

mp-17 commented Jul 2, 2024

I am not sure why this happens, I will need to debug this as soon as I have time :-)

@Tanishqgithub
Copy link
Author

Tanishqgithub commented Jul 3, 2024

@mp-17

For fst i tried the solution in #260 i,e changing -O3 to -O0 but the simulation sort of stopped midway and gave this o/p :

Set `ram TOP.ara_tb_verilator.dut.i_ara_soc.i_dram 10 0x80000000 0x100000 write with offset: 0x0 write with size: 0x81ff8

Simulation of Ara

=================



Tracing can be toggled by sending SIGUSR1 to this process:

$ kill -USR1 2515

Tracing enabled.

Writing simulation traces to sim.fst



Simulation running, end by pressing CTRL-c.



=============

  FMATMUL  =

============







------------------------------------------------------------

Calculating a (4 x 4) x (4 x 4) matrix multiplication...

-----------------------------------------------------------



alculating fmatmul...

The execution took 744 cycles.

he performance is 0.172043 FLOP/cycle (2.150538% utilization).



------------------------------------------------------------

Calculating a (8 x 8) x (8 x 8) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 931 cycles.

The performance is 1.099893 FLOP/cycle (13.748657% utilization).



------------------------------------------------------------

Calculating a (16 x 16) x (16 x 16) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 2593 cycles.

The performance is 3.159275 FLOP/cycle (39.490936% utilization).



------------------------------------------------------------

Calculating a (32 x 32) x (32 x 32) matrix multiplication...

-----------------------------------------------------------



Calculating fmatmul...

The execution took 11446 cycles.

[121474] %Warning: ara_tb_verilator.sv:45: TOP.ara_tb_verilator: Core Test *** FAILED *** (tohost = 2)

- /home/tanishq/Documents/ARA_new/ara/hardware/tb/ara_tb_verilator.sv:52: Verilog $finish

Received $finish() from Verilog, shutting down simulation.



Simulation statistics

=====================

Executed cycles:  ed41

Wallclock time:   147.126 s

Simulation speed: 412.823 cycles/s (0.412823 kHz)

Trace file size:  231eb5a B



You can view the simulation traces by calling

$ gtkwave sim.fst

make: *** [Makefile:232: simv] Error 2

make: Leaving directory '/home/tanishq/Documents/ARA_new/ara/hardware'

and while viewing sim.fst in terminal :

tanishq@Tanishq:~/Documents/ARA_new/ara$ gtkwave sim.fst

Gtk-Message: 08:07:46.919: Failed to load module "canberra-gtk-module"



GTKWave Analyzer v3.3.104 (w)1999-2020 BSI



GTKWAVE | Could not initialize 'sim.fst', exiting.

and by manually opening the sim.fst in the file location i could see only signals of CVA6

  1. Along with this other query is how can i see the signals for the ARA [vector processor] only .

  2. And in the same issue at last it is suggested to bump down verilator version to v4.214 , i did changes to make file and again ran make verilator &> verilator_install_4.log and the error occured is in :
    verilator_install4.txt

Can you please tell me how to encounter this issue and if the solution is bumping down the verilator version then please tell me what is the solution for the error i am occuring.

@Tanishqgithub
Copy link
Author

@mp-17
Respected sir

A kind request to reply for the last query

Thanking you
Tanishq.S

@Tanishqgithub
Copy link
Author

@mp-17

I did bump down the verilator version to 4.214 and i tried the fst simulation :

# Compile fmatmul for Ara                                                                      
make -C apps bin/fmatmul        
# Verilate the design
make -C hardware verilate trace=1 -B 
# Run fmatmul on Ara through Verilator
make -C hardware simv app=fmatmul trace=1 

i got this o/p in terminal :
fst_new.txt
I didn't sense any error in execution and seems that .fst has been successfully created.
But when i run gtkwave sim.fst i got this o/p :

Gtk-Message: 11:14:57.588: Failed to load module "canberra-gtk-module"

GTKWave Analyzer v3.3.104 (w)1999-2020 BSI

GTKWAVE | Could not initialize 'sim.fst', exiting.

and then i tried to open sim.fst in ara/hardware manually then i got this submodules in simulation wave:
image

Am i getting the correct .fst file or anything is missing ?

@Tanishqgithub
Copy link
Author

@mp-17

Can you please update on the fst file i have got in the previous message

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants