Skip to content

Add systemverilog support,Fix verilator build error#3944

Merged
waruqi merged 1 commit intoxmake-io:devfrom
leesum1:verilator_fix
Jul 10, 2023
Merged

Add systemverilog support,Fix verilator build error#3944
waruqi merged 1 commit intoxmake-io:devfrom
leesum1:verilator_fix

Conversation

@leesum1
Copy link
Contributor

@leesum1 leesum1 commented Jul 9, 2023

  1. Add extension .sv to enable systemverilog support
  2. Remove unnecessary flags in test

我不知道为什么要在 config 函数中测试正确性的时候将 verilator.flags 读取进来,加入到那个很简单的 main.v 的编译参数中来,这样做的目的是什么?

进行内部测试的时候应该也不需要读取用户的参数,这就导致了很多问题,用户输入的参数会影响内部测试的结果。verilator 很多参数不能用,模块重命名等等。

关于原先 --top 参数不能用的问题(Adding Support for Verilog and SystemVerilog · Issue #3257 · xmake-io/xmake · GitHub),其实是参数解析的问题,我不知道为什么参数解析遇到空格就罢工了,是就这样子设计的吗?

经过修改后,我把一些工程改造为使用 xmake 编译运行,都没有问题,包括 systemverilog 工程。

文档应该说明的内容

verilator 参数参考
verilator Arguments — Verilator 5.012 documentation

verilator 参数 --top 选项问题

--top keyval 需要分别添加,不能放在一个字符串里面,可能是 xmake 解析问题。

-- 正确用法
add_values("verilator.flags", "--top","test_top"-- 错误用法
add_values("verilator.flags", "--top test_top"

如果使用 "--top test_top" 传递参数时,会直接传递为 "--top test_top",由于多个双引号,导致参数解析错误

verilator 参数 --prefix

这种 --key val 形式的参数,都需要将 --keyval 分别存放在两个字符串里。

--prefix 选项在 xmake 中已经被使用了,如果需要更改,可以改 target

-- 错误用法
target("Vtop")
    add_rules("verilator.binary")
    set_toolchains("@verilator")
    add_files("src/*.cpp")
    add_files("src/*.sv")
    add_values("verilator.flags", "--prefix","my_prefix")

-- 正确用法
target("my_prefix")
    add_rules("verilator.binary")
    set_toolchains("@verilator")
    add_files("src/*.cpp")
    add_files("src/*.sv")

verilator 参数 --Mdir

--Mdir 已经被 xmake 内部使用,不能自定义,一个具体的编译命令如下(使用 xmake -v 得到 )

verilator --cc --make cmake --prefix Vtop --Mdir build/.gens/Vtop/linux/x86_64/release/rules/verilator

verilator 参数 --trace-fst 问题

fst 格式的波形文件比 vcd 格式的大小小的多,并且依赖于 libz ,如果启用,需要手动添加 libz 依赖。

-- 正确用法
add_values("verilator.flags", "--trace-fst"add_syslinks("z")
--错误用法
add_values("verilator.flags", "--trace-fst"

xmake 参数 add_includedirs() add_defines() 问题

xmake.luaadd_includedirsadd_defines 都只作用于 cpp 文件,对于 .v .sv 文件是没有效果的。
如果要为 rtl 文件添加 includedirs 或者 defines 需要在 add_values 里面添加。具体方法如下:

verilator 宏定义

-D<var>=<value>
Defines the given preprocessor symbol. Similar to +define, but does not allow multiple definitions with a single option using plus signs. “+define” is relatively standard across Verilog tools, while “-D” is similar to gcc -D.

+define+<var>=<value>
+define+<var>=<value>[+<var2>=<value2>][...]
Defines the given preprocessor symbol, or multiple symbols if separated by plus signs. Similar to -D; +define is relatively standard across Verilog tools while -D is similar to gcc -D.

verilator includedir

-y <dir>
Add the directory to the list of directories that should be searched to find include files or libraries. The three flags -y, +incdir+<dir> and -I<dir> have a similar effect; +incdir+<dir> and -y are relatively standard across Verilog tools while -I<dir> is used by many C++ compilers.

Verilator defaults to the current directory “-y .” and any specified --Mdir, though these default paths are used after any user-specified directories. This allows ‘-y “$(pwd)”’ to be used if absolute filenames are desired for error messages instead of relative filenames.
-- 错误用法
add_includedirs("vsrc/include")
add_defines("VERILOG_DEFINE")
-- 正确方法 1
add_values("verilator.flags", "-DVERILOG_DEFINE""-y""vsrc/include"-- 正确方法 2
add_values("verilator.flags", "-DVERILOG_DEFINE""-Ivsrc/include"-- 正确用法 3
add_values("verilator.flags", "-DVERILOG_DEFINE""+incdir+vsrc/include"

@waruqi
Copy link
Member

waruqi commented Jul 9, 2023

我不知道为什么要在 config 函数中测试正确性的时候将 verilator.flags 读取进来,加入到那个很简单的 main.v 的编译参数中来,这样做的目的是什么?

做测试的目的,就是为了获取对应用户完整配置下的各种宏开关,比如用户通过 verilator.flags 额外配置 --threads,通过测试从生成的 test.cmake 提取 VM_THREADS=XX 的定义值,其他还有 --trace, --timing, --coverage, --sc, --trace-fst 等扩展的用户设置 flags ,都有影响实际编译的 macros 定义,VM_COVERAGE , ...,这些都需要从测试中,同步用户的配置后,获取到对应定义值。。

进行内部测试的时候应该也不需要读取用户的参数,这就导致了很多问题,用户输入的参数会影响内部测试的结果。verilator 很多参数不能用,模块重命名等等。

一些有干扰的 flags ,目前只能通过内部忽略掉,之前已经忽略了 -Gxxx,你可以继续忽略一些 flags,但是这个设置逻辑不能删 。。

table.insert(argv, flag)
end
end
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不能删

-- @see https://github.com/xmake-io/xmake/issues/3257
rule("verilator.binary")
set_extensions(".v")
set_extensions(".v",".sv")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

逗号两边空格

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I don't know why verilator.flags should be read in when testing correctness in the config function, and added to the very simple compilation parameters of main.v. What is the purpose of doing this?

The purpose of the test is to obtain various macro switches corresponding to the complete configuration of the user. For example, the user additionally configures --threads through verilator.flags, and extracts VM_THREADS=XX from the generated test.cmake through the test. Define the value, and other extended user setting flags such as --trace, --timing, --coverage, --sc, --trace-fst, etc., all affect the actual compilation The definition of macros needs to obtain the corresponding definition value from the test after synchronizing the user's configuration. .

There should be no need to read user parameters when conducting internal tests, which leads to many problems. The parameters entered by users will affect the results of internal tests. Many parameters of verilator cannot be used, module renaming and so on.

Some interfering flags can only be ignored internally at present, -Gxxx has been ignored before, you can continue to ignore some flags, but this setting logic cannot be deleted. .

@leesum1
Copy link
Contributor Author

leesum1 commented Jul 9, 2023

我不知道为什么要在 config 函数中测试正确性的时候将 verilator.flags 读取进来,加入到那个很简单的 main.v 的编译参数中来,这样做的目的是什么?

做测试的目的,就是为了获取对应用户完整配置下的各种宏开关,比如用户通过 verilator.flags 额外配置 --threads,通过测试从生成的 test.cmake 提取 VM_THREADS=XX 的定义值,其他还有 --trace, --timing, --coverage, --sc, --trace-fst 等扩展的用户设置 flags ,都有影响实际编译的 macros 定义,VM_COVERAGE , ...,这些都需要从测试中,同步用户的配置后,获取到对应定义值。。

进行内部测试的时候应该也不需要读取用户的参数,这就导致了很多问题,用户输入的参数会影响内部测试的结果。verilator 很多参数不能用,模块重命名等等。

一些有干扰的 flags ,目前只能通过内部忽略掉,之前已经忽略了 -Gxxx,你可以继续忽略一些 flags,但是这个设置逻辑不能删 。。

为什么要将这些参数通过 test.cmake 得到呢?用户输入了自己的 verilog 项目文件,直接编译用户输入的就行了,在 buildcmd_vfiles 中编译用户的 verilog 文件,也会得到 .cmake 文件,直接从这里提取信息就好了吧。

并且由于上面的 --top 问题 --top 后面跟着的 val 参数,也让忽略变得有点复杂。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I don't know why the verilator.flags should be read in when testing the correctness in the config function, and added to the compilation parameters of the very simple main.v. What is the purpose of doing this?

The purpose of the test is to obtain various macro switches corresponding to the complete configuration of the user. For example, the user additionally configures --threads through verilator.flags, and extracts VM_THREADS=XX from the generated test.cmake through the test The definition value of --trace, --timing, --coverage, --sc, --trace-fst and other extended user setting flags, all of which affect the actual compilation The definition of macros, VM_COVERAGE, ..., these all need to get the corresponding definition value from the test after synchronizing the user configuration. .

There should be no need to read the user's parameters when conducting internal tests, which leads to many problems. The parameters entered by the user will affect the results of the internal test. Many parameters of verilator cannot be used, module renaming and so on.

Some interfering flags can only be ignored internally at present, -Gxxx has been ignored before, you can continue to ignore some flags, but this setting logic cannot be deleted. .

Why should these parameters be obtained through test.cmake? The user has entered his own verilog project file, just compile the user input directly, compile the user's verilog file in buildcmd_vfiles, and also get the .cmake file, just extract the information directly from here.

And due to the --top problem above --top is followed by the val parameter, which also complicates the ignore a bit.

@waruqi
Copy link
Member

waruqi commented Jul 9, 2023

为什么要将这些参数通过 test.cmake 得到呢?用户输入了自己的 verilog 项目文件,直接编译用户输入的就行了,在 buildcmd_vfiles 中编译用户的 verilog 文件,也会得到 .cmake 文件,直接从这里提取信息就好了吧。

不能,buildcmd_vfiles 阶段已经是实际构建阶段了,滞后了,这个时候去添加其他 flags ,无法影响到其他 cpp files 编译,而且会干扰增量编译,并且无法对 project generator 生效这些设置。。只能在 on_config 阶段,提前设置

并且由于上面的 --top 问题 --top 后面跟着的 val 参数,也让忽略变得有点复杂。

就算复杂,目前也只能这么做

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Why should these parameters be obtained through test.cmake? The user has entered his own verilog project file, just compile the user input directly, compile the user's verilog file in buildcmd_vfiles, and also get the .cmake file, just extract the information directly from here.

No, the buildcmd_vfiles stage is already the actual construction stage, and it is lagging behind. Adding other flags at this time will not affect the compilation of other cpp files, and will interfere with incremental compilation. . It can only be set in advance in the on_config stage

@leesum1
Copy link
Contributor Author

leesum1 commented Jul 9, 2023

为什么要将这些参数通过 test.cmake 得到呢?用户输入了自己的 verilog 项目文件,直接编译用户输入的就行了,在 buildcmd_vfiles 中编译用户的 verilog 文件,也会得到 .cmake 文件,直接从这里提取信息就好了吧。

不能,buildcmd_vfiles 阶段已经是实际构建阶段了,滞后了,这个时候去添加其他 flags ,无法影响到其他 cpp files 编译,而且会干扰增量编译,并且无法对 project generator 生效这些设置。。只能在 on_config 阶段,提前设置

verilator 编译过程为 .v 文件转换为 cpp 文件,再用 gcc 编译 cpp 文件。这两部分是串行的,完全可以安装逻辑一路走下来,不用绕一个大弯。

如果是因为 config 阶段和 build 阶段不同的原因,不如把 buildcmd_vfiles 放在 config 阶段,增量编译也只是 cpp 文件的范围。和 .v 文件没关系,反正每次修改 verilog 代码后都要重新生成 cpp 文件。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Why should these parameters be obtained through test.cmake? The user has entered his own verilog project file, just compile the user input directly, compile the user's verilog file in buildcmd_vfiles, and also get the .cmake file, just extract the information directly from here.

No, the buildcmd_vfiles stage is already the actual construction stage, and it is lagging behind. Adding other flags at this time will not affect the compilation of other cpp files, and will interfere with incremental compilation, and these settings will not take effect on the project generator. . It can only be set in advance in the on_config stage

The verilator compilation process converts the .v file into a cpp file, and then compiles the cpp file with gcc. These two parts are serial, and it is completely possible to install the logic all the way down without making a big bend.

If it is because of the difference between the config phase and the build phase, it is better to put buildcmd_vfiles in the config phase, and the incremental compilation is only the scope of the cpp file. It has nothing to do with the .v file. Anyway, the cpp file must be regenerated every time the verilog code is modified.

@waruqi
Copy link
Member

waruqi commented Jul 9, 2023

verilator 编译过程为 .v 文件转换为 cpp 文件,再用 gcc 编译 cpp 文件。这两部分是串行的,完全可以安装逻辑一路走下来,不用绕一个大弯。

首先,buildcmd_vfiles 的模式是完全不支持这么做的,它是缓存了所有 cmds ,后面批量执行的,执行过程中,无法再脚本化处理 flags 设置,实际开始执行 buildcmd_vfiles 里面的命令的时候,已经完全过了 defines flags 等设置阶段。。

另外,处理 vfiles 阶段获取的 flags ,这个阶段尽管比如 cppfile 构建阶段靠前,但是已经过了 xmake 内部设置 flags 阶段,内部有各种缓存和复杂的逻辑处理,cppfile 阶段很难可靠的获取到完整真实的 flags 。。你只能 on_config 之前做配置

还有 build_cppfiles 是并行的,跟 buildcmd_cppfiles 不同

在 on_config 里设置,可以很好的对 build_cppfiles 和 buildcmd_cppfiles 同时生效配置,否则 这两个里面都要做一遍,而且也做不了。

如果是因为 config 阶段和 build 阶段不同的原因,不如把 buildcmd_vfiles 放在 config 阶段,增量编译也只是 cpp 文件的范围。和 .v 文件没关系,反正每次修改 verilog 代码后都要重新生成 cpp 文件。

不能,buildcmd_vfiles 主要是给 project generator 用的,所以只能串行。 但是 build_cppfiles 的 batchjob 构建模式,会并行处理。

放到 on_config 做,project generator 完全就不生效了,另外 build_cppfiles 并行化也失效了。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


The verilator compilation process converts the .v file into a cpp file, and then compiles the cpp file with gcc. These two parts are serial, and it is completely possible to install the logic all the way down without making a big bend.

First of all, the buildcmd_vfiles mode does not support this at all. It caches all cmds and executes them in batches later. During the execution process, the flags settings cannot be scripted. When the commands in buildcmd_vfiles actually start to be executed, it has completely passed Defines flags and other setup phases. .

If it is because of the difference between the config phase and the build phase, it is better to put buildcmd_vfiles in the config phase, and the incremental compilation is only the scope of the cpp file. It has nothing to do with the .v file. Anyway, the cpp file must be regenerated every time the verilog code is modified.

No, buildcmd_vfiles is mainly for the project generator, so it can only be serialized. But in the later period, the batchjob construction mode may be further added, and the full parallel processing, but it is not added for the time being.

Put it in on_config, the project generator will not take effect at all, and it cannot be parallelized in the later stage.

@leesum1
Copy link
Contributor Author

leesum1 commented Jul 9, 2023

我不清楚 project generator 是什么,我没有在 xmake doc 的 verilator 章节中看见。

我暂时理解 xmake 的编译模式是这样子的,verilator 负责把 vfile 转换为 cppfile,并且有一个 cmake 文件描述所转换 cppfile 的编译规则。这个过程 xmake 也不能干预什么,这是 verilator 内部的过程。

xmake 拿到 cmake 文件后,解析 cmake 转换为 xmake 自己内部的编译规则,然后再调用已经实现的 cpp 编译进行编译,增量编译等等优化都可以在编译 cpp 的时候进行。

主要就是拿到 verilator 转换 vfile 后的 cmake 罢了。这个 cmake 文件放在 build 阶段之前拿到不行吗?

目前把第一步 verilator 转换 vfile 的过程放在了 build 阶段,导致拿到 cmake 文件后只能得到 cpp 文件信息,一下宏定义也无法在 build 阶段进行设置。

导致必须在 config 阶段用一种很麻烦,和容易出问题的方式进行去拿到一些宏定义信息。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I don't know what the project generator is, I didn't see it in the verilator chapter of the xmake doc.

I temporarily understand that the compilation mode of xmake is like this, verilator is responsible for converting vfile to cppfile, and there is a cmake file describing the compilation rules of the converted cppfile. This process xmake can not interfere with anything, this is the internal process of verilator.

After xmake gets the cmake file, it parses cmake and converts it into xmake's own internal compilation rules, and then calls the implemented cpp compilation to compile, incremental compilation and other optimizations can be performed when compiling cpp.

The main thing is to get the cmake after the verilator converts the vfile. Can this cmake file be obtained before the build phase?

At present, the first step of converting vfile by verilator is placed in the build phase, so after getting the cmake file, only cpp file information can be obtained, and the macro definition cannot be set in the build phase.

As a result, it is necessary to use a troublesome and problem-prone way to obtain some macro definition information in the config stage.

@waruqi
Copy link
Member

waruqi commented Jul 9, 2023

主要就是拿到 verilator 转换 vfile 后的 cmake 罢了。这个 cmake 文件放在 build 阶段之前拿到不行吗?

不行,只能在 on_config 时候做

我不清楚 project generator 是什么,我没有在 xmake doc 的 verilator 章节中看见。

两种模式,build_cppfiles 用于直接构建,并行编译

buildcmd_cppfiles 用于 project generator,也就是工程生成器,可以用来生成 makefile, cmakelists.txt, build.ninja, compile_commands ,给第三方构建工具和 IDE 使用。。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


The main thing is to get the cmake after the verilator converts the vfile. Can this cmake file be obtained before the build phase?

No, it can only be done during on_config

I don't know what the project generator is, I didn't see it in the verilator chapter of xmake doc.

Two modes, build_cppfiles for direct build, parallel compilation

buildcmd_cppfiles is used for project generator, that is, project generator, which can be used to generate makefile, cmakelists.txt, build.ninja, compile_commands for third-party build tools and IDEs. .

@leesum1
Copy link
Contributor Author

leesum1 commented Jul 9, 2023

on_config

on_config 不是在 build 之前吗?反正我最开始每次都是在 on_cofnig 阶段编译 test.v 时报莫名其妙的错误。
on_config 应该就是拿到 cmake 里面的宏定义等等参数吧。

如果因为各种原因不能改变结构的话,那当前 on_config 是拿一个 test.v 加上用户输入的 verilator flags 进行编译得到 cmake 从而得到宏定义等等参数。

那为什么不直接使用用户输入的 vfiles 替代可能会有各种参数问题的 test.v 来生成 cmake 呢?报错的问题也是和用户文件相关的,并且也不用进行什么参数过滤

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


on_config

Isn't on_config before build? Anyway, every time I compiled test.v in the on_cofnig stage, I reported some inexplicable errors.
on_config should be to get the macro definition and other parameters in cmake.

If the structure cannot be changed due to various reasons, the current on_config is compiled with a test.v plus the verilator flags input by the user to obtain cmake to obtain macro definitions and other parameters.

Then why not directly use the vfiles input by the user instead of test.v which may have various parameter problems to generate cmake? The error problem is also related to the user file, and there is no need to filter any parameters

@leesum1
Copy link
Contributor Author

leesum1 commented Jul 9, 2023

我刚刚又看了一下 verilator.lua 源码,config 从 cmake 获取的宏定义就是下面几个

### Switches...
# SystemC output mode?  0/1 (from --sc)
set(Vtop_SC 0)
# Coverage output mode?  0/1 (from --coverage)
set(Vtop_COVERAGE 1)
# Timing mode?  0/1
set(Vtop_TIMING 0)
# Threaded output mode?  1/N threads (from --threads)
set(Vtop_THREADS 4)
# VCD Tracing output mode?  0/1 (from --trace)
set(Vtop_TRACE_VCD 0)
# FST Tracing output mode? 0/1 (from --trace-fst)
set(Vtop_TRACE_FST 1)

但其实这些宏定义并没有在生成的 cpp 文件里用到,所以我删掉那些代码也没有什么影响,经过测试,我的程序功能还是正常(因为我的程序里没有用到这些宏定义),但说不定用户会使用这些宏定义。

并且我看到 config 函数里也没有办法获取 vfile。我提到的用 vfile 替代 test.v 也不行,但其实还有一个更简单的方法可以解决这个麻烦的问题。

目前从 cmake 中获取的宏定义就那么几个,并且都有明确的参数来源,可以就只对这几个参数敏感,不用管其他的 verilator 参数。
修改前:

    if flags then
        for _, flag in ipairs(flags) do
            -- we need ignore some unused flags in this stub testing
            --
            -- e.g. add_values("verilator.flags", "-GWIDTH=4", "--trace")
            -- error: %Error: Parameters from the command line were not found in the design: WIDTH
            if not flag:startswith("-G") then
                table.insert(argv, flag)
            end
        end
    end

修改后示例:

    if flags then
        for _, flag in ipairs(flags) do
            -- we need ignore some unused flags in this stub testing
            --
            -- e.g. add_values("verilator.flags", "-GWIDTH=4", "--trace")
            -- error: %Error: Parameters from the command line were not found in the design: WIDTH
            if  flag:startswith("--sc")  or  flag:startswith("--coverage") then
                table.insert(argv, flag)
            end
        end
    end

这样子保证了 test.v 每次编译都是成功的,也解决了复杂的过滤问题,你看如何?

还有目前 add_values 里面的字符串不支持空格,如果有空格的话

--top Vtop” 会传递过去为 "--top Vtop" 而预期接受到的应该是 --top Vtop

这就导致了 verilator 参数错误,这是一个 bug 吗?还是带空格就必须将字符串分开吗?

@waruqi
Copy link
Member

waruqi commented Jul 10, 2023

修改后示例:
这样子保证了 test.v 每次编译都是成功的,也解决了复杂的过滤问题,你看如何?

这样改可以。

这就导致了 verilator 参数错误,这是一个 bug 吗?还是带空格就必须将字符串分开吗?

原本就是按空格分开的

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Modified example:
This ensures that every compilation of test.v is successful, and also solves the complicated filtering problem. What do you think?

This can be changed.

This causes the verilator parameter to be wrong, is this a bug? Or do I have to separate the strings with spaces?

Originally separated by spaces

table.insert(argv, flag)
for idx, flag in ipairs(flags) do
for _, prefix in ipairs(switches_flags) do
if flag:startswith(prefix) then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要双层 for-loop,用 map

import("core.base.hashset")

local switches_flags = hashset.of( "sc", "coverage", "timing", "trace", "trace-fst", "threads")

if flag:startswith("--") and switches_flags:has(flag:sub(3)) then
   -- ...
end

1. Add extension .sv to enable systemverilog support
2. Remove unnecessary flags in test
3. Add libz dependance when enable TRACE_FST
@waruqi waruqi merged commit 450568b into xmake-io:dev Jul 10, 2023
@waruqi
Copy link
Member

waruqi commented Jul 10, 2023

thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants