Here you will find tutorials that will help you use the Functional Coverage tools offered in chiselverify.

Code Coverage is one of the most important tools used by verification engineers. There are two main types of coverage:

• Statement Coverage: How many lines of code have been tested? (quantitative approach).
• Functional Coverage: Which features have been tested? (qualitative approach).

Statement Coverage is generally handled by the simulator and work has been done, in the scope of this project, in order to add statement coverage to the Treadle FIRRTL execution engine. This work can be found here.

Functional Coverage works by defining a in-test version of your design's specification. This representation of what you are trying to achieve is called a Verification Plan.
There are two main ways to get functional coverage with ChiselVerify and each one has its own pros and cons.

• Explicit Verification Plan: The first way is similar to how it is done in SystemVerilog. Here it is up to the user to define his own Verification plan BEFORE testing. Using this, the user will only have access to coverage as defined in their verification plan.

• Implicit Verification Plan a.k.a Queryable Coverage: The second way to get functional coverage with ChiselVerify is to use QueryableCoverage. This means that the user simply says that he wants to sample a DUT and then the tool can be queried AFTER testing. Using this, the user has access to all of the ports in the DUT, however this comes at a slight performance cost when sampling, since all ports are constantly sampled.

An explicit Verification Plan is defined using a CoverageReporter that is linked to the Device Under Test (DUT).

val cr = new CoverageReporter(dut)

The CoverageReporter can then be used to register a set of Cover constructs, which are grouped together in CoverGroups. This is done using the CoverageReporter's register method. Each call to said method generates a new CoverGroup.

def register(points: Cover*): CoverGroup

Once the different CoverGroups have been registered, they must be sampled throughout the test suite by calling the sample() method a the right time. This should be done whenever a set of input pokes should cause covered output ports to change values.

Once the test suite is done and that we are ready to see our coverage report, we can generate it using the report method. This will return a CoverageReport which contains a set of binHits. These define how many times each bin in has been hit.

def binNHits(groupId: BigInt, pointName: String, binName: String): BigInt

These results can then be used to update constraints used in Constrained Random Verification. If you simply wish to see a readable coverage report all you need to do is call the printReport() method.

There are multiple ways to define a cover construct. All of these share a similar API and inherit from the Cover abstract class.

abstract class CoverConst(val pointName: String, val ports: Seq[Data])

Here is a list of the different Cover constructs that we offer. The different pages can be explored if the reader wants to know more about the different internal representations used for the cover points:

• CoverPoint
• CoverCondition
• CrossPoint
• TimedCross

A Verification Plan is declared by creating Cover constructs and registering them in the CoverageReporter. Creating a Cover construct can be done using the following function:

cover(name: String, ports: Data*)(bins: Bin*)

This will generate a CoverPoint or a CrossPoint, depending on the number of ports that are used. This is the simplest form of a Cover construct.

cover(name: String, ports: Data*)(delay: DelayType)(bins: Bin*)

Adding a delay will generate a TimedCross, which can be used to compare two different ports sampled at different cycles.

cover(name: String, ports: Data*)(conditions: Condition*)

Using a Condition instead of a Bin will generate a CoverCondition, which can be used to compare multiples ports using a user defined predicate.

On top of this, the bin and cross functions can be used to generate Bin, Condition and CrossBin.

//Generates a Bin or a Condition depending on if a Range is used
bin(name: String, range: Range, condition: Seq[BigInt] => Boolean, expectedHits: BigInt) 

//Generates a CrossBin
cross(name: String, ranges: Seq[Range], expectedHits: BigInt = 0)

Let's show a simple example showing how to write a verification plan using our functional coverage tool.
Given the following DUT:

class TimedToyDUT(size: Int) extends Module {
    val io = IO(new Bundle {
        val a = Input(UInt(size.W))
        val b = Input(UInt(size.W))
        val c = Input(UInt(size.W))
        val outA = Output(UInt(size.W))
        val count = Output(UInt(size.W))
        val outB = Output(UInt(size.W))
        val outC = Output(UInt(size.W))
    })
    val regA = Counter(5)

    regA.inc()

    io.outA := io.a
    io.count := regA.value
    io.outB := io.b
    io.outC := io.c
}

We can test it with:

val cr = new CoverageReporter(dut)
cr.register(
    //Declare CoverPoints with conditional bins
    cover("accu", dut.io.outA)(
        bin("lo10even", 0 until 10, (x: Seq[BigInt]) => x.head % 2 == 0),
        bin("First100odd", 0 until 100, (x: Seq[BigInt]) => x.head % 2 != 0)),
    //Declare CoverPoints without conditional bins
    cover("test", dut.io.outB)(
        bin("testLo10", 0 until 10)),
    //Declare CoverConditions
    cover("aAndB",dut.io.outA, dut.io.outB)(
        bin("aeqb", condition = (x: Seq[BigInt]) => x.head == x(1)),
        bin("asuptobAtLeast100", condition = (x: Seq[BigInt]) => x.head > x(1), expectedHits = 100)),
    //Declare cross points
    cover("accuAndTest", dut.io.outA, dut.io.outB)(
        cross("both1", Seq(1 to 9 by 2, 1 to 1))),
    //Declare timed cross points
    cover("timedAB", dut.io.outA, dut.io.count)(Exactly(3))(
        cross("ExactlyBoth3", Seq(3 to 3, 3 to 3))),
    cover("EventuallyTimedAB", dut.io.outB, dut.io.count)(Eventually(3))(
        cross("EventuallyBoth1", Seq(1 to 1, 1 to 1))),
    cover("AlwaysTimedAB", dut.io.outC, dut.io.outA)(Always(3))(
        cross("AlwaysBoth3", Seq(3 to 3, 3 to 3)))
)

//Dummy test just to get a report
def test(): Unit = {
    dut.io.a.poke(3.U)
    dut.io.b.poke(1.U)
    dut.io.c.poke(3.U)
    cr.step(3)
    dut.io.c.poke(0.U)
    cr.sample()
    cr.step()
    dut.io.c.expect(0.U)
}

test()

//Generate report in a usable form
val report = cr.report

//Simply print out the report
cr.printReport()

The coverage report we obtain is:

============ COVERAGE REPORT ============
============== GROUP ID: 1 ==============
COVER_POINT PORT NAME: accu
BIN lo10even COVERING Range 0 until 10 WITH CONDITION onlyEven HAS 0 HIT(S) = 0,00%
BIN First100odd COVERING Range 0 until 100 WITH CONDITION onlyOdd HAS 1 HIT(S) = 1,00%
=========================================
COVER_POINT PORT NAME: test
BIN testLo10 COVERING Range 0 until 10 HAS 1 HIT(S) = 10,00%
=========================================
COVER_CONDITION NAME: aAndB
CONDITION aeqb HAS 0 HITS
CONDITION asuptobAtLeast100 HAS 5 HITS EXPECTED 100 = 5.0%
=========================================
CROSS_POINT accuAndTest
BIN both1 COVERING: CROSS(Range 1 to 1, Range 1 to 1) HAS 0 HIT(S) = 0,00%
=========================================
CROSS_POINT timedAB WITH AN EXACT DELAY OF 3 CYCLES
BIN ExactlyBoth3 COVERING: CROSS(Range 3 to 3, Range 3 to 3) HAS 1 HIT(S) = 100,00%
=========================================
CROSS_POINT EventuallyTimedAB WITH AN EVENTUAL DELAY OF 3 CYCLES
BIN EventuallyBoth1 COVERING: CROSS(Range 1 to 1, Range 1 to 1) HAS 1 HIT(S) = 100,00%
=========================================
CROSS_POINT AlwaysTimedAB WITH AN ALWAYS DELAY OF 3 CYCLES
BIN AlwaysBoth3 COVERING: CROSS(Range 3 to 3, Range 3 to 3) HAS 1 HIT(S) = 100,00%
=========================================
=========================================

An implicit verification plan is much easier to define than an explicit one and is done simply in one line:

val coverage = new QueryableCoverage(dut)

After that, just as in the explicit version, the coverage tool must sample the DUT throughout the test suite. This is done simply by:

coverage.sample()

Note that every call to sample() inserts a new value in the coverage database for every port in the DUT. Some longer test times might occur if the DUT contains many ports and is sampled often.

The main difference between the two methods is that this tool must be queried after the fact in order to obtain specific coverage data. This is done using the following method:

coverage.get(port: Data, expected: Option[Int] = None, range: Option[Range] = None) : CoverageResult

Here the port argument is the port that we want to get coverage from and expected is the expected number of hits for our port. The latter will be used to compute the final coverage %. Finally, we can also query for coverage info over a specific range, this allows the user to not have to rely on default bins for coverage.

This method will return a CoverageResult which has the following signature:

case class CoverageResult(name: String, valueCycles: List[(BigInt, BigInt)], hits: BigInt, coverage: Double) {
    val report : String
    def print(): Unit = println(report)
}

Here, name is the name of the port, valueCycles is the list of (value, cycle) pairs that ended in a hit, hits is the number of distinct hits and coverage is the coverage % based on the expected number of hits.

Given the following example DUT:

class BasicToyDUT(size: Int) extends Module {
    val io = IO(new Bundle {
        val a = Input(UInt(size.W))
        val b = Input(UInt(size.W))
        val outA = Output(UInt(size.W))
        val outB = Output(UInt(size.W))
        val outAB = Output(UInt((size + 1).W))
    })

    io.outA := io.a
    io.outB := io.b
    io.outAB := io.a + io.b
}

We can gather coverage as follows:

val coverage = new QueryableCoverage(dut)

def testOne(): Unit = {
    for (fun <- 0 until 50) {
        dut.io.a.poke(toUInt(fun))
        dut.io.b.poke(toUInt(fun % 4))
        coverage.sample()
    }
}

testOne()

coverage.get(dut.io.outA, 50).print()
coverage.get(dut.io.outB).print()
coverage.get(dut.io.outAB).print()
coverage.get(dut.io.outA, range = 0 to 4).print()

//One can also print everything at once  
coverage.printAll()

This results in the following:

Port io_outA has 50 hits = 100.0% coverage.
Port io_outB has 4 hits.
Port io_outAB has 26 hits.
Port io_outA for Range 0 to 4 has 5 hits = 100.0% coverage.  

==================== COVERAGE REPORT ====================
Port io_outAB has 26 hits.
Port io_outB has 4 hits.
Port io_outA has 50 hits.
Port io_b has 4 hits.
Port io_a has 50 hits.
=========================================================

Here only the first one outputs a coverage %, since all of the other percentages are negligible (ex: 4 hits over 2^32 possible values is rounded down to 0%).

Next topic: CoverPoint or return home.