Coverage Analysis Techniques For HDL Design Validation: Jing-Yang Jou and Chien-Nan Jimmy Liu
Coverage Analysis Techniques For HDL Design Validation: Jing-Yang Jou and Chien-Nan Jimmy Liu
Coverage Analysis Techniques For HDL Design Validation: Jing-Yang Jou and Chien-Nan Jimmy Liu
Jing-Yang Jou
and
Abstraction
Until now, the functional verification at RTL is still
mostly done by simulating the HDL designs with a
massive amount of test patterns. In a typical design
environment, the quality of the test mainly depends on the
designers understanding of the design and is not
measurable. Therefore, more objective methods, which
use some well-defined functional coverage metrics to
perform a quantitative analysis of simulation completeness,
are proposed and rapidly getting popular. For this purpose,
many functional coverage metrics are proposed to verify
the designs written in HDL. We will present a survey on
several popular functional coverage metrics first in this
paper. In order to monitor the coverage during simulation,
a dedicated tool is required besides the simulator. Several
commercial tools for Verilog and VHDL code coverage
are now available. We will then introduce three popular
approaches of coverage measurement in this paper.
1. Introduction
Due to the increasing complexity of modern circuit
design, verification has become the major bottleneck of
the entire design process [1]. Most verification efforts are
put on verifying the correctness of the initial
register-transfer level (RTL) descriptions written in
hardware description language (HDL). Until now, the
functional verification is still mostly done by simulation
with a massive amount of test patterns even though some
formal verification techniques claim to be able to verify
the equivalence of a design across several different design
levels. Before the high computing complexity of the
formal verification techniques is dramatically reduced, it
appears that the simulation method will continue to play
an important role in the verification process.
During simulation, an important question is often asked
by the designers and the verification engineers: Are we
done yet? In a typical design environment, the verification
task is completed when the engineers think they have done
a thorough simulation. The quality of the test mainly
depends on the designers understanding of the design and
is not measurable. Therefore, more objective methods,
which use some well-defined functional coverage metrics
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
A. Statement Coverage
C. Decision Coverage
B. Block Coverage
Block coverage, which is also called basic block
coverage or segment coverage, is similar to statement
coverage, except that the measured unit of code is a
sequence of non-branching codes which is executed
D. Path Coverage
Path coverage measures the coverage of all possible
paths through the HDL code. A path is defined as a unique
sequence of branches or decisions from the beginning of a
end
if ( b ) begin
end
Figure 2 : A Verilog code fragment.
Path coverage is considered to be more complete than
decision coverage because it can detect the errors related
to the sequence of decisions. However, it is considered to
be too cumbersome in practice. There are often a very
large number of possible paths through a design such that
making 100 percent path coverage becomes impractical. It
implies that trade-off between the speed and the accuracy
is often needed in practice regarding to those coverage
metrics.
E. Expression Coverage
Expression coverage, which is also called condition
coverage, reveals how the variables or sub-expressions in
conditional statements are evaluated during simulation. It
can find the errors in the conditional statements that
cannot be found by other coverage analysis with
insufficient tests. For example, consider the two examples
shown in Figure 3. If the test pattern is a = 1 , b = 1, the
results are identical for both conditions. However, if the
actual expression in the code is a && b but your
intention is a || b, you will not discover the error unless
you test it by a = 1 , b = 0 or a = 0 , b = 1.
if ( a && b ) begin
end
if ( a || b ) begin
end
assign e = ( a = = b ) & ( ~c ) | ( d! = 2 ) ;
Expression coverage will check for the cases "a == b",
"a != b", "c == 0", "c != 0", "d == 2" and "d != 2".
Sometimes, it will also check for the combinations of
those cases to make the analysis more complete.
F. Event Coverage
Most HDL simulators are event-driven. Therefore, it is
nature for designers to care about the possible events that
may occur in a design. Events are typically associated
with the change of a signal. For example, as shown in the
line 7 of Figure 1, there is an event always @ ( posedge
clk ) which waits for the clk signal changing from low to
high. Event coverage is the measurement of events that
have been fired for each of the variables to which it has
been sensitized. This coverage is not commonly used but
sometimes useful for some special designs.
B. SFSM Coverage
It is impractical to use the conventional FSM coverage
to verify the behavior of a FSM because the size of the
STG is often too large to be traversed completely. Some
techniques [16, 17] have been proposed to reduce the huge
C1 : reset
C2 : (! reset) load
count
=0
C4
C1
C2
C1
C3
count =
C4
count+1
C3
C4
C2
count
= in
C2
B. Toggle Coverage
Toggle coverage, which is one of the oldest
measurements of coverage in hardware design, measures
the bits of logic that have toggled during simulation. It can
be used as a crude method of functional coverage because
a bit may not be properly tested if it never toggles from 0
to 1 or from 1 to 0. Since the measured unit is bit, it is
well suited to gate-level designs. More often, it is used as
the foundation for power analysis tools [23].
C. Variable Coverage
The extension of toggle coverage to the RTL is called
as variable coverage. In other words, variable coverage
# time2
<new_value><variable1>
<new_value><variable2>
.....
.....
# time1
<new_value><variable1>
<new_value><variable2>
(b)
a=b+c;
d=a+2;
if ( a == 3 | b == 5 )
a=b+c;
d=a+2;
Top.id = InstanceID ;
Top.index = 2 ;
$count_stmt ;
Tmp1 = a == 3 ;
Tmp2 = b == 5 ;
Tmp3 = Tmp1 | Tmp2;
Top.id = InstanceID ;
Top.index = 4 ;
$sample_values(Tmp1,Tmp2,Tmp3) ;
if (Tmp3)
Instrumentation
enb
signals
change
if (reset)
M
if (enb)
out2
=0
if (enb)
out1=
data
out1=
in
out2
=1
out1=
0
previous
result used
if (reset)
if (enb)
re-evaluated
out2
=0
if (enb)
out1=
data
out1=
in
out2
=1
keep
modified
out1=
0
executed code
4. Conclusions
The coverage-driven functional verification is rapidly
getting popular. Many functional coverage metrics have
been proposed to verify the designs written in HDL. A
survey on several popular functional coverage metrics has
been presented in this paper. However, although so many
different functional coverage metrics have been proposed,
there is still not a single metric similar to the stuck-at fault
model in the manufacturing test being popularly accepted
as complete and reliable. A lot of efforts are still required
in developing better functional coverage tests.
In order to monitor the coverage during simulation, a
dedicated tool is required besides the simulator. Several
commercial tools for Verilog and VHDL code coverage
are now available. We have introduced three popular
approaches of coverage measurement in this paper.
However, due to the limited simulator supports in PLI 1.0,
the extra overhead incurred by the coverage measurement
is still high. Using the extended PLI 2.0 with more
supports seems to be a good solution, but it is not
available now in some Verilog simulators. Many
engineers and researchers are still trying to find a better
solution for more convenience and better performance on
coverage measurement.
Besides those standalone coverage measurement tools,
some simulator vendors, such as Cadence, Synopsys, and
Model Technology [13] , are trying to integrate the
coverage measurement feature into their simulation
environments. Because the coverage measurement is
basically to monitor the action of a simulator, it would be
more convenient to integrate those monitors into the
simulation environment. However, the existing simulators
have been optimized for speed consideration. Some
information may be lost in the optimization so that the
simulators cannot analyze the coverage. Therefore, it
would be a big challenge for those simulator vendors to
integrate the coverage measurement feature into their
simulators without speed degradation.
5. Acknowledgments
This work was supported in part by NOVAS Software
Incorporation and R.O.C. National Science Council under
Grant NSC89-2215-E-009-009. Their supports are greatly
appreciated.
References
[1]
[2]
[3]
[4]
[5]
[16]
[17]
[18]
[19]
[20]
[21]
(http://www.isdmag.com/Editorial/1998/CodeCoverage9806.html)
[6]
[7]
[22]
( http://www.covermeter.com )
[8]
[9]
[23]