ASIC Verification

Introduction to Verification

Fall 2011
Meeta Yadav

©2011, Meeta Yadav 1

 Why do we need Functional Verification?

The age of digital

convergence

• Designs are becoming more complex

 Increased functionality increases
the number of transistors and the
possibility of error in the design
• The cost of undetected problems grow
over time

Cost
 There is little cost in detecting a
problem during device verification
but the cost increases substantially
if the bug is detected by the Verification Systems Customer
customer Test
Time
Source: Will, Goss, Roesner: Comprehensive Functional Verification: Elsevier

©2011, Meeta Yadav 2

 Which is more complex?

Saturn V moon rocket 45 nm ASIC

300,000 parts in 70,000 1 Billion transistors with ~10 Billion

assemblies Sub-compoments

©2011, Meeta Yadav 3

 Design Failures: Example

• Pentium floating point bug* (1994)

 Chip generated inaccurate computations to certain floating point
calculations
x = 4195835, y = 3145727, and z = x - (x/y)*y
gave the answer as 256 instead of 0
 Error occurred due to omission of entries and was hard to detect since
only 1 in 9 billion calculations were affected

• Explosion of Ariane 5** rocket (1996)

 Unmanned Ariane 5 rocket of the European Space Agency exploded 40
seconds after lift-off (Cost $7 billion + $500 million)
 Error in conversion of 64 bit horizontal velocity number to a 16 bit signed
integer
* http://www.maa.org/mathland/mathland_5_12.html
** http://www.cnn.com/WORLD/9606/04/rocket.explode/

©2011, Meeta Yadav 4

Verification Trends
Principle contributors:
- Functional bugs
- Clocking related bugs
IC/ASIC Designs Requiring Re-Spins by Type of Flaw

Logic/Functional
Clocking
Tuning Analog Circuit
Fast Path
Yield/Reliability
Delays/Glitches
Slow Path
Mixed-Signal Interface
Power Consumption
IR Drops
Firmware 2002 Market Study
Other 2004 Market Study

0% 20% 40% 60% 80% 100%

Percent of Designs Requiring Two or More Silicon Spins
[Collet 2005]

50% of ASICS require more than one respin

5
©2011, Meeta Yadav 5
The Verification Challenge
• Two major verification challenges are:
 The challenge of state space explosion There are more theoretical
► Formal Verification states in today’s design than
there are atoms in the
universe!!!

 The challenge of detecting incorrect behavior

► Functional Verification
– Constrained Random Tests
– Coverage
– Assertions

6
©2011, Meeta Yadav 6
The Verification challenge

Verification Design
Gap Gap

Ability to Fabricate
Design Size in Millions of Gates

Ability to Design

Ability to Verify (Directed)

 The Design & Verification Gap

— Many companies still using 1990’s techniques
— Traditional verification techniques can’t keep up with today’s designs

* Based on data from the Collett International 2004 FV Survey

7
©2011, Meeta Yadav 7
 Traffic Controller
Design description
 Light should stay green for
1 minute in each direction
when the intersection is
busy

Main Street

Elm Street

©2011, Meeta Yadav 8

 Traffic Controller
Algorithm Implemented

Wait 60
seconds

no yes
Main St. traffic?

no yes Main St.

Elm St. traffic?
turns green

Is there a problem
Elm St.
with the design? turns green

Algorithm for a Traffic Controller by Eagelton Signal Controllers

And Parking Engineering Solutions (ESCAPES)
Source: Will, Goss, Roesner: Comprehensive Functional Verification: Elsevier

©2011, Meeta Yadav 9

 Functional Verification
• Goals of Functional verification are to Customer
requirements
ensure
 The design accurately performs the tasks General
intended by the overall system architecture specification
and architecture
► To detect a missing feature
► To detect a missing corner condition
High level
 There are no bugs in the tasks chip design
implemented in the design
HDL
Functional
implementation
verification
at RTL level
Fixes to
HDL
Functional Verification testbenches
should be built from the general Physical circuit
design via Fabricated Chip
specifications and architecture synthesis

Source: Will, Goss, Roesner: Comprehensive Functional Verification: Elsevier

©2011, Meeta Yadav 10

 Functional Verification
Customer
Customer
A well-verified chip reduces cost by requirements

avoiding:
 Re-fabrication General
specification
 Re-calls and architecture

High level
chip design
Manufacturing
Fixes to
HDL HDL Functional
implementation
verification
at RTL level

Physical circuit
Timing Analysis design via Fabricated Chip
synthesis

System Testing
Source: Will, Goss, Roesner:
Comprehensive Functional
Verification: Elsevier Design Process
©2011, Meeta Yadav 11
Cost of Bugs

Cost

Verification Systems Customer

Test

Time

The cost of undetected problems grow over time

Source: Will, Goss, Roesner: Comprehensive Functional Verification: Elsevier

12
©2011, Meeta Yadav 12
Verification Productivity

Productivity improvements
drive early problem discovery
Number of Bugs

Verification Systems Test

Time

Verification productivity reduces cost and schedule

Source: Will, Goss, Roesner: Comprehensive Functional Verification: Elsevier

13
©2011, Meeta Yadav 13
 Overall Testbench Functionality
• Generate stimulus
• Apply stimulus to the Design Under Test (DUT)
• Capture the response
• Check for correctness
• Measure the progress against the overall verification goals

Progress Check and Control of

Verification Process
Stimulus Generation

Correctness Check
Golden Model
Application

Response
Stimulus

Capture
Design
Under
Test

©2011, Meeta Yadav 14

 Directed Testing Approach
• Directed Test Progress
 Writing stimulus vectors to exercise various individual features of the design
 Is a time and resource consuming exercise
 Good for small test space with limited variations

100%
Coverage

Directed
Test

Time
Directed Test Progress

Time and resource consuming

©2011, Meeta Yadav 15

 Directed Testing Approach
• Directed Testing Coverage
 Individual test cases cover individual features in the design space
 Over time the entire design space can be covered
 Increase in the complexity of the design causes an increase in the time to
cover the entire design space

Uncovered
Covered Features
Features

Bug

Directed Test Coverage

©2011, Meeta Yadav 16

 Randomization

• Why Randomize?
 Create VALID “corner” states : system states of maximum device
discomfort
• What do you randomize?
 Device Configuration: Move device to states of least probability
 Input data
 Error handling:
► Exception handling by protocols (bus protocols, handshaking, memory
interface)
► Recovery from errors in system states
 Relative timing between blocks
► Blocks operating at different rates

©2011, Meeta Yadav 17

 Stimulus: Constrained Random Testing

• Constrained Random Test Progress

• Constrain stimulus to VALID limits and allow tool to generate values at random within limits
• Random stimulus is required to exercise a complex design
• Useful in finding unanticipated bugs
Less time to achieve
100% coverage
100%
Coverage

Random
Test

Directed
Test

Time
More time to write
Constrained random tests
Constrained Random Test Progress

Achieves coverage faster

©2011, Meeta Yadav 18

 Stimulus: Constrained Random Testing
• Constrained Random Test Coverage
 Random tests cover wider design space than directed tests
 Tests may overlap
 Directed tests are still needed to test uncovered features

New Area

?
Test Overlap

Directed
testcase
? ?
Constrained-Random Test Coverage

©2011, Meeta Yadav 19

 Coverage Convergence
How do we achieve coverage convergence?
 Start with a fully randomizable testbench
► Run many randomized simulations
► Analyze cumulative coverage and coverage holes
 Then with minimal code changes
► Add constrained stimulus to fill coverage holes
 Finally:
► Make few directed tests to hit the remaining holes

Constrained Random Many runs

Tests different seeds

Add Directed Functional

constraints Tests Coverage

Minimal Code Identify

Modifications holes

©2011, Meeta Yadav 20

 Coverage Feedback

• For a large verification space consider coverage feedback

Coverage (% of possible bugs

Constrained Random without
Coverage Feedback

Constrained Random with

checked)

Coverage Feedback

Directed tests

Latency for coding

constrained random tests

Time (Code Writing and output

checking)

©2011, Meeta Yadav 21

 SystemVerilog Assertions in Verification Strategy

• Assertions:
1. Captures Designer Intent
2. Allows protocols to be defined and verified
3. Reduces time to market
4. Greatly simplifies the verification of reusable IP
5. Facilitates functional coverage metrics

©2011, Meeta Yadav 22

 SystemVerilog Assertions in Verification Strategy

Block Design System Integration

Bugs

Ship much
earlier with less Ship much later with margin of error
risk and cost
Ship

Upfront Cost Time to Market

Time

Assertions reduce time to market

©2011, Meeta Yadav 23

 Writing an Effective Testbench

• Testbench:
 Emulates the environment of the DUT
• Features of an effective testbench
 Re-usable and easy to modify for different DUTs
► Object-oriented!!
 Layered: Orthogonalize concerns
► Transactors (encapsulation of protocol) and Interfaces!!!!!
 Catches bugs or achieves coverage quickly
► Randomizable!!

©2011, Meeta Yadav 24

 Benefits of testbench environment

• Environment updating takes less time

• Testbench is easy to constrain from the top level file
• All Legal Device Configurations are tested
 Regression can select different DUT configurations
 Configuration object is randomized and constrained
• Enables reuse

©2011, Meeta Yadav 25

 Levels of Verification

System

System
level
Board Backplane Node
Board
level
Peripheral Processor Local Bus Memory PCI South Peripheral
memory arbiter controller bridge
Chip
level
DMA Cache ALU FPU X Y
Unit
level
Designer Memory Memory Bus Snoop
Access unit Unit
level
Hierarchical Diagram of Multiple Node System

Figure Courtesy: Will, Goss, Roesner: Comprehensive Functional Verification: Elsevier

©2011, Meeta Yadav 26

 Levels of Verification and Bugs Detected

Bugs

Time

Designer Unit Chip System

Figure Courtesy: Will, Goss, Roesner: Comprehensive Functional Verification: Elsevier

Lower levels of verification tend to uncover more bugs since they occur earlier
in the design cycle and because verification of each designer or unit level occurs in
parallel with the others. It is a good practice to wait until the bug rate begins to drop
in the low levels before moving to the next level

©2011, Meeta Yadav 27

 Thank You

©2011, Meeta Yadav 28