Abstract

The speed of coverage convergence plays an important role in verification productivity. However, traditional approaches to achieve coverage convergence rely on manual efforts. This manual approach does not scale given the ever increasing amount of functionality that must be covered by today’s verification environment.

This paper discusses a new approach that enables rapid coverage convergence with Synopsys Coverage Convergence Technology (CCT). The design illustrated in this paper is a Video Output Controller that is widely used in multimedia SOCs in Verisilicon, and the verification environment is based on VMM methodology. With CCT, the verification productivity improves greatly by reducing both the number of testcases and regression time.

Key words: VMM, Coverage, Coverage Convergence Technology, VCS, Synopsys

1. Introduction

The progress of a verification project is measured by the number of functional features that are confirmed as functionally correct. Therefore, verification productivity is a measure of how many such features are confirmed as functionally correct over a given time, including the time necessary to build verification environment, create tests, debug and fix any functional errors, and analyze the functional coverage.

VMM (Verification Methodology manual) is a SystemVerilog based verification methodology proposed by Synopsys. It improves the verification productivity through four different mechanisms: abstraction, automation, reuse and assertions. VMM emphasizes “Coverage Driven Verification”, as illustrated in Figure 1. With less code, more testing and checking can be done to maximize the verification quality.

Figure 1: Coverage Driven Verification

Traditional approaches to achieve coverage convergence rely on manual efforts. An engineer would have to perform coverage analysis and recalibrate the constraints solver to generate different stimulus for higher coverage. This manual work costs much time in verification cycle. A new Coverage Convergence Technology (CCT) is developed by Synopsys, which automates the loop between constrained-driven stimulus generation and coverage-hole analysis. The automation allows the VCS constraints-solver to adapt to the coverage achieved in a given regression. This adaptation leads to higher coverage in a given time. Deployment of CCT in VCS enables rapid coverage convergence, eliminates coverage redundancy across testbenches, and increases the efficiency of the regression farm.

2. VMM based Verification Environment

The design under test of this verification project is Video Output Controller, which is widely used in Multimedia SOCs of Verisilicon. Figure 2 is the previous Verilog based verification environment for this project.

Figure 2: Verilog Based Verification Environment

Where Vo_top is the Video Output Controller (DUT), Ahb_ master, Sdram_mdl , vo_interface, Lcd_model are verilog behavior models that drive or monitor the DUT interface signals. Ahb_master works as the ARM , is used to configure the registers; Sdram_mdl works as the memory, provides video input data to DUT; lcd_model works as a LCD Monitor, checks the video timing from DUT; vo_interface receives the video output data from DUT, writes into text files, and compared with golden text files manually.

There are many limitations of this Verilog based verification environment: layers are not clear, difficult for maintenance and improvement; test specified codes are mixed with the verification environment, hard to understand, and difficult to create new testcases; all the testcases are direct-test, can only find expected bugs, difficult to find unexpected bugs; Needs collecting the coverage information manually, don’t know whether the testcases are enough to cover all the features.

A new VMM based verification environment has been adopted for this project, as illustrated in Figure 3. Testcases are implemented on the top of the verification environment. The verification environment implements the abstraction and automation functions that help minimize the number and details of tests that need to be written.

Figure 3: VMM Based Verification Environment

There are many benefits to build a verification environment based on VMM: layers are very clear, includes test layer, generator layer, function layer and command layer; transactors are connected by vmm_channel, testbench and DUT are connected by Interface, which makes the verification environment much easier for implementation and maintenance; tests are separated from the testbench, test-specified codes are only added in tests, won’t pollute the whole verification environment; it is a constraint-random, coverage-driven verification environment, verification goals are easily measured by functional coverage.

3. Challenge of Coverage-Driven Verification

In a coverage-driven verification process, the analysis of the collected coverage data is used to identify where to focus the next verification efforts. Thus, a feedback mechanism is required to translate uncovered areas in the coverage model into stimulus.

Theoretically, there are mechanisms in SystemVerilog to dynamically query the coverage rating of different coverage groups. The information can be used to alter, at run time, state variables in constraints on stimulus. A simulation could therefore automatically focus on the uncovered areas of the coverage space and reduce the redundancy stimulus that covered the same coverage point repeatedly.

But in practice, this automated feedback mechanism is not easy to implement. It must be coded and debugged to ensure that the stimulus converges toward greater coverage, more testcases should be created, and more simulations with different seeds should be run to achieve coverage convergence.

Thus, the challenge for coverage-driven verification is to find techniques that can close the coverage feedback loop automatically, with fewer additional requirements from the user.

4. Traditional Coverage Convergence Method

Before we deploy Synopsys Coverage Convergence Technology, we built a manual coverage feedback mechanism for this verification project: run tests with different seeds; check the coverage rating, if coverage does not converge, analyze the coverage holes; create new tests, in which modify the constraints or define new scenarios; then start another run loop with different seeds. The verification process is illustrated in Figure 4.

Figure 4: Traditional Coverage Convergence Verification Process

Where, steps of “setting random seed”, “run” and “check loops over” are done via run-script; while other steps must be done manually.

There are some disadvantages of this method: the number of run loops is difficult to decide, setting too large may cause waste of simulation runtime, while setting too small may cause little increase for coverage rate and waste coverage analysis time; coverage holes analysis must be done manually; more tests needed to be created for adding or modifying constraints, which leads to more manual efforts and time before coverage converges.

5. Synopsys Coverage Convergence Technology

A new Coverage Convergence Technology (CCT) is developed by Synopsys, which automates the loop between constrained-driven stimulus generation and coverage-hole analysis. The automation allows the VCS constraints-solver to adapt to the coverage achieved in a given regression. This adaptation leads to higher coverage in a given time.

In this verification project, a new verification process is deployed: compile with CCT enable, start the first run loop; generate the CCT bias file, in which it records the coverage holes that have not been covered yet; check the coverage rating, if coverage does not converge, start another run loop with the generated bias file. The new verification process with CCT is illustrated in Figure 5.

Figure 5: Verification Process with Synopsys Coverage Convergence Technology

Where, all the steps can be done in a single run-script, there are few manual efforts needed. CCT builds feedback mechanism that identifies and systematically targets functional coverage holes, eliminates redundancy in stimulus generation. CCT improves the speed of coverage convergence by several ways: in an individual simulation, CCT collects the coverage data of each randomization, analyzes the coverage holes automatically, and eliminates the redundancy in stimulus generation dynamically; while between simulations, CCT collects the coverage data in simulation, analyzes the coverage holes and records the analysis results into a bias file, with the bias file, CCT adjusts the generation of stimulus to only focus on the uncovered area across multiple simulations .

6. Improvement of Verification Productivity with CCT

In this verification project, we have tried both the Traditional Coverage Convergence Method and Synopsys Coverage Convergence Technology. The verification productivity improves greatly with Synopsys CCT. Test result is illustrated as following:

7. Conclusion

Synopsys Coverage Convergence Technology in VCS addresses coverage-driven verification challenge by automating the loop between constrained-driven stimulus generation and coverage-hole analysis. Synopsys Coverage Convergence Technology improves our verification productivity greatly by reducing the manual efforts and reducing verification time to reach the verification goal.

8. References

[1] Janick Bergeron, Eduard Cerny, Alan Hunter, Andrew Nightingale, “Verification Methodology Manual for SystemVerilog”. Springer 2005

[2] VCS® / VCS® MX Beta Feature Descriptions, Version C-2009.06 Beta, December 2008.