An Innovative methodology to accelerate the verification for complex design
Wei Jian
Digital Baseband Department, COMMIT Inc.
Jian.wei@commit.cn
Abstract
As the chip size increases, the verification task plays a more important role in the ASIC design process. The quality of the more and more complex verification is critical to the success of our current 3G TD-SCDMA terminal chip design. We need to cover from the module-level verification to the system chip-level verification. To improve the quality of the verification, we set up a reusable testbench generation mechanism and test flow using Synopsys VERA and VCS. Vera language is more abstract to help the verification engineer focuses on increasing functional verification coverage and finding bugs in their early design stage. .
1. Introduction
As we known, the SoC verification is becoming more and more difficult to current design today. We need to verify not only all the modules and chip integration, but also complex protocols and applications on the chip. To improve the quality of verification to meet our requirements, we must use advanced verification methodology in our project in a reasonable cost.
2. Verification challenges
For large size chip with multi-chip architecture, the verification task is very heavy. Cooperated with algorithm engineers, more than 10 engineers are involved in the verification from module-level to chip-level and SoC level, and more than 10,000 tests have been developed to verify the design. Theall verification tasks take several months.
The overview block diagram of this chip is shown as Figure 1:
It’s a TD-SCDMA/GSM Dual mode terminal chip for digital baseband processing. It contains two parts – one is Tempest coprocessor, the other is OMAP. This system provides all TD-SCDMA/GSM functions such as receiver and transmitter of data processing (L1 to L3).
◆ Tempest Coprocessor
The TEMPEST Coprocessor includes the function blocks of TD-SCDMA/GSM dual mode modem digital baseband processing, clock & reset management, interrupt handling, system timebase management, etc.
It is partitioned as follows:
● Block1: TD-SCDMA part
– Cell search, tracking, path search, joint detection, AGC, AFC, measurement, transmit, TD-SCDMA RF/ABB I/F, TD-SCDMA Radio I/F, etc.
● Block2: GSM part
– GSM RF/ABB I/F, SPI, etc.
● Block3: TD-SCDMA/GSM general part
– CLK & Reset Management, Power management, SIM I/F, Interrupt handler, System time base control, Data I/F, Control I/F, etc.
◆ OMAP platform
OMAP platform has a DSP core (C55xx) and a MCU core (ARM9). Working with TEMPEST coprocessor, the DSP core handles physical layer processing for TD-SCDMA and GSM as follows:
● TD-SCDMA/GSM L1 SW
● Receiver/Transmit real time control
● Voice coder/decoder
● GSM radio data receive/transmit
The MCU core controls the OMAP engine and handles layer2 and layer3 for TD-SCDMA and GSM as follows:
● TD-SCDMA/GSM L2/L3 protocol stack
● TD-SCDMA/GSM L1 interface
● Simple MMI and applications
● System control
As we can see, there are many complex communication protocols involved in the verification. We must adopt an innovative verification methodology to address these verification challenges for this SoC design.
3. Verification plan and methodology
3.1 what is the main task of the verification?
Our verification flow is shown in Figure 2.
Verification planning
The verification plan identifies the main verification phases, objectives and goals for the SoC verification, and defines the technical approaches to accomplish the verification objectives.
◆ Identify Test Cases
This stage covers from hardware function verification to system application verification, such as the module’s functionality, the interfaces between the modules, whole system’s dataflow…etc. So we must decompose the test development for all modules and the chip-level testbenches into appropriate test suites that covers all functions need to be tested.
● Function test suite (gray-box verification)
The DUT test list is a checklist of all functionalities based on the design specification. It guarantees that the documented features will be tested, but may miss some corner cases derived from the interaction of the features and the implementation choices. In our design, it is called gray –box verification. Many modules (CLK&Rst, Power management, Interrupt handler, etc.) have this pure function test suites.
● Interfaces test suite (black-box verification)
A DUT typically has multiple external interfaces such as EMIFS, etc. Testing on the interfaces needs to list all the transactions on all interfaces. This kind of test works well with the bus protocol testing. In our design, it is called black-box verification. The verification focuses on the interactions and bus interfaces to the related function blocks.
● Corner cases test suite (white-box verification)
The corner cases come from the boundary conditions of the DUT’s functionality and the way of its implementation. In our design, it is called white-box verification (In our design, one example is the frequency control and the receiver signal gain control for RF/ABB devices. Normally the control is done per sub-frame (5ms). However, we have the corner case that the adjustment is done per timeslot (675us). If we just test the long interval adjustment case, it definitely cannot cover the short time adjustment case, as the internal HW and SW mechanism are different.)
As corner cases test plan focus on the corner cases of the DUT implementation, it is hard to work out without the detailed knowledge of the implementation.
For the above three test suites, each has its own function coverage. So we combine those three test suites to a complete test plan. The classification of the three kinds of test suite will help us to maintain and re-use the testbenches easily.
◆ Stimulus generation policy
Once test cases are enumerated in the test plan, the stimuli need to be generated and applied to the DUT. From the DUT specification and the test plan, we get the test requirements, and then the stimulus generation scheme is defined based onthese requirements. When all the tests are executed to the completion, we will have the required confidence in the function correctness of the design.
Below are the considerations of the stimulus generation:
◆ Directed testing
Directed testing targets specifically to certain functionality or a set of functionality. Directed testing works well with modules that have simple and completely dedicated features. (For example, the interrupt generation and detection scheme are dedicated features) In this scheme, verification engineers usually know the input constraints, target output behaviors and they will translate them to test vectors on a test-by-test basis. However, the disadvantage of the directed testing is that, it is easy to miss interesting test cases and bugs.
◆ Random testing
◎Random testing
A random testing environment can help to uncover the hidden test cases and bugs. In a random environment, the testbench knows how to apply every type of transaction and randomizes the continuous sequences of those transactions. In this scheme, the verification engineer must program an automatic test generator and an automatic result checker. It is extremely important for the random testing environment to be able to hit all test cases. However, some features of the DUT (low-probability test case) are so difficult to hit that they may never be tested in a finite simulation time. We use random testing for the bus interaction testing in our design.
◎Constraints random testing
In this scheme, the random testbench is enhanced to include variable probability weights and constraints that steer the random choices. This scheme is built on random environments by stimulating low-probability test cases and bugs. It must hit all test cases in the test plan as well as all test cases generated by unconstrained random environments. It is the best way to improve the automation of the testbenches.
Because each test generation scheme has its advantages and disadvantages, it is usually the best way to use more than one scheme.
◆ Result checking
We have three methods to check the results to make sure whether the output of the DUT is what we expect:
● Visual checking.
● Automatic post-processing: dump the output of the DUT and run scripts to process them.
● Implement real-time monitors that checks results on the fly.
Visual checking works only for small modules manually.
Post-processing of results needs not to add so many codes in the testbenches, however, will impact the simulation performance for the excessive dumps(the file I/O slows down simulation).
Real-time monitors require additional development efforts. It is useful for debugging because it can flag an error as it happens. It is a kind of self-checking process and can be created using assertions or Vera language.
In our design, we use a combination of post-processing and real-time monitor to do result checking. Wemake a trade-off between the performance and the debug competence with the development cycle. Real-time monitor also help us to increase automation of the testbench.
◆ Build Testbench
We develop the testbenches from the module-level to the chip-level and SoC level (We have the TEMPEST chip, OMAP chip platform and RF/ABB chipsets), which synchronizes with the development of the RTL codes.
In module-level testing, each sub-module (TD-SCDMA demodulator, TD-SCDMA cell search time tracking, AGC, AFC…) testbench is set up, which focuses on the module’s functionality testing.
In chip-level testing, The modules are integrated, and the tesbench is developed to test the connectivity and interaction between the modules.
In SoC –level testing, the testbench contains the tests for the TEMPEST and the OMAP model. We focus on system data flow and the complex protocol testing.
With the help of OMAP model, we take the advantage of the multiple-layer testbench configuration to enhance the efficiency, reusability of the testbench. We will introduce our multi-layer testbbench the next session.
◆ Regression Phase
Once almost all test cases are run successfully, verification moves into the regression phase. In this phase, all tests will be reused to keep the consistence with each other when some modification may be necessary later.
3.2 How to improve the quality of verification?
The function coverage and the code coverage are introduced into our verification methodology as the important measurements to check and improve the quality of the verification.
The code coverage analysis provides an assessment of the quality of the test suites and identifies untested areas of a design. Code coverage analysis can be performed at individual block level or the full-chip level. The analysis provides a coverage number in percentage for each attribute being assessed and a list of untested or partially tested areas of the design. The code coverage assesses various types of coverage including: statement coverage, toggle coverage, FSM arc coverage, triggering coverage, branch coverage, expression coverage, path and signal coverage.
The functional coverage helps you to find out that whether all specific properties of the design are executed. It usually includes the testbench coverage to the functional specification and the test plan.
The code coverage and the function coverage provide different and complementary view of how well the DUT being tested. The very important factor is the analysis from the different metrics always helps us to find out the ‘holes’ in the verification plan. For example, if you have high code coverage but low test plan coverage, you may need to add in more constrained test cases.
4. Verification development
4.1 Testbench Architecture & Bottom-up Verification Approach
Tests are connected to DUTs through the testbench, which in turn is connected to the DUT through customer configurations. This testbench architecture is a hierarchical structure with clearly defined functionalities for each layers. The architecture is shown as Figure 3.
Based on the hierarchical testbench architecture, we adopt a bottom-up verification approach, which is shown as Figure 4.
◆ Layer0
Layer0 provides signal-level connectivity to the physical representation of the DUT. In layer0 testing, DUTs are tested for individual components, blocks or units. The intent is to test the component exhaustively without regard to the environment into which it will be integrated. In this case, layer0 testing might include code coverage analysis to ensure the quality of the design.
◆ Layer1
Layer1 configures the tests and provides a HW bus-abstraction view of the hardware. It includes BFM (Bus Function Model), bus protocol checker and some behavioral models for the DUT. In layer1 testing, the internal interconnect of the modules tested in layer0 and the system memory map are tested. This test checks all interconnections in the design by performing write & read-backs across all on chip buses. In addition, all registers in the design are tested by write & read-backs through the on-chip processor.
◆ Layer2, Layer3
In these two layers, the application models of both sub-model (layer2) and whole system (layer3) are configured.
In layer2 testing, the basic functionalities of the design and the external interconnections are verified. Tests are written to exercise the main functional paths between functional blocks and the I/O pins.
In layer3 testing, the whole design is verified at the system level. The goal is to test the functionalities of the integrated design exhaustively.
To construct the multiple-layer testbenches , we adopt VERA in our verification flow. Based on OpenVera language, a mature hardware verification language with rich resources focusing on design verification, we can easily establish each layer by classes to encapsulate related property and methods. Also a lot of verification IPs created using OpenVera can be found from lots of IP Vendors, This is helpful for us in the whole project.
4.2 Module-level verification development
The steps of module-level verification development are follows:
● First do code coverage analysis. The target is to obtain 100% code coverage at the most of the modules.
● Collect the module’s port information.
● Envelop all ports with different tasks except the system clock signals. Only these tasks need to visit the ports directly. In our design, these tasks are called port-related task.
● In the module verification, if we want to input the stimulus, transact bus protocol… etc., all these transactions can be carried out through the port-related tasks without caring the details of the related ports.
● Define our specific tasks which will execute the specific module’s function, these tasks include the specific functional tasks and the interface related tasks.
● Define more general-purpose tasks, which are used frequently such as file operation, error handling, … etc.
● Define tasks to check the simulation result.
● Construct a program, which invokes all the tasks necessary, and add some flag registers for the communication between the tasks.
A complete coverage report becomes a basic criterion of the completion of a milestone in our verification methodology. Besides the coverage, the performance is also very important to our project. VCS simulator, the perfect integration of the fast simulation engine and coverage matrix, is used in our design flow.
With the awareness of the testbench reuse, we develop some reusable module interface related tasks, general-purpose tasks and result checking tasks.Table 1 lists the module-level testbench tasks. The resuable tasks are also used in the top-level testbench.
4.3 Chip-level & SoC-level verification development
After the module-level testing, the top-level testbench is built based on the testbench architecture in Figure 3. Figure 5 shows the components of the top-level testbench. As you can see, some module-level testbench tasks will be reused in the top-level testbench.
Figure 6 shows the top-level verification flow.
To shrink the verification development time, module-level testbench reuse is very important in the top-level testbench:
● Task_EMIFS, task_vlynq, task_ulpd, …etc. are directly reused from module-level verification. We need to make some modifications to meet the top-level requirements.
● Task_MCU is for whole system processing control. You can regard it as a big FSM. This task models the OMAP’s system behavior. The Co-processor’s data processing, data control and pure data transactions and layer1 software processing are all embedded in this complex task. This task uses ITC to control parallel threads. In this system task, FLAG Registers are used for the communication between the tasks.
● Task_AutoCheck is for result self-checking of the system-level simulation. It includes both results post-processing and real-time monitors.
Table 2 shows the top-level testbench tasks we develop for our design. Some module-level tasks are directly reused in the top-level testbench.
5. Conclusion
In this paper, we present the verification methodology and flow, which are currently being used for our design. The hierarchical approaches to the verification development decompose the difficulties into each layer’s testbench development, and make it easier to manage and resue them in different levels. With the help of VCS and VERA, we set up a smart and fast verification platform, on which the coverage-aware testbenches can be developed to meet the coverage goals. Meanwhile, the embedded monitors help the verification engineer focus on increasing functional verification coverage and find bugs in the early design cycle, and the tests automation benefits the fast regression test for the complex protocols on the chip.
6. Acknowledgements
I am appreciating for Mr. Li Guohong’s effort in the verification methodology and resolving many specific relevant issues.
7. References
<
> by Faisal Haque, Khizar Khan and Jonathan Michelson
<> by Synopsys
<> by Synopsys
<> on Verification avenue (issue 3)
Insight China
-
SNUG
-
White Paper
-
Good Chip Series
