World of dynamic simulation based functional verification is not real. Verification architects abstract reality to create the virtual worlds for design under test (DUT) in simulation. Sometimes it reminds me of the architectures created in the movie inception. It’s easy to lose track of what is real and what is not. This leads to critical items being missed in the functional coverage plan. In this article we will look at three point of views to look at feature while doing functional coverage planning to increase chances of discovering those critical scenarios.

Huh..! In the world of functional verification how do I know, what is real and what is not?

In simple words if your design under test (DUT) is sensitive to it then its real and if it’s not sensitive to it then it’s not real. Requirements specification generally talks about the complete application. Your DUT may be playing one or few of the roles in it. So it’s important to understand what aspects the application world really matter to your design.

The virtual nature of the verification makes it difficult to convey the ideas. In such cases some real life analogies come in handy. These should not be stretched too far but should be used to make a focused point clear.

In this article, we want to talk about how to look at the requirements specifications and micro-architecture details while writing coverage plan. We want to emphasize the point micro-architecture details are equally important for functional coverage plan. Often verification teams ignore these. It affects the verification quality. Let’s look at the analogy to specifically understand why micro-architecture details matter to verification engineer? How much should he know? Let’s understand this with analogy.

Analogy

Let’s say patient approaches orthopedic surgeon to get treatment for severe back pain. Orthopedic surgeon prescribes some strong painkillers, is it end of his duty? Most painkiller tablets take toll on stomach and liver health. Some can even trigger gastric ulcers. What should orthopedic surgeon do? Should he just remain silent? Should he refer him to gastroenterologist? Should he include additional medications to take care of these side effects? In order to include additional medications to take care of side effects how far should orthopedic surgeon get into field of gastroenterology? Reflect on these based on real life experiences. If you are still young then go talk to some seniors.

The same dilemma and questions arise when doing functional coverage planning. Functional coverage planning cannot ignore the micro-architecture. Why?

When the requirements specifications are translated to micro-architecture, it introduces its own side effects. Some of these can be undesirable. If we choose to ignore it, we risk those undesirable side effects showing up as bugs.

Dilemma

Well we can push it to designers saying that they are the ones responsible for it. While it’s true to some extent but only when designer’s care for verification and verification cares for design first pass silicon dreams can come true. We are not saying verification engineers should be familiar with all intricacies of the design but the basic control and data flow understanding cannot be escaped.

Functional coverage planning: 3 points of view

All the major features have to be thought out from three points of views while defining functional coverage. There can be certain level of redundancy across these but this redundancy is essential for the quality.

Those three points of view are:

1. Requirements specification point of view
2. Micro-architecture point of view
3. Intersection between requirements and micro-architecture point of view

For each of these how deep to go is dependent on the sensitivity of the design to the stimulus being thought out.

Let’s take an example to make this clearer.

Application of 3 points of view

Communication protocols often rely on timers to figure out the lost messages or unresponsive peers.

Let’s build simple communication protocol. Let’ say it has 3 different types of request messages (M1, M2, M3) and single acknowledgement (ACK) signaling successful reception. To keep it simple unless acknowledged the next request is not sent out. A timer (ack_timer) is defined to takes care of lost acknowledgements. Range of timer duration is programmable from 1ms to 3ms.

In micro-architecture this is implemented with simple counter that is started when any of requests (M1, M2, M3) is sent to peer and stopped when the acknowledgement (ACK) is received from peer. If the acknowledgement is not received within the pre-defined delay a timeout is signaled for action.

So now how do we cover this feature? Let’s think through from all three points of view and see what benefits do each of these bring out.

Requirements specification point of view:

This is the simplest and most straightforward among the three. Coverage items would be:

• Cover successful ACK reception for all three message types
• Cover the timeout for all three message types
• Cover the timeout value for Min(1 ms), Max(3 ms) and middle value( 2ms)

Don’t scroll down or look down.

Can you think of any more cases? Make a mental note; if they are not listed in below sections, post them as comments for discussion. Please do.

Micro-architecture point of view:

Timer start condition does not care about which message type started it. All the message types are same from the timer point of view. From timer logic implementation point of view, timeout due to any one message type is sufficient.

Does it mean we don’t need to cover timeout due to different message types?

It’s still relevant to cover these from requirements specification point of view. Remember through verification we are proving that timeout functionality will work as defined by specification. Which means, we need to prove that it will work for all three-message types.

Micro-architecture however has its own challenges.

• Timeout needs to covered multiple times to ensure the mechanism of timer reloading takes place correctly again after timeout
• System reset in the middle of timer running followed by timeout during operation needs to be covered. It ensures that system reset in middle of operation does reset the timer logic cleanly without any side effects (difference between power on reset vs. reset in middle)
• If the timer can be run on different clock sources it needs to be covered to ensure it can generate right delays and function correctly with different clock sources

Requirement specification may not care about these but these are important from micro-architecture or implementation point of view.

Now let’s look at intersection.

Intersection between requirements and micro-architecture point of view

This is the most interesting area. Bugs love this area. They love it because of intersection, shadow of one falls on another creating dark spot. Dark spots are ideal places for the bugs. Don’t believe me? Let’s illuminate and see if we find one.

Generally synchronous designs have weakness around +/- 1 clock cycle around key events. Designers often have to juggle lot of conditions so they often use some delayed variants of some key signals to meet some goals.

Micro-architecture timeout event intersecting with external message reception event is interesting area. But requirements specification cares for acknowledgement arriving within timeout duration or after timeout duration.

What happens when acknowledgement arrives in following timing alignments?

• Acknowledgement arrives just 1 clock before timeout
• Acknowledgement arrives at the same clock as timeout
• Acknowledgement arrives just 1 clock after timeout

This is area of intersection between the micro-architecture and requirement specification leads to interesting scenarios. Some might call these as corner cases but if we can figure out through systematic thought process, shouldn’t it be done to the best possible extent?

Should these cases of timing alignment be covered for all message types? Its not required because we know that timer logic is not sensitive to message types.

Another interesting question is why not merge the point views #1 and #2? Answer is thought process has to be focused on one aspect at a time, enumerate it fully and then go to other aspect enumerate it fully then go figure out the intersection between them. Any shortcuts taken here can lead to scenarios being missed.

Closure

This type of intent focused step-by-step process enhances the quality of functional coverage plans and eases the maintenance when features are updated.

Some of you might be thinking, it will take long time and we have schedule pressure and lack of resources. Let’s not forget quality is remembered long after the price is forgotten.

Coverage is one of the important metrics in measuring verification quality and helping with verification closure. Lack of functional coverage can lead to expensive bugs like pentium fdiv being discovered late in silicon.

There are two categories of the coverage metrics. First one is code coverage. The simulator tool generates it automatically. Second is functional coverage. Verification engineers develop this by coding.

Code coverage consists of line coverage, condition coverage, expression coverage, FSM coverage and toggle coverage of DUT. Code coverage is topic by itself. Focus of this blog is on the functional coverage.

Functional coverage is not built-in and needs to be manually coded. It’s additional effort for the verification teams. The effort has to justify by providing the additional value on the top of code coverage. So it needs to be carefully planned.

Functional coverage vs. Code coverage

Code coverage has certain limitations. Some of the limitations of code coverage are:

• It does not address concurrency
• Various sequence combinations coverage cannot be determined

Code coverage as the name says looks at design just as a piece of code. A network switch or processor or fundamentally every RTL design looks same to the code coverage.

Code coverage does not differentiate them from application use case perspective. This means we cannot completely rely on code coverage. Thus code coverage is necessary but not sufficient.

This is where the functional coverage comes in. Functional coverage looks at the design from its application use perspective. A network switch is not same as processor to functional coverage.

In a coverage driven constrained random verification approach the randomness of the stimulus and configuration brings in the uncertainty. Uncertainty of, whether some scenarios really got exercised in certain configuration or not. This is where the functional coverage comes in handy to figure it out with certainty.

Code coverage is based on the DUT code. Significant part of the functional coverage is based testbench code to check if it created the intended scenarios to ensure the correct functionality of the DUT.

Coverage plan writing process

Functional coverage may have certain level of overlap with code coverage. This may have to be done in certain areas for the completeness of the functional coverage. This does not mean, duplicates the entire code coverage in functional coverage.

Functional coverage is luxury. But you cannot have lot of it. A careful thought has to be given where it can provide the maximum impact. The process for coverage plan also should follow “think theoretically and execute practically” as used in test plan writing process.

Coverage plan creation takes into consideration 3 inputs:

• Requirements specifications: This is the same requirement specifications used for designing DUT and creating the verification plan and test bench architecture
• Design under test (DUT): Various aspects of DUT such as control & state registers, interface and key FSMs etc.
• Test plan: Test plan listing the various scenarios in the form of stimulus and checks

Please note DUT’s micro-architecture functional coverage is equally important as requirements specification coverage.

Using these inputs create the complete functional coverage space using the following 7 steps process described.

Step#1: List all independent random variables

List all the variables from the specifications that can take multiple values.

Ideally each variable randomized in test bench should be covered. Random variables of each class have to be covered independently.

Typically in the current verification environments, which are object oriented programming based, these variables are encapsulated in to classes. These classes fall in to configurations and transactions.

Configurations can be structure configuration, functional configurations or error configurations. Transactions are encapsulations of request and responses from DUT at various levels of abstractions. All the properties of these classes declared as random should be covered by the functional coverage.

Step#2: List all applicable combinations of random variables within class

Look at the usefulness of combinations of two or more variables related to same unit of information. Same unit of information transforms to variables within the same class.

This could be combination of variables within configurations and transactions may additionally have to be covered in all combinations of values with the other variables.

For example in a full duplex communication protocol it’s not sufficient to cover all transaction types but also cover it in combination with the direction. This is to ensure all legal transactions types have been both transmitted and received by design. Note that transaction type and directions are properties of the same transaction class.

Step#3: List all necessary combinations of random variables across class

Cover the combinations of two or more variables across different units of related information.

This could be combination of variables across the configuration and transaction classes or across various transactions and across various configurations.

For example, to cover all applicable error injection types for all transaction types, cover combination of error injection type in error configuration and transaction type in transaction.

Step#4: List all state variables

Covering certain scenarios or sequences require covering state variables.

State variables can be stored in the test bench or the DUT. As a part of this list all independent state variables that needs to be covered.

Note that to cover certain type of sequences and scenarios new state machines exclusively used for the coverage may have to be developed in the test bench.

For example in USB host subsystem total ports connected state count coverage for active ports coverage.

Step#5: List all necessary sequences of same state variable

Cover the sequences of the same state variables across multiple sampling events.

For example to cover back-to-back stimulus transaction sequences, store last two stimulus transactions as a state. The combinations of these two transaction types stored in state have to be covered.

DUT FSMs are covered by default in code coverage. Limitation of code coverage is, it cannot tell if specific sequences of state transitions have taken place. So some of the key DUT FSMs will have to be additionally covered by the functional coverage.

For example cover sequence of state transitions of a port state machine. USB port going through one of the many state transitions such as: disconnected -> connected -> reset -> enumerated -> traffic -> low power etc.

Step#6: List all necessary combinations of different state variable

Cover the combinations of the different state variables across multiple sampling events.

For example for a N-port USB hub verification environment, need to cover combinations of all port state. This coverage provides insight into, if the hub has experienced various variations of use case scenarios across its multiple ports. This combination can help find out if scenarios such as multiple port connections taking place at same time, one port under rest while data traffic flowing to the other ports, one port under reset while other in low power state etc.

Another example for link layer is combinations of traffic outstanding state and credit state. This combination can help find out if there was outstanding traffic when credits had run out or insufficient credits for outstanding traffic etc.

Step#7: List all necessary combinations of different random variables and state variables

Cover the combinations of random variables and state variables.

For example communication interface such as MIPI UniPro uses PHY that allows multiple power modes. Idea of power modes is to trade speed for power. It also supports multiple traffic classes for supporting quality of service. Covering combinations of the power mode state and randomized traffic class variable of data traffic transaction tells us if all traffic class were active in all power modes.

After the coverage space is created, check if some items already have sufficient coverage in the code coverage. For items that have sufficient coverage in code coverage mark it so and focus on the items that are not covered or partially covered by the code coverage.

Tool’s like curiosity can significantly cut down your functional coverage writing effort with its built-in functional coverage models. Not only its brings time to write it but various built-in coverage models can act as triggers for the functional coverage plan thought process.