Verification plans – 3 Plans

Verification plan is process of translating the verification requirement specifications into verifiable description.  A verifiable description has to be aligned to technology being used for the verification. This alignment helps improve the chances of success.

Verification plan is one of the key deliverables of the functional verification. Starting verification without good verification plan is strict no. Test bench architecture is dependent on the verification plan. So cutting corners on creating verification plan will directly affect testbench architecture and that will impact the functional verification quality.

One of the current popular approach to functional verificaiton is coverage driven constrained random verification. Verification plan aligned to this approach is primarily driven by three plans: Test plan, coverage plan and checks plan.

Fundamentally functional verification is about two things. First one is stimulus generation. Second is doing the response checks. In the constrained random verification approach, there is uncertainty about stimulus generated. Hence there is third dimension of functional coverage  to confirm indeed all interesting scenarios have been generated.

These three dimensions are covered by three key plans. They are test plan, checks plan and functional coverage plan.

Let’s look into details about each of these three plans.  Feel free to jump to specific plan of interest for you.

Verification plan: Test plan

Test plan will capture the various scenarios to be verified. Test plan should be written as executable document. Many times test plans are written at a very high level. High level test plan will lack the details and hence affects accuracy of whole planning activity. In high level plan a single high level item may result in one test or hundred tests. This level of uncertainty is not good for the planning. Consider this form of test plan organization and follow this 8 step test plan writing process after understanding these test plan prerequisites.

Different areas of the specification requires specific approach targeted to it. Error injection for example should be approached with this thought process.

Verification plan: Checks plan

Checks plan will list all the response checks to be done. This is the second key element of the functional verification.  If the DUT has multiple physical interfaces for the interaction with external world, response checks have to be implemented on all the physical interfaces. Any deviation from the specification either the standard specification or design micro-architecture specifications will have to be flagged as errors.

Some of the checks could be simple signal level checks. While others could be  complex sequence of events or data integrity checks .

Irrespective of which test bench component the check is going to be implemented all the checks have to be listed in the checks plan.

Verification plan: Coverage plan

Functional coverage plan should capture the functional coverage requirements.

The individual parameters, transactions, sequential and concurrent scenarios information have to be captured. Functional coverage plan will also bias the seeding of the constrained random tests.

Along the requirements specifications, micro-architecture coverage and their intersection has to be considered. Not doing so leads missing some critical side effect scenarios which are often the cause of late discovered bugs.

Incomplete functional coverage plan can results in premature closure of verification with the bugs. The cost of these bug fixes increases as time passes and also leads to accumulation of functional coverage debt. Functional coverage debt can be paid periodically during the bug fixes verification .  Bottom line is, either do it right from start or pay it back with high interest rates later.

Sequential and concurrent scenarios, state machine transitions sequences any design pipeline coverage  have to be paid special attention. This is the key value proposition of the functional coverage plan. Multi instance coverage requirements have to be clearly identified.

Which aspects of the specification affect DUT’s behavior and which does not have to be very clearly understood. DUT may be implementing only part of the specifications. Rest may be handled in the software for instance. In this case what is handled by software, DUT may not be sensitive to to it. Blindly developing the coverage model for everything in specification can unnecessarily bloat the overall verification effort.

Functional coverage has to be carefully balanced without missing important coverage and ruthlessly cutting on irrelevant coverage with right balance of both functional coverage types.

Good verification plans is essentially art of balancing between, theoretically enumerating all cases while ensuring irrelevant cases being kept to minimum. Irrelevant cases are like weed.  We know while farming, along with the stuff we want the unwanted weed also grows. It cannot be avoided. Key is to keep cleaning it so that resources are available to good crops.

Practically only about 80 % of the verification plan can be completed during the initial planning phase. Remaining 20 % evolves during the course of project. This happens due to two reasons.

First reason being, it takes time to understand the new requirements specification thoroughly and execution cannot wait till every details is absorbed.

Second reason being, requirements specification undergo changes during the course of execution.

Both of these reasons mean, a verification plan should not written out as document that you write it once and keep it aside. It should part of the everyday verification execution and work alongside the testbench and test suite development. Make sure you have traceable verification plan.

Poor verificaiton plan affects the quality of the verification results and makes closing the last 20% longer.

Find out about 8 pitfalls to avoid while doing verification planning.