Verification plan debate – Part IV

Continued from: Verification debate Part III

Ananta could not sleep although he was really tired. His body was tired but his mind was still racing at full throttle. He started reflecting.

He was pondering on how sheriff was able to get his hands on escaped convict so quickly? How it prevented breach of the confidential informers data which could have lead to further crime.

Action of patrol teams in motion, teams that were stationed at sensitive points, investigation and emergency response handled this case had made deep impression on Ananta’s mind. Their contribution was evident and appreciated.

However the informers who played equally important role in the whole process almost went unnoticed. This was one of the missing pieces of puzzles he felt. He continued to reflect on this fact.

What would be informers in the context of functional verification? He had a sudden flash of insight. He finally figured out it was coverage plan. Coverage is like your informers network. You need omnipresence of coverage by combining both the code and functional coverage.

With coverage we gain clarity as to where we need target our stimulus and checks. Without it we are blind.

He thought is it just me who had ignored the coverage plan or rest of world is doing the same?

He thought of doing a quick check. It was already late in night.

He messaged his friend Achyuta, Are you awake? Hey what’s up Ananta came back a quick reply.

Ananta replied can you send me number of page views on the blogs of verification planning that you had pointed me earlier at once if convenient. If inconvenient, send all the same.

In next 10 minutes following were the number sent out by Achyuta. All these blogs got online almost around same timelines. Being read by readers over a year now. Here are the page views statistics:

From August 2016 – September 2017 duration pageviews:

Page tile	Page Views
Test plan	799
Checks or assertion plan	354
Coverage plan	271

Test plan has 200% more views compared to checks or assertion plan. Test plan also has close to 300 % more views compare to coverage plan.

This data makes it clear thought Ananta. He smiled, I am not alone.

Coverage plan is given least importance among three. Remember coverage plan is doing the job of informer. At times you can raid and even capture criminals without informers but you would lack the accuracy and speed of response.

With all three verification plans getting right level of attention, we also should be able to bring down the bugs rate.

Ananta got excited and he had to share these realizations with Achyuta. It’s the debates with him that had aroused his curiosity in first place so he had share it with him. He called him up. He knew it would interest him as well.

Achyuta looks like finally I understand the riddle of verification plan now. He shared his findings connecting all the dots in single breath.

We are excessively focusing on stimulus. Our focus on checks or assertion is lacking. Our focus on the functional coverage is even lower. That explains bugs rate. We are working hard and we are making our machine farms work even harder by running regression with thousands of tests. We are not doing it effectively. What is happening in regression is staying in regression. We need more transparency into effectiveness of our regressions.

Many times, our best players are fighting hard on wrong battles. Our ad hoc approach is taking toll on them. Let’s accept it we only have handful of A+ players. We need to invest part of their bandwidth on strategic tasks of writing and maintaining the verification plan.

Verification plan is not something that is written once at beginning of project and finished for good. It evolves. This is a plan to cope with the changes not a line set in stone. We are innovating and by definition it means things will change and evolve. No one knows requirements completely from start. We have to deal with ambiguity. Our only chance is we have verification plan that can evolve and adapt to these changes. We need to provide the equal importance to all three plans.

If we do get these three plans right then these are three strings that can be pulled to control the entire execution of the functional verification. It can greatly streamline execution and enable us to adapt to changing priorities without dropping any balls on the floor. It will also bring in a good level predictability and ease to make informed decisions to on what features to cut down to meet schedules.

Bravo Bravo shouted Achyuta being unable to control his excitation.

This woke up his wife who looked straight at him with her red eyes in half asleep.

Achyuta’s heart skipped a beat. There was moment of silence. Ananta was quick to sense it.

He said, I am coming back this weekend anyway so let’s catch up at our usual coffee place we have lot more to talk. Good night, I mean Good morning.

See you soon…