Saturday, 17 November 2007

Statement Coverage & Decision Coverage

ISEB Foundation Certification Syllabus Covers three TEST DESIGN TECHNIQUES,
The 3 categories are :
1) Specification based or Black box testing.
2) Structured-based or White-Box Techniques.
3) Experienced Based Techniques.

As per the request of my blog readers I would like to post the "Structured-based or White-Box Techniques" first then later on continue with other design techniques.

TEST DESIGN TECHNIQUES for Structured-based (or) White-Box Techniques are:
-> Statement Testing Coverage
-> Decision Testing Coverage

{Statement Coverage & Decision Coverage : These 2 topics are covered in ISEB foundation Syllabus in 4th chapter "TEST DESIGN TECHNIQUES".}

Structured-based or White-Box Techniques :

White Box Testing :
-> Testing based on Knowledge of internal structure and logic.
-> Logic errors and incorrect assumptions are inversely proportional to a path’s execution probability.
-> We often believe that a path is not likely to be executed, but reality is often counter intuitive.
-> Measure Coverage.

Structured-based or White-Box Techniques is based on an identified structure of the software or system, as seen in the following examples:
Component level: The structure is that of the code itdelf, ie., statements, decisions or branches.
Integration level : The structure may be a call three (a diagram in which modules call other modules).
System level : the structure may be a menu structure, bussiness process or webpage structure.

Structure-based or white-box testing can be applied at different levels of testing. Here we will be focussing on white-box testing at the code level, but it can be applied wherever we want to test the structure of something - for example ensuring that all modules in a particular system have been executed.

** Further we will discuss about two code-related structural techniques for code coverage, based on statement and dicision, are discussed.

** For decision testing, a control flow diagram may be used to visualize the alternatives for each decision.

As said earlier, I focus main on code-related structural techniques. These techniques identify paths through the code tha need to be excercised in oredr to acheive the required level of code average.

These are methods that can be deployed that can make the identification of white-box test cases easier - one method is control-flow graphing, control flow graphing uses nodes, edges and regions.. I will show them in detail with examples here..

Now comming to the actual topic :

TEST DESIGN TECHNIQUES for Structured-based (or) White-Box Techniques are:
-> Statement Testing Coverage
-> Decision Testing Coverage

1. Statement testing & Coverage :
A Statement is:
>> 'An entity in a programming language, which is typically the smallest indivisible unit of execution' (ISTQB Def).
A Statement coverage is:
>> 'The percentage of executable statements that has been exercised by a test suite' (ISTQB Def)

Statement coverage:
-> Does not ensure coverage of all functionality
->
The objective if the statement testing is to show that the executable statements within a program have been executed at least once. An executable statement can be described as a line of program sourse code that will carry out some type of action. For example:


If all statements in a program have been executed by a set of tests then 100% statement coverage has been acheived. However, if only half of the statement have been executed by a set of tests then 50% statement coverage has been acheived.

The aim is to acheive the maximum amount of statement coverage with the mimimum number of test cases.

>>Statement testing test cases to execute specific statements, normally to increase statement coverage.
>>100% statement coverage for a component is acheived by executing all of the execuatbel statements in that component.

If we require to carry out statemtnt testing, the amount of statement coverage required for the component should be stated in the test coverage requirements in the test plan.We should aim to acheive atleast the minimim coverage requirements with our test cases. If 100% statement coverage is not required, then we need to determine which ares of the component are more important to test by this method.

>>Consider the following lines of code:

>> 1 test would be required to execute all three executable statements.

If our component consists of three lines of code we will execute all with one test case, thus acheiving 100% statement coverage. There is only one way we can execute the code - starting at line number 1 and finishing at line number 3.

>>statement testing is more complicated when there is logic in the code
>>For example..

>>Here there is one executable statement i.e., "Display error message"
>> hence 1 test is required to execute all executable statements.

Program code becomes tough when logic is introduced. It is likely what a component will have to carry out different actions depending upon circumstances at the time of execution. In the code examp,e shown, the component will do different things depending on whether the age input is less than 17 or if it is 17 and above. With the statement testing we have to determine the routes through the code we need to take in order to execute the statements and the input required to get us there!

In this example, the statement will be executed if the age is less than 17, so we would create a test case accordingly.

>> For more complex logic we could use control flow graphing
>> Control flow graphs consists of nodes, edges and regions

Control flow graphs describes the logic structure if the software programs - it is a method by which flows through the program logic are charted, usign the code itseld rather than the program specification. Each flow graph nodes and egdes The nodes represent computational statements or expressions, and the edges represent transfer of control between the nodes. Together the nodes and edges encompass an area known as a region.

In the diagram, the structure represents an 'If Then Else Endif' costurct. NOdes are shown for the 'If' and the 'Endif'. Edges are shown for the 'Then' ( the true path) and the 'Else ( the false path). The region is the area enclosed by the nodes and the edges.

>>All programs consists of these basic structures..

This is hetzel notation that only shows logic flow.

There are 4 basic structures that are used withn control-flow graphong.

The 'DoWhile' structure will execute a section of code whilst a feild or indicator is set to a certain value. For example,


The 'Do until' structure will execute a section of code until a field or indicator is set to a certain value. Foe example,

The evaluation of the condition occurs after the code is executed.

The 'Go To' structure will divert the program execution to the program section in question. For example

>> SO the logic flow code could now be shown as follows:

If we applied control-flow graphing to our sample code, then 'if Then Else' structure is applicable.
However, while it shows us the structure of the code, it doesn`t show us where the executabel statements are, and so it doesn`t help us at the moment with determining the tests we required for statement coverage.

>> we can introduce extra nodes to indicate where the executable statements are

>> And we can see the path we need to travel to execute the statemen in the code.

What we can do is introduce extra nodes to indicate where the statements occur in the program code.
NOw in our example we can see that we need to answer 'yes' to the question being posed to traverse the code and execute the statement on line 2.

>> Now consider this code and control flow graph:

>> We will need 2 tests to acheive 100% statement coverage.

Program logic can be a lot more complicated than the examples I have given so far!
In the source code shown here, We have executable statements associated with each outcome of the question being asked. We have to dosplay an error message if the age is less than 17( answering 'yes' to the question), and we have display 'costomer OK' if we answer 'No'.
We can only traverse the code only once with a given test; therefore we require two tests to acheive 100% statement coverage.

>> And this example...

>> We will need 3 tests to acheive 100% statement coverage.

NOw it get even more complecated!
In this example, we have a supplementary question, or what is know as a 'nested if'. If we answer 'yes' to 'If fuel tank empty?' we then have a further question asked, and each outcome of this question has an associated statement.

Therefore we will need two tests that answer 'yes' to 'if fuel tank empty'
* Fuel tank empty AND petrol engine ( to execute line 3)
* Fuel tanl empty AND NOT petrol engine( to execute line 5)
one further test will be required where we anser 'no' to 'if fuel tank empty' to enable us to execute the statement at line 8.

>>And this will be the last example for statement coverage.. we will then go for decision coverage.

>> We will need 2 tests to acheive 100% statement coverage.

In this example,,, we ahve two saperate questions that are being asked.

The tests have shown are
* A coffee drinker who wants cream
* A non coffee drinker who doesn`t want cream

Our 2 tests acheive 100% statement coverage, but equally we could have had 2 tests with:
* A coffee drinker who doesn`t want cream
* A no-coffee drinker who wants cream

If we were being asked to acheive 100% statement coverage, and if all statements were of equal importance, it would n`t matter which set if tests we chooose.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Checking your calculation values:

Minimum tests required to acheive 100%

Decision coverage >= Statement coverage

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Decision testing & Coverage
A Decision is :
>> ' A program point at which the control flow has two or more alternative routes. A node with two or more links to saperate branches.'(ISTQB Def)
A Decision Coverageis :
>> ' The percentage if the decision outcomes that have been exercised by a test suite. 100% decision coverage implies both 100% branch coverage and 100% statement coverage.' (ISTQB Def)

Decision Coverage :
>>

The objective of decision coverage testing is to show all the decisions within a component have been executed at least once.
A decision can be described as a line of source code that asks a question.
For example:

If all decisions within a component have exercised by a given set of tests then 100% decision coverage has been achieved. However if only half of the decisions have been taken with a given set of tests then you have only achieved 50% decision coverage.
Again, as with statement testing, the aim is to achieve the maximum amount of coverage with the minimum number of tests.

>> Decision testing derives test cases to execute specific decision outcomes, normally to increase decision coverage.
>> Decision testing is a form of control flow testing as it generates a specific flow of control through the decision points.

If we are required to carry out decision testing, the amount of decision coverage required for a component should be stated in the test requirements in the test plan.
We should aim to achieve atleast the minimum coverage requirements with our test cases. If 100% decision coverage is not required, then we need to determine which areas of the component are more important to test by this method.

>> Decision coverage is stronger than statement coverage.
>> 100% decision coverage for a component is achieved by exercising all decision outcomes int he component.
>> 100% decision coverage guarantees 100% statement coverage, but not vice versa.

Decision testing can be considered as the next logical progression from statement testing in that we are not much concerned with testing every statement but the true and false outcomes from every decision.
As we saw in our earlier examples of the statement testing, not every decision outcome has a statement( or statements) to execute.
If we achieve 100% decision coverage, we would have executed every outcome of every decision, regardless of whether there were associated statements or not..

>>Lets take earlier example we had for statement testing:

>>This would require 2 tests to achieve 100% decision coverage, but only 1 test to achieve 100% statement coverage.

In this example there is one decision, and therefore 2 outcomes.
To achieve 100% decision coverage we could have two tests:
* Age less than 17(answer 'yes')
* Age equal to or greater than 17 (answer 'no')
This is a greater number of tests than would be required for statement testing as statements are only associated with one decision outcome(line 2).

>> Again, consider this earlier example :

>> we will need 2 tests to achieve 100% decision coverage & also 2 tests to achieve 100% statement coverage.

This example would still result in two tests, as there is one decision therefore 2 outcomes to tests.
However, we would need two tests to achieve 100% statement coverage, as there are statements with each outcome of the decision.
So,in this instance, statement and decision testing would give us the same number of tests. NOte that if 100% coverage is required, statement testing can give us the same number of tests as decision testing, BUT NEVER MORE!

>>Lets look at some more examples now..

>> We will need 3 tests to achieve 100% decision coverage, but only 1 test to achieve 100% statement coverage.

Here we have an example of a supplimentary question, or a 'nested if'.
We have 2 decisions, so you may think that 4 tests may be required to achieve 100% decision coverage( two for each decision).
This is NOT the case! We can achieve 100% decision coverage with three tests - we need to exercise the 'Yes' outcome from the first decision ( line 1) twice, in order to subsequently exercise the 'Yes' and then the 'No' outcome from the supplementary question(line 2).
We need a further third test to ensure we exercise the 'No' outcome of the first decision( line 1 ).
There is only one decision outcome that has an associated statement - this means that 100% statement coverage can be achieved with one test.

>> As more statements are added, the tests for decision coverage are the same:

>> 3 tests to achieve 100% decision coverage, and 2 tests to achieve 100% statement coverage.

We have now introduced a statement that is associated with 'No' outcome of the decision on line 2.
This change affects the number of tests required to achieve 100% statement coverage, but does NOT alter the number of tests required to achieved 100% decision coverage - it is still three!

>And again an example..

>> 3 tests to achieved both decision and statement coverage.

Finally, we have statements associated with each outcome of each decision - the number of tests to achieve 100% statement coverage and 100% decision coverage are now the same.

>> And Last Example..

>> We will need 2 tests to achieve 100% decision coverage and 100% statement coverage.

We looked at this example of the "if Then Else' structure when considering statement testing.
As the decisions are separate questions we only need two tests to achieve 100% decision coverage( the same as the number required for statement coverage).
You may have thought that four tests were required - exercising the four different routes through the code, but remember, with decision testing our concern is to exercise each outcome of each decision atleast once - as long as we have answered 'Yes' and 'No' to each decision we have satisfied the requirements of the techinique.
The tests we have illustrated would need the following input conditions:
* Coffee drinker wanting cream.
* Non Coffee drinker not wanting cream ( but milk).

Equally, we could have chosen the following input conditions:
* Coffee drinker not wanting cream( but milk).
* Non coffee drinker wanting cream.

>> Then What about loops?

>> If we choose an initial value of p=4, we only need 1 test to achieve 100% statement and 100% decision coverage.

The control-flow graphs we showed earlier depicted a 'Do While' construct.
To reiterate, thw 'Do While' structure will execute a section of code whist a field or indicator is set to a certain value. For example,

The evaluation of the condition occurs before the code is executed.

Unlike the 'If Then Else', we can loop around the 'Do While' structure, which means that we exercise different routes through the code with one test.
As in the above diagram, if we set 'p' with an initial value '4', the first time through the code will :
* Go from line 1 to line 2
* Answer 'Yes' to the 'If' on line 2 ( if p<5)
* Execute the statement in line 3 (p=p*2, so p now equals 8)
* Go from line 3, through line 4 to line 5
* Execute the statement on line 5 ( which adds 1 to 'p', making it`s value '9')
* Execute the statement on line 6, which takes it back up to line 1.

Again we execute the code, with the value of 'P' now '9'
* GO from line1 to line2
* Answer 'NO' to the 'if' on line 2 (If p>5)
* Go from line 4 to line 5
* Execute the statement on line 5( which adds 1 to 'p', making it`s value '10')
* Execute the statement on line 6, which takes it back up to line 1.

Once more we execute the code
* Line 1 - 'P' is not less than '10' ( it is equal to 10), therefore, we exit this structure.

1 test - it achieves 100% statement coverage and 100% decision coverage.

>> And it`s same for 'Do until' structure

>> IF we choose an initial value of A =15, we only need 1 test to achieve 100% Decision coverage and 100% statement coverage.

The control flow structures we showed earlier also depicted a 'Do Until' structure.
To reiterate, the 'Do Until' structure will execute a section of code until a field or indicator is set to a certain value. For example,

The evaluation of the condition occurs after the code is executed.
Unlike the 'If Then Else', we can loop around the 'Do Until' structure, which means that we exercise different routes through the code with one test.
In the example above, If we set 'A' with an initial value of '15', the first time through the code will:
* Go from line 1 to line 2
* Answer 'Yes' to the 'If' on line 2 (If A<20)
* Execute the statement on line 3 ( A=A*2,which makes A=30)
* GO from line 3, through the line 4 to line 5
* Execute the statement on line 5( which adds 1 to 'A', making its value '31'.
* Execute the statement on line 6, Which takes back to line 1

Again we execute the code, with the value of 'A' now '31'
* Go from line 1 to line 2
* Answer 'No' to the 'If' on line 2 (If A < 20)
* Go from line 2, through line 4 to line 5
* Execute the statement on line 5( which adds 1 to 'A', making it`s value '32')
* Execute the statement on line 6, which exits the structure('A' is greater than 31)

1 test - it achieves 100% statement coverage and 100% decision coverage.

END...

82 comments:

SureshKumar said...

Nice info... Very useful.

Thanks,
Suresh Kumar K

Hang said...

It is very helpful to me. Thanks a lot.

Mahesh said...

Gud information but needed more examples

charu said...

Really helpful.I never found such a good clarity on the statement and decision coverage as this blog.Thanks Shailaja.

Anonymous said...

many thanks Shailajakiran, for the examples.
I never understand anything without examples. this helps a lot.
But I guess the example where you started with the while wend loop, I think it shoulda been if p<10.
Regards,
Karthik

chandrika said...

Nice info...
But plz explain a little bit,d coffee n cream example in stmt covg.
Why only 2 number of test cases n not 4?

Anonymous said...

It is an excellent description,
Thanks for ur effort Shailaja

Regards
Amit Ranjan

Amit said...

It is an excellent description
I never found such a good clarity.
Thank u very much ShailajaKiran

Regards
Amit

Anonymous said...

Nice info..thanx....

cheers,

Anonymous said...

thxxx a lot for d info.. i never understood these concepts untill today.. :-)

Anonymous said...

Nice info indeed....But could have had a few more examples....

Gaurav said...

really really helpful document.thankx shaliajakiran

Xuyenht said...

Oh, it helps me clear the concepts of statement and decision testing.
Thanks a lot!

Mahesh said...

Please Correct the While Loop Example (the first one) in Decision Coverage.
It should be
WHILE (P<10)
IF (P<5)

Rajesh said...

Really good explanation of Stmnt and Decision coverage. I must say, ISEB publisher should include you and your content in ISEB Testing book.

Anonymous said...

Thanks a lot... easy to understand...

Anonymous said...

Nice useful info shailaja

Pavlina said...

Thank you for the explanation so much Shailaja, it is very helpful

divya said...

Its really very useful information, examples really helped me having a fair knowledge about statement coverage and decision coverage. I also referred to all the sample papers provided. I passed ISTQB Exam.Yippie!!

Anonymous said...

VERY very good explanation, really thanks a lot!
Dave

Anonymous said...

It was way too good.....

Vikram said...

Thanks Shailaja.. this helped a LOT!

Geeta said...

Thanks a lot... its really easy to understand

Anonymous said...

It's very clearly, thanks

Anonymous said...

I really want to appreciate your time and dedication and research into the statement and decision coverage -- vikram

sithu said...

Hi Shailaja,

Really nice explanation thanks a lot for clarifying my doubts.

Naresh said...

Hey, the examples are very clear and very well explained...

vishal said...

I was really confused about Statement & Decision coverage bt not now .
Thankx :)

vishal said...

Thank a looooooooooottttt.
:)

Anonymous said...

i completely echo vishal.kudos ,Shailaja

Ashu said...

Thanks alot for explanation.. This is really helpfull..
-Ashu Barve

Anonymous said...

Very Good Informatioon. Thanks a lot :)

Nisha said...

Concept has been very nicely explained.Thanks a lot

Anonymous said...

Wonderfull thoughts bro... keep it up

Anonymous said...

Thanks a lot, this will help me for the ISEB-exam, found it hard to understand without the examples.

Priyanka said...

Thanks a lot.... excellent explanation with examples :)

Anonymous said...

seriously this was very much helpful!!

aswini said...

It is very helpful for me and easy to understand.

aruna said...

Hai shailajaKiran,
what u posted info is very good to understand who doesnot know about the coverages. Keep it up......

aparna_ku said...

Thanx a lot...... :)

Kopresh said...

Hi Shailaja,

Grt work....
Thanks for providing good examples in understanding the concepts

Shailja Ganeriwala said...

Thanks a lot.....very nice explanation

Arun said...

Thanks.You saved me from a head ache. I never understood it until now

Anonymous said...

very useful and easy to understand..MANY THANKS for this

Deeps said...

NIce...thanx a lot...but still am confused....

Article Submissions said...

Thanks for the great explanation. I didn't understand it before I read this.

Anonymous said...

thanks...for the wonderful explanation.

Adil said...

Thanks alot very helpful...
Very nicely explained....
Easy to understand....

ammu said...

thanks a million....was really helpful!!!!

manikanth said...

extremely helpful...thanx a lot

Anonymous said...

Thankyou.....its really wat i wanted compared 1 situation for both SC and DC.....that cleared my all doubts....

GilchristP said...

Thanks a lot...it really helped me

Anonymous said...

Nice examples. Thanks

Anonymous said...

Thanks

Saranya said...

thanks for these examples. Realy it made me to attend the concepts easily now.

Anonymous said...

Thanks..Clear Defintions and examples..

Anonymous said...

Thanks so much! You saved my day!!

Anonymous said...

Thanks a lot Shalijja .excellent descriptions with examples.. especially the explanation of the loop structured models..

Chanchal said...

Good information

Manjunath said...

Shailaja really owe you tons of thanks for the wonderful explaination. Was finding it extremely difficult to understand these basic concepts in other websites until I found your blog. I wish you good health and loads of happiness and good luck .

Anonymous said...

excellent

Anonymous said...

Really good explanation Shailaja Kiran...
Thank you soo much

Anonymous said...

Very helpful information with example.Helped me a lot to understand the concept.Big thanks to you Shailaja.

Priya said...

Very helpful.. thanks a lot

Satya said...

Is there any relation between Statement and decision coverage?

anna said...

I think the only relationship between the statement coverage and decision coverage, is that when you have decision coverage you have statement coverage too. But these two techniques does not detect the same failures.

Anonymous said...

thanks very useful to understand the concept

srividya said...

very informative

Thanks,
Srividya

Anonymous said...

Thanks for the info Shailaja..

just have a small ques for the last example of the Decision coverage calculation..

In this example,,, we ahve two saperate questions that are being asked.

What if the 2nd decision is dependent on the outcome of the first decision?

For example..
IF A > B THEN
C = A – B
ELSE
C = A + B
ENDIF
Read D
IF C = D Then
Print “Error”
ENDIF

What is the decision coverage for the above code? How to calculate it? Can you please explain in detail?

JP said...

This is very useful for freshers

Anonymous said...

Thanks a lot....

Anonymous said...

Very nice and thanks for help.

Anonymous said...

hi i think the Branch/decision coverage for the while.... Endwhile example should be 2. I got to this result by drawing the control flow graph and there are minimum two paths to cover all the edges and 1 path does not cover all the edges. Can you plz check?

Anonymous said...

Its really helpful. Thanx

Anonymous said...

Thank you very much! Very clear explanation!!!

Lady J said...

It helped me to understand my homework assignment. Thank you very much.

Akhil Chauhan said...

Amazing Explanations...Can't thank you enough...everything was crystal clear..
Only one slight update needs to be made..which someone else has pointed out as well...:
Instead of P>10, it should be P<10..

Thanks so much again,
Akhilesh

active40cymbal said...

I cleared my ISTQB Foundation level sometimes back and below are some of the tips I can give you:

1. The sample Questions will only give you the feel of the questions. Go thru the 5/6 sets of questions provided above and this will help. Original question sets will be similar but not exactly the same.

2. Go thru the syllabus quite a few times. Many questions will come from there..(see the other post in this section for syllabus pdf)

3. Some questions will come from glossary.(see the other post in this section for Glossary pdf)

4. From Static testing if you can remember the defintions and properties of reviewer, inspector etc you will get quite a few questions.

5. Finally If you study completely the second book,I can assure you that you can clear the exam very easily.

Also check the following links for free practice tests of ISTQB:

ISTQB Practice Test Part 1
ISTQB Practice Test Part 2
ISTQB Practice Test Part 3
ISTQB Practice Test Part 4
ISTQB Practice Test Part 5
ISTQB Practice Test Part 6
ISTQB Practice Test Part 7
ISTQB Practice Test Part 8
ISTQB Practice Test Part 9

Anonymous said...

its great help for me to understand sc and dc concept clearly for passing advance istqb
thanks a lot
khushboo

jai said...

Hi Shailaja iam new to this field . Can you explain and give me some questions of sc and dc . It is really confusing and hard to understand. can you send me some important questions.My email id is kksjsk@yahoo.com

Thanks in advance,
jai

Anonymous said...

Hi,
The information very useful , but i have doubt regarding while and dountil i think their should a 2 test cases to cover branch testing
1. For IF condition yes
2. For IF condition no

Please could you explain its will be very helpful

Thanks
Geetha

Anonymous said...

Absolutely brilliant explanations !!!Thanks a ton