Static Techniques : Reviews and the test process

What are Reviews?

> A way of testing software work products( e.g. documents, code)
> Can be Performed well before dynamic test execution.
> They are most beneficial when performed as early as possible in the development life cycle.

There are different levels of review, from very informal through to very formal, they all have their benefits and should be used through out the software development life cycle.

ISTQB DEFINES Review : An evaluation of a product or project status to ascertain discrepancies from planned results and to recommend improvements. Examples include Management Reviews, Informal Reviews, Technical reviews, Inspection ans walkthrough.

> Defects found during reviews are often much cheaper to remove than those found during dynamic execution.
> A review be done as a completely manual activities, but there is tool support.
> The main manual activity is to examine a work product and make comments about it.


The cost of finding and fixing defects increases over time. For example, If an error is made it is defect detected in the requirement at the specification stage, then it is relatively cheap to fix. However, not finding and fixing the defect until acceptance test or live use will be much more expensive.
Tools can assist in administering the review activities, but the reviewers will still be required to manually examine the work product.

What can be reviewed?

Any software work product can be reviewed :
> Requirement Specification
> Design Specification
> Code
> Test Plans
> Test Specification
> Test cases
> Test Scripts
> User Guides
> Web pages

Any other type of design document can also be reviewed such as date flow diagrams and entity relationship diagrams.
But it does not stop there, there is also benefit for the test plan and tests to be reviewed as well the code itself.
Lastly, the system documentation such as user manual and help text can also benefit from reviews.

Benefits of reviews

> Early defect detection and correction
> Fewer defects
> Reduced testing time and cost
> Reduced development time scales.
> Development productivity improvements
> Life time cost reductions
> Improved communication


Analysis have been performed on projects using reviews and it is estimated that ongoing reviews cost around 15% of the total development budget.
These costs include all the preparatory work required for the reviews to take place. For example who will be involved in the review, agreeing the date and time for the review and sending the document out prior to the review. Time much be allowed for each reviewer to read the document prior to the review meeting.
The costs also include the actual review meeting any follow up work required. Follow-up work can sometimes involve investigation of issues, correcting the document and producing a report detailing the outcome of the review. The report would detail the actions required, by whom, whether another review meeting is necessary. Metrics would be kept of the number of faults found which, when analyzed later could lead to process improvements.

> They can find omissions in requirements which are unlikely to be found in dynamic testing
> Remember verification and validation?


Well founded statistics estimate that 60% of the defects have already been made before coding/implementation has started, therefore it makes sense to carry out early lifecycle testing activities.
Early defects(faults) are often the most important to find; they have the characteristic to multiply thenselves top-down.
Tom De Marco said that "One of the main reasons for project failure is ambiguity in the requirements".

Objectives of reviews

> Reviews have the same objective as static analysis and dynamic anlysis of software
****** TO identify defects.
> The different techiniques are complementary
****** They can find different types of defects effectively and efficiently.
> Remember : Reviews identify defects, dynamic testing reveals failures.

In particular, the work product is being reviewed to validate it against actual requirements and to verify that it has been created to the appropriate standard and contains everything it should.
It is vital that the review team work together to acheive a consensus of opinion regarding the work product being reviewed.
Another objective of the review process is that the review team work together to improve the quality of the item being reviewed - it is a constructive, not destructive process.

Typical defects that are easier to find in reviews

> Deviations from standards
> Requirements defects
> Design defects.
> insufficient maintainability
> Incorrect interface specification

We will look at different types of reviw later but the reviews of documents such as user requirements, functional specifications and technical specifications are usually incremental.
The first review will usually raise sme questions and issues, points of clarification as to how a particular requirement will be tested. Or it could be an inconsistency in naming say 'a customer' or a client. These will need to be resolved and the document re-reviewed.
It may also be the case that all issues cannot be resolved at once. one issue may need to be resolved before others can be addressed, or having resolved one issue, this is turn raises further issues.
Reviews are therefore an iterative process.

The Benefits of early test design

Benefits of early test design

> 60% of the defects have already been made before coding/implementation has started.
> The thought process of designing tests early int eh lifecycle helps to verify the test basis inconsistencies and omissions.
> Their removal will increase the quality of the test basis and help to prevent defects from being introduced into the code.


As we know that the later in the lifecycle defects are detected, the more expensive they are to fix. It is most cost effective to find and fix requirements defects at the requirements stage. Building the right software correctly gives a lower cost overall.

The thought process of designing tests in the early lifecycle can help to prevent defects from being introduced into code. We some times refer to this as 'verifying the test basis via the test design'. The test basis includes documents like the requirements and design specification.

According to ISTQB
Requirement : A condition or capability needed by a user to solve a problem or acheive an objective that must be met or prossessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed document.

Test Basis : All documents from which the requirements of a component or system can be inferred. The documentation on which the test cases are based. If a document can be amended only by way of formal amendment procedure, then the test basis is called a frozen test basis.

Test Case : A set of input values, execution preconditions, expected results and execution post cnditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement.

> Documents reviewed can help to prevents defects occuring in to the code.
> Static analysis of code can identify defects in the cde and give a measure of complexity of the code.
> Time invested in early life cycle test activities will reap many benefits
* Quality of product and documentation improved
* Defects found are relatively cheap to fix.
* Fewer defects by reducing fault multiplication.

There are different types of reviews we can carry out, but an objective of all types of reviews is to find defects.

Time invested in early lifecycle test activities will reap many benefits:
* Quality if product improved at all stages in the process
* Reduce the number of faults to detect by reducing fault multiplication
* Less rework at later development stages.
* System implemented on time and to budget.

ISTQB defines
Review : An evaluation of the product or project status to ascertain discrepancies from planned results and to recommend improvements. Examples include management review, informal review, technical review, inspection, and walkthrough.

When we look at the test principles you will see the following test principle is related to this discussion:
Testing Principle - Early Testing : Testing activities should start as early as possible in the software or system development lifecycle, and should be focused on defined objectives.