Static Techniques : Reviews and the test process

What are Reviews?

> A way of testing software work products( e.g. documents, code)
> Can be Performed well before dynamic test execution.
> They are most beneficial when performed as early as possible in the development life cycle.

There are different levels of review, from very informal through to very formal, they all have their benefits and should be used through out the software development life cycle.

ISTQB DEFINES Review : An evaluation of a product or project status to ascertain discrepancies from planned results and to recommend improvements. Examples include Management Reviews, Informal Reviews, Technical reviews, Inspection ans walkthrough.

> Defects found during reviews are often much cheaper to remove than those found during dynamic execution.
> A review be done as a completely manual activities, but there is tool support.
> The main manual activity is to examine a work product and make comments about it.


The cost of finding and fixing defects increases over time. For example, If an error is made it is defect detected in the requirement at the specification stage, then it is relatively cheap to fix. However, not finding and fixing the defect until acceptance test or live use will be much more expensive.
Tools can assist in administering the review activities, but the reviewers will still be required to manually examine the work product.

What can be reviewed?

Any software work product can be reviewed :
> Requirement Specification
> Design Specification
> Code
> Test Plans
> Test Specification
> Test cases
> Test Scripts
> User Guides
> Web pages

Any other type of design document can also be reviewed such as date flow diagrams and entity relationship diagrams.
But it does not stop there, there is also benefit for the test plan and tests to be reviewed as well the code itself.
Lastly, the system documentation such as user manual and help text can also benefit from reviews.

Benefits of reviews

> Early defect detection and correction
> Fewer defects
> Reduced testing time and cost
> Reduced development time scales.
> Development productivity improvements
> Life time cost reductions
> Improved communication


Analysis have been performed on projects using reviews and it is estimated that ongoing reviews cost around 15% of the total development budget.
These costs include all the preparatory work required for the reviews to take place. For example who will be involved in the review, agreeing the date and time for the review and sending the document out prior to the review. Time much be allowed for each reviewer to read the document prior to the review meeting.
The costs also include the actual review meeting any follow up work required. Follow-up work can sometimes involve investigation of issues, correcting the document and producing a report detailing the outcome of the review. The report would detail the actions required, by whom, whether another review meeting is necessary. Metrics would be kept of the number of faults found which, when analyzed later could lead to process improvements.

> They can find omissions in requirements which are unlikely to be found in dynamic testing
> Remember verification and validation?


Well founded statistics estimate that 60% of the defects have already been made before coding/implementation has started, therefore it makes sense to carry out early lifecycle testing activities.
Early defects(faults) are often the most important to find; they have the characteristic to multiply thenselves top-down.
Tom De Marco said that "One of the main reasons for project failure is ambiguity in the requirements".

Objectives of reviews

> Reviews have the same objective as static analysis and dynamic anlysis of software
****** TO identify defects.
> The different techiniques are complementary
****** They can find different types of defects effectively and efficiently.
> Remember : Reviews identify defects, dynamic testing reveals failures.

In particular, the work product is being reviewed to validate it against actual requirements and to verify that it has been created to the appropriate standard and contains everything it should.
It is vital that the review team work together to acheive a consensus of opinion regarding the work product being reviewed.
Another objective of the review process is that the review team work together to improve the quality of the item being reviewed - it is a constructive, not destructive process.

Typical defects that are easier to find in reviews

> Deviations from standards
> Requirements defects
> Design defects.
> insufficient maintainability
> Incorrect interface specification

We will look at different types of reviw later but the reviews of documents such as user requirements, functional specifications and technical specifications are usually incremental.
The first review will usually raise sme questions and issues, points of clarification as to how a particular requirement will be tested. Or it could be an inconsistency in naming say 'a customer' or a client. These will need to be resolved and the document re-reviewed.
It may also be the case that all issues cannot be resolved at once. one issue may need to be resolved before others can be addressed, or having resolved one issue, this is turn raises further issues.
Reviews are therefore an iterative process.