Testing Estimation method

Software Testing Estimation Techniques

In order to be a successful in estimating, the software test project and proper execution are significant as the software development life cycle. Software testing estimation techniques play a very important role in making the good reputation with the client while bidding the project for testing.
Each developed software application is unique in its own domain, and it is inconceivable to estimate software accurately at first attempt. Even if the scope is understood and requirements are clear, it is still arduous to estimate a complex system that is going to be built with the finite requirements.
One of the most important factors while estimating testing efforts is the experience on varied projects for the software test life cycle. Apparently one cannot just put some number of days for any task or taking the old time formula of one third of the development effort. This is one of the most widely used estimation technique by the companies offering software testing services. It is merely due to the fact that this method is not based on any scientific principle or technique. Unfortunately the development versus testing effort method has given many failures in the software projects, thereby compromising the efforts on quality.
For any software testing estimation technique, it is highly recommended that following factors should be taken into account:

1. Domain Knowledge and core requirements
2. Risks and complexity of the application
3. Team Knowledge on the subject/skills
4. Historical data for the previous estimation for improvement and accuracy
5. Estimation should include buffer time
6. Bug cycles for the project
7. Resources availability (Like vacations, holidays, and sick days can have a great impact on your estimates)

However the techniques just provide the means for estimating but rely heavily on the team productivity variations, individual skills, complexity of the unknown factors like system environment and downtime.
In the recent years there were many techniques that have been developed for estimating the software testing. These techniques are: 3-Point Software Testing Estimation Technique, Use-Case Point Method and Wide Band Delphi Method.
Following are the different popular test estimation techniques:-

i)                    3-Point Software Testing Estimation Technique

     ii)                  Use – Case Point Method:
    iii)                Work Breakdown Structure
    iv)                Wideband Delphi technique
    v)                  Function Point/Testing Point Analysis
   vi)                Percentage of development effort method
   vii)              Percentage distribution

   viii)            Best Guess

   ix)                Ad-hoc method

   x)                  Experience Based

1)      1)3-Point Software Testing Estimation Technique:

3-Point Software Testing Estimation Technique is based on statistical methods in which each testing task is broken down into sub tasks and then three types on estimation are done on each tasks.

 The formula used by this technique is:
 Test Estimate = P + (4*N) + E / 6
 Whereas
 P = Positive Scenarios or Optimistic Estimate (Best case scenario in which nothing goes wrong and all conditions are optimal.)

• N = Negative Scenarios or Most Likely Estimate (most likely duration and there may be some problem but most of the things will go right.)
• E = Exceptional Scenarios or Pessimistic Estimate (worst case scenario which everything goes wrong.)

 Standard deviation for the technique is calculated as,
 Standard Deviation (SD) = (N – E)/6

2)      Use – Case Point Method:
Use-Case Point Method is based on the use cases where we calculate the unadjusted actor weights and unadjusted use case weights to determine the software testing estimation.
Use case is a document which well specifies different users, systems or other stakeholders interacting with the concerned application. They are named as ‘Actors’. The interactions accomplish some defined goals protecting the interest of all stakeholders through different behavior or flow termed as scenarios.
The formula used for this technique is:

• Unadjusted actor weights = total no. of actors (positive, negative and exceptional)
• Unadjusted use case weight = total no. of use cases.
• Unadjusted use case point = Unadjusted actor weights + Unadjusted use case weight
• Determine the technical/environmental factor (TEF) ( if not available take as 0.50)
• Adjusted use case point = Unadjusted use case point * [0.65+ (0.01 * TEF]
• Total Effort = Adjusted use case point * 2

 3)      Work Breakdown Structure:
 It is created by breaking down the test project into small pieces. Modules are divided into sub-modules. Sub modules are further divided into functionalities and functionalities are divided in sub-functionalities.

Review all the requirements from Requirement Document to make sure they are added in WBS. Now you figure out the number of tasks your team needs to complete. Estimate the duration of each task.

 4)      Wideband Delphi technique:
 Same as above WBS.  In Wideband Delphi Method, work breakdown structure is decomposed for each task and is distributed to a team comprising of 3-7 members for re-estimating the task. The final estimate is the result of the summarized estimates based on the team consensus. This method speaks more on experience rather than any statistical formula. This method was popularized by Barry Boehm to emphasize on the group iteration to reach to a consensus where the team visualized on the different aspects of the problems while estimating the test effort.
 5)      Function Point/Testing Point Analysis:
 The FP technique is a direct indicator of the functionality of software application from the user's perspective. This is the most accepted technique used to estimate the size of a software project. 

This technique is a part of TMap. Base of this technique is function point technique. Here we convert function points into test points. In Test Point analysis, we usually carry out the following:

• Dynamic Test Points
• Static Test Points
• Environmental Factor
• Productivity Factor
• Primary Test Hours
• Control Factor
• Total Test Hours

 In Testing, This estimation is based on requirement specification document, or a previously created prototype of the application. To calculate FP for a project, some major components are required. 

The major components are:

Unadjusted Data Function Points: i) Internal Files, ii) External Interfaces 

Unadjusted Transaction Function Points: i) User Inputs, ii) User Outputs & iii) User Inquiries
Capers Jones basic formula:
Number of Test cases = [Number of Function Points] x 1.2
Total Actual Effort, TAE = (Number of Test cases) * (Percentage of development effort /100) 

This method is done in a case when a detailed low level design document or requirement document is available (i.e measure of function point is available) & Previous data for development and testing is available. But now a days, when we are using agile and iterative methodologies to deliver projects, so most of the times all this documentation is not available.

 6)      Percentage of development effort method:
 Here the assumption is that a more complex business application may require more testing effort. The test effort required is a direct proportionate or percentage of the development effort.

Note: The development effort can be estimated using line of code (LOC) or function point (FP) which is not in the our scope.
Example:
If a previous project with 500 FPs required 50 man hours for testing, the percentage of testing effort is calculated as:
P = (50 / 500) * 100 =10%
For the current project with a development effort, say 1500 FPs, the testing effort is:
Total Actual Effort, TAE = 1500 * (P/100) = 1500 * (10/100) = 150 man hours.

7)      Percentage distribution:
Here all the phases of SDLC are divided in parts and assigned effort in %. Like – 

Project management 7%
Requirements 9%
Design 16%
Coding 26%
Test (all test phases) 27%
Documentation 9%
Installation and training 6%
Now testing % is further distributed into all testing phases: 

All phases	%
Component testing	16
Independent testing	84
Total	100
Independent testing	%
Integration testing	24
System testing	52
Acceptance testing	24
Total	100
System testing	%
Functional system testing	65
Non-functional system testing	35
Total	100
Test Planning and Design Architecture	50%
Review phase	50%

8. Best Guess:
This technique is purely guesswork and based on the some sort of experience. 
The method is very common, but since it is based on your gut feeling, its uncertainty contingency is probably around 200% or even higher.

9. Ad-hoc method:
The test efforts are based on tentative timeframe. The timeline set by managerial or marketing personnel or by client without any guess/experience. Alternatively, it is done until the budgeted finances run out. 
This is very common practice in extremely immature organizations and has error margins of over 100% at times.

10. Experience Based:
Analogies and experts: 

• Metrics collected from previous tests.
• You already tested similar application in previous project.
• Inputs are taken from Subject Matter experts who know the application (as well as testing) very well.

 Following Rules should be remembered while estimating a testing project:
1st Rule: Estimation should be always based on the software requirements
All estimation should be based on what would be tested, i.e., the software requirements.
Normally, the software requirements were only established by the development team without any or just a little participation from the testing team.  After the specification have been established and the project costs and duration have been estimated, the development team asks how long would take for testing the solution.  The answer should be said almost right away.
Then, the software requirements shall be read and understood by the testing team, too.  Without the testing participation, no serious estimation can be considered.
2nd Rule: Estimation should be based on expert judgment
Before estimating, the testing team classifies the requirements in the following categories:
¨      Critical:  The development team has little knowledge in how to implement it;
¨      High:  The development team has good knowledge in how to implement it but it is not an easy task;
¨      Normal:  The development team has good knowledge in how to implement.
The experts in each requirement should say how long it would take for testing them.  The categories would help the experts in estimating the effort for testing the requirements.
3rd Rule: Estimation should be based on previous projects
All estimation should be based on previous projects experience/ knowledge. If a new project has similar requirements from a previous one, the estimation is based on that project.
4th Rule: Estimation should be based on metrics
Organization should be create an OPD, Organization Process Database, where the project metrics are recorded. 

5th Rule: Estimation should never forget the past
The testing team continues using the old process and the spreadsheet.  After the estimation is done following the new rules, the testing team estimates again using the old process in order to compare both results.
Normally, the results from the new estimate process are cheaper and faster than the old one in about 20 to 25%. If the testing team gets a different percentage, the testing team returns to the process in order to understand if something was missed.
6th Rule: Estimation should be recorded
All decisions should be recorded.  It is very important because if requirements change for any reason, the records would help the testing team to estimate again.  The testing team would not need to return for all steps and take the same decisions again.  Sometimes, it is an opportunity to adjust the estimation made earlier.
7th Rule: Estimation should be supported by tools
A spreadsheet containing metrics should be created that help to reach the estimation quickly.  The spreadsheet calculates automatically the costs and duration for each testing phase. 
The template could contain some sections such as: cost table, risks, and free notes to be filled out.  This tool could show the different options for testing that can help the customer decides which kind of test he needs.
8th Rule: Estimation should always be verified
All estimation should be verified.  Another spreadsheet for recording the estimations should be created.  The estimation is compared to the previous ones recorded in a spreadsheet to see if they have similar trend.  If the estimation has any deviation from the recorded ones, then a re-estimation should be made.
9th Rule: Estimation should cover risk factors
Estimation should cover all kind of risk like resource availability, product down time, skill improvements, learning capability etc.

Following Scenarios will give good understanding effects by resource availability 

Scenario:
Suppose you have a project that you estimate will take a 10 person team 300 days to complete. (3000 workdays - Sunny day scenario)

Knowing that there are 265 working days in a year (52 weeks x 5 days), without holidays, means that the project, at best, will take 1.13 years or 13.6 months to complete

Suppose the average year includes 12 days company holidays (total impact = 12x10 people = 120 daysx1.13years = 136 days)

Suppose also that the average employee takes 4 sick days and 3 comp days off a year (total impact = 7 x 10 people = 70 x 1.13 = 79)

Suppose also that the average employee takes 7 work days of vacation a year (total impact = 7 x 10 people = 70 x 1.13 = 79)

Suppose also that each employee loses 3 days out of each year for training (30 days)

Total duration change due to employee holidays, sick time, vacation and training = 136+79+79+30=324

Total workdays for the project is now 3000+324 = 3324 or a 10.8% increase in duration (300 day schedule is now 333 days or 15.1 months.)