How to get maximum value out of automated testing during a limited period of time?

Question

I have a relatively large project, which up to now had almost no automatic tests.

Now the management has given us several weeks of time to stabilize the product, including test automation.

In order to get the most value of these X weeks we can spend on automatic testing, I need to know what classes/methods to test first.

How can I prioritize testing efforts (decide, which class/method to test now, which later) apart from the approaches listed below?

1. Calculate the dependents for each class (how many other classes use class, including transitive dependencies). Classes with the greatest number of dependent classes should be tested first.

2. Find out, which classes change most frequently (according to the version control system). Frequent changes may be a symptom of either lots of bugs or active development in these classes. In both cases, it makes sense to write unit tests for them.

3. Find out, which classes are involved in bug reports from testers and/or customers.

Answer 1

Other than the above answer, I would like to add few more points regrading priority and coverage for automation: 1. Before you start any coding, understand what all test cases can give you maximum coverage like a full flow or end to end test scenario. 2. Always maintain the usability of automation test suite - like code those test cases first which can be utilised all the time and can cover most of regression part rather than concentrating a specific feature of application. 3. When you've a limited time, try to avoid implementing fancy things like logger, email summary report, html report etc... 4. Better to have data driven framework rather than having a keyword or hybrid framework because you can cover many test cases by just varying your test data in a limited time. 5. Maintain an excel sheet or csv for test data, test results, you can use JXL Lib to handle excel sheet in java. 6. Read excel sheet, write excel sheet are the most common method which you may want to use in your automation code. You can get reference about this from blog: http://testingmindzz.blogspot.in/

Answer 2

All of your ideas seems good. This article can help you with prioritizing and automating.

This is formula of how to do Estimating Testing effort:

Method for Testing Process (based on use case driven approach).

Step 1 : count number of use cases (NUC) of system 

step 2 : Set Avg Time Test Cases(ATTC) as per test plan 

step 3 : Estimate total number of test cases (NTC) 

Total number of test cases = Number of usecases X Avg testcases per a use case

Step 4 : Set Avg Execution Time (AET) per a test case (idelly 15 min depends on your system) 

Step 5 : Calculate Total Execution Time (TET) 

TET = Total number of test cases * AET 

Step 6 : Calculate Test Case Creation Time (TCCT)

usually we will take 1.5 times of TET as TCCT

TCCT = 1.5 * TET

Step 7 : Time for ReTest Case Execution (RTCE) this is for retesting

usually we take 0.5 times of TET

RTCE = 0.5 * TET

Step 8 : Set Report generation Time (RGT

usually we take 0.2 times of TET

RGT = 0.2 * TET

Step 9 : Set Test Environment Setup Time (TEST)

it also depends on test plan

Step 10 : Total Estimation time = TET + TCCT+ RTCE + RGT + TEST + some buffer...;)

Here is example how it works:

Total No of use cases (NUC) : 227

Average test cases per Use cases(AET) : 10

Estimated Test cases(NTC) : 227 * 10 = 2270

Time estimation execution (TET) : 2270/4 = 567.5 hr

Time for creating testcases (TCCT) : 567.5*4/3 = 756.6 hr

Time for retesting (RTCE) : 567.5/2 = 283.75 hr

Report Generation(RGT) = 100 hr

Test Environment Setup Time(TEST) = 20 hr.

Total Hrs 1727.85 + buffer

4 means Number of test cases executed per hour

i.e 15 min will take for execution of each test case

And since you are going to automate almost from scratch

up to now had almost no automatic tests

I think you may consider not only benefits, but Myths about Automated testing too:

• Automation cannot replace the human

• Once automated, cost savings is a given

• Every test case can be automated

• Testing can be fully automated

• One test tool is suitable for all tasks

• Automated Testing doesn’t mean Automatic

  After all Testing ... It means Computer aided testing.