Software Test Automation

Automation is the buzzword in the entire QA spectrum. Firms in India and abroad have got a new outlook on how the QA efforts can be saved by automation. Some companies have begun to mature in this field and utilizing automation in most effective ways.

It is important to educate the senior management about the expectations from Automation. Identifying what to automate and when to do it is as important as it is to plan the entire effort. Managing an automation project is similar to managing a software development project. The most critical part is to write maintainable scripts – scripts which can be reused across multiple releases of the same product with slight modification. That requires defining a set of guidelines before automation begins. Here are a few tips on how to get started with an Automation project:

1. The Manager and the entire team should attend trainings on the product and become familiar with the product functionality to the maximum. One of the reasons for the automation efforts going waste is the scriptor’s lack of knowledge of the component which is being automated.

2. Next the team should conduct brainstorming sessions on what can be automated and when. The outcome of all such sessions would be a detailed Automation Master Test Plan

3. By the time the planning activities are in progress, the team members should attend all the meetings associated with the component for which they are assigned – SRS Review meeting, DDS review meeting or any other meetings which could be with Manual testing team. This gives an insight on what can be automated and in which stages

4. Another activity which goes in parallel would be designing the Automation framework. The framework is one task which would be allocated a majority of effort, since it forms the crux of entire automation effort in future. The framework should be designed in such a way that it could handle all the ways in which an application could fail during execution of scripts and still recover smoothly to execute the next set of testcases. This is known as the Recovery System. At the same time, the framework should be flexible enough to accommodate any changes into the application functionality. Since Automation scripts would always need to be up-to-date with the application’s functionality.

5. Next comes the automation of scripts – how to get started? A senior member of the team should automate a few testcases for a particular area and demonstrate the same to entire team. The team should ensure that the automation has been done in accordance with the guidelines defined for automation of scripts.

6. Once every member is involved in writing automated scripts, two steps need to be performed:

a. Peer review of scripts every week or fortnight.

b. A check-in mechanism needs to be defined using some Version Control System such as VSS or CVS, so that
scripts are always up-to-date. One member in the team should be assigned this responsibility of Check-in
Administrator. Before every check-in the Admin has to ensure that the check-in does not break the existing
framework.


7. Be in regular contact with the Manual Test Engineer and get the scripts reviewed from him. It is important to gain their trust.

8. It is not important to automate everything, but it is most important to ensure that whatever is automated should be in working condition on the current build. Due to this reason, the maintenance of scripts is a constantly on-going task.

9. There are many times when the scripts fail due to UI/functionality changes and not due to bug in the code. Those are trying times for Automation Engineers and they need to cope with all such changes. The Manager should ensure the motivation levels of the team are always up in such cases and a failure due to a bug in the software should never go unnoticed

10. To keep the automation efforts visible to management, it is important to produce some deliverables at different stages during the SDLC. The deliverables could be in the following order:

a. Automated set of Smoke testcases – a set of high-level testcases which verify that the basic functionality of the
product is correct. Once the smoke tests are automated, they should be executed on every weekly build (some
places it is also done on Daily Builds to identify the side-effects of any check-ins). Such tests could include – UI
tests, Command-line. For tests which cannot be automated, maintain such a list separately so that they could
be executed manually on every build. The results of execution should be published to all concerned teams –
including Dev, PMG and QA

b. Automated set of command-line tests – Such tests one automated are resistant to change. Since the
functionality is command-line which is not expected to change frequently, maintenance of such testcases
requires minimal effort

c. Automated set of Functionality tests – This set should be automated based on the priority. For example – we
associate a rotation with every testcase – R1, R2, R4, R8, R12, R16. R1- smoke test (Every 1 week); R2,R4,R8
– High priority functionality tests; R12,R16 – Low priority, low risk functionality tests.

d. Automated set of Regression tests – This requires keeping a track of all critical defects. A critical defect could
be a showstopper (P0), P1 or even a P2. I always recommend to automate all P1s, selected P2s and selected
P0s. This kind of Automation would always give a high visibility, since no manual tester would regress his fixed
defects again. An execution report of such testcases could be published every fortnight or so.

e. Automation of workflows, shortcut keys, localization tests, performance tests