Software QA and Testing

Testing always spans all stages of software lifecycle. It is much too late to start testing after software product is implemented. The cost of fixing faults introduced in earlier phases would be very high.

Software testing requires careful testing and diligent execution. It starts with a test plan. Test plan is a document that identifies the high-level project information and software components that will correspond to test cases. It describes project's testing strategy, identifies QA resources, efforts, schedule, and cost. Test cases describe definitive sequence of steps to break the software. Each functional module has one or more test cases defined.

The most effective testing is conducted by a separate dedicated QA group that is charged with responsibility for product quality. The first pass at testing is always done by the developers themselves. They are responsible for unit testing, and initial integration testing. However individual developers always remain somewhat blindfolded by the work they have done to implement their modules, making certain assumptions, and missing some border conditions.

The more testing is packed into early development stages the better payoff.

Testing begins as soon as requirements and design documentation matures. Their testing procedures consist of walkthroughsinspections, implemented as formal reviews.

Each formal review is a planned meeting that target a particular part of the systems documentation. Formal reviewer studies documents, and raises questions during the meeting. The goal is to decide, if each issue represents a problem, without concentrating on a solution.
Thanks to the 'team synergy' and friendly creative atmosphere reviews are a great tool in early detection and correction of many design faults.

Execution-based testing starts once builds (early releases of the software) are made available.
Execution-based testing of a typical project includes:

  • User interface testing
  • Database testing
  • Security and authorization testing
  • Performance testing
  • Stress testing
  • Fault-tolerance and fail-over testing
  • Compatibility and configuration testing
  • Installation testing
  • There are two kinds of execution-based testing:
  • Testing to spec (black-box testing). Testers assume no knowledge of internals of the software. They test functionality purely based on the requirements and specifications, i.e. intended behavior. The software is given expected input, and results are analyzed for errors.
  • Testing to code (white-box testing). This type of testing requires close interaction of our developers and QA engineers. Employs knowledge of the internal program logic to understand required input to exercise all execution paths in the software.

Testing to code is assisted by code coverage tools, such as Rational Pure Coverage. These tools provide information on what parts of source code were executed and how extensively. Code coverage is also indispensable as an integral part of test results, and in assessing readiness of software modules.

Testing to spec and testing to code tend to discover differnet categories of errors, and complement one another

Incremental development not only means incremental integration of software modules, but also implies incremental or regression testing. Regression testing is re-execution of the previous test cases with the same data on every new release of the software. In simple terms it means: what was working in the previous release should NOT become broken in the new release.

There are always many builds throughout the project lifecycle. Thus automation of regression testing becomes an important task. Automation testing achieves:

  • Increased productivity
  • Reduced omissions, and repeatedly introduced software faults
  • Shorter development cycle

Automation testing is implemented using various tools such as QTP, Functional Tester, Silk Test, and Test Complete for GUI and Web-based interfaces, and programmatic tools and scripts for command line, and internal modules, such as database layers, business logic, and 'server-side'.