Software Testing

What is Software Testing?

• Testing is intended to show that a program does what it is intended to do and to discover program defects before it is put into use.
• The testing process has two distinct goals:

1. Demonstrate to the developer and the customer that the software meets its requirements.
2. To discover situations in which the behavior of the software is  incorrect, undesirable. (Defect Testing). In order words, to ensure that the software system is Defect free by removing the undesirable faults.

What is main focus on Software Testing?

• Correctness of software with respect to requirements or intent
• Performance of software under various conditions
• Robustness of software, its ability to handle erroneous input and unanticipated conditions
• Installation and other facets of a software release

here

• In simple terms, Software Testing means Verification of

Application Under Test (AUT). It can be either done manually or using automated tools.

Why is Software Testing Important?

• Testing is impor tant because software bugs could be

expensive or even dangerous. Software bugs can potentially cause monetar y and human loss, and histor y is full of such examples.

• Software testing helps to identify errors, gaps or missing

requirements in contrar y to the actual requirements.

Verification and Validation (V&V)

• Testing is par t of a broader process of software verification and validation (V & V).
• Verification
  • The aim of is to check that the software meets its stated functional and non-functional requirements.
  • Are we building the product right?
• Validation
  • Is a more general process, with the aim of ensuring that the software meets the customer’s expectations.
  • Are we building the right product ?

• BLACK BOX TESTING is defined as a testing technique in which functionality/features of the Application Under Test (AUT) is tested without looking at the internal code structure, implementation details and knowledge of internal paths of the software.

• This type of testing is based entirely on software requirements

and specifications.

• In BlackBox Testing we just focus on inputs and output of the

software system without bothering about internal knowledge of the software program.

There are many types of Black Box Testing but the following are the prominent ones -

• Functional testing - This black box testing type is related to the functional requirements of a system; it is done by software testers.
• Non-functional testing - This type of black box testing is not related to testing of specific functionality, but non-functional requirements such as performance, scalability, usability.
• Regression testing - Regression Testing is done after code fixes, upgrades or any other system maintenance to check the new code has not affected the existing code.

Black-Box Testing

• Advantages
  • Test case selection is done before the implementation of a program.
  • Help in getting the design and coding correct with respect to the specification.

Black-Box Testing Techniques Following are the prominent Test Strategy amongst the many used in Black box Testing

• Decision Table Testing: A decision table puts causes and their effects in a matrix. There is a unique combination in each column.
• Equivalence Class Testing: It is used to minimize the number of possible test cases to an optimum level while maintains reasonable test coverage.
• Boundary Value Testing: Boundar y value testing is focused on the values at boundaries. This technique determines whether a cer tain range of values are acceptable by the system or not. It is ver y useful in reducing the number of test cases. It is most suitable for the systems where an input is within cer tain ranges.

1. Analyze the given test inputs or requirements and list out the various conditions in the decision table.
2. Calculate the number of possible combinations (Rules)
3. Fill columns of the decision table with all possible combinations (Rules)
4. Find out cases where the values assumed by a variable are immaterial for a given combination Fill the same by "don't care" symbol (X).
5. For each of the combination of values, find out the action or expected result.
6. Create at least one Test case for each rule. If the rules are binary, a single test for each combination is probably sufficient. Else if a condition is a range of values, consider testing at both the low and high end of range.

• Decision Table Testing is Important because it helps to test different combinations of conditions and provide better test coverage for complex business logic. When testing the behavior of a large set of inputs where system behaviour differs with each set of input, decision table testing provides good coverage and the representation is simple so it is easy to interpret and use.

• Equivalent Class Par titioning is a black box technique (code is not

visible to tester) which can be applied to all levels of testing like unit, integration, system, etc. In this technique, you divide the set of test condition into a par tition that can be considered the same.

• It divides the input data of software into different equivalence data

classes.

• You can apply this technique, where there is a range in the input field.

Which set of test cases derive from the following problem if we apply equivalence classes partitioning: Our application expects the user to enter his age. Teenagers (ages from 12 to 16) are given a 10% discount ?

a. Testcases: inputs { 9, 12, 16 } b. Testcases: inputs { 0, 9, 14, 45 } c. Testcases: inputs { -1, 9, 12, 16, 21 } d. Testcases: inputs { 3, 14, 58 }

The number of items in an order of a stock control system can range between 100 and 9999 inclusive. What are the test cases that should be derived from this problem if we are applying boundary value testing? a. 99, 100, 101, 400, 9999, 10000 b. 50,, 99, 100, 9999, 10000, 10500 c. 100, 101, 9999, 10000 d. 99, 100, 101, 9998, 9999, 10000

To test an input field that accepts an integer that represents a day. What are the test cases that should be derived from this problem if we are applying boundary value testing?

a. 0, 1, 31, 32 b. -1, 0, 1, 30, 31, 32 c. 0, 1, 2, 30, 31, 32 d. 0, 1, 2, 29, 30, 31

Basis Path Testing A White Box Method for Designing Test Cases

• Basis Path testing is a structural testing method that involves using the source code of a program in order to find every possible executable path.
• It helps to determine all faults lying within a piece of code.
• This method is designed to execute all or selected path through a computer program.

Basis Path Testing A White Box Method for Designing Test Cases

• Basis path testing involves execution of all possible blocks in a program and achieves maximum path coverage with the least number of test cases.
• The analysis of the code can be used to find how many test cases are needed.

Basis Path Testing Why Basis Path Testing?

• Any software program includes, multiple entr y and exit points. Testing each of these points is ch allen gin g as well as time-consuming.
• In order to reduce the redundant tests and to achieve maximum test coverage, basis path testing is used.

CYCLOMATIC COMPLEXITY

• CYCLOMATIC COMPLEXITY is a software metric used to measure the paths complexity of a program.
• It is a quantitative measure of independent paths in the source code of the program.
• Independent path is defined as a path that has at least one edge which has not been traversed before in any other paths.
• Cyclomatic complexity can be calculated with respect to functions, modules, methods or classes within a program.
• It is based on a control flow representation of the program. Control flow depicts a program as a graph which consists of Nodes and Edges.

The following steps should be followed for computing Cyclomatic complexity and test cases design.

Step 1: Draw a control graph with nodes and edges from the source code (to determine different program paths) Step 2: Calculate Cyclomatic Complexity (metric to determine the number of independent paths) V(G) = E - N + 2 or also V(G) = P + 1 (n: nb of nodes, e: nb of edges, p: number of decision nodes) Step 3: Identify basis set of paths Step 4: Generate Test Cases to execute all the paths.

Basis Path Testing: Step 4

• Prepare a test case for each independent path.
• In this example:
  • [ 1 – 2 – 6 ]: N=0, A[]
  • [ 1 – 2 – 3 – 5 – 2 – 6 ]: N=2, A[5,7]
  • [ 1 – 2 – 3 – 4 – 5 – 2 – 6]: N=3, A[5,3,4]

Computing mathematically, V(G) = 9 - 7 + 2 = 4 V(G) = 3 + 1 = 4 (Condition nodes are 1,2 and 3 nodes) Basis Set - A set of possible execution path of a program 1, 7 1, 2, 6, 1, 7 1, 2, 3, 4, 5, 2, 6, 1, 7 1, 2, 3, 5, 2, 6, 1, 7

• Basis Path testing is one of White box technique and it guarantees to execute each statement at least once during testing.
• It checks each “linear ly independent path” of the program, which means that the number of test cases will be equal to the cyclomatic complexity of the program.

• Helps developers and testers to determine independent path executions
• Developers can assure that all the paths have been tested at least once
• Helps to focus more on the uncovered paths
• Improve code coverage in testing
• Evaluate the risk associated with the application or program
• Using these metrics early in the cycle reduces risks of project failure

• Unit Testing is a type of software testing where individual units or components of a software are tested. The purpose is to validate that each unit of the software code performs as expected.

• A Unit is a smallest testable por tion of system or

application (individual function, method, procedure) which can be compiled and executed.

• The aim is to test each par t of the software by separating

it. It checks that component are fulfilling functionalities and verify its correctness. Usually this kind of testing is performed by developers.

Why Unit Testing?

• Sometimes software developers attempt to save time by doing minimal unit testing. This is a myth because skipping on unit testing leads to higher Defect fixing costs during System Testing, Integration Testing and even Beta Testing after the application is completed.
• Proper Unit tests help to fix bugs early in the development cycle and save both time and money in the end.
• It helps the developers understand the code base and enables them to make changes quickly.
• Good unit tests ser ve as project documentation.
• Unit tests help with code re-use. Migrate both your code and your tests to your new project.

Unit Testing Code Review

• Code walkthrough

1. It's a type of Semi Formal Review.
2. Author is Presenter.
3. Lead by Author only.
4. Reviewers are not aware of the subject/topic.

• Code inspection It's totally a Formal Review. Author is not presenter. Someone else is giving the presentation. Lead by Moderator. Reviewers are aware & well prepared for the subject/topic.

Unit Testing

The Best Team Size for Inspections

• The preparation level, not the team size, determines inspection effectiveness
• The team’s effectiveness and efficiency depend on their familiarity with their product

• The purpose of this level of testing is to expose defects in the

interaction between these software modules when they are integrated.

• Integration Testing focuses on checking data communication amongst

these modules.

Sample Integration Test Cases for the following scenario:

• Application has 3 modules say 'Login Page', 'Mailbox' and 'Delete emails'

and each of them is integrated logically.

• The software tester does not concentrate much on the Login Page testing

as it's already been done in Unit Testing. But he/she checks how Login Page is linked to the Mail Box Page.

• Similarly check how Mail Box Page is integrated with the Delete Mails

Module.

Types of Integration Testing

• Big-Bang Approach
• Incremental Approach: which is fur ther divided into the following
  • Top Down Approach
  • Bottom Up Approach

• Incremental Approach • In this approach, testing is done by joining two or more modules that are logically related. • Then the other related modules are added and tested for the proper functioning. • The process continues until all of the modules are joined and tested successfully.

• Incremental Approach is carried out by two different methods:

• Bottom Up
• Top Down

• What is Stub and Driver?

• Incremental Approach is carried out by using dummy programs

called Stubs and Drivers. Stubs and Drivers do not implement the entire programming logic of the software module but just simulate data communication with the calling module.

• Stub: a special-purpose program to simulate the activity of the missing component. Is called by the Module under Test.
• Driver: A routine that calls a par ticular component and passes a test case to it. Calls the Module to be tested.

• Stubs and drivers both are dummy modules and are only created for

testing purposes.

Example:

Consider a computer program that queries a database to obtain the min price of all products that are ”cars” stored in the database.

In this example, the quer y is slow and consumes a large number of system resources.

Secondly, tests may include values outside those currently in the database. The method (or call) used to perform this is get_mi n.

For testing purposes, the source code in get_mi n

can be

temporarily replaced with a simple statement that returns a specific value. This would be a test stub.

• On the other hand, Drivers are the " calling" programs. Drivers are used in bottom up testing approach. Drivers are dummy code, which is used when the sub modules are ready but the main module is still not ready.

System Testing

• System Functional Test
  • Test entire system against the functional requirements.
• System Performance Test
  • Test the non-functional requirements of the system. For example,
  • Response times, load testing etc.

• NON-FUNCTIONAL Software testing is defined as a type of Software testing to check non-functional aspects (performance, usability, reliability, etc) of a software application.
• An example of non-functional test is to check how many users can simultaneously use a software.
• Non-functional testing is equally important as functional testing and affects client satisfaction.

• The purpose of Performance Testing is not to find functional defects but to eliminate performance bottlenecks in the software or device.
• Performance Testing checks the speed, response time, reliability, resource usage, scalability of a software program under their expected workload.
  • Speed - Determines whether the application responds quickly
  • Scalability - Determines maximum user load the software application can handle.
  • Stability - Determines if the application is stable under var ying loads
• Performance Testing is popularly called “Perf Testing”.

Security: Test how a system is protected from deliberate attacks from internal and external sources.

Reliability: Test if the system continuously performs the specified functions without failure.

Survivability (Resiliency): Test if the software system continues to function and recovers itself in case of system failure.

Examples:

• When an application is receiving data from the network, unplug the connecting cable. After some time, plug the cable back in and analyze the application’s ability to continue receiving data from the point at which the network connection was broken.
• Restart the system while a browser has a definite number of sessions open and check whether the browser is able to recover all of them or not

Availability: Test the degree to which a user can depend on the system during its operation. It involves exercising the system with a heavy number of users and measuring the performance parameters to verify whether the system can support the anticipated load (Stress tests).

Usability: Test the ease with which the user can learn, operate, prepare inputs and outputs through interaction with a system.

Scalability: Test if software application can expand its processing capacity to meet an increase in demand by rapidly and gradually increasing the load on the application until it reaches a threshold/limit.

• Acceptance testing is beta testing of the product done by

the actual end users.

• Here, only black-box Testing techniques are used.

• Acceptance testing is a test conducted to find if the

requirements of a specification or contract are met as per its deliver y.

• Acceptance testing is basically done by the user or

customer. However, other stakeholders can be involved in this process.

When new software is released then

• the need to test new functionality is obvious. Equally

impor tant is the need to re-run old tests that the application previously passed, to ensure that new software does not re-introduce old defects or create new side effects: called regressions.

• While regression testing is an apparently simple concept,

it can be quite challenging in practice due to limited resources such as time, personnel, and test systems.