📋 Software Testing

Comprehensive Guide to Quality Assurance

"Testing can only show the presence of errors, not their absence" - Dijkstra et al.

🎯 What is Software Testing?

Definition

Software Testing is intended to show that a program does what it is intended to do and to discover program defects before it is put into use.

The Testing Process Has Two Distinct Goals:

🎖️ Goal 1: Validation

Demonstrate to the developer and the customer that the software meets its requirements.

Requirements Met

🔍 Goal 2: Defect Testing

To discover situations in which the behavior of the software is incorrect or undesirable. Ensure that the software system is defect-free by removing undesirable faults.

Find Defects

📌 Main Focus Areas

  • Correctness: Software behavior with respect to requirements or intent
  • Performance: Software behavior under various conditions
  • Robustness: Ability to handle erroneous input and unanticipated conditions
  • Installation: Other facets of a software release

📚 Basic Definitions

Failure

A deviation of the observed behavior of a program or system from its specification.

Example: square(3) = 6 ❌

Fault/Defect

An incorrect step, process, or data definition in the code.

Example: return x * 2; instead of return x * x;

// Example: Fault in Code int square(int x) { return x * 2; // <-- FAULT: Should be x * x } // Test Cases square(2) = 4 ✓ Correct (by coincidence) square(3) = 6 ✗ Failure (reveals the fault)

✅ Verification and Validation (V&V)

Testing is part of a broader process of software verification and validation.

Verification

"Are we building the product right?"

The aim is to check that the software meets its stated functional and non-functional requirements.

  • Checks specifications
  • Internal quality
  • Process-oriented

Validation

"Are we building the right product?"

A more general process ensuring that the software meets the customer's expectations.

  • Checks customer needs
  • External quality
  • Product-oriented

🔬 Types of Testing

⬛ Black-Box Testing

Testing based on external specifications without knowledge of internal code structure.

Functional Requirements-Based

⬜ White-Box Testing

Testing based on internal code structure and implementation details.

Structural Code-Based

⬛ Black-Box Testing

Definition

Black-Box Testing is defined as a testing technique in which functionality/features of the Application Under Test (AUT) is tested without looking at the internal code structure, implementation details, and knowledge of internal paths of the software.

🎯 Key Characteristics

  • Based entirely on software requirements and specifications
  • Focuses on inputs and outputs without bothering about internal knowledge
  • Test case selection done before implementation
  • Helps get design and coding correct with respect to specification

📋 Black-Box Testing Process

1. Examine Requirements
→
2. Choose Valid/Invalid Inputs
→
3. Determine Expected Outputs

↓

4. Construct Test Cases
→
5. Execute Tests
→
6. Compare Results

🔧 Black-Box Testing Techniques

1️⃣ Decision Table Testing

A decision table puts causes and their effects in a matrix. There is a unique combination in each column.

Example: Login Screen

Condition: User must provide correct username AND password to access homepage. Any incorrect input shows error message.

Conditions Rule 1 Rule 2 Rule 3 Rule 4
Username (T/F) F T F T
Password (T/F) F F T T
Output (E/H) E E E H

Legend: T = Correct, F = Wrong, E = Error message, H = Home screen

2️⃣ Equivalence Class Partitioning

Used to minimize the number of possible test cases to an optimum level while maintaining reasonable test coverage. It divides the input data into different equivalence classes where system behavior can be considered the same.

Example: Order Pizza (1-10 pizzas allowed)

Instead of testing all possible values, divide into partitions:

  • Invalid: < 1 (e.g., test with -1)
  • Valid: 1-10 (e.g., test with 8)
  • Invalid: > 10 (e.g., test with 33)

3️⃣ Boundary Value Testing

Focused on the values at boundaries. This technique determines whether a certain range of values are acceptable by the system or not.

For range [1, 10]: • Invalid: 0 (just below minimum) • Valid: 1 (minimum), 2 (just above minimum) • Valid: 9 (just below maximum), 10 (maximum) • Invalid: 11 (just above maximum)

✅ Advantages of Black-Box Testing

  • Test cases can be designed early
  • No programming knowledge required
  • Tests from user's perspective
  • Helps verify specifications

Limitations

  • Limited code coverage
  • Can't test hidden functions
  • May miss logical errors

⬜ White-Box Testing

Definition

Test cases are derived from the internal design specification or actual code of the program. It involves using the source code to find every possible executable path.

🎯 Key Features

  • Tests internal details of the code
  • Checks all paths a program can execute
  • Requires programming knowledge
  • Performed by developers

🔍 Basis Path Testing

Purpose

Basis Path Testing is a structural testing method that involves using the source code to find every possible executable path and achieve maximum path coverage with the least number of test cases.

📊 Cyclomatic Complexity

A software metric used to measure the path complexity of a program. It's a quantitative measure of independent paths in the source code.

Formula 1: V(G) = E - N + 2 Where: E = Number of edges N = Number of nodes Formula 2: V(G) = P + 1 Where: P = Number of decision/predicate nodes

🔢 Complexity Interpretation

Complexity Meaning Testability
1-10 Structured and well-written code High
10-20 Complex code Medium
20-40 Very complex code Low
>40 Not testable Very Low

📝 Basis Path Testing Steps

Step 1: Draw Control Flow Graph

Create a graph with nodes and edges from the source code to determine different program paths.

Step 2: Calculate Complexity

Use V(G) = E - N + 2 or V(G) = P + 1 to determine the number of independent paths.

Step 3: Identify Basis Set

Identify the basis set of independent paths through the program.

Step 4: Generate Test Cases

Create test cases to execute all identified paths.

💡 Example: Finding Minimum Value

min = A[0]; i = 1; while (i < N) { if (A[i] < min) min = A[i]; i = i + 1; } print min
Cyclomatic Complexity Calculation: • Nodes (N) = 6 • Edges (E) = 7 • Decision nodes (P) = 2 V(G) = 7 - 6 + 2 = 3 V(G) = 2 + 1 = 3 Independent Paths (3 required): 1. [1 → 2 → 6] (empty array) 2. [1 → 2 → 3 → 5 → 2 → 6] (no update to min) 3. [1 → 2 → 3 → 4 → 5 → 2 → 6] (update min) Test Cases: 1. N=0, A=[] 2. N=2, A=[5,7] (ascending order) 3. N=3, A=[5,3,4] (min in middle)

🎯 Uses of Cyclomatic Complexity

  • Helps determine independent path executions
  • Ensures all paths tested at least once
  • Focuses on uncovered paths
  • Improves code coverage in testing
  • Evaluates risk associated with the application
  • Reduces risks of project failure when used early

⚖️ Black-Box vs White-Box Testing

Aspect Black-Box White-Box
Focus Functional requirements validation Internal structure and working
Knowledge No code knowledge required Programming language essential
Abstraction Abstraction from code Based on code structure
Communication Facilitates inter-module testing Does not facilitate communication
Performed By Testers Developers

🏗️ Levels of Testing

There are different levels of testing executed by different persons at different moments:

1️⃣ Unit Testing

Test Individual Components

Developer Level

2️⃣ Integration Testing

Test Integrated Components

Module Level

3️⃣ System Testing

Test the Entire System

System Level

4️⃣ Acceptance Testing

Test the Final System

User Level

1️⃣ Unit Testing

Unit Testing is where individual units or components of software are tested. The purpose is to validate that each unit of the software code performs as expected. A unit is the smallest testable portion of a system (individual function, method, procedure).

🎯 Why Unit Testing?

  • Skipping unit testing leads to higher defect fixing costs later
  • Helps fix bugs early in development cycle
  • Saves both time and money
  • Helps developers understand code base
  • Enables quick changes
  • Serves as project documentation
  • Helps with code reuse

📋 Code Review Types

Code Walkthrough

  • Semi-formal review
  • Author is presenter
  • Led by author only
  • Reviewers not prepared beforehand

Code Inspection

  • Formal review
  • Author is NOT presenter
  • Led by moderator
  • Reviewers well-prepared

2️⃣ Integration Testing

Integration Testing involves building a system from its components and testing it for problems that arise from component interactions. Different software modules are combined and tested as a group.

Key Principle

To simplify error localization, systems should be incrementally integrated.

🎯 Why Integration Testing?

  • Modules designed by different developers may have varying logic
  • Requirements may change during development
  • Database interfaces could be erroneous
  • External hardware interfaces could be faulty
  • Inadequate exception handling could cause issues

🔧 Integration Testing Approaches

1️⃣ Big Bang Approach

✅ Advantages

  • Convenient for small systems
  • All modules tested at once

❌ Disadvantages

  • Fault localization is difficult
  • Interfaces may be missed
  • Critical modules not prioritized

2️⃣ Incremental Approach

Testing done by joining two or more logically related modules, then adding related modules progressively.

🔑 Key Concepts: Stubs and Drivers

Stub: Dummy program that simulates a called module (used in top-down testing)

Driver: Dummy program that calls a module (used in bottom-up testing)

📊 Top-Down Integration

Testing from top to bottom following control flow

Uses: Stubs for testing

Pros: Early prototype, critical modules tested first

Cons: Needs many stubs, lower modules inadequately tested

📊 Bottom-Up Integration

Testing from bottom to top, each lower module tested with higher modules

Uses: Drivers for testing

Pros: Easier fault localization, no waiting for modules

Cons: Critical modules tested last, no early prototype

📝 Example Integration Test

Test Case ID Objective Description Expected Result
1 Login → Mailbox link Enter credentials and click Login Directed to Mailbox
2 Mailbox → Delete link Select email and click Delete Email in Trash folder

3️⃣ System Testing

System Testing validates the complete and fully integrated software product. It evaluates the end-to-end system specification.

🔍 Types of System Testing

Functional Testing

Test entire system against functional requirements

Requirements-based

Performance Testing

Test non-functional requirements (response times, load testing, etc.)

Performance-based

⚡ Non-Functional Testing

Testing non-functional aspects like performance, usability, reliability, etc.

🔒 Security

Protection from attacks

🔄 Reliability

Continuous function without failure

💪 Survivability

Recovery from system failure

📊 Scalability

Handle increased demand

🎯 Performance Testing

Checks speed, response time, reliability, resource usage, and scalability under expected workload.

  • Speed: Does the application respond quickly?
  • Scalability: Maximum user load the application can handle
  • Stability: Is the application stable under varying loads?

4️⃣ Acceptance Testing

Acceptance testing is beta testing of the product done by actual end users. It's a test to find if requirements of a specification or contract are met as per delivery.

🎯 Key Features

  • Only black-box testing techniques used
  • Conducted by actual end users
  • Other stakeholders can be involved
  • Final validation before deployment

🔄 Regression Testing

Definition

When new software is released, there's a need to re-run old tests that the application previously passed to ensure new software does not re-introduce old defects or create new side effects (regressions).

⚠️ Challenges

  • Limited resources (time, personnel, test systems)
  • Large number of test cases to re-execute
  • Balancing thorough testing with time constraints

🎯 When to Perform Regression Testing

  • After code fixes
  • After upgrades
  • After system maintenance
  • When new features are added

✨ Testing Best Practices

✅ Do's

  • Start testing early
  • Test throughout development
  • Automate repetitive tests
  • Document test cases thoroughly
  • Use both black-box and white-box
  • Involve end users early
  • Track and analyze defects

❌ Don'ts

  • Don't skip unit testing
  • Don't test without a plan
  • Don't ignore failed tests
  • Don't test only happy paths
  • Don't assume tests pass
  • Don't test in production only
  • Don't forget edge cases

📚 Summary

Key Takeaways

  • Testing is Essential: Critical for software quality and user satisfaction
  • Multiple Approaches: Black-box and white-box complement each other
  • Testing Levels: Unit → Integration → System → Acceptance
  • Early Testing: Catches defects early, reducing costs
  • Continuous Process: Testing throughout development lifecycle
  • Cyclomatic Complexity: Helps measure code testability
  • Test Coverage: Aim for comprehensive coverage of all paths

Testing Process Flow

Requirements
→
Unit Testing
→
Integration

↓

System Testing
→
Acceptance
→
Deployment