Agile Testing - Quadrants

As in the case of Traditional Testing, Agile Testing also need to cover all the Test Levels.

• Unit Testing
• Integration Testing
• System Testing
• User Acceptance Testing

Unit Testing

• Done along with Coding, by Developer
• Supported by Tester who writes Test Cases ensuring 100% Design Coverage
• Unit Test Cases and Unit Testing results need to be reviewed
• Unresolved major defects (as per priority and severity) are not left
• All Unit Tests are automated

Integration Testing

• Done along with Continuous Integration as the Sprints progress
• Done at the end after all the Sprints are completed
• All Functional Requirements are tested
• All Interfaces between Units are tested
• All the Defects are Reported
• Tests are automated where possible

System Testing

• Done as the Development progresses
• Users Stories, Features and Functions are Tested
• Testing done in Production Environment
• Quality Tests are executed (Performance, Reliability, etc.)
• Defects are reported
• Tests are automated where possible

User Acceptance Testing

• Done at the end of each Sprint and at the end of the project
• Done by the Customer. Feedback is taken by the Team
• Feedback will be an input to subsequent Sprints
• User Stories in a Sprint are pre-verified to be testable and are with defined Acceptance Criteria

Test Types

• Component Tests (Unit Tests)
• Functional Tests (User Stories Tests)
• Non-functional Tests (Performance, Load, Stress, etc.)
• Acceptance Tests

Tests can be fully Manual, fully Automated, Combination of Manual and Automated or Manual supported by Tools.

Support Programming and Critique Product Tests

Tests can be for −

• Supporting Development (Support Programming) − Support Programming Tests are used by the Programmers.
  • To decide on what code they need to write to accomplish a certain behavior of a System
  • What Tests need to be run after Coding to ensure the new Code does not hamper the rest of the behaviors of the System
• Verification only (Critique Product) − Critique Product Tests are used for discovering inadequacies in the finished Product

Business Facing and Technology Facing Tests

To decide on what tests to be performed when, you need to determine whether a test is −

• Business Facing, or
• Technology Facing

Business Facing Tests

A Test is a business-facing test if it answers the questions framed with words from business domain. These are understood by the business experts and would interest them so that behavior of the system can be explained in the real time scenario.

Technology Facing Tests

A Test is a technology-facing test if it answers the questions framed with words from technology domain. The programmers understand what needs to be implemented based on the clarifications on technology.

These two aspects of test types can be viewed using the Agile Testing Quadrants defined by Brian Marick.

Agile Testing Quadrants

Combining the two aspects of Testing Types, the following Agile Testing Quadrants are derived by Brian Marick −

The Agile Testing Quadrants provide a helpful taxonomy to help teams identify, plan and execute the testing needed.

• Quadrant Q1 − Unit Level, Technology Facing, and supports the developers. Unit tests belong to this Quadrant. These tests can be Automated tests.
• Quadrant Q2 − System level, business facing, and conform product behavior. Functional tests belong to this quadrant. These tests are either manual or automated.
• Quadrant Q3 − System or User Acceptance Level, Business Facing and focus on real time scenarios. User Acceptance Tests belong to this quadrant. These tests are manual.
• Quadrant Q4 − System or Operational Acceptance Level, Technology Facing and Focus on Performance, Load, Stress, Maintainability, Scalability Tests. Special tools can be used for these tests along with automation testing.

Combining these, the Agile Testing Quadrants that reflect What-Testing-When can be visualized as follows −

Agile Testing - Scrum

Scrum advocates Whole Team Approach, in the sense that every team member has to take part in every project activity. Scrum team is self-organizing with accountability to the project deliverables. Decision-making is left to the team that results in appropriate actions being taken at the right time without any time delays. This approach also encourages proper use of the team talent instead of restricting to one activity. Testers also participate in all the project and development activities contributing their expertise in testing.

The whole team works together on Test Strategy, Test Planning, Test Specification, Test Execution, Test Evaluation, and Test Results Reporting.

Collaborative User Story Creation

Testers participate in User Story Creation. Testers contribute their ideas on possible behavior of the system. This helps in the customer and/or end user understanding the system in the real environment and thus getting clarity on what actually they want as the outcome. This results in faster freezing of requirements and also reduces the probability of changes in the requirements later on.

Testers also come up with the Acceptance Criteria for every scenario agreed by the customer.

Testers contribute to the creation of testable user stories.

Release Planning

Release Planning is done for the entire project. However, Scrum framework involves iterative decision making as more information is obtained in the due course of executing sprints. Hence, Release Planning session at the beginning of the project need not produce a detailed release plan for the entire project. It can be updated continually, as relevant information is available.

Every sprint-end need not have a release. A release can be after a group of sprints. The main criteria of a release is to deliver business value to the customer. The team decides on the sprint length with release planning as an input.

Release Planning is the basis of test approach and test plan for release. Testers estimate Test Effort and plan Testing for the release. When release plans change, testers must handle changes, obtain an adequate test basis considering larger context of release. Testers also provide testing effort that is required at the end of all the sprints.

Sprint Planning

Sprint planning is done at the beginning of each sprint. The sprint backlog is created with the user stories picked up from the product backlog for implementation in that particular sprint.

The testers should −

• Determine the testability of the user stories selected for the sprint
• Create acceptance tests
• Define test levels
• Identify test automation

Testers update the test plan with the estimates for testing effort and durations in the sprint. This ensures provision of time for required testing during the time-boxed sprints and also accountability of the testing effort.

Test Analysis

When a sprint begins, as the developers carry on story analysis for design and implementation, testers perform test analysis for the stories in the sprint backlog. Tester create the required test cases – both manual and automated tests.

Testing

All the members of the Scrum team should participate in testing.

• The developers execute the unit tests as they develop code for the user stories. Unit Tests are created in every sprint, before the code is written. The unit test cases are derived from low level design specifications.
• The testers perform functional and non-functional features of the user stories.
• The testers mentor the other members in the scrum team with their expertise in testing so that the entire team will have a collective accountability for the quality of the product.
• At the end of the sprint, customer and/or end users carry out User Acceptance Testing and provide feedback to the scrum team. This forms as an input to the next sprint.
• Test results are collected and maintained.