Testing Management

1. What is a test plan?

A test plan is a document that defines the scope, approach, resources, and schedule for the testing effort of a project. It is the blueprint for testing: what will be tested, how, with what, by whom, and by when.

The test plan covers the testing objectives, the scope (what is in and out of testing), the testing strategy (the levels and types of testing to be performed), test environments and tools, entry and exit criteria, roles and responsibilities, schedule and effort, risks, and the deliverables testing will produce. It answers the management questions — how much testing, at what cost, with what resources — and gives the testing effort a formal, agreed shape.

The test plan is created early, typically during the requirements or design phase, and reviewed and updated as the project evolves. It is the contract between testing and the rest of the project. A project without a test plan tests by improvisation — unclear scope, unmanaged resources, and no defensible answer to “when is testing done?“

2. What is a test strategy?

A test strategy is a high-level document that describes the overall approach to testing for the project — the big decisions about how testing will be done, at what levels, with which techniques, and how risk and resources will be balanced. It sets the direction; the test plan fills in the detail.

The strategy decides the testing levels (unit, integration, system, acceptance), the testing types (functional, performance, security, usability), the test design techniques (equivalence partitioning, boundary value, state-transition), the environment needs, the automation approach, and how testing aligns with risk — which areas get deep testing because they are risky.

The common interview distinction: strategy versus plan. The strategy is the approach — the “how we will test” philosophy, relatively stable and reusable across projects. The test plan is the project-specific execution detail — scope, schedule, resources, entry/exit criteria — derived from the strategy and unique to each project. The strategy answers “how will we test in general?” The plan answers “what exactly will we do in this project, when, and with what?“

3. What is the difference between a test plan and test strategy?

The test strategy defines the approach — the high-level decisions about how testing will be performed, including the levels and types of testing, the techniques, the environment, and the automation philosophy. It is relatively stable, often reusable across projects, and answers “how will we test?”

The test plan is the project-specific execution document — it takes the strategy and fills in the detail: the exact scope, the schedule, the resources and roles, the test environment, entry and exit criteria, risks, and deliverables. It answers “what exactly will we do in this project, when, and with what?”

A useful way to separate them: the strategy is the policy, the plan is the practice. The strategy is set once and stays consistent; the plan is created per project and updated as the project evolves. The strategy can exist without a specific project; the test plan is meaningless without one. In interviews, lead with the strategy-as-approach, plan-as-detail distinction.

4. What are the contents of a test plan?

A standard test plan follows the IEEE 829 format and contains: the test plan identifier; an introduction with objectives; the scope of testing — what is in and out; and the testing strategy — the levels, types, and techniques to be applied.

It also specifies the test environment and the hardware/software/tools needed, the schedule and effort estimates, roles and responsibilities, and the entry criteria (when testing can begin) and exit criteria (when testing can be declared complete). The plan lists risks and their mitigations, the deliverables testing will produce — test cases, reports, logs — and the procedures for defect tracking and reporting.

Beyond the boilerplate, the test plan’s real content is the management decisions: how much testing is enough, which areas get priority, what the test team needs, and how defects are handled. A good test plan is specific enough to guide execution and reviewable enough to be challenged. A test plan full of generic boilerplate is a checkbox; the value is in the concrete choices it records.

5. What are entry criteria?

Entry criteria are the conditions that must be satisfied before testing can begin — the gate that testing passes through to start. They define “is the software ready to be tested?”

Typical entry criteria include: the build is available and deployable, the requirements and design documents are complete enough to test against, the test plan and test cases are ready, the test environment is set up and validated, and the necessary tools and data are in place. Some teams also require that unit testing has been completed with acceptable results before system testing begins.

Entry criteria protect testing from wasted effort. Testing a build that is not deployable, or testing against requirements that keep changing, burns the test budget on work that has to be redone. The gate forces the project to prepare before testing invests. Entry criteria are the mirror of exit criteria — entry says when testing may start, exit says when testing may stop. Both are defined in the test plan and agreed with stakeholders.

6. What are exit criteria?

Exit criteria are the conditions that must be satisfied before testing can be declared complete — the gate that defines “when is testing done?” They protect the project from shipping untested software and prevent testing from running forever.

Typical exit criteria include: all planned test cases have been executed; the defined levels of coverage have been met — requirements coverage, code coverage, or both; all high-priority and critical defects have been fixed and retested, and the remaining defects are documented and accepted as known issues; the number of open defects is below an agreed threshold; and the test report has been reviewed and signed off.

Exit criteria are the answer to the project’s most difficult question — “are we done?” — and they must be agreed up front, not improvised at the end. Without them, testing either stops too early (because the schedule pressure wins) or continues indefinitely (because nothing defines enough). Good exit criteria are objective and measurable, tied to risk — the critical defects must be gone; the nice-to-have polish can be deferred with stakeholder agreement.

7. What is test coverage?

Test coverage measures how much of the software or its requirements has been exercised by testing. It answers “how much did we test?” and is expressed as a percentage — the proportion of the total that was actually covered.

Coverage is measured against different bases. Requirements coverage is the percentage of requirements that have at least one test case and have been executed. Code coverage is the percentage of code — statements, branches, conditions, paths — executed by the test suite. Risk coverage tracks the proportion of identified risks that have been mitigated by testing.

Coverage is the primary quantitative answer to “is testing complete?” But it must be interpreted with care. High coverage is necessary but not sufficient — testing 90% of the statements tells you nothing about whether the 10% of uncovered paths hide a critical defect, and even 100% coverage does not guarantee correctness. Coverage tells you what you have looked at; it does not tell you that the look was deep enough. The discipline: define coverage targets tied to risk, measure them honestly, and use them as inputs to the exit criteria, not as a substitute for judgment.

8. What is requirements coverage?

Requirements coverage measures the percentage of requirements that have been tested — that have at least one test case mapped to them, and that the test has actually been executed. It answers “did we test everything we agreed to build?”

The measure is computed from the traceability matrix: each requirement is linked to its test cases, and requirements coverage is the proportion whose tests have been executed and passed. It is typically tracked during execution — as tests run, requirements move from “tested” to “passed.”

Requirements coverage is the most important coverage metric for acceptance, because it ties testing directly to what was specified. It answers the question the project cares about: everything we promised is either proven working or has a known status. Its limitation is that it measures breadth, not depth — a requirement can have one weak test and count as covered. Good practice pairs requirements coverage with code coverage: requirements coverage proves the agreed functionality was tested; code coverage proves the test suite was thorough enough to exercise the implementation.

9. What is code coverage?

Code coverage measures the percentage of the source code that is executed by the test suite. It answers “how much of the implementation did our tests actually run?” and is measured by instrumenting the code and tracking which parts execute during the tests.

There are several levels. Statement coverage measures the proportion of executable statements run. Branch coverage measures the proportion of decision outcomes (both true and false) taken. Condition coverage measures the proportion of boolean conditions tested both ways. Path coverage measures the proportion of logical paths through the code. Each level is stricter and more expensive than the last.

Code coverage is a white-box measure — it requires the source and tooling. Its purpose is to find the untested parts of the implementation — branches, error paths, edge conditions — that black-box testing misses. The caution is the coverage paradox: 100% statement coverage says nothing about the correctness of the tests, and high coverage can be gamed with assertions that check nothing. The discipline is to use coverage as a finding tool (which code is untested?) and a minimum standard, not as proof of quality. A common target is 80–90% branch coverage for critical code, with the risky paths covered fully.

10. What is a Requirements Traceability Matrix?

A Requirements Traceability Matrix (RTM) is a document that maps requirements to their corresponding test cases — and often further to design and code — so that every requirement can be traced from origin to implementation to verification. It is the spreadsheet that connects what was asked to what was built to what was tested.

The matrix has requirements on one side and test cases on the other: each row links a requirement (with its ID) to the test case or cases that verify it. Extended matrices also link to design documents and code modules. The mapping works in both directions: every requirement has tests proving it, and every test traces back to a requirement it exists to verify.

The RTM is built during test design and maintained throughout the project. It provides the evidence for coverage — untested requirements are visible at a glance — and it is the backbone of change management: when a requirement changes, the matrix shows exactly which tests must be updated. In regulated industries, the RTM is mandatory audit evidence. Its cost is maintenance — it is only valuable if it stays current.

11. Why is an RTM important?

The Requirements Traceability Matrix is important because it gives testing (and the whole project) four things: coverage, evidence, change management, and accountability.

First, coverage: the RTM makes untested requirements visible. If a requirement has no test case, or its test has not been executed, the gap is visible in a column of the matrix. It is the quantitative proof of “have we tested everything we agreed?” Second, evidence: it provides the auditable trail linking each requirement to the test proving it — the documentation regulators and certification bodies require.

Third, change management: when a requirement changes, the matrix immediately shows which test cases, design elements, and code are affected, so the ripple is controlled rather than discovered later. Fourth, accountability: each requirement has tests, and each test has a requirement, so nothing is built without justification and nothing is verified without purpose.

The RTM’s importance in interviews: it is the tool that makes the three biggest testing questions answerable — Are we covering everything? What breaks when a requirement changes? Can we prove we tested what we agreed?

12. How do you prioritize test cases?

Test cases are prioritized so the most important testing happens first, protecting the project when time runs out. The ranking balances three things: the risk associated with the feature, the impact of a defect there, and the likelihood of a defect occurring.

High-priority test cases cover: critical functionality (anything whose failure stops the business — login, payment, data integrity); high-risk, frequently used areas; requirements the customer cares most about; and the results of risk analysis — features with complex logic, recent changes, or past defect history. These are tested first and exhaustively. Lower priority goes to rarely used, low-risk, or nice-to-have functionality.

The prioritization uses the risk-based approach: likelihood of failure multiplied by impact of failure. Test cases for high-risk, high-impact areas are P1; low-risk, low-impact areas are P3. The purpose is ruthless pragmatism — if the schedule shrinks, the team can drop the P3 tests confident that the critical paths are already proven. Prioritization is also about sequencing, not just dropping: the top tests run first in the cycle, so if anything is cut, it is the least important coverage.

13. What is risk-based testing?

Risk-based testing is the strategy of focusing the testing effort on the areas of highest risk — where the likelihood of a defect is high, the impact of a defect is severe, or both. It allocates testing resources in proportion to risk: high-risk areas get deep, extensive testing; low-risk areas get light coverage.

The approach begins with a risk analysis: for each feature or requirement, assess the likelihood of failure (complexity, newness, history of defects, frequency of change) and the impact of failure (business criticality, data sensitivity, user exposure). Risk = likelihood × impact. High-scoring features are tested first, most thoroughly, and with the strongest techniques; low-scoring features get smoke-level coverage.

Risk-based testing is valuable because resources are always limited — no project can exhaustively test everything, so the question is where to spend the effort. It aligns testing with what matters most: the areas where a failure would hurt. It also drives prioritization of test cases and exit criteria — a defect in a high-risk area blocks release; a defect in a low-risk area may not. The interview distinction to make: risk-based testing is not “test the hardest parts” but “spend testing budget where failure hurts the most.”

14. What are test metrics?

Test metrics are quantitative measures that track the testing effort, its effectiveness, and the quality of the product being tested. They convert the testing activity into numbers that can be monitored, compared, and used for decisions.

Metrics fall into categories. Effort and progress metrics: test cases written, executed, passed, failed, and blocked — showing how testing is progressing against the plan. Defect metrics: total defects found, defect density, defects found per phase, defect leakage, defects by severity, open defect trends. Coverage metrics: requirements coverage and code coverage. Effectiveness metrics: defect detection percentage, test case effectiveness, escaped defect rate.

The purpose of metrics is management visibility and improvement. They answer questions like: is testing on schedule? Is the product getting healthier? Where do defects cluster? Are we testing the right things? The discipline is choosing the metrics that drive decisions — a metric nobody acts on is noise — and interpreting them carefully. Metrics can be gamed (defect counts reward finding trivial bugs, coverage can be inflated with useless tests), so they are tools for insight, not targets to optimize blindly.

15. How do you measure testing effectiveness?

Testing effectiveness measures how well the testing effort finds defects and prevents them from reaching the customer. It answers “is our testing actually working?”

The core measures are defect-based. Defect Detection Percentage (DDP) is the proportion of all defects — found in testing plus escaped to production — that testing caught before release. Defect leakage measures the defects that escaped to production, usually normalized per thousand lines of code. A low leakage and high DDP means testing is effective. Defect removal efficiency extends this across phases — what proportion of defects was caught at each phase, which shows whether reviews and early testing are working.

Other measures include test case effectiveness (the percentage of test cases that found a defect) and test coverage achieved. Effectiveness is also judged by the cost angle: finding defects early is more effective than finding them late because the fix cost is lower. The interview nuance: testing effectiveness is about finding the right defects at the right time, not just finding many defects — volume of defects found can actually indicate poor early quality, while a low escape rate indicates effective testing.

16. What is defect leakage?

Defect leakage is the number of defects that escape the testing phases and reach the customer or production environment. It is the measure of the testing process’s biggest failure — bugs that the customer finds instead of the testers.

Defect leakage is usually expressed as a rate — defects found in production per thousand lines of code (KLOC), or as the percentage of all defects that escaped. The formula: leakage = defects found in production / (defects found in testing + defects found in production) × 100. A high leakage rate means testing missed too much; a low one means testing is catching defects before release.

Defect leakage matters because production defects are the most expensive — they affect real users, require hotfixes and incident response, damage trust, and cost far more than the same defect caught in testing. Analyzing leakage — where the escaped defects came from, which phase should have caught them — drives process improvement: better reviews, better test design, better coverage. The interview point: leakage is the honest measure of testing effectiveness, and analyzing it is more valuable than just counting it.

17. What is defect density?

Defect density is the number of confirmed defects divided by the size of the software — typically defects per thousand lines of code (KLOC) or per function point. It measures how many bugs exist in a given amount of code.

The formula is simple: defect density = total defects / software size. Its value is comparative and predictive. It lets a team compare quality across modules — the module with the highest defect density needs attention or rewrite. It can be compared against the organization’s historical baseline — is this project cleaner or buggier than usual? And industry and internal norms give a sense of whether the product is healthy.

Defect density must be interpreted with care. A low defect density could mean good code — or poor testing that never found the bugs. A high density in a small, complex module is different from a high density across the whole codebase. It also punishes small modules and rewards trivial fixes. The interview point: defect density is a relative quality indicator, meaningful for comparison and trend, not an absolute measure of goodness — it must be paired with testing effectiveness measures like leakage.

18. What is test case effectiveness?

Test case effectiveness measures how productive the test suite is at finding defects — the proportion of test cases that actually detect a defect. It answers “are our test cases any good?”

The simplest form: test case effectiveness = (number of test cases that found at least one defect / total number of test cases executed) × 100. A low percentage means the test suite is mostly confirming working behavior — useful as regression coverage but inefficient as defect detection. A high percentage means the tests are finding real problems.

The nuance is that “effective” does not mean “as high as possible.” A suite where every test finds a defect is often a suite testing a broken product. The metric is best used over time: newly written test cases should have a high detection rate (they are targeting new or risky functionality), while the overall suite’s rate falls as it matures and mainly does regression protection. The interview point: test case effectiveness measures the detection power of individual test cases, and it must be balanced against coverage — a suite that finds no defects might still be doing its job preventing regressions.

19. How do you determine whether testing is complete?

Testing is complete when the exit criteria are met — the conditions agreed in the test plan have been satisfied. The answer is never “when we run out of time” or “when we feel like it”; it is an objective, agreed decision.

The exit criteria typically combine several measures: all planned test cases have been executed; requirements coverage and code coverage have reached their agreed targets; all critical and high-priority defects are fixed and retested, and the remaining open defects are documented and accepted by the stakeholders; defect leakage risk has been assessed; and the test report has been reviewed and signed off.

The interview nuance: completion is a risk-based decision, not a mathematical certainty. Testing can never prove the absence of defects — it can only reduce risk to an acceptable level. So “complete” means the agreed level of risk has been reached: the critical paths are proven, the coverage targets are met, the open defects are known and accepted. Determining completion also involves the cost-benefit balance — the point where finding another defect costs more than the defect would cost — which is why exit criteria are agreed before testing starts, not improvised at the end.

20. How do you decide which test cases to automate?

Test cases are selected for automation based on their stability, frequency, and value — automation is an investment, and it should be spent where it pays back.

The candidates for automation: test cases that run repeatedly — regression suites that re-run after every change; test cases that are stable — functionality that is not changing, so the test does not need constant rewriting; core and critical paths that must never break; tests that are slow or laborious to run manually; and tests needed in continuous integration — they must run on every build, automatically, fast.

The cases to keep manual: new functionality still being explored; complex usability and visual testing that needs human judgment; exploratory and ad-hoc testing by nature; one-off tests; and tests whose expected behavior is still changing. Automating unstable tests is a treadmill — the maintenance cost exceeds the savings.

The key measure is the cost-benefit ratio: automation costs development and maintenance; it pays back through repetition. A test that runs once is a poor automation candidate; a test that runs a hundred times is ideal. The interview point: automate the stable, repetitive, critical tests; keep the unstable, exploratory, judgment-based testing manual. And automation replaces the execution of tests, not the test design — the test cases are designed first, then automated.

12. What is the defect lifecycle?

The defect lifecycle (bug lifecycle) tracks a bug from discovery to closure through a defined set of states:

1. New — a tester logs the defect with a clear description, steps to reproduce, expected vs actual result, and severity/priority.
2. Open / Assigned — the bug is triaged and assigned to a developer.
3. Fixed — the developer resolves the root cause and marks it fixed (optionally with statuses like In Progress, Deferred, Rejected/Not a Bug, Duplicate along the way).
4. Retest / Verified — the tester verifies the fix against the original test case.
5. Closed — the fix is confirmed; or Reopened — if the bug persists, the defect cycles back to Open.

New → Open → Fixed → Verified → Closed
                 ↕ (Reopened if the fix fails)

Along the way defects can be Deferred (valid but low priority, pushed to a later release), Rejected (not actually a bug / by design), or Duplicate (already reported).

Two terms to distinguish in interviews: Severity — how impactful the bug is (critical/major/minor, e.g. a crash vs a typo); Priority — how urgently it must be fixed (immediate/high/low, driven by business value). A cosmetic defect on the login page can be high priority (blocks a demo) yet low severity.

The interview answer: the defect lifecycle is the controlled journey of a bug (new → open → fixed → verified → closed, with reopen loops), and a defect is only truly “closed” after the tester independently verifies the fix.