Inspection

Goal

An inspection exists to take defects out of a work product before they get expensive, and BABOK gives it that focus by name: to remove defects and create a high quality work product (§10.37.3.2). It adds in the same sentence the nuance that has to be kept: an inspection is usually performed by peers, and it can also be used for stakeholder reviews. It is the most formal and most systematic of the peer reviews, Wiegers writes, and it is the only one he says has been identified as a software industry best practice.

What an inspection buys has to be paid for, and Wiegers makes the arithmetic explicit. Defects are not spread evenly through a product: one study he cites found 58% of customer-reported defects in 31 modules out of 425 and another that roughly 20% of modules hold roughly 80% of the defects. His conclusion is the technique's selection rule: use inspections for high-risk work products and rely on cheaper techniques for components whose risk is lower. BABOK says the same thing from the other side, early identification of defects removing the need for the expensive removal of defects discovered later in the life cycle (§10.37.4.1).

The figures he reports give the order of magnitude of the gain. A telecommunications company was finding 16 to 20 defects per thousand lines of code by inspection, against three when the review was run informally. At Ford, inspections found 50% more defects per thousand lines than walkthroughs. And on a requirements document that had already circulated several times among reviewers, a formal inspection of the assembled text found an average of 4.5 further defects per page. A pass around takes the easy errors out quickly and cheaply; the inspection that follows is paid for on the defects that remain.

Wiegers' objectives table compares six review methods across fourteen objectives, and it marks eight of them for the inspection, more than for any other method (Table 3-3). The formal walkthrough carries seven, the informal walkthrough seven as well, and they are not the same seven.

• Find product defects.
• Check conformance to a specification.
• Check conformance to a standard.
• Verify that the product is complete and correct.
• Assess the understandability and maintainability of the document.
• Demonstrate the quality of a critical or high-risk component.
• Collect data for process improvement.
• Measure document quality.

The two rows in bold, Wiegers marks for none of the five other methods. The first is the objective that justifies the expense: a critical component is inspected so that it can be established, and shown, that it holds. The second requires counting, and it is to the inspection alone that Wiegers reserves the measurement of a document's quality (Table 3-3); it is also the only one for which he marks data collection and analysis without reservation (Table 3-2: yes, where the team review gets a "maybe").

What an inspection leaves behind is a closed defect log, a product appraisal and measurements. The log and the appraisal come out of the session; the closing of the log comes out of the follow-up. The formal walkthrough pronounces an appraisal and stops there (Wiegers, Table 3-1: correction, yes; verification, no).

Usage

When to use it

• A high-risk work product: a defect discovered later is expensive to remove (BABOK §10.37.4.1).
• A quality gate or an approval has to rest on evidence: only a documented and verified review leaves any.
• Conformance to a standard has to be demonstrated: Wiegers' Table 3-3 marks that objective for the inspection.
• Defect metrics are expected: collecting data and measuring document quality are among its objectives (Table 3-3).
• The product will be a base for others: a reused component, a template, a reference model, a load-bearing architecture.

When not to use it

• The product is still a draft and still moving: an inspection assumes a stable artefact. An informal walkthrough has the author run the group through the draft and collects feedback.
• Low risk or a cheap decision: the most expensive review on the spectrum will not repay itself there. A desk check puts one competent reader in front of the document, a pass around puts several, each on their own.
• No trained moderator, no trained team, no checklist: the technique rests on all three, and without them it produces a walkthrough's result at an inspection's price. A formal walkthrough needs neither a reader nor trained inspectors.

Description

The roles

BABOK gives the inspection four roles (Table 10.37.1). The author answers questions, listens and incorporates the changes after the session. The reviewer examines the product against the review's objectives, and BABOK spells out what that means in a defect-detection review: the reviewer examines the work product before the session and keeps track of the defects found as well as suggestions for improvement. The facilitator, which is BABOK's word in that table where its prose says moderator once, runs the session. The scribe documents all defects, suggestions, comments, issues, concerns and outstanding questions raised during the session.

Wiegers, IEEE and Fagan say moderator. BABOK writes that this neutral facilitator should not be the author, in order to avoid compromising the review. The rule is harder than that with the authorities of the practice: Wiegers writes that in an inspection the author is not permitted to serve as the moderator, and IEEE 1028-2008, which defines the inspection by name, distributes five roles (inspection leader, recorder, reader, author, inspector) and holds that the author shall take none of the first three.

The reader is the role that separates the inspection from the whole rest of the family. The material is presented to the team by a participant other than the author, one small portion at a time. Wiegers' reason for it is mechanical: the reader helps the team reach the same interpretation of each portion of the product, because each inspector can compare their own understanding to the one the reader expresses. An author walking through his own document says what he meant to say. A reader says what is written, and the gap between the two is a defect. A review where the author presents is an informal walkthrough, whatever word is printed on the invitation.

The entry criteria

Wiegers lists explicit entry and exit criteria among the characteristics of the most formal reviews. Concretely: the work product is complete and stable; it is numbered by line or by page, failing which a defect cannot be located; the defect checklist exists; the inspectors are assigned and trained; and the moderator has confirmed the material is ready. BABOK gives the facilitator a duty that is an entry criterion under another name: they validate that reviewers have examined the work product before the session begins (Table 10.37.1).

Fagan's six operations

The stage lists of the inspection do not overlap, and each has to be named to its source rather than melted into the others. BABOK names five in its definition (overview, individual review, logging the defects, team consolidation, follow-up). Wiegers' Table 3-1 marks five others, which are activities and not stages (planning, preparation, meeting, correction, verification). His prose describes seven, adding causal analysis. The breakdown used is Fagan's, who defined the technique in 1976 and whose six operations are the ones the rest of the practice took up.

1. Planning
   The moderator picks the work product and the team, sizes the chunks, sets the pace and schedules the sessions. The pace is a parameter: past a few pages an hour the defect yield collapses.
2. Overview
   The author gives the team the context of the product. Fagan makes an operation of it, BABOK names it first among its five activities, and it is skipped when the team already knows the domain.
3. Individual preparation
   Each inspector examines the product alone, against a defect checklist and records what they find. Wiegers writes that inspections rely on checklists of defects commonly found in different types of work products, and his Table 3-2 marks the use of those checklists for the inspection. This is the operation that does most of the work and it is the first one to be sacrificed. The checklist is what gives that reading its shape; without it, the inspector reads with whatever is in his head.

   The checklist is built in-house. A generic list taken from elsewhere describes another organisation's mistakes. The one that is worth having is built from the logs of past inspections: the defects already found are grouped by type and the ones that recur are kept. Each line kept becomes a question the inspector puts to the document. "Does every capped amount cite the article that sets it?" "Does every age band say whether its boundary is included?" "Is every rate dated?" A question the document does not answer is a defect, and a question that has found nothing in three inspections comes off the list. Maintenance runs on the same beat: after each session the moderator pours the log into the corpus, and a defect that has appeared three times without being on the list goes onto it. A list that never changes has stopped describing what the organisation gets wrong. Of the seven reviews, the inspection is the one whose definition names the logging of defects (§10.37.3.2) and therefore the only one that leaves behind the corpus this work feeds on.

4. The inspection meeting
   The reader walks the product in small chunks (Table 3-2: granularity of material presented, small chunks); the inspectors raise what they found; the scribe logs each point. The session logs, and the correction happens afterwards. The session ends on three things: the defect log, the product appraisal (Table 3-2: product appraisal determined, yes) and the disposition of the verification.
5. Rework
   The author corrects, alone, outside the session, and he corrects against the log: every line gets a change or the written reason for not getting one. A line the author disputes goes back to the moderator; it does not evaporate. This is where the separation between finding and repairing pays out: the room logged without discussing solutions, so the author has the time, the context and the quiet to find them.
6. Follow-up
   The moderator carries out the disposition the session settled on, line by line, which closes the log.

The rework and its verification

This is where the technique stands apart, and the exact wording matters, because the short version is false. What the follow-up requires is decided in the session, by the team and recorded. Wiegers writes it without ambiguity: at the end of the inspection meeting the team agrees on an appraisal of the work product and judges whether the changes the author will make during rework must be verified through a second inspection, by a single person or not at all. Fagan says the same: a trivial correction may be accepted without verification, and a non-trivial case calls for a full re-inspection by the team.

The verification of the rework is therefore a disposition, settled case by case. What the inspection has of its own: it is the only one of BABOK's seven reviews whose definition carries a follow-up stage, the question of verification is asked in it, settled and assigned, and the log closes on the answer. Elsewhere the question stays outside the technique. A finding from a formal walkthrough that has to survive the session goes into item tracking, and it is another tool, held by somebody else, that carries it. In an inspection the finding does not leave the technique before it is closed.

What the moderator does under each of the three dispositions

Under re-inspection, he reconvenes the team or the part of it that raised the defects concerned, on the reworked portions only: the reader walks through what changed, the team checks that the correction settles the defect and introduces no other, then the lines concerned close in session. Under verification by a single person, he names the verifier, most often himself, sometimes the inspector who found the defect, and that person compares the reworked version to the log, line by line, then signs off. Under accept as is, there is nothing to verify: the line closes on the change the author declares, and the disposition itself is the record of what the team judged. In all three cases the line ends up carrying a verifier, a date and a state, and a line missing any of the three is open, whatever the author says about it.

The exit criteria fit in one sentence: the defect log is closed out according to the disposition settled in the session, and the product appraisal stands.

The measurements

Wiegers' Table 3-2 marks the row data collected and analyzed for the inspection: yes. For the team review, which is BABOK's formal walkthrough, he writes maybe. For the walkthrough, which is the informal walkthrough, no. His Table 3-3, which is a table of objectives and not of activities, marks collect data for process improvement for two methods only, the inspection and the team review, and it reserves measure document quality for the inspection alone. Those two rows sit in two different tables and they say two different things: one is a characteristic of the technique, the other a reason to convene it.

What can be computed, from the log and the attendance sheet: the number of defects per page, the defect density by severity, the inspection rate in pages per hour and the effort per defect found. An organisation that inspects three comparable documents owns a basis for comparison; one that does not inspect owns impressions. It is the third reason to inspect that Wiegers cites from IEEE: to provide metrics on defects and inspection effort that can lead to improvements in both the inspection process and the organisation's engineering process. Wiegers also adds a seventh stage to his own inspection process, causal analysis, which is what one does with those measurements: work out why the defects found came to be written.

Where the inspection sits

BABOK lists its seven review types by decreasing formality and calls the first three formal techniques, the next four informal techniques (§10.37.3.2). Two binary questions run across that ordering: is the session planned, and does the reviewer examine the product alone before speaking? The inspection answers yes to both, and there it is indistinguishable from the formal walkthrough and the single issue review. A third question separates them: does the technique's definition carry a follow-up stage on the rework? Of BABOK's seven definitions, only one does.

The third question rests on an elimination. Wiegers' Table 3-1 marks verification for the inspection and refuses it to the team review, to the walkthrough, to the peer deskcheck grouped with the passaround and to the ad hoc review. His spectrum carries no row for the single issue review, which comes out of the IEEE and ISO technical-review tradition. Cover for that cell is therefore BABOK's: each of its seven types gets a one-sentence definition, and the follow-up stage appears in exactly one of them. That is a fact about BABOK's text, which says nothing about the other six on this point. Wiegers' Table 3-1 also marks verification for pair programming, which is not one of BABOK's seven reviews and which Wiegers himself puts outside the category.

The pitfalls

Fixing defects in the session

The session logs, and the first interesting defect carries off the hour if the moderator lets it. The price of an inspection is the price of everyone in the room at the same time, and discussing a correction is exactly the work the author will do alone, later, more cheaply.

Inspectors who did not prepare

BABOK gives the facilitator the duty of validating that reviewers have examined the work product before the session begins (Table 10.37.1), and it is the only task in the technique that happens before the room fills. Four inspectors discovering the document in session produce a first collective read, at an inspection's price.

Inspecting the author

BABOK states the rule in one sentence: each review is focused on a work product, not on the skills or actions of the participants (§10.37.2). An inspection that turns into an assessment of a person stops producing defects, because nobody reports any.

The senior in the room

BABOK writes that when a supervisor or a manager is a reviewer, the moderator is careful to avoid adversely affecting the level of candour of the other participants or inappropriately affecting the decisions of the team (§10.37.3.3). The wording is cautious and it stays cautious here: it is a vigilance of the moderator's, and it is for him to judge whether the presence is workable.

Too many pages an hour

The pace is the parameter that silently destroys the yield. An inspection that swallows thirty pages in two hours has cost what an inspection costs and returned what a proofread returns.

Skipping the follow-up

The one operation that distinguishes the technique is the one that falls under schedule pressure.

Letting the author present

Wiegers opens his chapter on an organisation that had turned its walkthrough process into an inspection process by globally replacing the word walkthrough with the word inspection. The reader, the individual preparation, the checklist and the follow-up are what the word denotes. Without them it denotes nothing.

AI considerations

The inspection separates finding from deciding, and it is that separation which says where a language model helps and where it cannot go.

Individual preparation

It is the operation AI changes most. A model reading a specification against a defect checklist is doing exactly the mechanical half of that work: internal contradictions, terms used without being defined, a rule stated twice with two different thresholds, a missing boundary case, a figure the text does not match. It produces candidate defects, for nothing, before a human hour is spent. The inspector then spends his reading on what only a reader of the business can see.

Building the checklist from the history

An inspection leaves behind a corpus of logged defects, and it is the only corpus an organisation has for knowing what it gets wrong. A model groups the past logs, brings together the different wordings of one defect and surfaces what recurs, which is the work nobody does by hand between two inspections.

Consolidating and de-duplicating

Several inspectors log the same defect in different words, and merging those lines is drudgery the session does by hand.

Instrumenting the follow-up

Comparing the reworked product to the defect log, line by line, to show that every logged defect has a change against it, is a mechanical check. It is also the operation teams skip because it is tedious, which makes it the technique's best candidate.

What a model cannot do

The product appraisal is the judgement of a group of qualified people. The severity of a defect is contextual, and Wiegers says so: whether a defect is major or minor depends on its context and on the impact it would have if left uncorrected; a model does not know what the product controls. A candidate defect produced by a model stays a candidate, and it enters preparation, never the log directly: on a business rule, an LPP conversion rate or a coordination deduction, the model is confident and wrong where the reader is least equipped to contradict it. Nor can a model be the moderator or supply the property the technique is built on, which is an examination made by somebody who is not the author and who answers for having made it.

Data protection

A benefit-calculation specification carries internal rules and often personal data. It does not go off to a service that trains on what it is given, and the organisation's rule, which model, which data, hosted where, is settled before the product goes out to the inspectors.

Examples

What an inspection produces is a defect log whose last column closes.

Forty-one defects over the twelve pages inspected, which is 3.4 defects per page. Twelve pages covered in three hours of session, which is four pages an hour. Those two numbers are what none of BABOK's six other reviews can supply.

The inspection ends when the "Verification" column is closed on each of the 41 lines. The disposition was settled in the session, line by line, and it is not the same for all of them.

Visualizations

The formality spectrum is a spatial object: seven types placed on an axis, three marks per cell. The first two gates do not separate the inspection from its two formal neighbours, and the third isolates it alone. An axis and positions do not render as cells.

The inspection's process is made of boxes and arrows, and one of those arrows comes back where it came from: the follow-up hands the work product back to the team or releases it, according to the disposition settled in the session. A loop renders neither as prose nor as a table. The label on that arrow carries the three possible dispositions, which is the only way to show that the follow-up is a decision.

The defect log renders as cells, rows and columns. Its last column is what the technique produces and what no other review in the family produces: the six others return comments, a consolidated set of findings, a resolution, a marked-up copy or nothing at all. The choice between the seven belongs to the reviews technique.

Cost

Tooling

An inspection first requires a work product that can be located. A defect is logged at a place, and a document without line or page numbers does not give that place: the paginated export, or the tool that exposes stable anchors, is the technique's first condition. It is also what will let the follow-up find again what it has to verify.

It then requires a defect checklist, which a spreadsheet is enough to carry, and a defect log carrying the location, the defect, the severity, the rework and the verification. A shared spreadsheet does that too. What leaves the technique, an open question, a decision to be taken elsewhere, belongs to item tracking.

Finally it requires something to count with. The inspection's measurements are computed from the log and the attendance sheet, and a spreadsheet holds them while there are few. They are only worth anything compared, so the format is fixed at the first inspection and does not move again.

The specialised tools for an inspection are the tools of the documented reviews. A code review platform, a pull request comment thread, a document's revision thread: all three carry the same thing and return the same thing to an inspection. They give stable line-level anchors, and they give them without a paginated export. They hang the defect thread on the location of the defect rather than on a spreadsheet row: the argument about defect 12 is read next to the text it disputes, and the author reworking it sees what the inspector saw. And they carry a verification state that can be assigned, dated and closed, which is the technique's discriminator expressed as a tooling requirement: under re-inspection, the thread stays open until the second session; under verification by one person, it carries the verifier's name and the date of his sign-off; under accept as is, it closes on the author's change.

What those tools do not supply is the pace. A platform will take thirty pages in two hours, and it is for the moderator to size the chunks. The measurement sheet also stays to be kept alongside: a comment thread can say that a defect is closed, it cannot say how many defects per page the organisation finds.

Remotely, the technique holds on two conditions: the reader shares his screen and moves in small chunks at the agreed pace, and the log is visible to everybody during the session, so that each participant sees what the scribe wrote of what he said. Nothing in an inspection depends on a physical artefact.

Sources

• IIBA, A Guide to the Business Analysis Body of Knowledge (BABOK Guide) v3, §10.37 Reviews (§10.37.2, the object of the review; §10.37.3.1, the seven objectives; §10.37.3.2, the seven types and the definition of the inspection; §10.37.3.3, conducting the review; Table 10.37.1, the roles; §10.37.4, strengths and limitations).
• Karl E. Wiegers, Peer Reviews in Software: A Practical Guide, Addison-Wesley, 2002, chapter 3, "Peer Review Formality Spectrum" (Table 3-1, activities by review type; Table 3-2, compared characteristics; Table 3-3, objectives and methods; the characteristics of the most formal reviews, the roles, the reader, the disposition of the verification and the effectiveness data).
• M. E. Fagan, "Design and code inspections to reduce errors in program development", IBM Systems Journal 15(3), 1976, 182-211 (the six operations, the rework and the follow-up).
• IEEE, IEEE Std 1028-2008, IEEE Standard for Software Reviews and Audits (the definition of the inspection, its five roles and the exclusion of the author).
• ISO/IEC, ISO/IEC 20246:2017, Software and systems engineering: Work product reviews (the inspection among the standard's most rigorous review types).