Decision Matrix

Purpose

The decision matrix replaces a tacit preference with an explicit evaluation: you write down the options, you write down the criteria you will judge on, you score each option on each criterion, and the ranking falls out of an addition. A choice between several options gets made either way, with or without a method; without one, it goes to the option that speaks loudest in the room, to the last argument heard or to the criterion the decision maker happens to have in mind that day. Its value lies in what it forces into the open: the criteria you keep, the relative importance you grant them and the score you set against each option. Each of those three is a judgement, and the grid puts them on the table while they are still open to challenge.

The decision it supports is a choice between competing alternatives assessed on multiple and often conflicting criteria: the cheapest is rarely the safest, the most complete is rarely the easiest to adopt. The grid does not remove that trade-off, it makes it legible. It answers a precise question, which of these options best serves the whole set of criteria once their importance is declared, and it leaves a trace of how the answer was reached.

The deliverable is the filled grid and two things that make it defensible. The weights, written down, each with its reason: they carry the value judgement, and a grid whose weights have no known provenance cannot be reread. And the sensitivity analysis: does the ranking hold if you move the contested weights a little, or does it tip. A ranking that tips at the slightest adjustment has shown that the decision was close, which is also an answer.

Usage

When to use it

• Several viable options on conflicting criteria: cost against quality against risk, where none dominates on everything.
• Criteria of unequal importance: when the priority order changes the outcome, weighting is exactly what expresses it.
• A decision to justify to a committee or an audit: tender or vendor selection, where the choice must be traceable.
• Decision makers with diverging preferences: the grid separates disagreement on weights from disagreement on scores, and makes each trade-off explicit.
• A long list to trim: shortlist quickly before committing a heavier method to the two or three finalists.
• A recurring choice to structure: software, site or scenario selection, where the same frame will serve again.

When not to use it

• Weights themselves contested and needing to be defensible: when the importance of the criteria is the point in dispute, prefer the analytic hierarchy process, which derives weights from pairwise comparisons and tests their consistency.
• Two options and few criteria: the grid costs more than it returns, pro versus con analysis settles it faster.
• An adoption or resistance stake: with nothing to score but forces to weigh for and against a change, map it with force field analysis.

Description

The grid and the two forms of the total

The matrix is a table of options against criteria. One option per column or per row, a criterion on the other axis and in each cell a score that says how well the option satisfies the criterion, on a scale fixed in advance, usually 1 to 5 or 1 to 10. An option's total is computed two ways, and the difference between them is the whole subject.

The simple form adds an option's scores across all criteria. Every criterion counts as much as the others. It is the form to keep when the criteria are genuinely of equal importance, and it has the merit of transparency: the total is a sum anyone can redo in their head. The weighted form assigns each criterion a weight reflecting its importance, then computes the total as the sum of scores multiplied by their weight. A criterion with double weight counts double in the result. The relation between the two forms is exact and worth seeing: the simple form is the weighted form with all weights equal. As soon as the criteria are not of equal importance, the simple form asserts an equality nobody intended. In the literature this technique is called the weighted sum model or simple additive weighting, and it is the base case of multi-criteria decision analysis.

An even barer variant exists and deserves a line: the binary matrix, where the cell carries a plain « meets » or « does not meet », and the total is the number of criteria met. It is a quick sieve on eliminatory criteria. As soon as an option can satisfy a criterion halfway, a score is needed, and the weighted form takes over again.

Designing the criteria and the scale

A matrix's quality is settled before the first score, in the choice of criteria. Three faults settle in silently. The first is overlap: listing « licence price », « total cost of ownership » and « budget » on three rows amounts to counting cost three times and quietly giving it a weight no decision maker intended. Criteria should be as independent as possible; where two overlap, merge them or adjust the weight accordingly. The second is the unanchored scale: if no one has defined what a 3 is worth, two people will score the same option 2 and 4, and the total aggregates noise. Anchor each level of the scale with a descriptor, so scorers actually mean the same thing. The third is orientation: all criteria must run the same way, a high score always meaning « better ». A cost or risk criterion must therefore be reversed, the cheapest receiving the best score, otherwise the total adds quantities that pull in opposite directions and no longer means anything.

One honesty to keep in mind about the nature of the numbers: scores from 1 to 5 are often ranks, ordinal judgements, and adding them assumes the gap between 4 and 5 equals the gap between 1 and 2, which is not guaranteed. One remedy is to rescale each criterion onto a continuous scale, the best observed value at 100 and the worst at 0, the rest placed proportionally, before weighting. Failing that, at least hold the assumption consciously and distrust total gaps too thin to be real.

Eliciting the weights

The weights are the heart of the technique and the part one is most tempted to rush. They are a value judgement, that of the decision makers on what matters. Arbitrary weights produce an arbitrary ranking, dressed as arithmetic and all the more dangerous for looking objective. So elicit them from the real decision makers, and keep a trace of the reasoning. Two simple methods usually suffice: distributing a fixed budget, where each person spreads a hundred points across the criteria, which forces choices; and pairwise comparison, where each criterion is judged against each other, a method the analytic hierarchy process pushes as far as a consistency check.

One rule of conduct protects the grid from its commonest bias, retrofitting: setting the weights and scores after the fact to recover the answer already in mind. The defence is to set the weights before looking at the scores, in session and in the open, then freeze them. A last piece of hygiene comes before any calculation: eliminate the dominated options, those that another option equals or beats on every criterion. A dominated option cannot win, whatever the weights, and it only clutters the reading.

Reading the result without falling for false precision

A total of 4.15 against 3.70 looks sharp, and it rests on subjective scores from 1 to 5. The decimal lends an assurance the input data does not have. The right reflex is to test the gap with a sensitivity analysis: nudge the most debatable weights by one step and see whether the ranking holds. If it holds, the decision is robust and the grid has shown it. If it tips, the grid has said something equally useful: the choice is close, the two options are roughly equal, and you must decide on a criterion the matrix does not capture or accept that the gap is not significant. The ranking is a starting point for the decision maker's discussion.

Its place in the decision-analysis family

The decision matrix is a tool of decision analysis, the approach that frames a problem, defines the alternatives, evaluates them, chooses and implements. The matrix is the machinery of the evaluation step, and it sits alongside other instruments chosen by the shape of the problem. The analytic hierarchy process is the rigorous cousin: the same grid of options and criteria, but weights and scores derived from pairwise comparisons on a structured scale, with a consistency ratio that flags incoherent judgements. You reach for it when the weights themselves are contested and must be defensible. The Pugh matrix is the design variant: each option is rated better, equal or worse than a reference option, and the aim is to converge and hybridise concepts rather than to name a winner on the first pass. It is the same grid, scored against a reference. And multi-criteria decision analysis is the whole family, of which the weighted sum model, hence the weighted matrix, is the simplest member. For decisions that are not comparisons of options, you leave the matrix: pro versus con analysis weighs the advantages and disadvantages of a single choice, and force field analysis sets the forces pushing a change against those holding it back.

Running the technique

1. State the problem and list the alternatives genuinely in play. A matrix over two phantom options wastes everyone's time.
2. Choose the evaluation criteria. Few, independent and all oriented the same way, a high score always meaning better.
3. Fix and anchor the scoring scale. One descriptor per level, so two scorers hear the same thing by a 3.
4. Elicit the weights from the decision makers, before the scores. A hundred points to spread or pairwise comparison, and write down the reason for each weight.
5. Eliminate the dominated options, those another beats on every criterion.
6. Score each option on each criterion, from verifiable facts.
7. Compute the simple total and the weighted total and read the ranking.
8. Run the sensitivity analysis: move the contested weights, see whether the winner holds and hand the decision maker a ranking together with its robustness.

AI considerations

The first useful use is preparing the criteria. A language model given a specification, a tender or a framing note proposes a list of candidate criteria and, more valuable, spots the overlaps, the two or three criteria that at bottom measure the same thing and, left in place, would count one concern twice. That is exactly the design fault a human eye lets through and a systematic reread catches.

The second use is filling a first pass of scores from documented sources, a product's spec sheets, tender responses, measured costs, which humans then only have to challenge. The third is the most rewarding and the most mechanical: the arithmetic and the sensitivity analysis. Recomputing totals, sweeping a range of weights, flagging where the ranking is fragile, keeping the simple total and the weighted total consistent when a score changes, all of that is calculation the machine does without error and continuously, where a room miscounts a twenty-row grid.

What AI must not do goes to what makes the technique exist. It does not set the weights: the weights are the stakeholders' value judgement, and automating them removes the one thing the matrix exists to make explicit. It does not fabricate scores or facts about an option, and does not turn an impression into a confident decimal. And it does not make the decision: the decision maker must understand the model's assumptions and own the choice, failing which they will grant a number more trust than subjective scores deserve. Finally, a vendor-scoring grid holds confidential commercial data, sometimes protected by contract; it calls for a tool whose data handling is under contractual control, which a consumer AI service does not offer.

Examples

A physiotherapy practice in French-speaking Switzerland is choosing its management and billing software among three offers. Four criteria, weights summing to 1.00, the same scores from 1 to 5 in both totals.

The concept the grid makes visible is that the ranking depends as much on the weights as on the scores. The scores are identical in both columns: the simple form, which treats the four criteria equally, names offer A, the cheapest and the easiest to adopt. The weighted form changes no score, it merely declares that billing compliance and Swiss data hosting together weigh 0.65, and that single change moves the Swiss vendor B ahead. The A to B switch is the information: it says this choice turns entirely on the importance granted to compliance and hosting. The weighted gap of 0.45 between B and A resists small adjustments: shifting five points of weight from security to adoption leaves B ahead, and it takes nine to bring A and B level. The grid therefore names B, and the decision maker knows how far their priorities would have to move for that ranking to change. A decision maker who still wanted to separate them more finely would decide on a criterion outside the grid, the quality of support in French-speaking Switzerland for instance, without over-reading a decimal that scores from 1 to 5 do not carry.

Visualisations

The matrix is made of rows and columns: the deliverable is the table itself. Carrying it as HTML rather than an image has practical consequences: it sorts, its totals recompute when a score changes, and it stays legible at zoom and to a screen reader. An image of a grid loses all that and freezes numbers that the slightest score revision makes false.

Three conventions make the grid legible. The weights appear in plain sight, in the header, next to the criterion they weigh, since they carry the value judgement and a grid that hides them is not verifiable. The two totals read side by side, the simple and the weighted, because their gap is precisely what the technique teaches. And the winning option stands out to the eye in each total column, which makes the switch visible without commentary.

The switch itself draws better than it tabulates, because it is a movement, the passage from one ranking to another under the effect of the weights. A diagram showing the same scores leading to two different rankings carries the technique's central idea in one image.

Cost

Tools

The spreadsheet is the honest and sufficient choice for almost every case. Totals are formulas, weighting is a multiplication, and the sensitivity analysis is done by changing one weight cell to watch the ranking recompute before your eyes. Conditional formatting that colours the head of each total column makes the switch immediately visible. It is also the tool that keeps the simple total and the weighted total consistent without effort.

Tools dedicated to multi-criteria decision analysis, such as 1000minds, Criterium DecisionPlus or Expert Choice for the pairwise-comparison variant, earn their place when the decision is heavy, the criteria are many and hierarchical, or the consistency of the judgements must be documented for a committee. They bring structured weight elicitation and a tooled sensitivity analysis, at the price of a learning curve and a licence the spreadsheet does not require.

For the workshop defining the criteria and weights, a whiteboard, physical or shared, remains the best support: you write the criteria on it, spread the hundred points in full view, and freeze them before moving to the scores. The support to avoid is the results slide, which photographs a ranking without its weights or its sensitivity, and presents it as a verdict when it is only a starting point.

Sources

• IIBA, A Guide to the Business Analysis Body of Knowledge (BABOK Guide) v3, §10.16 Decision Analysis: decision analysis as an approach, its generic activities, the four components of a decision, the two example matrices (simple and weighted), the trade-off methods including elimination of dominated options and proportional scoring, as well as the recognised strengths and limitations, notably the risk of false certainty.
• Stuart Pugh, Total Design: Integrated Methods for Successful Product Engineering, Addison-Wesley, 1991: the Pugh matrix and controlled convergence, the design variant where each option is scored relative to a reference. One author's method, honestly attributed.
• Institute for Manufacturing, University of Cambridge, Controlled Convergence: an academic account of Pugh's method, scoring relative to a datum and iterating to hybridise concepts.
• Thomas L. Saaty, The Analytic Hierarchy Process, McGraw-Hill, 1980: the primary source for the analytic hierarchy process, the derivation of weights and scores by pairwise comparison and the consistency ratio.
• Valerie Belton and Theodor Stewart, Multiple Criteria Decision Analysis: An Integrated Approach, Springer, 2002: the reference work on multi-criteria decision analysis, which situates the weighted sum model and grounds the recommendations on weights, scale and sensitivity.