Multi-Criteria Decision Analysis (MCDA)

Goal

Multi-criteria decision analysis supports a selection decision under several objectives, where no single number settles it. A financial criterion, functional coverage, a compliance requirement and a supplier risk cannot be reduced to one unit, and yet a choice has to be made. The technique makes that trade-off explicit instead of leaving it to intuition or to whoever holds the floor. It converts a contested, multi-dimensional judgement into a structured, repeatable and challengeable comparison.

The deliverable has two parts, and it is the second that makes the technique. First a scored, weighted options-against-criteria matrix, which produces a ranking. Then, inseparable from it, the record of choices: the criteria retained, the weights and their justification, the scoring scale and the aggregation rule. The ranking alone is an opinion dressed as a number. The ranking together with its assumptions is a decision that can be put to a steering committee, defended before an audit and rerun when a value changes. BABOK places multi-criteria decision analysis among the tools of decision analysis and gives its weighted tabular form. The most common use is supplier evaluation: several bids, several criteria that pull against each other and a recommendation that will have to be justified.

Usage

When to use it

• Several options, several conflicting criteria: make a trade-off between cost and non-financial factors visible.
• High-stakes choice, evidence required: a tender, an award, a tool selection, where the matrix is the supporting record.
• Stakeholders disagree on priorities: weighting surfaces the disagreement and negotiates it in the open.
• Recommendation to justify to a committee or an audit: the record of weights is its written trace.
• Decision meant to be rerun: options or criteria change, the matrix recomputes rather than being rebuilt.

When not to use it

• One criterion truly dominates, usually cost: compare on that criterion or run a pro versus con analysis.
• The uncertainty is in the outcomes, with branches and probabilities: a matrix hides it, use a decision tree.
• Two options and a fast call: MCDA over-engineers it, a force field analysis is enough.

Description

Multi-criteria decision analysis is a frame that houses several methods. The decision matrix is its simplest tabular instrument, the one that implements a weighted sum; the analytic hierarchy process is a more rigorous method, deriving the weights and scores through pairwise comparisons with a consistency check. Both belong to the decision analysis family.

The elements of the analysis

A multi-criteria analysis combines four elements. The options are a finite set of real alternatives, comparable with one another. The criteria are the dimensions that matter to the decision. Keep the ones that discriminate between options and drop the ones every option meets equally, because they add a column without adding information. The criteria should also stay as independent as possible: two criteria that cover the same concern count it twice and skew the result in favour of whoever carries it.

The weight expresses the relative importance of a criterion; the set of weights is normalised to sum to one, or to a hundred per cent. Weighting is a stakeholder value judgement, not a fact to be measured, and it is elicited before scoring. The methods range from direct point allocation, where a hundred points are distributed across the criteria, to structured pairwise comparison. The score, finally, places each option on each criterion on a common scale, one to five or zero to a hundred, say. Benefit criteria, where a higher raw value is better, and cost criteria, where a lower value is better, are scored in the same direction: a cheaper option must receive a higher cost score, otherwise the aggregation adds figures that do not point the same way.

Normalise, then aggregate

Normalisation brings heterogeneous raw values onto the common scale. Proportional scoring is one way: the best value observed takes the maximum, the worst the minimum, the others interpolate linearly between them. Max normalisation is another. The choice of normalisation method can change the ranking, because it changes the relative distance between options on a given criterion. It is therefore a decision, taken and documented explicitly.

Aggregation gathers the weighted scores into one number per option. The weighted sum adds the product of each weight and its score: it is the simplest and most common model. The weighted product multiplies the scores raised to the power of their weight. It is heavier, but it has a property that both Belton and Stewart and Triantaphyllou note: being multiplicative, it is dimensionally consistent and avoids adding quantities of different natures when the units differ. For scores brought onto a common, dimensionless scale, the weighted sum is enough in the great majority of cases.

The sequence

1. Frame the decision and list the options. A finite, real and comparable set of alternatives.
2. Define the criteria. The dimensions that discriminate, as independent as possible; drop the ones every option meets equally.
3. Weight the criteria. The stakeholder value judgement, elicited and fixed before any score, and summing to one.
4. Score each option on each criterion. On the common scale, cost and benefit criteria oriented the same way.
5. Normalise. Bring raw values onto the scale, recording the method used.
6. Aggregate, then rank. Weighted sum or weighted product, one number per option, a rank.
7. Test the ranking with a sensitivity analysis. Vary the weights within a plausible range and check that the winner holds.

Sensitivity analysis

A ranking is not a conclusion until its robustness is known. Sensitivity analysis moves the weights within a defensible range and watches the rank. A winner that survives a reasonable shift in weighting is a robust winner, and the decision can rest on it. A winner that tips over the moment a weight moves by a few hundredths is settled by the fine-tuning of the weights, that is, by the preference that weighting was meant to make explicit. A close gap, a total of 3.90 against 3.75, is reported to the decision-maker as sensitive to the weights. This is the method's own contribution: it says how far the answer depends on the assumptions, beyond the winner alone.

The traps that distort the result

• Weights set after seeing the scores. This is the commonest way of manufacturing the result you wanted: the weighting is adjusted until the preferred supplier wins. Weights are fixed first, and they are written down before the first score is entered.
• False precision. Treating a gap of 3.90 against 3.75 as decisive lends the scores an exactness they do not have. A gap of that order calls for a sensitivity analysis.
• Double counting. Overlapping criteria inflate a single concern. "Price" and "total cost of ownership" on two separate rows count cost twice.
• Criteria inflation. Twelve criteria dilute the three that actually separate the options. The column every option meets in the same way reassures without informing.
• Silent normalisation. The scaling method changes the result, and nobody recorded which one was used. The matrix becomes irreproducible.
• The number replacing the judgement. The aggregate informs the decision maker, it does not decide in their place. BABOK states this limit itself: the decision maker remains the owner of the assumptions. A committee that ratifies the total without questioning the weights has delegated its decision to a spreadsheet.

AI considerations

The first useful use is preparing the matrix. From a specification, a tender or a set of requirements, a language model proposes a first list of criteria, spots the ones that visibly overlap and flags the criteria every option seems to meet equally. This is a rough cut, then reviewed by the analyst.

The second use is extraction. Hundred-page supplier bids read poorly by hand; a model pulls out the comparable elements and places them in the matching cells, with the analyst checking each entry against the source. The third is computation at scale: running the sensitivity analysis and Monte-Carlo simulations over the weights and flagging dominated options automatically, those another beats on every criterion and which therefore need not stay in the comparison any longer.

What AI must not do turns on the nature of the weights. Weighting encodes stakeholder priorities, a value choice and sometimes a political one. A model that proposes weights launders an opinion into a calculation: it gives a preference the look of an objective result, and it hides the step the technique exists to make explicit. In the same way, a subjective score does not become objective because a model produced it, the normalisation choice does not disappear behind a number, and the decision does not shrink to the single total the committee would only have to sign off. The human remains the owner of the weights and of the trade-off. That leaves data sensitivity: a comparison of bids contains confidential material and personal data within the meaning of the revised Federal Act on Data Protection (nFADP), which does not go into a public tool.

Examples

A small firm in French-speaking Switzerland, around 300 employees, is selecting payroll and HR software from three shortlisted vendors, on five weighted criteria. Scores run from one to five, five being best, and cost is scored so that a cheaper offer receives a higher score. The indicative three-year total cost drives the cost row alone: about CHF 180'000 for vendor A, CHF 240'000 for B, CHF 310'000 for C. The matrix is the artifact.

The concept the matrix makes visible fits in one phrase: the cheapest loses. Vendor A takes the top cost score and yet finishes last: its 0.60 weighted-point lead on cost is exactly cancelled by what it loses on legal compliance and data hosting in Switzerland, and it gives ground on the three remaining criteria as well. Vendor B leads outright on only one of the five criteria, compliance; it moves ahead of C because its leads on cost and compliance, 0.50 point together, exceed C's leads on functional coverage and longevity, 0.35 point. This is exactly what MCDA is there to prevent: the decision falling back by default to the cheapest option because cost is the one number that is easy to compare.

The ranking of B and C is close and is reported as such. If the weighting shifts, cost from 0.30 to 0.20 and functional coverage from 0.25 to 0.35, C moves ahead of B: B rises to 4.00 and C to 4.05. Neither offer has changed, only the preference has moved. A decision this close is put to the committee as sensitive to the weights, with both scenarios.

Visualizations

The matrix is made of rows and columns: the deliverable is the table itself. The form carries the real use of the artifact, which reads two ways. A row shows how the offers compare on one criterion, a column reads the full profile of one offer. Those two readings and the recomputation of the total, a formula that updates the rank the moment a weight changes, are exactly what sensitivity analysis draws on.

The sequence of the method, by contrast, does not fit in a table, because it is a chain of steps. It is drawn: frame, define the criteria, weight, score, normalise, aggregate, rank, test. The order is the argument, in particular the fact that weighting precedes scoring, and a diagram shows it where a grid cannot.

Cost

Tools

The spreadsheet is the default choice and it fits almost every case: a weighted sum is written in a single SUMPRODUCT formula, and the same sheet is the ground for sensitivity analysis, a weight changed and a rank recomputed. For the weighting step, where the discussion matters more than the arithmetic, a shared whiteboard or a workshop tool such as Miro carries the elicitation of criteria and weights, remotely as much as in the room.

Software dedicated to multi-criteria analysis earns its place when the criteria are many, when weighting is collective, when a formal sensitivity analysis is required or when an audit calls for a full audit trail. The categories matter more than the products: pairwise-comparison weighting tools, such as those that apply the PAPRIKA method or the analytic hierarchy process, and open-source packages for the R statistical language for aggregation and the comparison of methods. The price of these tools is a learning time and a dependency; it is paid only where the decision and its need for traceability deserve it.

Sources

• IIBA, A Guide to the Business Analysis Body of Knowledge (BABOK Guide) v3, §10.16 Decision Analysis: multi-criteria decision analysis among the tools of decision analysis, the components of the decision, the weighted decision matrix and the limit that the decision maker remains the owner of the assumptions. A descriptive source: it names the technique and does not fix the method.
• Belton, V. and Stewart, T. J., Multiple Criteria Decision Analysis: An Integrated Approach, Springer, 2002: the discipline's reference work, anchoring weighting, value functions, aggregation and sensitivity analysis.
• Triantaphyllou, E., Multi-Criteria Decision Making Methods: A Comparative Study, Springer, 2000: the anchor for the weighted sum and the weighted product, with their properties, including the dimensional consistency of the multiplicative model.
• Keeney, R. L. and Raiffa, H., Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Cambridge University Press, 1993: multi-attribute value theory, the foundation of the explicit trade-off between criteria and of weighting.