SD&A · Sheet Set 08.01
Chapter 08 · Lecture 01 · Architecture and Design Evaluation

Before you build it, put it on trial.

Architecture evaluation catches expensive mistakes while they're still cheap to fix. ATAM is the courtroom — quality attributes are cross-examined against every architectural decision.

ATAM · SAAM · ARID Quality Attribute Tradeoffs 9 Steps · 4 Phases Utility Tree
FIG. 01 — WHY EVALUATE EARLY

The earlier a design flaw is found, the cheaper it is to fix.

Architecture must be evaluated before construction begins — not after. Architecture is shaped by three drivers: functional requirements, non‑functional requirements, and business constraints.

01 · Benefits

Why bother evaluating architecture?

1Prioritization

Forces prioritization of conflicting goals.

2Clarity

Forces clear explanation of the architecture.

3Documentation

Improves quality of architecture documentation.

4Reuse

Uncovers opportunities for cross‑project reuse.

5Practice

Results in improved architecture practices overall.

MEM
"Priorities get Clear, Docs get better, Reuse gets found, Practice improves." (PCDRP)
02 · The Three Methods

ATAM · SAAM · ARID

All three lean on the same two ingredients: scenarios and quality attributes. The difference is depth and focus, not the underlying idea.

AATAM

Architecture Tradeoff Analysis Method. The most‑used method — full focus on quality‑attribute tradeoffs and risk discovery. This chapter's main subject.

SSAAM

Software Architecture Analysis Method. ATAM's predecessor — simpler, earlier scenario‑based analysis technique.

AARID

Active Reviews for Intermediate Designs. Reviews designs that aren't complete architectures yet — earlier in the process, more informal.

MEM
"Architects Sometimes Argue" — ATAM, SAAM, ARID, in that order.
03 · ATAM Purpose

What ATAM is actually looking for

ATAM assesses the consequences of architectural decisions against quality‑attribute requirements and business goals — which really means: hunting for risk where one quality attribute is affected by a decision made for another.

The core insight

In complex systems, quality attributes are never achieved in isolation. Chasing one quality often moves another — sometimes helping it, sometimes hurting it. So every architectural decision is really a tradeoff between quality attributes, made in service of the business goals.

MEM
"You can't max out every quality at once — architecture is the art of which tradeoffs you're willing to make."
04 · Quality Attributes, Grouped

3 perspectives on quality

EEnd‑user perspective

Performance, Availability, Usability, Security — PAUS. What the person using the system feels.

TTechnical perspective

Modifiability, Portability, Reusability, Testability, Interoperability — MPRTI. What the engineering team feels.

BBusiness perspective

Time to market, cost & benefits, projected lifetime, project budget — TLCB. What the business feels.

Bridge to Chapter 3: to actually evaluate any of these, you characterize them as a quality attribute scenario (Source → Stimulus → Artifact → Environment → Response → Response Measure). That scenario is then what gets prioritized and traced through the architecture.
05 · Who's in the Room

3 groups, 5 roles

ATAM only works if these three groups cooperate — leave one out and the evaluation is blind to part of the system's reality.

EEvaluation team

3–5 experienced architects, external to the project. They run the process.

PProject decision makers

People with authority to change the project — project manager, customer, senior manager.

AArchitecture stakeholders

Developers, testers, integrators, maintainers, performance engineers, users.

MEM
"EPA protects the architecture" — Evaluation team, Project decision makers, Architecture stakeholders.
Role Responsibility, in one line
Team leader Sets up the evaluation, coordinates with the client, forms the team, delivers the final report.
Evaluation leader Runs the evaluation — facilitates scenario elicitation, prioritization, analysis, on‑site work.
Scenario scribe Writes scenarios on the flipchart/whiteboard; locks the exact agreed wording.
Proceedings scribe Types everything electronically — raw scenarios, motivating issues, resolutions; hands out the adopted list.
Questioner Raises architecturally interesting issues, usually tied to their own quality‑attribute expertise.
MEM
"Team leads, Evaluation runs, Scribe writes it on the wall, Proceedings types it up, Questioner pokes holes." (TESPQ)
06 · The Process

The 9 ATAM steps

Notice the shape: it opens with three "Present" steps, works through Identify → Generate → Analyze, loops back for Brainstorm → Analyze again, then closes with one more Present.

Phase 1 — Steps 1 to 6
1

Present the ATAM method

Overview of steps, techniques, and expected outputs (utility tree, scenarios, risks, sensitivity points).

2

Present the business drivers

Goals, stakeholders, high‑level requirements, top quality scenarios, constraints — from the decision maker.

3

Present the architecture

Lead architect walks through it — often using the 4+1 view model — and flags known risks.

4

Identify architectural approaches

Evaluation team names the patterns in play — client‑server, layered, pipe‑filter, etc.

5

Generate quality attribute utility tree

Top‑down prioritization of quality goals into concrete, scored scenarios.

6

Analyze architectural approaches

Trace the highest‑ranked scenarios through the architecture; capture decisions, risks, sensitivities, tradeoffs.

Phase 2 — Steps 7 to 9
7

Brainstorm & prioritize scenarios

Now the full stakeholder community adds scenarios and votes — checking whether they agree with the utility tree.

8

Analyze architectural approaches

Same analysis as step 6, now against the newly brainstormed, stakeholder‑driven scenarios.

9

Present results

Report + slides: business context, architecture documentation, prioritized scenarios, utility tree, risks/non‑risks/sensitivities/tradeoffs.

MEM
"3 Presents open it, I‑G‑A builds it, Brainstorm‑Analyze stress‑tests it, one more Present closes it."

Present ×3 (1,2,3) → Identify, Generate, Analyze (4,5,6) → Brainstorm, Analyze (7,8) → Present (9).

Step 6 vs Step 8 — the trap question: they use the same analysis technique. The only difference is whose scenarios: Step 6 analyzes the utility‑tree scenarios (architect‑driven), Step 8 analyzes the brainstormed scenarios (stakeholder‑driven).
07 · Zooming Out

The 4 phases that wrap the 9 steps

Phase Activity Participants Typical duration
0 Partnership & preparation: logistics, planning, stakeholder recruitment, team formation. Evaluation team leadership + key decision‑makers Informal, over a few weeks
1 Evaluation: Steps 1–6 Evaluation team + project decision‑makers 1–2 days, then a 2–3 week hiatus
2 Evaluation: Steps 7–9 Evaluation team, decision‑makers, stakeholders 2 days
3 Follow‑up: report generation & delivery, process improvement. Evaluation team + evaluation client 1 week
MEM
Phase 0 = Prep. Phase 1 = Steps 1–6. Phase 2 = Steps 7–9. Phase 3 = Wrap‑up. "0 preps, 1 does 6, 2 does 3 more, 3 reports."
08 · Step 5 in Detail

The Utility Tree — turning fuzzy goals into scored scenarios

A top‑down structure: Utility → Quality Attributes → Refinements → concrete Scenarios. Every leaf scenario gets scored (Importance, Difficulty) — e.g. (H,H) means high business importance, high architectural difficulty. That's the pair the team prioritizes first.

FIG. 02 — UTILITY TREE (SAMPLE)
Utility
Performance
(L,M)Reduce storage latency on customer DB to < 200ms.
(M,M)Deliver video in real time.
Modifiability
(H,H)Add CORBA middleware in < 20 person‑months.
(H,L)Change the Web user interface in < 4 person‑weeks.
Availability
(H,H)Power outage at site 1 → traffic redirected to site 2 in < 3 seconds.
(H,H)Network failure detected & recovered in < 1.5 minutes.
Security
(H,M)Credit card transactions are secure 99.999% of the time.
(H,L)Customer DB authorization works 99.999% of the time.
MEM
Score order is always (Importance, Difficulty) — "I before D," same order as the letters in the word "AID" minus the A.

High‑High leaves are triaged first in Step 6 — they're the scenarios most likely to reveal real risk.

09 · Step 6/8 in Detail

Sensitivity · Tradeoff · Risk · Non‑risk

These four labels are what the evaluation team attaches to every architectural decision while tracing a scenario. Getting these definitions exact is one of the most commonly tested parts of ATAM.

SSensitivity Point

A decision that has a significant effect on one quality attribute. Tune this decision and that one attribute moves noticeably.

TTradeoff Point

A decision that's a sensitivity point for more than one attribute — improving one costs another.

RRisk

A potentially problematic architectural decision — something that could threaten a quality goal if left unaddressed.

NNon‑risk

A decision the team is confident is safe — reviewed and found sound.

MEM
"Sensitivity = affects one. Tradeoff = affects two+, pulling opposite ways. Risk = might go bad. Non‑risk = confirmed fine."
FIG. 03 — WORKED SCENARIO A12
Scenario Detect and recover from HW failure of the main switch.
Attribute Availability
Stimulus → Response One CPU fails → 0.999999 availability of the switch.
Backup CPU(s) Sensitivity S2 · Risk R8
No backup data channel Sensitivity S3 · Tradeoff T3 · Risk R9
Watchdog / Heartbeat / Failover routing Sensitivities S4–S6 · Non‑risks N12–N14
Reading this fast: everything in the decisions column got a sensitivity number — that's expected, almost every decision is sensitive to something. What separates a healthy design from a risky one is whether that sensitivity resolves into a Risk or a Non‑risk.
10 · Deliverables

What ATAM must hand back

SADSoftware Architecture Document

The key artifact documenting the selected architecture itself.

REPEvaluation report

Recaps the process, captures the scenario analysis, and explains the candidate architectures with the rationale behind the final choice.

11 · The Fast Version

Lightweight ATAM

Same 9‑step skeleton, built for smaller / lower‑risk projects — sometimes done in a single half‑day, entirely with people already inside the organization.

+What you gain

Speed. Internal‑only participants mean faster shared understanding — no external onboarding needed.

What you lose

Depth and objectivity. Internal‑only reviewers probe the architecture less deeply than an outside evaluation team.

Step Time What changes vs. full ATAM
1. Present ATAM 0h Skipped — participants already know the process.
2. Business drivers 0.25h Quick refresh only.
3. Present architecture 0.5h Module + C&C views, 1–2 scenarios traced.
4. Architectural approaches 0.25h Often folded into step 3.
5. Utility tree 0.5–1.5h May reuse an existing tree from earlier work.
6. Analyze approaches 2–3h Bulk of the time — unchanged in spirit from full ATAM.
7. Brainstorm scenarios 0h Omitted — same internal people already contributed in step 5.
8. Analyze approaches 0h Omitted — all analysis already happened in step 6.
9. Present results 0.5h Review risks/non‑risks/sensitivities/tradeoffs; check for new risk themes.
MEM
Lightweight ATAM zeroes out steps 1, 7, and 8 — the "teach the process" step and both stakeholder‑brainstorm steps — because everyone's already internal and aligned.
12 · Memory Toolkit

All the mnemonics, in one place

1Benefits

PCDRP — Priorities, Clarity, Docs, Reuse, Practice.

2Methods

"Architects Sometimes Argue" — ATAM, SAAM, ARID.

3QA perspectives

PAUS (end‑user) / MPRTI (technical) / TLCB (business).

4Participants

EPA — Evaluation team, Project decision‑makers, Architecture stakeholders.

5Roles

TESPQ — Team leader, Evaluation leader, Scribe, Proceedings, Questioner.

69 steps shape

3 Presents → Identify‑Generate‑Analyze → Brainstorm‑Analyze → Present.

74 phases

0 preps, 1 does steps 1–6, 2 does steps 7–9, 3 reports.

8Utility tree score

Always (Importance, Difficulty) — "I before D."

9S·T·R·N

Sensitivity=one attribute · Tradeoff=two+ opposite · Risk=maybe bad · Non‑risk=confirmed fine.