Requirements definition is
where failure occurs most commonly.
Two lectures in one deck. It covers the five key issues of data gathering, the six techniques, the kinds of requirements, the artifacts used to express them - personas, scenarios, use cases, HTA - and the qualitative and quantitative analysis that turns data into requirements.
LEARNING OUTCOMES
- Explain and describe the process of data gathering.
- Plan and run a successful data gathering programme.
- Distinguish the different kinds of requirements.
- Use scenarios, use cases and essential use cases to articulate work practices.
- Apply hierarchical task analysis and simple qualitative and quantitative analysis.
Getting requirements right is crucial
Requirements work has two aims: understand as much as possible about users, task and context, and produce a stable set of requirements.
- Data gathering activitiesCollect evidence from users, documents and existing products.
- Data analysis activitiesInterpret, categorise and structure what was collected.
- Expression as 'requirements'Turn findings into statements that can be checked.
- All of this is iterativeNone of the three happens once.
Requirements definition is the stage where failure occurs most commonly. That single sentence is the justification for the whole deck, and it is worth quoting.
"Establish" is used deliberately rather than "collect": requirements need clarification, refinement, completion and re-scoping. The input is a requirements document (maybe); the output is stable requirements. Requirements arise from understanding users' needs, and can be justified and related back to the data.
The five key issues
Every data-gathering exercise, whichever technique is used, must settle these five things first.
| Issue | What it means |
|---|---|
| 1. Setting goals | Decide how you will analyze the data once it is collected - before you collect it. |
| 2. Identifying participants | Decide who to gather data from. |
| 3. Relationship with participants | Keep it clear and professional; obtain informed consent when appropriate. |
| 4. Triangulation | Look at the data from more than one perspective. |
| 5. Pilot studies | Run a small trial of the main study first. |
G-P-R-T-P: Goals, Participants, Relationship, Triangulation, Pilot. Read it as "Good Planning Really Takes Practice". Notice the first and last are both about doing work before the real study.
Interviews, questionnaires, similar products, observation, ethnography, documentation
Each technique has a stated strength and a stated weakness. The exam pairs them.
1. Interviews
Props such as sample scenarios of use and prototypes can be used. Good for exploring issues, but time consuming and it may be infeasible to visit everyone. Focus groups are group interviews - good at gaining a consensus view and/or highlighting areas of conflict, but can be dominated by individuals.
2. Questionnaires
Often used in conjunction with other techniques. Can give quantitative or qualitative data. Good for answering specific questions from a large, dispersed group. Closed questions are easier to analyze and can be processed by computer. Distributed by paper, email or web. Sampling is a problem when the population size is unknown, as is common online.
3. Researching similar products
Good for prompting requirements - a cheap way to surface what a domain normally provides.
4. Observation
The action or process of observing something in order to gain information. Direct: gains insights into stakeholders' tasks and is good for understanding the nature and context of tasks, but requires time and commitment from a design team member and produces a huge amount of data; includes think-aloud techniques. Indirect: not often used in the requirements activity, but good for logging current tasks - diaries, interaction logs and web analytics.
5. Ethnography
A philosophy with a set of techniques including participant observation and interviews. Ethnographers immerse themselves in the culture they study, and their degree of participation varies along a scale from 'outside' to 'inside'.
6. Studying documentation
Procedures and rules are often written down in manuals. Good for understanding the steps in an activity, legislation and background information. Not to be used in isolation. Its unique advantage: no stakeholder time, which is the limiting factor on every other technique.
| Interview structure | Character |
|---|---|
| Unstructured / open-ended | Not directed by a script. Rich but not replicable. |
| Structured | Tightly scripted, often like a questionnaire. Replicable but may lack richness. |
| Semi-structured | Guided by a script, but interesting issues can be explored in depth. A good balance between richness and replicability. |
| Group interview | A small group guided by a facilitator. |
- IntroductionIntroduce yourself, explain the goals, reassure about ethical issues, ask to record, present the informed consent form.
- Warm-upMake the first questions easy and non-threatening.
- Main bodyPresent questions in a logical order.
- Cool-off periodA few easy questions to defuse tension at the end.
- ClosureThank the interviewee and signal the end - switch the recorder off.
Long questions. Compound sentences - split them into two. Jargon and language the interviewee may not understand. Leading questions that make assumptions - "why do you like...?". Unconscious biases such as gender stereotypes.
Questionnaire design: the impact of a question can be influenced by question order; you may need different versions for different populations; give clear completion instructions; strike a balance between white space and compactness; and decide whether phrases will be all positive, all negative or mixed. Response formats include yes/no checkboxes, multi-option checkboxes, rating scales (Likert, semantic - 3, 5, 7 or more points) and open-ended responses.
| Encouraging a good response | Online questionnaires: advantages | Online: problems |
|---|---|---|
| Make the purpose of the study clear; promise anonymity. | Responses received quickly. | Sampling is problematic if population size is unknown. |
| Ensure it is well designed; offer a short version. | No copying or postage costs. | Preventing individuals from responding more than once. |
| Include a stamped addressed envelope if mailed; follow up; provide an incentive. | Data collected straight into a database, reducing analysis time; errors easily corrected. | Individuals have been known to change questions in email questionnaires. |
40% is high; 20% is often acceptable. A concrete number the exam can ask for.
Frameworks, ethnography and contextual inquiry
Observation without a framework produces notes; with one it produces data.
The simple framework
The person - who? The place - where? The thing - what?
Goetz and LeCompte (1984)
Who is present? What is their role? What is happening? When does the activity occur? Where is it happening? Why is it happening? How is the activity organized?
- Web analytics is a system of tools and techniques for optimizing web usage by measuring, collecting, analyzing and reporting web data - typically focused on the number of visitors and page views.
- Ethnography requires the co-operation of the people being observed; informants are useful; data analysis is continuous; it is an interpretivist technique; questions get refined as understanding grows; and reports usually contain examples.
- Online ethnography (virtual, online, netnography) covers online and offline activity. Online interaction differs from face-to-face, virtual worlds have a persistence that physical worlds do not, and ethical considerations and presentation issues are different.
| Contextual inquiry: four principles | What it means |
|---|---|
| Context | See the workplace and what happens there. |
| Partnership | User and developer collaborate. |
| Interpretation | Observations are interpreted by user and developer together. |
| Focus | A project focus determines what to look for. |
It is an approach to ethnographic study in which the user is the expert and the designer is the apprentice. It is a form of interview, but held at the user's own workplace or workstation, and lasting two to three hours.
Contextual inquiry = C-P-I-F: Context, Partnership, Interpretation, Focus. And the one-line frame that generates all four: the user is the master craftsman, you are the apprentice.
What goes wrong, and how to plan around it
The lecture lists the practical failures of data gathering before giving the guidelines.
- Identifying and involving stakeholders - users, managers, developers, customer reps, union reps, shareholders - through workshops, interviews, workplace studies, or co-opting them onto the development team.
- Getting 'real' users, not managers - traditionally a problem in software engineering, though better now.
- Requirements management - version control and ownership.
- Communication between parties - within the development team, with the customer or user, and between users, since different parts of an organisation use different terminology.
- Domain knowledge distributed and implicit - difficult to dig up and understand. The knowledge-articulation problem: how do you walk?
- Availability of key people.
- Political problems within the organisation, and dominance of certain stakeholders.
- Economic and business environment changes.
- Balancing functional and usability demands.
| Data gathering guidelines | Data recording options |
|---|---|
| Focus on identifying the stakeholders' needs. | Notes + still camera - less intrusive than a keyboard and flexible, but tiring; hard to write and listen and observe at once; concentration lapses; biases creep in; handwriting is hard to read; writing speed is limited. Solution: a partner or second observer. |
| Involve all the stakeholder groups, and more than one representative from each. | Audio + photographs - less intrusive than video; in observation it lets observers focus on the activity rather than the words; in interviews it lets the interviewer attend to the interviewee. Transcribing is time consuming, though not all sections may be needed. |
| Use a combination of data gathering techniques; support with props such as prototypes and task descriptions; run a pilot session; and consider carefully how to record the data. | Video - captures both audio and video, but raises where to fix the camera or whether to rove, where to point it, and the impact of recording on participants' behaviour. |
"You will need to compromise on the data you collect and the analysis to be done, but before you can make sensible compromises, you need to know what you'd really like." Plan the ideal study, then cut it - not the reverse.
Functional, data, environment, users
Different kinds of requirement matter for interaction design, and each has its own sub-structure.
Functional
What the system should do. Historically the main focus of requirements activities. (Non-functional requirements cover memory size, response time and similar.)
Data
What kinds of data need to be stored, and how they will be stored - for example in a database.
Environment / context of use
Physical: dusty, noisy, vibration, light, heat, humidity - an outdoor kiosk or an ATM. Social: sharing of files, displays, paper; across great distances; working individually; privacy for clients. Organisational: hierarchy, the IT department's attitude and remit, user support, communications structure and infrastructure, availability of training.
Users
Characteristics - ability, background, attitude to computers. System use - novice, expert, casual, frequent. Novice: step-by-step prompted interaction, constrained, clear information. Expert: flexibility, access and power. Frequent: short cuts. Casual/infrequent: clear instructions, e.g. menu paths.
Users' capabilities vary in many dimensions, and each dimension is a design constraint: size of hands affects the size and positioning of input buttons; motor abilities affect the suitability of input and output devices; height matters when designing a physical kiosk; strength matters - a child's toy needs little strength to operate but greater strength to change the batteries; and disabilities of sight, hearing and dexterity change everything.
The slides ask which environmental, user and usability factors would affect: a self-service petrol filling and payment system (physical: weather, gloves, fumes; users: everybody; usability: safety and learnability); an on-board ship data analysis system for geologists (physical: vibration, motion; users: domain experts; usability: efficiency and utility); and a fashion clothes website (environment: anywhere, any device; users: casual; UX goals dominate).
Personas, scenarios, use cases and task analysis
Four artifacts, each answering a different question about the work being supported.
Personas
Capture user characteristics. They are not real people, but are synthesised from real user characteristics. They should not be idealised. Bring them to life with a name, characteristics, goals and personal background, and develop multiple personas.
Scenarios
An informal narrative story - simple, 'natural', personal, and not generalisable. The Thomson family sailing-holiday scenario is the slides' worked example: four family members of different ages gather round a travel organizer, the system suggests a flotilla, the children object, descriptions from other children persuade them, and Will asks for the details to be printed because it is getting late.
Use cases
Assume interaction with a system and assume a detailed understanding of the interaction. Written as a numbered sequence of system and user steps, plus alternative courses for the failure paths - if the country name is invalid, display an error and return to step 3; if no visa information is found, display a message and return to step 1.
Essential use cases
Abstract away from the details and do not carry the same assumptions as use cases.
Task analysis completes the set. Task descriptions are often used to envision new systems or devices; task analysis is used mainly to investigate an existing situation. It is important not to focus on superficial activities: ask what people are trying to achieve, why they are trying to achieve it, and how they are going about it. The most popular technique is Hierarchical Task Analysis (HTA).
- HTA breaks a task into subtasks, then sub-sub-tasks, grouped as plans specifying how the tasks might be performed in practice.
- It focuses on physical and observable actions, and includes actions not related to software or an interaction device.
- Start with a user goal, examine it, and identify the main tasks for achieving it; then subdivide the tasks into subtasks.
0. In order to buy a DVD → 1. locate DVD → 2. add DVD to shopping basket → 3. enter payment details → 4. complete address → 5. confirm order. plan 0: If regular user do 1-2-5. If new user do 1-2-3-4-5. The plan is the part students forget, and it is the part that carries the analysis.
Persona = who. Scenario = a story about them. Use case = the exact steps. Essential use case = the steps with the details stripped out. HTA = the goal broken down, plus a plan.
Quantitative, qualitative, and three theoretical frameworks
Analysis starts soon after the data gathering session, with initial interpretation before deeper analysis. Different approaches emphasize different elements - class diagrams for object-oriented systems, entity-relationship diagrams for data-intensive systems.
| Quantitative | Qualitative | |
|---|---|---|
| Data | Expressed as numbers. | Difficult to measure sensibly as numbers - counting words to measure dissatisfaction is the lecture's warning example. |
| Analysis | Numerical methods to ascertain size, magnitude and amount. | Expresses the nature of elements, represented as themes, patterns and stories. |
| Simple techniques | Averages - mean (add and divide), median (middle value when ranked), mode (most frequent). Percentages. Graphical representations. | Recurring patterns or themes; categorizing data with an emergent or pre-specified scheme; looking for critical incidents to focus on key events. |
Mean, median and mode are different kinds of average and can give very different answers for the same data. And the standing warning: be careful how you manipulate data and numbers.
Grounded theory
Aims to derive theory from systematic analysis of data, based on a categorization approach called coding, in three levels: open - identify categories; axial - flesh out and link to subcategories; selective - form a theoretical scheme. Researchers are encouraged to draw on their own theoretical backgrounds to inform analysis.
Distributed cognition
Used as an analytic framework - the same framework introduced in Lecture 2.
Activity theory
Explains human behaviour in terms of our practical activity in the world. Provides a framework focusing analysis around the concept of an activity and helps identify tensions between the different elements of the system. Two key models: one outlining what constitutes an activity, and one modelling the mediating role of artifacts.
- Tools: spreadsheets (simple, basic graphs); statistical packages such as SPSS; qualitative data analysis tools for categorization and theme-based analysis such as N6; and the CAQDAS Networking Project at the University of Surrey.
- Presenting the findings: only make claims your data can support. The best presentation depends on the audience, the purpose, and the gathering and analysis undertaken. Techniques include rigorous notations such as UML, using stories to create scenarios, and summarizing the findings.
Presentation of the findings should not overstate the evidence. The data analysis that can be done depends on the data gathering that was done - you cannot analyse your way out of a badly designed study.
Mistakes students usually make
Each claim below is the wrong answer; the line beneath it is the correction, in the wording this course marks against.
Shortest correct answers
The night-before table: every term in this lecture with the smallest answer that still earns the mark.
| Concept | Shortest correct answer |
|---|---|
| Two aims of requirements | Understand users, task and context; produce a stable set of requirements. |
| Five key issues | Setting goals, identifying participants, relationship with participants, triangulation, pilot studies. |
| Interview types | Unstructured (rich, not replicable), structured (replicable, may lack richness), semi-structured (balanced), group. |
| Interview stages | Introduction, warm-up, main body, cool-off, closure. |
| Response rate | 40% is high; 20% is often acceptable. |
| Observation types | Direct (think-aloud) and indirect (diaries, interaction logs, web analytics). |
| Web analytics | Measuring, collecting, analyzing and reporting web data to optimize web usage. |
| Goetz & LeCompte | Who is present, what is their role, what is happening, when, where, why, how is it organized. |
| Contextual inquiry | Ethnographic interview at the user's workplace, 2-3 hours; context, partnership, interpretation, focus; user is expert, designer is apprentice. |
| Kinds of requirements | Functional, non-functional, data, environment (physical/social/organisational), users. |
| Persona | Not a real person; synthesised from real user characteristics; not idealised; multiple personas. |
| Scenario | Informal narrative story - simple, natural, personal, not generalisable. |
| Use case | Assumes interaction with a system and detailed understanding; numbered steps plus alternative courses. |
| Essential use case | Abstracts away from the details and drops the use case's assumptions. |
| HTA | Break a task into subtasks grouped as plans; focuses on physical and observable actions. |
| Averages | Mean (add and divide), median (middle when ranked), mode (most frequent). |
| Grounded theory coding | Open (identify categories), axial (flesh out and link subcategories), selective (form theoretical scheme). |
| Activity theory | Explains behaviour through practical activity; identifies tensions; models the mediating role of artifacts. |
Exam-style application
Write your own answer first, then open the model answer. These are the longer-form questions this material generates.
You have two weeks and no budget to establish requirements for a hospital shift-handover tool. Design the programme, justifying each technique.
Write a persona, a scenario fragment and a use-case step for a pharmacy stock app, and say what each one adds that the others do not.
A colleague reports "the average task time was 4 minutes" from ten trials, eight of which took about 2 minutes and two of which took 15. Critique this, then say what analysis you would present.
Check yourself
6 questions. Every option is explained after submitting, including why the wrong ones are wrong.