Recent from talks
Nothing was collected or created yet.
Usability
View on WikipediaUsability or user friendliness can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience.[1] In software engineering, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.[2]
The object of use can be a software application, website, book, tool, machine, process, vehicle, or anything a human interacts with. A usability study may be conducted as a primary job function by a usability analyst or as a secondary job function by designers, technical writers, marketing personnel, and others. It is widely used in consumer electronics, communication, and knowledge transfer objects (such as a cookbook, a document or online help) and mechanical objects such as a door handle or a hammer.
Usability includes methods of measuring usability, such as needs analysis[3] and the study of the principles behind an object's perceived efficiency or elegance. In human-computer interaction and computer science, usability studies the elegance and clarity with which the interaction with a computer program or a web site (web usability) is designed. Usability considers user satisfaction and utility as quality components, and aims to improve user experience through iterative design.[4]
Introduction
[edit]The term "user friendly" must surely rate as the inanity of the decade. When was the last time you thought of a tool as "friendly"? "Usable" and "useful" are the appropriate operative terms.
The primary notion of usability is that an object designed with a generalized users' psychology and physiology in mind is, for example:
- More efficient to use—takes less time to accomplish a particular task
- Easier to learn—operation can be learned by observing the object
- More satisfying to use
Complex computer systems find their way into everyday life, and at the same time the market is saturated with competing brands. This has made usability more popular and widely recognized in recent years, as companies see the benefits of researching and developing their products with user-oriented methods instead of technology-oriented methods. By understanding and researching the interaction between product and user, the usability expert can also provide insight that is unattainable by traditional company-oriented market research. For example, after observing and interviewing users, the usability expert may identify needed functionality or design flaws that were not anticipated. A method called contextual inquiry does this in the naturally occurring context of the users own environment. In the user-centered design paradigm, the product is designed with its intended users in mind at all times. In the user-driven or participatory design paradigm, some of the users become actual or de facto members of the design team.[6]
The term user friendly is often used as a synonym for usable, though it may also refer to accessibility. Usability describes the quality of user experience across websites, software, products, and environments. There is no consensus about the relation of the terms ergonomics (or human factors) and usability. Some think of usability as the software specialization of the larger topic of ergonomics. Others view these topics as tangential, with ergonomics focusing on physiological matters (e.g., turning a door handle) and usability focusing on psychological matters (e.g., recognizing that a door can be opened by turning its handle). Usability is also important in website development (web usability). According to Jakob Nielsen, "Studies of user behavior on the Web find a low tolerance for difficult designs or slow sites. People don't want to wait. And they don't want to learn how to use a home page. There's no such thing as a training class or a manual for a Web site. People have to be able to grasp the functioning of the site immediately after scanning the home page—for a few seconds at most."[7] Otherwise, most casual users simply leave the site and browse or shop elsewhere.
Usability can also include the concept of prototypicality, which is how much a particular thing conforms to the expected shared norm, for instance, in website design, users prefer sites that conform to recognised design norms.[8]
Definition
[edit]ISO defines usability as "The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use." The word "usability" also refers to methods for improving ease-of-use during the design process. Usability consultant Jakob Nielsen and computer science professor Ben Shneiderman have written (separately) about a framework of system acceptability, where usability is a part of "usefulness" and is composed of:[9]
- Learnability: How easy is it for users to accomplish basic tasks the first time they encounter the design?
- Efficiency: Once users have learned the design, how quickly can they perform tasks?
- Memorability: When users return to the design after a period of not using it, how easily can they re-establish proficiency?
- Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?
- Satisfaction: How pleasant is it to use the design?
Usability is often associated with the functionalities of the product (cf. ISO definition, below), in addition to being solely a characteristic of the user interface (cf. framework of system acceptability, also below, which separates usefulness into usability and utility). For example, in the context of mainstream consumer products, an automobile lacking a reverse gear could be considered unusable according to the former view, and lacking in utility according to the latter view. When evaluating user interfaces for usability, the definition can be as simple as "the perception of a target user of the effectiveness (fit for purpose) and efficiency (work or time required to use) of the Interface"[citation needed]. Each component may be measured subjectively against criteria, e.g., Principles of User Interface Design, to provide a metric, often expressed as a percentage. It is important to distinguish between usability testing and usability engineering. Usability testing is the measurement of ease of use of a product or piece of software. In contrast, usability engineering (UE) is the research and design process that ensures a product with good usability. Usability is a non-functional requirement. As with other non-functional requirements, usability cannot be directly measured but must be quantified by means of indirect measures or attributes such as, for example, the number of reported problems with ease-of-use of a system.
Intuitive interaction or intuitive use
[edit]The term intuitive is often listed as a desirable trait in usable interfaces, sometimes used as a synonym for learnable. In the past, Jef Raskin discouraged using this term in user interface design, claiming that easy to use interfaces are often easy because of the user's exposure to previous similar systems, thus the term 'familiar' should be preferred.[10] As an example: Two vertical lines "||" on media player buttons do not intuitively mean "pause"—they do so by convention. This association between intuitive use and familiarity has since been empirically demonstrated in multiple studies by a range of researchers across the world, and intuitive interaction is accepted in the research community as being use of an interface based on past experience with similar interfaces or something else, often not fully conscious,[11] and sometimes involving a feeling of "magic"[12] since the course of the knowledge itself may not be consciously available to the user . Researchers have also investigated intuitive interaction for older people,[13] people living with dementia,[14] and children.[15]
Some have argued that aiming for "intuitive" interfaces (based on reusing existing skills with interaction systems) could lead designers to discard a better design solution only because it would require a novel approach and to stick with boring designs. However, applying familiar features into a new interface has been shown not to result in boring design if designers use creative approaches rather than simple copying.[16] The throwaway remark that "the only intuitive interface is the nipple; everything else is learned."[17] is still occasionally mentioned. Any breastfeeding mother or lactation consultant will tell you this is inaccurate and the nipple does in fact require learning on both sides. In 1992, Bruce Tognazzini even denied the existence of "intuitive" interfaces, since such interfaces must be able to intuit, i.e., "perceive the patterns of the user's behavior and draw inferences."[18] Instead, he advocated the term "intuitable," i.e., "that users could intuit the workings of an application by seeing it and using it". However, the term intuitive interaction has become well accepted in the research community over the past 20 or so years and, although not perfect, it should probably be accepted and used.
ISO standards
[edit]ISO/TR 16982:2002 standard
[edit]ISO/TR 16982:2002 ("Ergonomics of human-system interaction—Usability methods supporting human-centered design") is an International Standards Organization (ISO) standard that provides information on human-centered usability methods that can be used for design and evaluation. It details the advantages, disadvantages, and other factors relevant to using each usability method. It explains the implications of the stage of the life cycle and the individual project characteristics for the selection of usability methods and provides examples of usability methods in context. The main users of ISO/TR 16982:2002 are project managers. It therefore addresses technical human factors and ergonomics issues only to the extent necessary to allow managers to understand their relevance and importance in the design process as a whole. The guidance in ISO/TR 16982:2002 can be tailored for specific design situations by using the lists of issues characterizing the context of use of the product to be delivered. Selection of appropriate usability methods should also take account of the relevant life-cycle process. ISO/TR 16982:2002 is restricted to methods that are widely used by usability specialists and project managers. It does not specify the details of how to implement or carry out the usability methods described.
ISO 9241 standard
[edit]ISO 9241 is a multi-part standard that covers a number of aspects of people working with computers. Although originally titled Ergonomic requirements for office work with visual display terminals (VDTs), it has been retitled to the more generic Ergonomics of Human System Interaction.[19] As part of this change, ISO is renumbering some parts of the standard so that it can cover more topics, e.g. tactile and haptic interaction. The first part to be renumbered was part 10 in 2006, now part 110.[20]
IEC 62366
[edit]IEC 62366-1:2015 + COR1:2016 & IEC/TR 62366-2 provide guidance on usability engineering specific to a medical device.
Designing for usability
[edit]Any system or device designed for use by people should be easy to use, easy to learn, easy to remember (the instructions), and helpful to users. John Gould and Clayton Lewis recommend that designers striving for usability follow these three design principles[21]
- Early focus on end users and the tasks they need the system/device to do
- Empirical measurement using quantitative or qualitative measures
- Iterative design, in which the designers work in a series of stages, improving the design each time
Early focus on users and tasks
[edit]The design team should be user-driven and it should be in direct contact with potential users. Several evaluation methods, including personas, cognitive modeling, inspection, inquiry, prototyping, and testing methods may contribute to understanding potential users and their perceptions of how well the product or process works. Usability considerations, such as who the users are and their experience with similar systems must be examined. As part of understanding users, this knowledge must "...be played against the tasks that the users will be expected to perform."[21] This includes the analysis of what tasks the users will perform, which are most important, and what decisions the users will make while using your system. Designers must understand how cognitive and emotional characteristics of users will relate to a proposed system. One way to stress the importance of these issues in the designers' minds is to use personas, which are made-up representative users. See below for further discussion of personas. Another more expensive but more insightful method is to have a panel of potential users work closely with the design team from the early stages.[22]
Empirical measurement
[edit]Test the system early on, and test the system on real users using behavioral measurements. This includes testing the system for both learnability and usability. (See Evaluation Methods). It is important in this stage to use quantitative usability specifications such as time and errors to complete tasks and number of users to test, as well as examine performance and attitudes of the users testing the system.[22] Finally, "reviewing or demonstrating" a system before the user tests it can result in misleading results. The emphasis of empirical measurement is on measurement, both informal and formal, which can be carried out through a variety of evaluation methods.[21]
Iterative design
[edit]Iterative design is a design methodology based on a cyclic process of prototyping, testing, analyzing, and refining a product or process. Based on the results of testing the most recent iteration of a design, changes and refinements are made. This process is intended to ultimately improve the quality and functionality of a design. In iterative design, interaction with the designed system is used as a form of research for informing and evolving a project, as successive versions, or iterations of a design are implemented. The key requirements for Iterative Design are: identification of required changes, an ability to make changes, and a willingness to make changes. When a problem is encountered, there is no set method to determine the correct solution. Rather, there are empirical methods that can be used during system development or after the system is delivered, usually a more inopportune time. Ultimately, iterative design works towards meeting goals such as making the system user friendly, easy to use, easy to operate, simple, etc.[22]
Evaluation methods
[edit]There are a variety of usability evaluation methods. Certain methods use data from users, while others rely on usability experts. There are usability evaluation methods for all stages of design and development, from product definition to final design modifications. When choosing a method, consider cost, time constraints, and appropriateness. For a brief overview of methods, see Comparison of usability evaluation methods or continue reading below. Usability methods can be further classified into the subcategories below.
Cognitive modeling methods
[edit]Cognitive modeling involves creating a computational model to estimate how long it takes people to perform a given task. Models are based on psychological principles and experimental studies to determine times for cognitive processing and motor movements. Cognitive models can be used to improve user interfaces or predict problem errors and pitfalls during the design process. A few examples of cognitive models include:
Parallel design
[edit]With parallel design, several people create an initial design from the same set of requirements. Each person works independently, and when finished, shares concepts with the group. The design team considers each solution, and each designer uses the best ideas to further improve their own solution. This process helps generate many different, diverse ideas, and ensures that the best ideas from each design are integrated into the final concept. This process can be repeated several times until the team is satisfied with the final concept.
GOMS
[edit]GOMS stands for goals, operators, methods, and selection rules. It is a family of techniques that analyzes the user complexity of interactive systems. Goals are what the user must accomplish. An operator is an action performed in pursuit of a goal. A method is a sequence of operators that accomplish a goal. Selection rules specify which method satisfies a given goal, based on context.
Human processor model
[edit]Sometimes it is useful to break a task down and analyze each individual aspect separately. This helps the tester locate specific areas for improvement. To do this, it is necessary to understand how the human brain processes information. A model of the human processor is shown below.
Many studies have been done to estimate the cycle times, decay times, and capacities of each of these processors. Variables that affect these can include subject age, aptitudes, ability, and the surrounding environment. For a younger adult, reasonable estimates are:
| Parameter | Mean | Range |
|---|---|---|
| Eye movement time | 230 ms | 70–700 ms |
| Decay half-life of visual image storage | 200 ms | 90–1000 ms |
| Perceptual processor cycle time | 100 ms | 50–200 ms |
| Cognitive processor cycle time | 70 ms | 25–170 ms |
| Motor processor cycle time | 70 ms | 30–100 ms |
| Effective working memory capacity | 2 items | 2–3 items |
Long-term memory is believed to have an infinite capacity and decay time.[23]
Keystroke level modeling
[edit]Keystroke level modeling is essentially a less comprehensive version of GOMS that makes simplifying assumptions in order to reduce calculation time and complexity.
Inspection methods
[edit]These usability evaluation methods involve observation of users by an experimenter, or the testing and evaluation of a program by an expert reviewer. They provide more quantitative data as tasks can be timed and recorded.
Card sorts
[edit]Card sorting is a way to involve users in grouping information for a website's usability review. Participants in a card sorting session are asked to organize the content from a Web site in a way that makes sense to them. Participants review items from a Web site and then group these items into categories. Card sorting helps to learn how users think about the content and how they would organize the information on the Web site. Card sorting helps to build the structure for a Web site, decide what to put on the home page, and label the home page categories. It also helps to ensure that information is organized on the site in a way that is logical to users.
Tree tests
[edit]Tree testing is a way to evaluate the effectiveness of a website's top-down organization. Participants are given "find it" tasks, then asked to drill down through successive text lists of topics and subtopics to find a suitable answer. Tree testing evaluates the findability and labeling of topics in a site, separate from its navigation controls or visual design.
Ethnography
[edit]Ethnographic analysis is derived from anthropology. Field observations are taken at a site of a possible user, which track the artifacts of work such as Post-It notes, items on desktop, shortcuts, and items in trash bins. These observations also gather the sequence of work and interruptions that determine the user's typical day.
Heuristic evaluation
[edit]Heuristic evaluation is a usability engineering method for finding and assessing usability problems in a user interface design as part of an iterative design process. It involves having a small set of evaluators examining the interface and using recognized usability principles (the "heuristics"). It is the most popular of the usability inspection methods, as it is quick, cheap, and easy. Heuristic evaluation was developed to aid in the design of computer user-interface design. It relies on expert reviewers to discover usability problems and then categorize and rate them by a set of principles (heuristics.) It is widely used based on its speed and cost-effectiveness. Jakob Nielsen's list of ten heuristics is the most commonly used in industry. These are ten general principles for user interface design. They are called "heuristics" because they are more in the nature of rules of thumb than specific usability guidelines.
- Visibility of system status: The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
- Match between system and the real world: The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
- User control and freedom: Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
- Consistency and standards: Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
- Error prevention: Even better than good error messages is a careful design that prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
- Recognition rather than recall:[24] Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
- Flexibility and efficiency of use: Accelerators—unseen by the novice user—may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
- Aesthetic and minimalist design: Dialogues should not contain information that is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
- Help users recognize, diagnose, and recover from errors: Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
- Help and documentation: Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.
Thus, by determining which guidelines are violated, the usability of a device can be determined.
Usability inspection
[edit]Usability inspection is a review of a system based on a set of guidelines. The review is conducted by a group of experts who are deeply familiar with the concepts of usability in design. The experts focus on a list of areas in design that have been shown to be troublesome for users.
Pluralistic inspection
[edit]Pluralistic Inspections are meetings where users, developers, and human factors people meet together to discuss and evaluate step by step of a task scenario. As more people inspect the scenario for problems, the higher the probability to find problems. In addition, the more interaction in the team, the faster the usability issues are resolved.
Consistency inspection
[edit]In consistency inspection, expert designers review products or projects to ensure consistency across multiple products to look if it does things in the same way as their own designs.
Activity Analysis
[edit]Activity analysis is a usability method used in preliminary stages of development to get a sense of situation. It involves an investigator observing users as they work in the field. Also referred to as user observation, it is useful for specifying user requirements and studying currently used tasks and subtasks. The data collected are qualitative and useful for defining the problem. It should be used when you wish to frame what is needed, or "What do we want to know?"
Inquiry methods
[edit]The following usability evaluation methods involve collecting qualitative data from users. Although the data collected is subjective, it provides valuable information on what the user wants.
Task analysis
[edit]Task analysis means learning about users' goals and users' ways of working. Task analysis can also mean figuring out what more specific tasks users must do to meet those goals and what steps they must take to accomplish those tasks. Along with user and task analysis, a third analysis is often used: understanding users' environments (physical, social, cultural, and technological environments).
Focus groups
[edit]A focus group is a focused discussion where a moderator leads a group of participants through a set of questions on a particular topic. Although typically used as a marketing tool, focus groups are sometimes used to evaluate usability. Used in the product definition stage, a group of 6 to 10 users are gathered to discuss what they desire in a product. An experienced focus group facilitator is hired to guide the discussion to areas of interest for the developers. Focus groups are typically videotaped to help get verbatim quotes, and clips are often used to summarize opinions. The data gathered is not usually quantitative, but can help get an idea of a target group's opinion.
Questionnaires/surveys
[edit]Surveys have the advantages of being inexpensive, require no testing equipment, and results reflect the users' opinions. When written carefully and given to actual users who have experience with the product and knowledge of design, surveys provide useful feedback on the strong and weak areas of the usability of a design. This is a very common method and often does not appear to be a survey, but just a warranty card.
Prototyping methods
[edit]It is often very difficult for designers to conduct usability tests with the exact system being designed. Cost constraints, size, and design constraints usually lead the designer to creating a prototype of the system. Instead of creating the complete final system, the designer may test different sections of the system, thus making several small models of each component of the system. Prototyping is an attitude and an output, as it is a process for generating and reflecting on tangible ideas by allowing failure to occur early.[25] prototyping helps people to see what could be of communicating a shared vision, and of giving shape to the future. The types of usability prototypes may vary from using paper models, index cards, hand drawn models, or storyboards.[26] Prototypes are able to be modified quickly, often are faster and easier to create with less time invested by designers and are more apt to change design; although sometimes are not an adequate representation of the whole system, are often not durable and testing results may not be parallel to those of the actual system.
The Tool Kit Approach
[edit]This tool kit is a wide library of methods that used the traditional programming language and it is primarily developed for computer programmers. The code created for testing in the tool kit approach can be used in the final product. However, to get the highest benefit from the tool, the user must be an expert programmer.[27]
The Parts Kit Approach
[edit]The two elements of this approach include a parts library and a method used for identifying the connection between the parts. This approach can be used by almost anyone and it is a great asset for designers with repetitive tasks.[27]
Animation Language Metaphor
[edit]This approach is a combination of the tool kit approach and the part kit approach. Both the dialogue designers and the programmers are able to interact with this prototyping tool.[27]
Rapid prototyping
[edit]Rapid prototyping is a method used in early stages of development to validate and refine the usability of a system. It can be used to quickly and cheaply evaluate user-interface designs without the need for an expensive working model. This can help remove hesitation to change the design, since it is implemented before any real programming begins. One such method of rapid prototyping is paper prototyping.
Testing methods
[edit]These usability evaluation methods involve testing of subjects for the most quantitative data. Usually recorded on video, they provide task completion time and allow for observation of attitude. Regardless to how carefully a system is designed, all theories must be tested using usability tests. Usability tests involve typical users using the system (or product) in a realistic environment [see simulation]. Observation of the user's behavior, emotions, and difficulties while performing different tasks, often identify areas of improvement for the system.
Metrics
[edit]While conducting usability tests, designers must use usability metrics to identify what it is they are going to measure, or the usability metrics. These metrics are often variable, and change in conjunction with the scope and goals of the project. The number of subjects being tested can also affect usability metrics, as it is often easier to focus on specific demographics. Qualitative design phases, such as general usability (can the task be accomplished?), and user satisfaction are also typically done with smaller groups of subjects.[28] Using inexpensive prototypes on small user groups provides more detailed information, because of the more interactive atmosphere, and the designer's ability to focus more on the individual user.
As the designs become more complex, the testing must become more formalized. Testing equipment will become more sophisticated and testing metrics become more quantitative. With a more refined prototype, designers often test effectiveness, efficiency, and subjective satisfaction, by asking the user to complete various tasks. These categories are measured by the percent that complete the task, how long it takes to complete the tasks, ratios of success to failure to complete the task, time spent on errors, the number of errors, rating scale of satisfactions, number of times user seems frustrated, etc.[29] Additional observations of the users give designers insight on navigation difficulties, controls, conceptual models, etc. The ultimate goal of analyzing these metrics is to find/create a prototype design that users like and use to successfully perform given tasks.[26] After conducting usability tests, it is important for a designer to record what was observed, in addition to why such behavior occurred and modify the model according to the results. Often it is quite difficult to distinguish the source of the design errors, and what the user did wrong. However, effective usability tests will not generate a solution to the problems, but provide modified design guidelines for continued testing.
Remote usability testing
[edit]Remote usability testing (also known as unmoderated or asynchronous usability testing) involves the use of a specially modified online survey, allowing the quantification of user testing studies by providing the ability to generate large sample sizes, or a deep qualitative analysis without the need for dedicated facilities. Additionally, this style of user testing also provides an opportunity to segment feedback by demographic, attitudinal and behavioral type. The tests are carried out in the user's own environment (rather than labs) helping further simulate real-life scenario testing. This approach also provides a vehicle to easily solicit feedback from users in remote areas. There are two types, quantitative or qualitative. Quantitative use large sample sized and task based surveys. These types of studies are useful for validating suspected usability issues. Qualitative studies are best used as exploratory research, in small sample sizes but frequent, even daily iterations. Qualitative usually allows for observing respondent's screens and verbal think aloud commentary (Screen Recording Video, SRV), and for a richer level of insight also include the webcam view of the respondent (Video-in-Video, ViV, sometimes referred to as Picture-in-Picture, PiP)
Remote usability testing for mobile devices
[edit]The growth in mobile and associated platforms and services (e.g.: Mobile gaming has experienced 20x growth in 2010–2012) has generated a need for unmoderated remote usability testing on mobile devices, both for websites but especially for app interactions. One methodology consists of shipping cameras and special camera holding fixtures to dedicated testers, and having them record the screens of the mobile smart-phone or tablet device, usually using an HD camera. A drawback of this approach is that the finger movements of the respondent can obscure the view of the screen, in addition to the bias and logistical issues inherent in shipping special hardware to selected respondents. A newer approach uses a wireless projection of the mobile device screen onto the computer desktop screen of the respondent, who can then be recorded through their webcam, and thus a combined Video-in-Video view of the participant and the screen interactions viewed simultaneously while incorporating the verbal think aloud commentary of the respondents.
Thinking aloud
[edit]The Think aloud protocol is a method of gathering data that is used in both usability and psychology studies. It involves getting a user to verbalize their thought processes (i.e. expressing their opinions, thoughts, anticipations, and actions)[30] as they perform a task or set of tasks. As a widespread method of usability testing, think aloud provides the researchers with the ability to discover what user really think during task performance and completion.[30]
Often an instructor is present to prompt the user into being more vocal as they work. Similar to the Subjects-in-Tandem method, it is useful in pinpointing problems and is relatively simple to set up. Additionally, it can provide insight into the user's attitude, which can not usually be discerned from a survey or questionnaire.
RITE method
[edit]Rapid Iterative Testing and Evaluation (RITE)[31] is an iterative usability method similar to traditional "discount" usability testing. The tester and team must define a target population for testing, schedule participants to come into the lab, decide on how the users behaviors will be measured, construct a test script and have participants engage in a verbal protocol (e.g., think aloud). However it differs from these methods in that it advocates that changes to the user interface are made as soon as a problem is identified and a solution is clear. Sometimes this can occur after observing as few as 1 participant. Once the data for a participant has been collected the usability engineer and team decide if they will be making any changes to the prototype prior to the next participant. The changed interface is then tested with the remaining users.
Subjects-in-tandem or co-discovery
[edit]Subjects-in-tandem (also called co-discovery) is the pairing of subjects in a usability test to gather important information on the ease of use of a product. Subjects tend to discuss the tasks they have to accomplish out loud and through these discussions observers learn where the problem areas of a design are. To encourage co-operative problem-solving between the two subjects, and the attendant discussions leading to it, the tests can be designed to make the subjects dependent on each other by assigning them complementary areas of responsibility (e.g. for testing of software, one subject may be put in charge of the mouse and the other of the keyboard.)
Component-based usability testing
[edit]Component-based usability testing is an approach which aims to test the usability of elementary units of an interaction system, referred to as interaction components. The approach includes component-specific quantitative measures based on user interaction recorded in log files, and component-based usability questionnaires.
Other methods
[edit]Cognitive walkthrough
[edit]Cognitive walkthrough is a method of evaluating the user interaction of a working prototype or final product. It is used to evaluate the system's ease of learning. Cognitive walkthrough is useful to understand the user's thought processes and decision making when interacting with a system, specially for first-time or infrequent users.
Benchmarking
[edit]Benchmarking creates standardized test materials for a specific type of design. Four key characteristics are considered when establishing a benchmark: time to do the core task, time to fix errors, time to learn applications, and the functionality of the system. Once there is a benchmark, other designs can be compared to it to determine the usability of the system. Many of the common objectives of usability studies, such as trying to understand user behavior or exploring alternative designs, must be put aside. Unlike many other usability methods or types of labs studies, benchmark studies more closely resemble true experimental psychology lab studies, with greater attention to detail on methodology, study protocol and data analysis.[32]
Meta-analysis
[edit]Meta-analysis is a statistical procedure to combine results across studies to integrate the findings. This phrase was coined in 1976 as a quantitative literature review. This type of evaluation is very powerful for determining the usability of a device because it combines multiple studies to provide very accurate quantitative support.
Persona
[edit]Personas are fictitious characters created to represent a site or product's different user types and their associated demographics and technographics. Alan Cooper introduced the concept of using personas as a part of interactive design in 1998 in his book The Inmates Are Running the Asylum,[33] but had used this concept since as early as 1975. Personas are a usability evaluation method that can be used at various design stages. The most typical time to create personas is at the beginning of designing so that designers have a tangible idea of who the users of their product will be. Personas are the archetypes that represent actual groups of users and their needs, which can be a general description of person, context, or usage scenario. This technique turns marketing data on target user population into a few physical concepts of users to create empathy among the design team, with the final aim of tailoring a product more closely to how the personas will use it. To gather the marketing data that personas require, several tools can be used, including online surveys, web analytics, customer feedback forms, and usability tests, and interviews with customer-service representatives.[34]
Benefits
[edit]The key benefits of usability are:
- Higher revenues through increased sales
- Increased user efficiency and user satisfaction
- Reduced development costs
- Reduced support costs
Corporate integration
[edit]An increase in usability generally positively affects several facets of a company's output quality. In particular, the benefits fall into several common areas:[35]
- Increased productivity
- Decreased training and support costs
- Increased sales and revenues
- Reduced development time and costs
- Reduced maintenance costs
- Increased customer satisfaction
Increased usability in the workplace fosters several responses from employees: "Workers who enjoy their work do it better, stay longer in the face of temptation, and contribute ideas and enthusiasm to the evolution of enhanced productivity."[36] To create standards, companies often implement experimental design techniques that create baseline levels. Areas of concern in an office environment include (though are not necessarily limited to):[37]
- Working posture
- Design of workstation furniture
- Screen displays
- Input devices
- Organization issues
- Office environment
- Software interface
By working to improve said factors, corporations can achieve their goals of increased output at lower costs, while potentially creating optimal levels of customer satisfaction. There are numerous reasons why each of these factors correlates to overall improvement. For example, making software user interfaces easier to understand reduces the need for extensive training. The improved interface tends to lower the time needed to perform tasks, and so would both raise the productivity levels for employees and reduce development time (and thus costs). Each of the aforementioned factors are not mutually exclusive; rather they should be understood to work in conjunction to form the overall workplace environment. In the 2010s, usability is recognized as an important software quality attribute, earning its place among more traditional attributes such as performance, robustness and aesthetic appearance. Various academic programs focus on usability. Several usability consultancy companies have emerged, and traditional consultancy and design firms offer similar services.
There is some resistance to integrating usability work in organisations. Usability is seen as a vague concept, it is difficult to measure and other areas are prioritised when IT projects run out of time or money.[38]
Professional development
[edit]Usability practitioners are sometimes trained as industrial engineers, psychologists, kinesiologists, systems design engineers, or with a degree in information architecture, information or library science, or Human-Computer Interaction (HCI). More often though they are people who are trained in specific applied fields who have taken on a usability focus within their organization. Anyone who aims to make tools easier to use and more effective for their desired function within the context of work or everyday living can benefit from studying usability principles and guidelines. For those seeking to extend their training, the User Experience Professionals' Association offers online resources, reference lists, courses, conferences, and local chapter meetings. The UXPA also sponsors World Usability Day each November.[39] Related professional organizations include the Human Factors and Ergonomics Society (HFES) and the Association for Computing Machinery's special interest groups in Computer Human Interaction (SIGCHI), Design of Communication (SIGDOC) and Computer Graphics and Interactive Techniques (SIGGRAPH). The Society for Technical Communication also has a special interest group on Usability and User Experience (UUX). They publish a quarterly newsletter called Usability Interface.[40]
See also
[edit]- Accessibility
- Chief experience officer (CXO)
- Design for All (inclusion)
- Experience design
- Fitts's law
- Form follows function
- Gemba or customer visit
- GOMS
- Gotcha (programming)
- GUI
- Human factors
- Information architecture
- Interaction design
- Interactive systems engineering
- Internationalization
- Learnability
- List of human-computer interaction topics
- List of system quality attributes
- Machine-Readable Documents
- Natural mapping (interface design)
- Non-functional requirement
- RITE method
- System Usability Scale
- Universal usability
- Usability goals
- Usability testing
- Usability engineering
- User experience
- User experience design
- Web usability
- World Usability Day
References
[edit]- ^ Lee, Ju Yeon; Kim, Ju Young; You, Seung Ju; Kim, You Soo; Koo, Hye Yeon; Kim, Jeong Hyun; Kim, Sohye; Park, Jung Ha; Han, Jong Soo; Kil, Siye; Kim, Hyerim (2019-09-30). "Development and Usability of a Life-Logging Behavior Monitoring Application for Obese Patients". Journal of Obesity & Metabolic Syndrome. 28 (3): 194–202. doi:10.7570/jomes.2019.28.3.194. ISSN 2508-6235. PMC 6774444. PMID 31583384.
- ^ Ergonomic Requirements for Office Work with Visual Display Terminals, ISO 9241–11, ISO, Geneva, 1998.
- ^ Smith, K Tara (2011). "Needs Analysis: Or, How Do You Capture, Represent, and Validate User Requirements in a Formal Manner/Notation before Design". In Karwowski, W.; Soares, M.M.; Stanton, N.A. (eds.). Human Factors and Ergonomics in Consumer Product Design: Methods and Techniques (Handbook of Human Factors in Consumer Product Design). CRC Press.
- ^ Nielsen, Jakob (4 January 2012). "Usability 101: Introduction to Usability". Nielsen Norman Group. Archived from the original on 1 September 2016. Retrieved 7 August 2016.
- ^ Rutkowski, Chris (October 1982). "An Introduction to the Human Applications Standard Computer Interface Part 1: Theory and Principles". BYTE. pp. 291–310. Retrieved 2024-12-30.
- ^ Holm, Ivar (2006). Ideas and Beliefs in Architecture and Industrial design: How attitudes, orientations, and underlying assumptions shape the built environment. Oslo School of Architecture and Design. ISBN 82-547-0174-1.
- ^ Nielsen, Jakob; Norman, Donald A. (14 January 2000). "Web-Site Usability: Usability On The Web Isn't A Luxury". JND.org. Archived from the original on 28 March 2015.
- ^ Tuch, Alexandre N.; Presslaber, Eva E.; Stöcklin, Markus; Opwis, Klaus; Bargas-Avila, Javier A. (2012-11-01). "The role of visual complexity and prototypicality regarding first impression of websites: Working towards understanding aesthetic judgments". International Journal of Human-Computer Studies. 70 (11): 794–811. doi:10.1016/j.ijhcs.2012.06.003. ISSN 1071-5819. S2CID 9051274.
- ^ Usability 101: Introduction to Usability Archived 2011-04-08 at the Wayback Machine, Jakob Nielsen's Alertbox. Retrieved 2010-06-01
- ^ Intuitive equals familiar Archived 2009-10-05 at the Wayback Machine, Communications of the ACM. 37:9, September 1994, pg. 17.
- ^ Alethea Blackler, ed. (2018). Intuitive interaction : research and application. Boca Raton, FL: CRC Press. ISBN 978-1-315-16714-5. OCLC 1044734346.
- ^ Ullrich, Daniel; Diefenbach, Sarah (2010-10-16). "From magical experience to effortlessness". Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries. NordiCHI '10. New York, NY, USA: Association for Computing Machinery. pp. 801–804. doi:10.1145/1868914.1869033. ISBN 978-1-60558-934-3. S2CID 5378990.
- ^ Lawry, Simon; Popovic, Vesna; Blackler, Alethea; Thompson, Helen (January 2019). "Age, familiarity, and intuitive use: An empirical investigation". Applied Ergonomics. 74: 74–84. doi:10.1016/j.apergo.2018.08.016. PMID 30487112. S2CID 54105210.
- ^ Blackler, Alethea; Chen, Li-Hao; Desai, Shital; Astell, Arlene (2020), Brankaert, Rens; Kenning, Gail (eds.), "Intuitive Interaction Framework in User-Product Interaction for People Living with Dementia", HCI and Design in the Context of Dementia, Human–Computer Interaction Series, Cham: Springer International Publishing, pp. 147–169, doi:10.1007/978-3-030-32835-1_10, ISBN 978-3-030-32834-4, S2CID 220794844, retrieved 2022-10-24
- ^ Desai, Shital; Blackler, Alethea; Popovic, Vesna (2019-09-01). "Children's embodied intuitive interaction – Design aspects of embodiment". International Journal of Child-Computer Interaction. 21: 89–103. doi:10.1016/j.ijcci.2019.06.001. ISSN 2212-8689. S2CID 197709773.
- ^ Hurtienne, J.; Klockner, K.; Diefenbach, S.; Nass, C.; Maier, A. (2015-05-01). "Designing with Image Schemas: Resolving the Tension Between Innovation, Inclusion and Intuitive Use". Interacting with Computers. 27 (3): 235–255. doi:10.1093/iwc/iwu049. ISSN 0953-5438.
- ^ Kettlewell, Richard. "The Only Intuitive Interface Is The Nipple". Greenend.org.uk. Archived from the original on 2012-01-30. Retrieved 2013-11-01.
- ^ Tognazzini, B. (1992), Tog on Interface, Boston, MA: Addison-Wesley, p. 246.
- ^ "ISO 9241". 1992. Archived from the original on 2012-01-12.
- ^ "ISO 9241-10:1996". International Organization for Standardization. Archived from the original on 26 July 2011. Retrieved 22 July 2011.
- ^ a b c Gould, J.D., Lewis, C.: "Designing for Usability: Key Principles and What Designers Think", Communications of the ACM, March 1985, 28(3)
- ^ a b c [1] Archived November 27, 2010, at the Wayback Machine
- ^ Card, S.K., Moran, T.P., & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.
- ^ "Memory Recognition and Recall in User Interfaces". www.nngroup.com. Archived from the original on 2017-01-05. Retrieved 2017-01-04.
- ^ Short, Eden Jayne; Reay, Stephen; Gilderdale, Peter (2017-07-28). "Wayfinding for health seeking: Exploring how hospital wayfinding can employ communication design to improve the outpatient experience". The Design Journal. 20 (sup1): S2551 – S2568. doi:10.1080/14606925.2017.1352767. ISSN 1460-6925.
- ^ a b Wickens, C.D et al. (2004). An Introduction to Human Factors Engineering (2nd Ed), Pearson Education, Inc., Upper Saddle River, NJ : Prentice Hall.
- ^ a b c Wilson, James; Rosenberg, Daniel (1988-01-01), Helander, MARTIN (ed.), "Chapter 39 - Rapid Prototyping for User Interface Design", Handbook of Human-Computer Interaction, North-Holland, pp. 859–875, doi:10.1016/b978-0-444-70536-5.50044-0, ISBN 978-0-444-70536-5, retrieved 2020-04-02
- ^ Dumas, J.S. and Redish, J.C. (1999). A Practical Guide to Usability Testing (revised ed.), Bristol, U.K.: Intellect Books.
- ^ Kuniavsky, M. (2003). Observing the User Experience: A Practitioner's Guide to User Research, San Francisco, CA: Morgan Kaufmann.
- ^ a b Georgsson, Mattias; Staggers, Nancy (January 2016). "Quantifying usability: an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics". Journal of the American Medical Informatics Association. 23 (1): 5–11. doi:10.1093/jamia/ocv099. ISSN 1067-5027. PMC 4713903. PMID 26377990.
- ^ Medlock, M.C., Wixon, D., Terrano, M., Romero, R., and Fulton, B. (2002). Using the RITE method to improve products: A definition and a case study. Presented at the Usability Professionsals Association 2002, Orlando FL.
- ^ "#27 – The art of usability benchmarking". Scottberkun.com. 2010-04-16. Archived from the original on 2013-11-04. Retrieved 2013-11-01.
- ^ Cooper, A. (1999). The Inmates Are Running the Asylum, Sams Publishers, ISBN 0-672-31649-8
- ^ "How I Built 4 Personas For My SEO Site". Seoroi.com. Archived from the original on 2013-11-03. Retrieved 2013-11-01.
- ^ "Usability Resources: Usability in the Real World: Business Benefits". Usabilityprofessionals.org. Archived from the original on 2013-10-31. Retrieved 2013-11-01.
- ^ Landauer, T. K. (1996). The trouble with computers. Cambridge, MA, The MIT Press. p158.
- ^ McKeown, Celine (2008). Office ergonomics: practical applications. Boca Raton, FL, Taylor & Francis Group, LLC.
- ^ Cajander, Åsa (2010), Usability - who cares? : the introduction of user-centred systems design in organisations, Acta Universitatis Upsaliensis, ISBN 9789155477974, OCLC 652387306
- ^ "UXPA - The User Experience Professionals Association". Usabilityprofessionals.org. 2013-03-31. Archived from the original on 2013-10-21. Retrieved 2013-11-01.
- ^ "STC Usability Interface - Newsletter Home Page". Stcsig.org. Archived from the original on 2013-10-23. Retrieved 2013-11-01.
Further reading
[edit]- R. G. Bias and D. J. Mayhew (eds) (2005), Cost-Justifying Usability: An Update for the Internet Age, Morgan Kaufmann
- Donald A. Norman (2013), The Design of Everyday Things, Basic Books, ISBN 0-465-07299-2
- Donald A. Norman (2004), Emotional Design: Why we love (or hate) everyday things, Basic Books, ISBN 0-465-05136-7
- Jakob Nielsen (1994), Usability Engineering, Morgan Kaufmann Publishers, ISBN 0-12-518406-9
- Jakob Nielsen (1994), Usability Inspection Methods, John Wiley & Sons, ISBN 0-471-01877-5
- Ben Shneiderman, Software Psychology, 1980, ISBN 0-87626-816-5
External links
[edit]Usability
View on GrokipediaFundamentals
Introduction
Usability refers to the ease with which people can employ a particular interface or tool to achieve a specified goal, encompassing aspects such as effectiveness, efficiency, and user satisfaction within a given context.[2] This concept is central to human-computer interaction (HCI), where it evaluates how intuitively users can navigate and interact with systems to accomplish tasks without undue frustration or effort.[5] The notion of usability emerged prominently in the 1980s alongside the rise of personal computing, as researchers and designers sought to make technology more accessible to non-expert users, building on foundational work in HCI that emphasized human-centered design principles.[10] Prior to this, influences from human factors engineering in the mid-20th century laid the groundwork, but the proliferation of graphical user interfaces in devices like the Apple Macintosh and IBM PC catalyzed a focused push toward usable systems.[11] In modern contexts, usability plays a critical role in enhancing user satisfaction by minimizing cognitive load and streamlining interactions, which in turn reduces errors and boosts productivity across diverse applications including software applications, websites, and even physical products like consumer electronics.[2] For instance, well-designed interfaces in e-commerce platforms can significantly lower abandonment rates by facilitating seamless navigation, directly impacting business outcomes.[12] Over time, usability has evolved to integrate closely with broader user experience (UX) design and accessibility considerations, ensuring that interfaces not only function efficiently but also accommodate diverse user needs, such as those with disabilities, thereby promoting inclusive technology adoption.[13] This progression reflects HCI's shift from isolated efficiency metrics to holistic evaluations of user well-being in interactive environments.[11]Definition
Usability refers to the extent to which a system, product, or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.[5] Effectiveness is defined as the accuracy and completeness with which users achieve their goals, ensuring that tasks are performed correctly and fully.[5] Efficiency measures the level of resources expended in relation to the accuracy and completeness of the goals achieved, such as time, effort, or cognitive load.[5] Satisfaction encompasses the users' comfort and acceptability of the system, reflecting their subjective experience of ease and appropriateness.[5] The term usability evolved from early concepts in ergonomics and human factors engineering, which focused on optimizing human performance in work environments, to more contemporary definitions within human-computer interaction (HCI) that emerged prominently in the 1980s.[10] This shift emphasized interactive systems and user interfaces, integrating psychological and cognitive principles to address how people learn and interact with technology.[10] Usability is a core subset of the broader user experience (UX), which includes additional elements like aesthetics, emotional response, and overall delight, whereas usability specifically targets practical aspects of task completion and interface ease.[2] Related attributes often associated with usability include learnability, which assesses how quickly users can accomplish basic tasks upon first encounter; memorability, indicating ease of re-establishment of proficiency after a period of non-use; and a low error rate, where the system minimizes user mistakes and supports recovery from them.[2]Key Concepts
Intuitive interaction in usability refers to the perceived naturalness of an interface, where users can perform tasks without extensive training by subconsciously applying prior knowledge and experiences. This concept emphasizes efficiency and minimal cognitive load, allowing seamless engagement as if the interaction aligns with innate human behaviors. For instance, drag-and-drop functionality in file management systems exemplifies intuitive interaction, as it leverages familiar physical actions like moving objects, enabling users to grasp and relocate items effortlessly without explicit instructions.[14] User mental models represent the internal representations or expectations that individuals form about how a system functions, shaped by prior experiences, analogies from similar domains, and cultural influences. These models guide user predictions and actions; when a system's conceptual model aligns closely with the user's mental model, interactions become predictable and effective, enhancing overall usability. Mismatches, however, such as an unfamiliar icon that contradicts a user's expectation based on real-world analogies, can lead to confusion, errors, and frustration, underscoring the need for designs that bridge these gaps. Affordances, as conceptualized by Donald Norman, describe the perceived and actual properties of an object or interface that indicate possible actions, such as a button's raised edge suggesting it can be pressed. Signifiers, a related but distinct element, are the cues—visual, auditory, or tactile—that communicate these affordances to users, ensuring that potential interactions are discoverable without ambiguity. In digital interfaces, for example, a scrollbar's appearance signifies the ability to scroll content, while poor signifiers like inconsistent icons can obscure affordances and degrade usability.[15] The context of use encompasses the environmental, social, and personal factors that shape how usability is experienced, including physical surroundings, user characteristics like expertise level, and task demands such as time constraints or complexity. These elements influence effectiveness and satisfaction; for instance, a mobile app's usability may diminish in a noisy outdoor environment if audio feedback is unclear, or for novice users if it assumes advanced knowledge. Designers must account for this variability to ensure robust performance across diverse scenarios.[16]History and Evolution
Origins in Human Factors Engineering
The roots of usability in human factors engineering trace back to the early 20th century, particularly through Frederick Winslow Taylor's principles of scientific management, which emphasized optimizing worker efficiency in industrial settings. Introduced in the 1910s, Taylorism applied systematic observation and experimentation to break down tasks into elemental motions, aiming to eliminate inefficiencies and standardize workflows to match human capabilities. This approach marked an initial recognition of the "human factor" in design, shifting from purely mechanical optimization to incorporating physiological limits and worker performance, thereby laying groundwork for later ergonomic principles.[17] World War II accelerated the development of human factors engineering, particularly in military aviation, where high error rates due to equipment design prompted interdisciplinary efforts to reduce pilot mistakes. For instance, psychologists like Alphonse Chapanis analyzed incidents such as wheels-up landings in aircraft like the P-47, B-17, and B-25 bombers, attributing them not to operator failure but to ambiguous cockpit controls. By 1943, Chapanis and colleagues implemented shape-coding for levers—such as wheels for landing gear and triangles for flaps—significantly decreasing errors and influencing broader equipment design standards. These interventions highlighted the need to align machine interfaces with human sensory and motor abilities, establishing human factors as a critical discipline for safety and performance in complex systems.[18] Post-war advancements in ergonomics built on these foundations, with Paul M. Fitts' 1954 formulation of what became known as Fitts' Law providing a predictive model for human movement in control tasks. Fitts' Law quantifies the time required to move to a target as a function of distance and target width, expressed as , where is movement time, is distance, is width, and and are empirical constants; this equation enabled designers to anticipate and mitigate performance limitations in analog interfaces like joysticks and switches. Widely adopted in post-war engineering, it underscored the quantifiable nature of human-motor interactions, informing layouts in aviation and industrial tools. By the 1960s, human factors engineering transitioned toward cognitive dimensions, reflecting the broader cognitive revolution in psychology that emphasized mental processes over purely physical ones. This shift addressed growing complexities in system design, such as increasing mental workloads from information overload in control rooms and early computing environments, prompting research into perception, decision-making, and attention. Pioneers integrated these insights to refine designs for reduced cognitive strain, marking a pivotal evolution from Taylorist efficiency to holistic human-system compatibility.[19]Development in Human-Computer Interaction
The emergence of usability as a core concern in human-computer interaction (HCI) gained momentum in the 1980s, driven by innovations in graphical user interfaces (GUIs) that prioritized intuitive visual elements over command-line inputs. At Xerox PARC, researchers developed the Alto system in the early 1970s, but its GUI concepts—including windows, icons, and mouse-driven navigation—profoundly influenced subsequent commercial systems by emphasizing user-centered design principles that reduced cognitive load and enhanced accessibility. This work laid foundational ideas for making computing more approachable, shifting focus from hardware efficiency to human performance in interactive environments.[20] Apple's Macintosh, released in 1984, commercialized these GUI advancements, integrating a desktop metaphor with point-and-click interactions that democratized personal computing and elevated usability as a competitive differentiator. By incorporating direct manipulation techniques, the Macintosh allowed non-expert users to perform complex tasks through familiar visual cues, significantly improving efficiency and user satisfaction compared to text-based systems. This influence extended HCI research toward empirical evaluation of interface designs, fostering a discipline that balanced technological innovation with psychological insights into user behavior.[21][22] A seminal contribution during this period was the 1983 publication of The Psychology of Human-Computer Interaction by Stuart K. Card, Thomas P. Moran, and Allen Newell, which formalized models of user cognition and task performance in interactive systems. The book introduced the Keystroke-Level Model for predicting user action times and bridged cognitive psychology with interface design, providing a scientific framework for assessing usability that has informed HCI methodologies ever since. Its emphasis on applied psychology helped establish usability as an interdisciplinary field, influencing evaluations of early GUIs and beyond.[23] In the 1990s, usability mainstreamed with the rise of the World Wide Web, where Jakob Nielsen's Alertbox column, launched in June 1995, disseminated practical insights on web interface design to a global audience. Nielsen's work highlighted common pitfalls in early websites, such as cluttered layouts and poor navigation, advocating for simplicity and user testing to optimize online experiences. Complementing this, his 10 Usability Heuristics, originally developed with Rolf Molich in 1990 and refined in a 1994 publication, offered a concise set of evaluation principles—like visibility of system status and error prevention—that became widely adopted for rapid usability inspections in web development.[24][6] From the 2000s onward, usability evolved with the proliferation of mobile and touch-based interfaces, exemplified by the iPhone's 2007 launch, which introduced multitouch gestures and responsive designs tailored to on-the-go contexts. These advancements necessitated HCI research into gesture recognition, screen real estate constraints, and context-aware interactions, resulting in guidelines that improved accessibility and reduced input errors on portable devices. Concurrently, usability integrated with agile methodologies, where iterative sprints incorporated user-centered techniques like lightweight prototyping and feedback loops to embed evaluation early in development cycles, enhancing software adaptability without compromising user needs.[25][26] In the 2020s, trends toward AI-driven personalization have further transformed usability in HCI, enabling adaptive interfaces that tailor content and interactions based on user behavior and preferences. Machine learning algorithms now power predictive features, such as dynamic layouts in e-commerce apps, which boost engagement by minimizing irrelevant information while raising challenges in privacy and bias mitigation. This shift underscores ongoing HCI efforts to balance personalization's benefits—like reduced task completion times—with ethical considerations, ensuring equitable and transparent user experiences.[27][28]Standards and Guidelines
International Standards (ISO and IEC)
The International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) have developed several key standards that define and guide usability practices in human-system interaction, emphasizing ergonomics, human-centered design, and risk mitigation. These standards provide frameworks for ensuring that interactive systems are effective, efficient, and satisfactory for users within specified contexts.[5] A cornerstone of these efforts is the ISO 9241 series, which addresses ergonomics of human-system interaction. ISO 9241-11:2018 specifically defines usability as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use," offering a conceptual framework to evaluate and apply these attributes during system design and assessment.[5] Complementing this, ISO 9241-210:2019 outlines requirements and recommendations for human-centered design processes in interactive systems, promoting iterative activities such as understanding user needs, specifying requirements, and evaluating prototypes throughout the product life cycle to enhance usability.[29] These parts of the series integrate usability into broader ergonomic principles, influencing global practices in software and hardware development.[30] Another influential document, ISO/TR 16982:2002, serves as a technical report providing guidance on selecting and applying human-centered usability methods for design and evaluation, including details on advantages, disadvantages, and contextual suitability of various techniques.[31] It remains a foundational reference for structuring usability efforts in alignment with human-centered approaches described in ISO 9241.[32] In the domain of medical devices, IEC 62366-1:2015, amended in 2020, establishes a usability engineering process for manufacturers to analyze, specify, design, and evaluate device interfaces, with a strong emphasis on identifying and mitigating risks associated with use errors to ensure patient and operator safety.[33] The 2020 amendment refines this process by updating linkages to risk management standards like ISO 14971 and clarifying formative and summative evaluation requirements.[34] As of late 2025, no major revisions have been issued for these core usability standards, maintaining their current editions as the primary references for international compliance and best practices.[35]Heuristics and Other Frameworks
Heuristics in usability engineering serve as informal, rule-of-thumb guidelines derived from expert experience to evaluate and improve user interfaces, offering practical complements to more formal international standards.[6] These frameworks emphasize broad principles that guide design decisions, focusing on common pitfalls and best practices without the binding requirements of regulatory norms. Jakob Nielsen's 10 Usability Heuristics, introduced in 1994, stem from an analysis of 249 usability problems across various interfaces and remain one of the most widely adopted sets for heuristic evaluation.[6] They include:- Visibility of system status: The system should always keep users informed about what is happening through appropriate feedback, such as progress indicators.[6]
- Match between system and the real world: The interface should speak the users' language, using words, phrases, and concepts familiar to them rather than system-oriented terms.[6]
- User control and freedom: Users often choose system functions by mistake and need a clearly marked "emergency exit" to leave unwanted states, with support for undo and redo.[6]
- Consistency and standards: Users should not wonder whether different words, situations, or actions mean the same thing, following real-world conventions and platform standards.[6]
- Error prevention: Design interfaces to prevent problems from occurring, even if it means limiting choices or confirming destructive actions.[6]
- Recognition rather than recall: Minimize the user's memory load by making objects, actions, and options visible, with instructions provided in context.[6]
- Flexibility and efficiency of use: Accelerators like shortcuts should be available for experienced users, while novices receive guidance.[6]
- Aesthetic and minimalist design: Dialogues should contain only relevant information, avoiding irrelevant or rarely needed data.[6]
- Help users recognize, diagnose, and recover from errors: Error messages should be expressed in plain language, precisely indicating the problem and constructively suggesting a solution.[6]
- Help and documentation: Information should be easily searchable and focused on the user's task, though ideally not needed.[6]
- Strive for consistency: Maintain uniform sequences of actions, terminology, and visual layouts across the interface to reduce cognitive load.[36]
- Enable frequent users to use shortcuts: Offer accelerators, such as keyboard commands, to speed up interaction for expert users without complicating the experience for novices.[36]
- Offer informative feedback: Provide clear, immediate, and meaningful responses to every user action to confirm its effect and guide further steps.[36]
- Design dialogs to yield closure: Structure interactions so that sequences of actions lead to a clear end, with confirmatory messages to signal completion.[36]
- Prevent errors: Anticipate potential mistakes by designing careful controls and avoiding error-prone conditions.[36]
- Permit easy reversal of actions: Allow users to undo recent choices easily, fostering confidence in experimentation.[36]
- Support internal locus of control: Design systems where users feel in command, initiating and controlling actions rather than reacting to the system.[36]
- Reduce short-term memory load: Promote recognition over recall by displaying necessary information and minimizing hidden options.[36]
Principles of Usable Design
User-Centered and Task-Based Approaches
User-centered design approaches prioritize the needs, goals, and contexts of end-users from the initial stages of the design process, ensuring that interactive systems align closely with human capabilities and expectations. This involves creating detailed representations of users through personas, which are fictional archetypes based on aggregated user research data to embody typical behaviors, motivations, and pain points. Developed by Alan Cooper in the 1990s as part of goal-directed design, personas help designers empathize with diverse user types and make informed decisions that avoid assumptions based solely on internal team perspectives.[39] Similarly, user scenarios provide narrative descriptions of how personas might interact with a system in realistic situations, highlighting sequences of actions, environmental factors, and potential challenges to guide the envisioning of system functionality. These scenarios, as outlined in scenario-based design frameworks, facilitate collaborative exploration of use cases without premature commitment to technical implementations. Stakeholder analysis complements these tools by systematically identifying and prioritizing individuals or groups affected by the system, such as end-users, developers, and organizational leaders, to map their interests and influence on design outcomes. This process ensures that diverse perspectives are integrated early, mitigating conflicts and enhancing system relevance across the user ecosystem.[40] Task-based approaches build on this user focus by decomposing complex activities into structured hierarchies, allowing designers to pinpoint inefficiencies and opportunities for support. Hierarchical Task Analysis (HTA) exemplifies this method, originating from ergonomics in the 1970s and refined for interactive systems, where high-level user goals are broken down into subtasks, operations, and decision points using diagrammatic notations. For instance, planning a trip might hierarchically include subtasks like searching options, comparing costs, and confirming bookings, revealing dependencies and cognitive demands at each level. HTA supports the identification of task flows that minimize unnecessary steps, ensuring designs facilitate seamless progression toward user objectives.[41] Participatory design extends user-centered and task-based strategies by actively involving users in the ideation and prototyping phases, fostering ownership and relevance in system development. Rooted in Scandinavian labor movements, this approach treats users as co-designers rather than passive informants, employing workshops and collaborative tools to generate ideas that reflect real-world practices. Pelle Ehn's seminal work emphasized designing computer artifacts as tools that empower skilled workers, promoting democratic participation to bridge gaps between technical possibilities and workplace needs. By integrating user input iteratively from the outset—while aligning with broader human-centered principles like those in ISO 9241-210—these methods ensure designs are not only usable but also meaningful to those who will employ them.[29] A key conceptual foundation for these approaches is the notion of bridging the gulf of execution and the gulf of evaluation, as articulated by Donald Norman. The gulf of execution refers to the cognitive distance between a user's intentions and the actions required by the system interface, while the gulf of evaluation describes the effort needed to interpret system feedback and assess goal progress. Effective user-centered and task-based designs narrow these gulfs through intuitive mappings, such as clear affordances for actions and immediate, unambiguous responses, thereby reducing mental workload and enhancing perceived control.Iterative and Empirical Design Processes
Iterative design in usability emphasizes repeated cycles of creating, testing, and refining interfaces based on user feedback and data, ensuring that designs evolve to meet user needs effectively. A foundational framework for this approach was outlined by John D. Gould and Clayton L. Lewis in their 1985 paper, which proposed four key principles: focusing early and continually on users, centering designs around user tasks, measuring product usage through empirical methods, and iterating on prototypes, simulations, or the final system to incorporate findings.[42] These principles underscore that usability cannot be achieved through a linear process but requires ongoing refinement to address unforeseen issues and optimize performance. By integrating user involvement from the outset, iterative design reduces the risk of costly rework later in development.[42] Prototyping plays a central role in iterative processes, progressing through stages of increasing fidelity to balance speed, cost, and detail. Low-fidelity prototypes, such as paper sketches or basic wireframes, are used in early iterations to quickly explore concepts, validate user flows, and identify major structural flaws without investing significant resources.[43] As iterations advance, designers shift to high-fidelity prototypes, which include interactive elements, visual styling, and simulated functionality to test more realistic user interactions and gather detailed feedback on usability aspects like navigation and responsiveness.[43] This staged progression allows teams to refine designs incrementally, ensuring that each cycle builds on validated insights while adapting to empirical evidence from prior tests. Empirical measurement is integral to iteration, involving the collection of real usage data to guide design changes and validate improvements. Techniques like A/B testing compare two versions of a design element—such as button placement or wording—by exposing them to different user groups and measuring outcomes like click-through rates or task completion times to determine which performs better.[44] This data-driven approach ensures that refinements are not based on assumptions but on quantifiable evidence of user behavior, enabling iterative cycles to systematically enhance usability.[44] In modern software development, iterative and empirical processes have been integrated into Agile methodologies through practices like usability sprints, where user experience activities are embedded within short development cycles of 1-4 weeks. During these sprints, teams prototype features, conduct quick empirical evaluations, and iterate based on findings to deliver incrementally usable increments.[45] This integration aligns usability with Agile's emphasis on flexibility and rapid delivery, allowing continuous refinement without disrupting overall progress.[45]Usability Evaluation Methods
Modeling and Cognitive Techniques
Modeling and cognitive techniques in usability evaluation involve predictive models that simulate human cognitive and motor processes to estimate user performance without involving actual users. These methods draw from cognitive psychology to forecast task completion times, error rates, and interaction efficiencies, enabling designers to compare interface alternatives early in development. By formalizing user behavior as computational processes, such models provide quantitative predictions grounded in empirical data on human information processing. The Goals, Operators, Methods, and Selection rules (GOMS) framework, introduced by Card, Moran, and Newell, represents a foundational family of models for analyzing skilled user performance in routine tasks. GOMS decomposes a task into hierarchical goals (high-level objectives like "edit document"), operators (primitive actions such as keystrokes or mouse movements), methods (procedural knowledge for achieving goals), and selection rules (heuristics for choosing among methods). This structure allows prediction of task execution time by summing operator durations, correlating well with observed user times in validation studies. CMN-GOMS, the original formulation, applies these elements in a textual, program-like description to estimate total performance time for expert users.[46] A simplified variant of GOMS, the Keystroke-Level Model (KLM), focuses on motor and cognitive operators to predict execution time for low-level interactions, assuming error-free performance by experts. The model represents tasks as sequences of physical-motor operators: K for keystroking (time ≈ 0.60 s), P for pointing with a mouse (1.10 s), H for homing hands to a device (0.40 s), and D for drawing a straight line (0.90 s plus adjustments for length). Mental operators M (1.35 s) are inserted via rules, such as before initiating a new command or after system feedback, while the I operator accounts for hardware initiation delays (system-specific, often 0.15–2.0 s). The total predicted time is calculated as: where subscripts denote instances of each operator in the sequence, adjusted by rules to omit redundant Ms (e.g., within anticipated units like command entry). For example, inserting text in a menu-driven editor might yield a sequence like I + M + K[menu] + P[insert] + M + K[text] + H[keyboard] + K[enter], totaling approximately 5.2 s. KLM has been validated against empirical data, predicting times within 20% accuracy for routine tasks like menu navigation. The Model Human Processor (MHP) underpins GOMS and KLM by conceptualizing human cognition as three interacting processors: perceptual (processing sensory input, cycle time ≈ 100 ms), cognitive (reasoning and decision-making, 70 ms), and motor (executing movements, 70–100 ms plus Fitts' law for pointing time). Memories include working memory (capacity 7 ± 2 chunks, decay 7–20 s) and long-term memory (unlimited, retrieval 70 ms). These parameters, derived from psychological experiments, enable simulations of information flow, such as a perceptual-cognitive-motor cycle taking about 240 ms for simple reactions. MHP facilitates broader predictions, like cognitive load in multitasking, by modeling processor bottlenecks.[46] Parallel design complements these predictive models by generating multiple interface alternatives concurrently to explore diverse solutions and enhance overall usability. In this technique, several designers independently create initial prototypes based on the same specifications, then merge the strongest elements into unified designs for evaluation. A comparative study found that parallel design improved usability scores by 70% from initial to merged versions, compared to 18% gains from traditional iterative approaches, due to broader idea exploration and reduced fixation on suboptimal paths. This method is particularly effective in early stages, fostering innovation while integrating cognitive modeling insights for refinement.[47]Inspection and Heuristic Methods
Inspection and heuristic methods are expert-driven usability evaluation techniques that identify potential issues in user interfaces without involving actual users, relying instead on the knowledge and judgment of experienced reviewers. These approaches are particularly valuable in early design stages for their efficiency and low cost, allowing teams to uncover violations of established usability principles before investing in user testing. Usability inspection, a broad category encompassing these methods, involves systematic reviews by specialists to detect problems related to interface design, consistency, and adherence to standards.[48] Usability inspection serves as the overarching term for a family of methods where experts examine an interface to pinpoint usability flaws, estimate their severity, and suggest remedies, often drawing from human factors principles. Developed in the early 1990s, these techniques emphasize informal yet structured analysis to complement more resource-intensive empirical evaluations. Key variants include heuristic evaluation, cognitive walkthroughs, and consistency inspections, each targeting different aspects of interface quality.[49] Heuristic evaluation involves a small group of usability experts independently reviewing an interface against a predefined set of recognized usability principles, or "heuristics," to identify violations that could hinder user performance. Introduced by Jakob Nielsen and Rolf Molich in 1990, this method typically employs Nielsen's 10 heuristics, such as visibility of system status, match between system and the real world, and user control and freedom, which guide evaluators in spotting issues like confusing navigation or inconsistent feedback.[6] The process begins with preparation, where 3-5 evaluators are selected and briefed on the interface scope, often focusing on specific tasks or components; evaluators then spend 1-2 hours independently inspecting the design, noting problems with descriptions, screenshots, and heuristic references. Findings are consolidated in a debriefing session using techniques like affinity diagramming to merge duplicates, discuss disagreements, and prioritize issues based on potential impact. To assess severity, problems are rated on a 0-4 scale: 0 (no problem), 1 (cosmetic), 2 (minor), 3 (major), or 4 (catastrophic), considering factors like frequency, impact, and persistence, with the average of multiple raters providing reliability. This approach can detect up to 75% of major usability problems with just five evaluators, making it a "discount usability" staple.[50][51][52] Cognitive walkthrough is a structured, task-oriented inspection method where experts simulate a user's learning process by stepping through a sequence of actions in the interface, evaluating whether the design supports intuitive goal achievement for novices. Originating from work by Peter Polson, Clayton Lewis, and colleagues in 1992, it applies principles from exploratory learning theory to predict points where users might fail, focusing on learnability rather than overall efficiency.[53] The method starts with defining a representative task scenario, breaking it into atomic steps, and assembling a team of 3-5 reviewers familiar with the target users. For each step, evaluators pose four key questions: (1) Will the correct action be evident to the user at this point? (2) Will the user understand that the action achieves their intended goal? (3) Will the user know the system's response confirms success? (4) Will the user encounter sufficient feedback to proceed confidently? Problems are flagged where answers indicate likely errors, such as unclear controls or ambiguous outcomes, and documented with rationale tied to cognitive principles. This step-by-step simulation helps reveal learnability barriers, like hidden features, without prototypes or users.[54][55] Pluralistic and consistency inspections are collaborative review techniques that emphasize group discussion and cross-interface alignment to enhance overall usability coherence. Pluralistic walkthrough, described by Randolph Bias in 1994, gathers stakeholders—including developers, users (in a simulated expert capacity), and experts—in a meeting to narrate and critique a task scenario step-by-step, fostering empathy and diverse perspectives to uncover overlooked issues like workflow disruptions.[49] Consistency inspection, also outlined by Nielsen in 1994, involves experts from related projects examining the target interface for alignment with established patterns, terminology, and behaviors across applications, preventing user confusion from discrepancies such as varying button placements or command synonyms. Both methods promote standardization; for instance, pluralistic sessions might reveal inconsistent error handling, while consistency checks ensure uniform navigation paradigms, ultimately supporting scalable design ecosystems.[48] Card sorts and tree tests provide targeted inspection tools for validating information architecture, allowing experts to assess content organization and findability without full user involvement. In an expert card sort, reviewers manually group and label content cards to simulate user categorization, identifying logical hierarchies or mismatches in site structure; this method, refined in usability practice since the early 2000s, helps detect overly broad categories or poor topical clustering. Tree testing complements this by having experts traverse a proposed menu tree to locate items, flagging deep nesting or misleading labels that could impede navigation efficiency. These techniques, often used iteratively in design reviews, ensure intuitive access to information, with expert validation serving as a precursor to broader testing.Inquiry and Feedback Techniques
Inquiry and feedback techniques in usability evaluation center on methods that directly solicit user perspectives, experiences, and observable behaviors to uncover preferences, challenges, and contextual nuances in human-computer interactions. These approaches prioritize user involvement to generate qualitative and quantitative insights, often through discussions, self-reports, or fieldwork, enabling designers to align systems with real-world needs. Unlike expert-driven inspections, these techniques emphasize empirical data from users themselves, fostering iterative improvements based on authentic feedback. Focus groups facilitate moderated discussions among small groups of users to elicit qualitative insights on preferences and attitudes toward interfaces or products. Involving 6 to 9 participants in sessions lasting approximately 2 hours, a facilitator guides conversations on predefined topics, encouraging diverse input while monitoring group dynamics to prevent dominance by individuals.[56] This method excels at surfacing spontaneous ideas and emotional responses, such as user reactions to documentation systems, making it valuable for early-stage requirements gathering.[56] However, focus groups are limited in evaluating actual task performance, as they capture expressed opinions rather than observed actions, and thus should complement other observational methods.[56] Questionnaires and surveys provide standardized instruments for quantifying user satisfaction and perceived usability, with the System Usability Scale (SUS) serving as a prominent example. The SUS consists of 10 items rated on a 5-point Likert scale, alternating between positively and negatively worded statements about ease of use, yielding a composite score from 0 to 100 where higher values indicate better usability.[57] Originally developed by John Brooke in 1986 for rapid assessments in electronic office systems, it offers a reliable, low-burden tool for benchmarking satisfaction across diverse applications.[58] Task analysis involves the systematic observational breakdown of user workflows in natural settings to map how individuals accomplish goals within their environments. Researchers conduct field observations and interviews to capture real-world episodes, then distill these into hierarchical structures or scenarios that highlight task sequences, decision points, and potential bottlenecks.[59] Scenario-based task analysis, for instance, uses stakeholder narratives to generate problem scenarios and claims about design tradeoffs, supporting iterative refinement.[60] This method, rooted in early HCI practices, aids in identifying inefficiencies without relying on controlled labs, though it requires careful synthesis to avoid oversimplification of complex behaviors.[59] Ethnography employs immersive field studies to deeply explore user contexts, embedding researchers in everyday settings to observe and participate in technology-mediated practices over extended periods. Drawing from anthropological traditions, this technique reveals tacit knowledge, social dynamics, and cultural influences on usability, such as how collaborative tools shape workplace interactions.[61] Pioneered in HCI through computer-supported cooperative work, ethnography challenges assumptions about isolated user actions by emphasizing mutual constitution of technology and practice, informing designs that respect contextual variability.[61] Activity analysis, informed by activity theory, examines tool-mediated activities to pinpoint opportunities for enhancing usability in dynamic, goal-oriented contexts. It decomposes human endeavors into hierarchical layers—activities driven by motives, actions by goals, and operations by conditions—focusing on how artifacts mediate subject-object relations and evolve through social interactions.[62] In HCI evaluations, this approach analyzes how systems support multitasking or collaboration, as in knowledge work environments, to redesign tools that better align with users' broader purposes.[63] Seminal applications highlight its strength in addressing contradictions within activity systems, promoting developmental improvements over static task models.[62]Prototyping and User Testing Methods
Rapid prototyping is a core method in usability evaluation that enables designers to create low-fidelity representations of interfaces quickly, allowing for early testing and iteration based on user feedback.[64] This approach emphasizes speed and cost-effectiveness, often using simple materials like paper sketches or wireframes to simulate user interactions without committing to full development. Seminal work highlights its role in revealing design flaws before implementation, reducing long-term costs by incorporating empirical observations into the iterative design process. Three key approaches to rapid prototyping facilitate this process: the Tool Kit approach, the Parts Kit approach, and the Animation Language Metaphor. The Tool Kit approach involves a library of reusable components, such as predefined interface elements, that designers assemble to build prototypes efficiently, promoting consistency and rapid customization. In contrast, the Parts Kit approach uses modular, interchangeable parts—like cutouts or templates—for assembling prototypes, enabling users to manipulate and reconfigure elements during testing to explore alternative layouts. The Animation Language Metaphor combines storyboarding with scripting techniques, where prototypes are depicted as sequences of frames or scenarios to convey dynamic interactions and user flows, akin to animating a narrative. These methods, rooted in human-computer interaction principles, support quick iterations and are particularly effective for eliciting user insights in early design stages. The thinking aloud protocol is a foundational technique integrated into prototyping sessions, where users verbalize their thoughts, decisions, and reactions in real-time as they interact with prototypes. This method, originally developed in cognitive psychology, reveals underlying cognitive processes, such as confusion or satisfaction, without relying on post-task recall, which can be biased. In usability testing, it enhances the validity of observations by providing direct access to users' mental models, with studies showing it uncovers 80-90% of usability issues when combined with prototypes. Facilitators prompt minimally to maintain natural flow, ensuring the protocol aligns with empirical design practices.[65] Rapid Iterative Testing and Evaluation (RITE) builds on prototyping by conducting short, successive tests with small user groups, typically 5-8 participants per cycle, to identify and fix issues immediately.[66] Developed for fast-paced environments like game design, RITE prioritizes high-impact problems, allowing teams to refine prototypes mid-session and retest in subsequent iterations, resolving a high percentage of issues through rapid cycles.[66] This method contrasts with traditional testing by emphasizing actionable changes over exhaustive data collection, making it suitable for agile development. The subjects-in-tandem, or co-discovery, method involves pairing users to collaborate on prototype tasks, where they discuss and assist each other, uncovering social and collaborative dynamics not evident in solo testing. This technique, an extension of think-aloud protocols, simulates real-world group usage scenarios, such as shared device interactions, and has been shown to detect interpersonal usability issues like communication barriers. Pairs naturally verbalize confusions, providing richer qualitative data while reducing individual pressure, though it requires careful task design to avoid dominance by one participant.[67] Component-based usability testing isolates specific user interface elements, such as buttons or menus, for targeted evaluation within a prototype, measuring their independent contribution to overall usability.[68] This approach uses metrics like task completion time for the component and subjective ratings of ease-of-use, enabling precise comparisons between design variants without full-system testing. Empirical studies validate its effectiveness, demonstrating higher sensitivity to localized improvements, such as reduced error rates in isolated navigation elements.[68] It supports modular design practices, aligning with iterative processes by focusing resources on high-priority components.Advanced and Remote Testing Methods
Remote usability testing enables researchers to evaluate user interfaces and experiences without requiring participants to visit a physical lab, leveraging digital platforms to conduct sessions synchronously or asynchronously. This approach has gained prominence due to its scalability and accessibility, particularly in distributed teams or global participant recruitment. Tools such as UserTesting.com facilitate unmoderated sessions, where users independently complete predefined tasks while recording their screens, audio, and sometimes video feedback, allowing for self-paced interaction without real-time researcher intervention.[69][70] Compared to traditional lab-based testing, remote unmoderated methods offer several advantages, including lower costs—often 20–40% less than moderated studies due to eliminated travel and facility expenses—and greater flexibility for participants to engage from their natural environments, which can yield more ecologically valid data. However, drawbacks include reduced ability to probe unexpected behaviors in real time, potential technical issues like connectivity problems, and challenges in ensuring participant attention without direct oversight, which may lead to lower data quality in complex tasks. Moderated remote testing, conducted via video conferencing tools like Zoom, mitigates some of these by allowing live observation and clarification, though it still lacks the nonverbal cues observable in-person.[71][72][73] For mobile applications, advanced remote testing emphasizes field studies in real-world contexts to capture mobility-specific interactions, such as multitasking or environmental distractions, which lab simulations often overlook. Techniques include session recording via built-in device tools or platforms like Lookback.io, which log user actions, timestamps, and errors during naturalistic use. Eye-tracking integration, enabled by wearable devices like Tobii glasses or mobile attachments, quantifies visual attention patterns, such as fixation duration and saccade paths, revealing how users navigate small screens amid movement. A 2023 study on a mobile AR app for urban cultural heritage used remote eye-tracking to identify usability issues in outdoor navigation tasks, highlighting longer task times in cluttered environments.[74][75][76] Usability benchmarking in advanced testing involves systematically comparing a product's performance metrics—such as task completion rates or error frequencies—against industry standards or direct competitors to establish relative effectiveness. For instance, the System Usability Scale (SUS) provides a standardized score for benchmarking, with meta-analyses indicating average scores of 68 for general software, allowing teams to gauge if a mobile app's 75 SUS outperforms e-commerce peers at 62. This method supports iterative improvements by highlighting gaps, such as slower navigation in a tested interface versus leading competitors, without requiring new primary data collection.[77][78][79] Meta-analysis enhances remote testing by statistically synthesizing findings from multiple usability studies, providing generalizable insights into patterns like error-prone interface elements across diverse contexts. In mobile usability, meta-analyses have identified patterns in interaction efficiencies and error-prone elements across contexts. This approach aggregates effect sizes from remote sessions, accounting for variability in participant demographics and devices, to yield robust evidence beyond single-study limitations.[80][81] Recent developments from 2024 to 2025 have integrated AI into remote session analysis for automated pattern detection, accelerating the identification of usability issues from large-scale unmoderated data. AI tools, such as those employing machine learning for sentiment analysis and anomaly detection in video recordings, can cluster user frustrations. A 2025 systematic review highlighted AI's role in remote UX evaluation, noting improvements in accuracy for predicting user behaviors over traditional methods, though ethical concerns around data privacy persist, with recent 2025 guidelines emphasizing consent in AI-analyzed remote sessions. These advancements enable scalable, real-time insights, particularly for mobile field tests.[82][83][84][85]Metrics and Benefits
Usability Metrics and Measurement
Usability metrics provide objective ways to quantify the quality of user interactions with systems, focusing on key attributes such as effectiveness, efficiency, and satisfaction as defined in the international standard ISO 9241-11. This standard describes effectiveness as the accuracy and completeness with which users achieve specified goals, efficiency as the level of resources expended relative to the accuracy and completeness of goal achievement, and satisfaction as the users' comfort and acceptability of the system in specified contexts.[86] These core metrics are typically measured during user testing to evaluate how well a system supports task performance without undue effort or frustration. Effectiveness is commonly assessed through the percentage of goals completed successfully, where users are observed attempting predefined tasks and the proportion of successful completions is calculated as (number of successful tasks / total tasks attempted) × 100.[87] For example, in e-commerce usability studies, effectiveness might be measured by the rate at which users add items to a cart without assistance, revealing barriers to task achievement. Error rates further refine this metric, capturing the frequency of user mistakes (e.g., number of errors per task or per session) and their severity on a scale from 0 (no effect) to 4 (catastrophic, preventing task completion), as outlined in established severity rating guidelines.[51] High error frequency or severity indicates design flaws that hinder accurate performance, such as confusing navigation leading to repeated wrong selections.[88] Efficiency metrics emphasize resource use, primarily time on task—the average duration to complete a task from start to successful end—and actions per task, counting steps or interactions required.[87] These are benchmarked against expert performance or prior iterations; for instance, if users take over 2 minutes to complete a simple search in a well-designed interface, it signals inefficiency. Satisfaction, the subjective component, is often quantified using the System Usability Scale (SUS), a 10-item questionnaire with responses on a 1-5 Likert scale yielding a score from 0 to 100.[57] The SUS formula adjusts responses for positive (odd-numbered) and negative (even-numbered) items: where odd items are scored directly minus 1 (0-4 range each), and even items are inverted (5 minus score, 0-4 range each), then summed and scaled.[57] Scores above 68 indicate above-average usability, based on aggregated data from thousands of studies. Learnability measures how quickly users acquire proficiency, typically via time to first success—the duration for novices to complete a task on their initial attempt—or the rate of performance improvement across repeated trials, following the power law of practice where task time decreases logarithmically with experience.[89] For example, if first-time users take 5 minutes for a task but reduce it to 2 minutes after three trials, the system demonstrates strong learnability.[90] Retention, or memorability, assesses long-term usability by evaluating performance after a break, such as time to re-complete tasks following a one-week absence; minimal increase in time or errors post-break signifies effective retention of learned skills.[2] This metric is crucial for infrequent-use systems like tax software, where users must recall interfaces without retraining. Meta-analysis aggregates usability metrics across multiple studies to establish benchmarks, using statistical techniques like effect size calculations to synthesize data on correlations between measures such as task time and SUS scores. For instance, meta-analyses of SUS data provide industry benchmarks, with average scores around 68 for general software, allowing comparisons to gauge relative performance. Benchmarking involves tracking these metrics over time or against competitors, enabling organizations to set improvement targets, such as reducing error rates below 5% through iterative design.[77]| Metric Category | Key Measures | Example Calculation/Application |
|---|---|---|
| Effectiveness | % Goal Completion, Error Frequency/Severity | (Successful tasks / Total tasks) × 100; Severity scale 0-4 per Nielsen guidelines. Used to identify task failure points in prototypes. |
| Efficiency | Time on Task, Actions per Task | Average seconds per task; Steps to completion. Shorter times indicate streamlined workflows. |
| Satisfaction | SUS Score | 0-100 scale via questionnaire formula. Benchmarks: >80 excellent, <50 poor. |
| Learnability & Retention | Time to First Success, Performance Post-Break | Initial vs. subsequent task times; Relearning delta after delay. Tracks skill acquisition and memory. |
