Empirical evaluation of user experience can be challenging in an agile environment since it typically relies on extensive access to users or complex modeling techniques. CogTool is a lean method for objectively evaluating designs with human performance models based on extensive cognitive psychology research. These models are useful for:
- Predicting the effectiveness of a proposed design prior to development.
- Comparing the efficiency of a design with that of a competitor’s.
- Measuring cost savings associated with improved usability.
- Understanding the cognitive processes associated with users’ work.
How CogTool Works
Defining a Storyboard
To start using CogTool, you’ll need to create a storyboard that illustrates your design and the way users interact with it. You can import the interfaces that make up a design from anything you happen to have on hand such as wireframes, mockups, and screenshots. CogTool provides an extensive set of widgets to represent interactive elements and lets you define the transitions that they afford. The example on the right shows some whiteboard sketches set up as a storyboard in CogTool.
Building the Keystroke-Level Model
Once a storyboard has been defined, CogTool lets you demonstrate a user’s workflow for accomplishing a task. During this process, CogTool automatically builds a keystroke-level model (KLM) by incorporating operators such as hand movements and mental preparation. Building a KLM by hand can be a tedious and error-prone activity; however, CogTool makes the technique accessible to anyone who can imagine a design. The example on the left shows a script in which CogTool has added several “Think” steps based on known heuristics for the model.
CogTool uses a cognitive architecture called ACT-R to predict the time it would take a skilled user to complete a task in seconds. It also produces a visualization of this performance model, which shows the following components:
- Visual encoding (light purple)
- Eye movement (dark purple)
- Cognition (gray)
- Hand movement (red)
This visualization is useful for comparing two designs and understanding the predictions made for a task. For example, in the visualization above, you can see that the design on the top forces the user to do significantly more eye movement than the design on the bottom. This eye movement adds enough time to a user’s workflow to decrease efficiency by several seconds in comparison to the other design.
CogTool is part of ongoing research by Bonnie John, a founding member of the HCI Institute at Carnegie Mellon University. KLM is one of a family of modeling techniques introduced by Card, Moran, and Newell in The Psychology of Human-Computer Interaction.