It
is a widely-supported tenet in human-computer interaction that
the meaningful unit of analysis is not the technical device alone,
but the technical device together with the person interacting
with it; the reason is that what is a relevant property of a technology
is only understandable with respect to the specific goals and
resources activated during its usage. This basic reflection should
also inspire the procedure followed to evaluate the usability
of a technology, namely its efficiency and satisfaction for a
specific class of users. The topic of this paper is precisely
to describe a method developed in compliance with this observation
and aimed at evaluating the usability of virtual environments.
Two main requirements were set forth: first, the method should
take the strong connection between humans and technology as its
building block, by linking a property of the virtual environment
to a particular use that makes that property relevant. To this
goal, action has been placed at the center of the analysis; the
functional properties of the VE are then observed in the general
economy of users' interaction with the technology and the whole
ensemble is the appropriate object of evaluation.
Such 'action-based' approach (Gamberini, Spagnolli, 2002) is reminiscent
of the Situated Action theory (Suchman, 1987) and Activity Theory
(Nardi, …); the former proposes a detailed analysis of the
sequential interaction with the technology and provides a rich
examination of the structure given to it by the users. The latter
focuses more on specific phenomena, such as contradictions and
breakdowns, identified by the evaluators; it allows to profit
from data poor in comments and verbalizations, and to analyze
the interaction with the technology from a structural and organizational
level.
As a second requisite for the method, we wanted it to benefit
from the advantages of both approaches; thus we decided to concentrate
on the breakdowns occurring during users' interaction with the
VE but to study these episodes from a situated point of view.
In our definition, breakdowns reveal an inappropriate interpretation
of the possibilities for action offered by the virtual environment
and are to be analyzed in their sequential, contextual unfolding.
This version of breakdown analysis highlights the spontaneous,
subjective problems in the use of a technology and connects them
to specific aspects of users' action. It renews the ergonomic
tradition of error studies (Reason, 1990; Rasmussen, 1980) with
an ethnographic contamination, that pays attention to users' contextualized
practices. It also suits the kind of data the interaction with
a virtual environment is mostly made of, namely bodily action
in a three-dimensional space. Few methods with these characteristics
have been employed so far to analyze the interaction with the
VE. After a brief introduction, the paper will describe the basics
of this approach and illustrate them with instances from the evaluation
of a virtual library.
