Scott Henninger
University of Nebraska
Department of Computer Science and Engineering
115 Ferguson Hall, Box 88011
Lincoln, NE 68588-0115 USA
scotth@cse.unl.edu
Nick Belkin
Rutgers University
The School of Communication, Information, and Library Studies
New Brunswick, NJ 08901-1071 USA
nick@belkin.rutgers.edu
The need for effective information retrieval systems becomes increasingly important as computer-based
information repositories grow larger and more diverse. In this tutorial, we will present the key issues
involved in the use and
design of effective interfaces to information retrieval systems. The process of satisfying information needs
is analyzed as a problem solving activity in which users learn and refine their needs as they interact with a
repository. Current systems are analyzed in terms of key interface and interaction techniques such as
querying, browsing, and relevance feedback. We will discuss the impact of information seeking strategies
on the search process and what is needed to more effectively support the search process. Retrieval system
evaluation techniques will be discussed in terms of their implications for users. We close by outlining some
user-centered design strategies for retrieval systems.
The field of information retrieval can be divided along the lines of its system-based and user-based
concerns. While the system-based view is concerned with efficient search techniques to match query and
document representations, the user-based view must
account for the
cognitive state of the searcher and the problem solving context. People are drawn to an information
retrieval system because they perceive that they lack some knowledge to solve a problem or perform a task.
This creates an "anomalous state of knowledge"
[1] or "situation of irresolution" [6] in which information seekers must find something they know little or
nothing about. Information retrieval systems must not only provide efficient retrieval, but must also support
the user in describing a problem
that they do not understand well. The process is not only one of providing a good query language, but also
supporting an iterative dialogue model. As users query and browse the repository, they learn more about
the problem and potential solutions and therefore refine their conceptualization of the problem. The
information being sought differs from that being sought at the beginning of the session. The user is engaged
in a problem solving session in which the problem to be solved, that of finding relevant
information, evolves and is refined through the process of seeing the results of intermediate queries.
Even in cases where the information is well-known, a vocabulary problem still exists. Users may know
what they are looking for, but lack the knowledge needed to articulate the problem in terms and abstractions
used by the retrieval system. An inherent problem is that people use a surprisingly diverse set of terms to
refer to the same object, such that the probability for choosing
the same term for a familiar object is less than 15 percent [3]. This problem is exacerbated by the fact that
information repositories are often indexed by experts and by the inherent properties of the objects. Expert
indexing causes problems because less knowledgeable users, who define the majority of people
experiencing an anomalous state of knowledge, are less likely understand the terminology used by experts.
Indexing by inherent properties causes problems because most information seeking is engaged
in some problem solving context. People are looking for information that is used for
something and are therefor more concerned with how an object is used, not its inherent properties [4].
Current information retrieval systems have addressed these inherent properties of information seeking and
indexing in a variety of ways. Browsing has been employed to facilitate the iterative and ill-defined nature
of information seeking, but can lead a loss of direction and overly narr
ow searches. Queries provide a means to direct the search, but often rely on the user understanding a
complex query language and proper vocabulary to be effective. An integration of these strategies is a
promising approach that can solve some of these problems [5]. Information visualization techniques such
as the Perspective Wall at Xerox PARC [2] can be used in interface design to improve both browsing and
querying.
Good information retrieval system design combines a combination of support for informat
ion seeking strategies, such as browsing and direct querying, in an interface that provides effective cues to
the location, use, and characteristics of the retrieved information. Feedback techniques are also crucial to
support the iterative refinement of
information needs.
Traditional retrieval system evaluation relies on the measures of recall (the proportion of
relevant items in the entire repository which have been retrieved) and precision
(the proportion of retrieved items which are relevant) for assessment of system effectiveness. There are
significant problems with these measures of effectiveness, and the criterion on which they are based,
relevance, including such issues as: who makes the relevance judgments, and how are these
judgments related to the user's context and the use of the items; lack of knowledge of the total number of
relevant items in the repository for any given information problem;
how to evaluate these measures over the course of an interaction; and, the validity of the measures
themselves as indicators of the effectiveness of the information interaction. This suggests the need for new
measures of retrieval effectiveness in interactive retrieval systems.
Furthermore, new evaluation techniques are needed that account not only for the accuracy of a retrieval
system, but also its interactive abilities and ease of use. A system that measures poorly in recall and
precision, but provides good browsing and iterative querying facilities may be more successful overall in
responding to a person's information problem than a system which is "more effective" in terms of recall and
precision.
Guidelines for the design of information retrieval systems must address not only issues of look-and-feel, but
also of effective interaction. Dialog models based on relevance feedback and query reformulation explicitly
address the ill-defined nature of information seeking by allowing users to learn from the repository and
iteratively refine the information need. Systems need to support a number of interaction styles, such as
querying and browsing, to accommodate the different kinds of search strategies users may need to use.
Design strategies for retrieval systems need to pay particular attention to the interaction between users and
the texts they retrieve. People's information seeking behavior needs to be analyzed in the problem solving
context in which their information nee
ds arise. For example, what are some of the common associations people make? What informa
tion is closely related? What different perspectives can a piece of information be viewed from? The
answers to these questions often differ, depending critically on the nature of the task and the individuals
performing the task. Task analysis, user mode
ling and interaction modeling are some of the strategies that can be used to improve the design of retrieval
systems.
Abstract
Keywords:
information retrieval, user interfaces, databases, information systems, interaction strategies.
INFORMATION RETRIEVAL AS A PROBLEM SOLVING PROCESS
THE VOCABULARY PROBLEM
INTERFACES FOR RETRIEVAL SYSTEMS
EVALUATION TECHNIQUES
DESIGN STRATEGIES
References