Claim V

Claim V states that user performance improves if IR systems retrieve broad results for exploratory tasks and narrower sets for lookup. Studies IV and V together confirmed the fifth key claim made in this thesis. It is possible to explain why the adaptation of exploration rate truly improves user performance by considering the AIF. With exploratory tasks, users have difficulty in formulating search queries that properly express their information need, so the search engine cannot trust the queries. Most IR systems ignore this fact and rank the documents that are most relevant for the search query at the top. This creates a gap between IR-system-predicted relevance and the actual user-perceived relevance. Particularly in the case of exploratory search tasks, this gap can become very large when user familiarity with the domain is low. This would result in an ecology wherein the result list contains only a few documents that the user perceives as relevant. When, on the other hand, the IR system returns more varied results, through employing a higher exploration rate, there is a better chance of encompassing more topics that the user would perceive as relevant. This affects the search strategy—with determination that more of the documents are relevant in the latter case.

Chapter 7

Discussion

Exploration is a natural human behavior motivated by our thirst for knowl-edge. With recent leaps forward in technology, we now have the opportunity to access a vast amount of information much more rapidly via the Web [157].

Although many IR systems and technologies have been designed to im-prove the retrieval of information, information-seekers still struggle when undertaking exploratory searches [7]. Hence, better search systems are im-portant, to enable faster and effortless exploration of information. Accord-ingly, I investigated exploratory and lookup search tasks for the purpose of revealing solutions to some of the prominent challenges encountered in information search.

This dissertation has addressed the challenges in information search in relation to three themes: 1) conceptualizing and understanding informa-tion search, 2) modeling and predicting informainforma-tion search behaviors, and 3) providing real-time adaptive support for exploratory and lookup search tasks. To conceptualize and understand the search strategies applied with different tasks, I used the adaptive interaction framework [123]. The the-sis contributes a model designed to predict dynamic information needs—

subjective specificity—in exploration, a classifier built from interaction logs to discriminate between exploratory and lookup tasks, and a reinforcement learning model to predict the adaptive interaction strategies pertinent for exploratory search. Possible approaches to provision of real-time support for both exploratory and lookup tasks were proposed through consideration of these models.

95

7.1 Summary of the Main Findings

The main findings yielded by this research are best summarized with ref-erence to the five key claims.

According to the first claim, exploratory search strategies emerge as an adaptation to ecology, mechanism, and utility in the AIF. In exploratory search tasks, there is no exact “correct” answer; therefore, the utility in-volves finding documents with high levels of user-expected information gain.

This results in an unpredictable distribution with regard to the ecology.

Since this scenario involves a search domain that is less familiar to the information-seeker, several cognitive constraints (such as comprehension time and memory) influence the mechanism alongside common constraints such as reading time, fixation time, and saccade time. When undertaking lookup tasks, on the other hand, the user has a clear target in mind, so the utility would generally involve finding a single document. Ecology follows a predictable pattern indicating that the topmost documents are the most rel-evant here, and the mechanism involves the common cognitive constraints (reading, fixation, and saccade times). When these differences in ecology, mechanism, and utility are borne in mind, it is possible to explain the differ-ences in rational information-seekers’ search strategy between exploratory and lookup tasks. Therefore, I was able to propose an approach for im-plementing a model to predict the rational exploratory search strategies by training a reinforcement learning agent. I applied formal models of the ecol-ogy to generate training data, of utility to define the reward function, and of the mechanism to compute the costs involved in the reward function (or human constraints that contribute negative reward). The model-predicted strategies of an exploratory searcher closely match the strategies applied by an actual user. These are exciting findings that confirm the first claim.

The second claim points to exploratory search as the most challenging and one of the most common search purposes. This claim was arrived at via case-study research that involved observation of information-seekers that was followed by a Web-based survey. The study found five common purposes for which academics initiate search activities: staying up to date on the field, exploring unfamiliar topics, collaborating, reviewing literature, and teaching. Of these purposes, exploring unfamiliar topics turned out to be among the most common search purposes yet one of the most difficult to address. These findings emphasize the importance of designing better tools and techniques to support exploration. Although there has been a large amount of interest in designing search systems, of various types, clear room for improvement remains.

7.2 Implications of the Research 97