4281

Segment-level display time as implicit feedback: a comparison to eye tracking

Georg Buscher, Ludger van Elst, Andreas Dengel

SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval ACM SIGIR Conference on Information Retrieval (SIGIR-09), 32nd, July 19-23, Boston,, MA, USA , Pages: 67-74 , ACM , 2009
We examine two basic sources for implicit relevance feed- back on the segment level for search personalization: eye tracking and display time. A controlled study has been conducted where 32 participants had to view documents in front of an eye tracker, query a search engine, and give explicit relevance ratings for the results. We examined the performance of the basic implicit feedback methods with respect to improved ranking and compared their performance to a pseudo relevance feedback baseline on the segment level and the original ranking of a Web search engine. Our results show that feedback based on display time on the segment level is much coarser than feedback from eye tracking. But surprisingly, for re-ranking and query expansion it did work as well as eye-tracking-based feedback. All behavior-based methods performed significantly better than our non-behavior-based baseline and especially improved poor initial rankings of the Web search engine. The study shows that segment-level display time yields comparable results as eye-tracking-based feedback. Thus, it should be considered in future personalization systems as an inexpensive but precise method for implicit feedback.

Show BibTex:

@inproceedings {
       abstract = {We examine two basic sources for implicit relevance feed-
back on the segment level for search personalization: eye
tracking and display time. A controlled study has been conducted where 32 participants had to view documents in front
of an eye tracker, query a search engine, and give explicit
relevance ratings for the results. We examined the performance of the basic implicit feedback methods with respect
to improved ranking and compared their performance to a
pseudo relevance feedback baseline on the segment level and
the original ranking of a Web search engine.
Our results show that feedback based on display time
on the segment level is much coarser than feedback from
eye tracking. But surprisingly, for re-ranking and query
expansion it did work as well as eye-tracking-based feedback. All behavior-based methods performed significantly
better than our non-behavior-based baseline and especially
improved poor initial rankings of the Web search engine.
The study shows that segment-level display time yields
comparable results as eye-tracking-based feedback. Thus, it
should be considered in future personalization systems as an
inexpensive but precise method for implicit feedback.},
       number = {}, 
       month = {}, 
       year = {2009}, 
       title = {Segment-level display time as implicit feedback: a comparison to eye tracking}, 
       journal = {}, 
       volume = {}, 
       pages = {67-74}, 
       publisher = {ACM}, 
       author = {Georg Buscher, Ludger van Elst, Andreas Dengel}, 
       keywords = {},
       url = {}
}