skip to main content
10.1145/952532.952698acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
Article

Web metasearch: rank vs. score based rank aggregation methods

Published: 09 March 2003 Publication History

Abstract

Given a set of rankings, the task of ranking fusion is the problem of combining these lists in such a way to optimize the performance of the combination. The ranking fusion problem is encountered in many situations and, e.g., metasearch is a prominent one. It deals with the problem of combining the result lists returned by multiple search engines in response to a given query, where each item in a result list is ordered with respect to a search engine and a relevance score. Several ranking fusion methods have been proposed in the literature. They can be classified based on whether: (i) they rely on the rank; (ii) they rely on the score; and (iii) they require training data or not. Our paper will make the following contributions: (i) we will report experimental results for the Markov chain rank based methods, for which no large experimental tests have yet been made; (ii) while it is believed that the rank based method, named Borda Count, is competitive with score based methods, we will show that this is not true for metasearch; and (iii) we will show that Markov chain based methods compete with score based methods. This is especially important in the context of metasearch as scores are usually not available from the search engines.

References

[1]
A. Aslam, Javed and Mark Montague. Models for metasearch. In ACM SIGIR-01, pages 276--284, 2001.
[2]
J. C. Borda. Mémoire sur les élections au scrutin. Histoire de I'Académie Royal des Sciences, 1781.
[3]
Sergey Brin and Lawrence Page. The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30(1--7):107--117, 1998.
[4]
Jamie Callan, Zhihong Lu, and Bruce W. Croft. Searching distribute collections with inference networks. In ACM SIGIR-95, pages 21--28, 1995.
[5]
Nick Craswell, David Hawking, and Paul Thistlewaite. Merging results from isolated search engines. In 10th Australian Database Conf., 1999.
[6]
Daniel Dreilinger and Adele E. Howe. Experiences with selecting search engines using metasearch. ACM Transactions on Information Systems, 15(3):195--222, 1997.
[7]
Cynthia Dwork, Ravi Kumar, Moni Noar, and D. Sivakumar. Rank aggregation methods for the web. In 10th International Conf. on the World Wide Web, pages 613--622. ACM Press and Addison Wesley, 2001.
[8]
Ronald Fagin and Edward L. Wimmers. Incorporating user preferences in multimedia queries. In Proc. of 6th International Conf. on Database Theory, LNCS 1186, 1997.
[9]
Joseph A. Fox, Edward Shaw. Combination of multiple sources: The TREC-2 interactive track matrix experiment. In ACM SIGIR-94, 1994.
[10]
Susan Gauch, Guijun Wang, and Mario Gomez. ProFusion: Intelligent fusion from multiple, distributed search engines. volume 2, pages 637--649, 1996.
[11]
Steve Lawrence and Lee C. Giles. Inquirus, the NECI meta search engine. Computer Networks and ISDN Systems, 30:95--105, 1998.
[12]
Joon Ho Lee. Analysis of multiple evidence combination. In ACM SIGIR-97, pages 267--276, 1997.
[13]
R. Manmatha, R. Rath, and F. Feng. Modeling score distributions for combining the outputs of search engines. In ACM SIGIR-01, pages 267--275, 2001.
[14]
D. G. Saari. The mathematics of voting: Democratic symmetry. The Economist, March 4 2000.
[15]
Gerard Salton and J. Michael McGill. Introduction to Modern Information Retrieval. Addison Wesley Publ. Co., 1983.
[16]
E. Selberg and O. Etzioni. The MetaCrawler architecture for resource aggregation on the Web. IEEE Expert, (January-February):11--14, 1997.
[17]
Christopher C. Vogt and Garrison W. Cottrell. Fusion via a linear combination of scores. Information Retrieval, 1(3):151--173, 1999.
[18]
Ellen M. Voorhees, Narendra K. Gupta, and Ben Johson-Laird. The collection fusion problem. In D. K. Harman, editor, Proc. 3rd Text Retrieval Cconference (TREC-3), number 500--225, 1994. National Institute of Standards and Technology.
[19]
Ronald R. Yager and Rybalov. On the fusion of documents from multiple collection information retrieval systems. Journal of the American Society for Information Science, 13(49):1177--1184, 1998.

Cited By

View all
  • (2024)A survey on rank aggregationProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/915(8281-8289)Online publication date: 3-Aug-2024
  • (2024)The Power of Linear Programming in Sponsored Listings Ranking: Evidence from Field ExperimentsSSRN Electronic Journal10.2139/ssrn.4767661Online publication date: 2024
  • (2024)How Normalization Strategies Affect the Quality of Rank Aggregation Methods in Recommendation SystemsProcedia Computer Science10.1016/j.procs.2023.10.174225:C(1843-1852)Online publication date: 4-Mar-2024
  • Show More Cited By

Index Terms

  1. Web metasearch: rank vs. score based rank aggregation methods

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SAC '03: Proceedings of the 2003 ACM symposium on Applied computing
      March 2003
      1268 pages
      ISBN:1581136242
      DOI:10.1145/952532
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 09 March 2003

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. meta-search
      2. rank list aggregation

      Qualifiers

      • Article

      Conference

      SAC03
      Sponsor:
      SAC03: ACM Symposium on Applied Computing
      March 9 - 12, 2003
      Florida, Melbourne

      Acceptance Rates

      Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

      Upcoming Conference

      SAC '25
      The 40th ACM/SIGAPP Symposium on Applied Computing
      March 31 - April 4, 2025
      Catania , Italy

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)46
      • Downloads (Last 6 weeks)9
      Reflects downloads up to 15 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)A survey on rank aggregationProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/915(8281-8289)Online publication date: 3-Aug-2024
      • (2024)The Power of Linear Programming in Sponsored Listings Ranking: Evidence from Field ExperimentsSSRN Electronic Journal10.2139/ssrn.4767661Online publication date: 2024
      • (2024)How Normalization Strategies Affect the Quality of Rank Aggregation Methods in Recommendation SystemsProcedia Computer Science10.1016/j.procs.2023.10.174225:C(1843-1852)Online publication date: 4-Mar-2024
      • (2024)Semantic deep learning and adaptive clustering for handling multimodal multimedia information retrievalMultimedia Tools and Applications10.1007/s11042-024-19312-7Online publication date: 25-May-2024
      • (2023)FLAGR: A flexible high-performance library for rank aggregationSoftwareX10.1016/j.softx.2023.10131921(101319)Online publication date: Feb-2023
      • (2022)Influence of Late Fusion of High-Level Features on User Relevance Feedback for VideosProceedings of the 2nd International Workshop on Interactive Multimedia Retrieval10.1145/3552467.3554795(17-24)Online publication date: 14-Oct-2022
      • (2022)ranx.fuse: A Python Library for MetasearchProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557207(4808-4812)Online publication date: 17-Oct-2022
      • (2022)Comparative analysis of managers’ perception in overseas construction project risks and cost overrun in actual cases: a perspective of the Republic of KoreaJournal of Asian Architecture and Building Engineering10.1080/13467581.2022.211694022:4(2291-2308)Online publication date: 4-Sep-2022
      • (2022)Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United StatesProceedings of the National Academy of Sciences10.1073/pnas.2113561119119:15Online publication date: 8-Apr-2022
      • (2021)Partition–Mallows Model and Its Inference for Rank AggregationJournal of the American Statistical Association10.1080/01621459.2021.1930547118:541(343-359)Online publication date: 8-Jul-2021
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media