Information retrieval (IR) is finding material, usually documents, of an unstructured nature, usually text, that satisfies an information need from within large collections. This is the definition in the textbook by Manning, Raghavan and Schütze (2008). Search engines are its best-known application.
What does information retrieval cover?
IR deals with how documents are collected, stored in an index and ordered by relevance for a query. IBM names three core techniques, indexing, weighting (for example TF-IDF) and relevance feedback. A database returns exact matches from structured data, while an IR system returns a list sorted by computed relevance.
Three families of models are common:
- Boolean. A document matches the query or it does not, with no ranking.
- Algebraic. The vector space model represents documents and queries as vectors and allows partial matches.
- Probabilistic. Models estimate how likely a document is to be relevant.
Wikipedia names precision and recall as standard measures. Precision is the share of retrieved documents that are relevant. Recall is the share of relevant documents that were retrieved.
Relevance for SEO and AI search
Search engines crawl and index pages before they rank them, which is the pipeline IR describes. Elastic points out that retrieval systems are moving from ranked lists toward synthesized answers, where retrieved documents feed a language model. This pattern, retrieval-augmented generation (RAG), was described by Lewis et al. in 2020. The related term rag covers it.
Example
A query for “cheap trail running shoes” is split into terms. The system looks up documents containing them in its index, weights the matches and returns them in order. The better the top results fit the intent, the higher the precision.
FAQ
What is an example of an information retrieval system? A web search engine, a library catalog or the search function of an online shop.
Who coined the term? According to reports, Calvin Mooers coined it in 1950.
Sources
- Introduction to Information Retrieval – Manning, Raghavan, Schütze, Cambridge University Press 2008nlp.stanford.edu
- Information retrieval – Wikipedia (secondary)en.wikipedia.org
- What is information retrieval? – IBM (secondary)ibm.com
- What is information retrieval? – Elastic (secondary)elastic.co
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks – Lewis et al. (2020)arxiv.org
Last updated: