A TDM reservation is a rights holder’s declaration that third parties may not use their content for text and data mining (TDM). Article 4 of Directive (EU) 2019/790 sets out an exception for text and data mining of lawfully accessible works. Under Article 4(3), it applies on condition that rights holders have not expressly reserved the use “in an appropriate manner, such as machine-readable means” for content made publicly available online. The TDM Reservation Protocol (TDMRep) is one technical way to do that.
At a glance
- Publisher
- W3C TDM Reservation Protocol Community Group
- Connects
- Rights holders and parties that carry out text and data mining
- Status
- Community Group Final Report of May 10, 2024, not a W3C Standard
- Transport
- File at /.well-known/tdmrep.json, HTTP header, HTML meta tag
- Specification
- TDMRep, Final Report
Last checked:
This is not legal advice.
How does TDMRep work?
TDMRep is a specification of a W3C Community Group. The final report dates from May 10, 2024. It is not a W3C Standard. It defines two properties. tdm-reservation is 1 when rights are reserved and 0 when they are not. tdm-policy is an optional URL pointing to a policy with licensing terms and contact details.
The protocol offers several places to state them:
- a file at
/.well-known/tdmrep.jsonwith rules per path pattern - HTTP response headers
- a
<meta>tag in HTML - metadata inside EPUB and PDF files
According to the report, a later level overrides an earlier one: header values override the file, and HTML meta values override headers. EDRLab, which maintains the protocol, says adoption is concentrated among publishers in Western Europe.
| File | Standard or proposal? | Location | Purpose | Who reads it (documented) |
|---|---|---|---|---|
| robots.txt | IETF standard: RFC 9309 (September 2022) | / | tells crawlers which URLs they may fetch; per the RFC not a form of access authorization | Crawlers; Google's common crawlers always respect it for automatic crawls |
| llms.txt | Proposal (Jeremy Howard, 2024; version 2 from August 2026) | / or in a subpath | Markdown overview linking to Markdown versions of pages, for AI agents | per the proposal mainly coding agents; Google Search ignores it, says Google; Lighthouse only checks the fetch |
| llms-full.txt | no specification; not part of the llms.txt proposal, a convention of individual vendors (e.g. Mintlify) | / (Mintlify also serves /.) | entire documentation as Markdown in one file | no reader documented; Google Search doesn't use such files, says Google |
| ai.txt | Individual IETF Internet-Draft (June 2026, expires 2026-12-14), not a standard; predecessor by Spawning (2023) | /. (draft), / (Spawning) | AI usage preferences (training, scraping, indexing, caching), licensing, attribution | no reader documented in primary sources; Google Search doesn't use 'AI text files', says Google |
TDMRep (tdmrep.) (this term) | W3C Community Group Final Report (2024-05-10), not a W3C standard | /., HTTP header tdm-reservation or HTML meta tag | machine-readable reservation of text and data mining rights (Art. 4 DSM Directive) | parties performing text and data mining (per the specification) |
| security.txt | RFC 9116 (April 2022, informational, not standards track) | /. | contact for reporting security vulnerabilities; required fields Contact and Expires | Security researchers |
| ads.txt | IAB Tech Lab specification, version 1.1 (August 2022) | / on the root domain, subdomains as needed | declares the companies authorized to sell the site's ad inventory | advertising systems and buyers (ad system crawlers) |
| humans.txt | Initiative (humanstxt.org), no formal specification | / next to robots.txt, optionally linked via rel="author" | names the people who contributed to the website | people who want to know who is behind the website |
Only robots.txt, security.txt and ads.txt are based on a finalised specification (IETF or IAB Tech Lab). Google states that Google Search doesn't use new machine-readable files or 'AI text files' and ignores llms.txt files, which neither help nor harm Search visibility. TDMRep is a technical way to express the reservation under Art. 4 of Directive (EU) 2019/790. Not legal advice. Sources: RFC 9309: Robots Exclusion Protocol; Google: Overview of Google crawlers and fetchers (last updated 2026-06-12); The /llms.txt file, v2 (llmstxt.org); llmstxt.org: Changes (v2, August 2026); Google Search Central: Optimizing for generative AI features (last updated 2026-07-10); Chrome for Developers: Lighthouse llms.txt audit (2026-05-05); Mintlify: llms.txt and llms-full.txt; IETF Internet-Draft draft-car-ai-txt-wellknown-00 (2026-06-12); W3C TDMRep Community Group: TDM Reservation Protocol, Final Report 2024-05-10; RFC 9116: A File Format to Aid in Security Vulnerability Disclosure; IAB Tech Lab: ads.txt; IAB Tech Lab: ads.txt Specification Version 1.1 (GitHub); humanstxt.org. Last checked: .
Example
<meta name="tdm-reservation" content="1">
<meta name="tdm-policy" content="https://example.com/tdm-policy.json">
The same values can be sent as HTTP headers (tdm-reservation: 1) or listed in tdmrep.json.
Common questions
Does a TDM reservation replace robots.txt? They do different jobs. robots.txt controls crawling. A TDM reservation states a rights position about using the content.
Do AI crawlers honor it? That depends on the operator. The protocol defines the signal, and no obligation for crawlers follows from the specification itself.
Sources
Last updated: