Teneo page drift survey, 9 November 2026
How fast does a stored copy of a public web page go stale?

pages.csv         The 60 public pages, chosen before the run.
drift-survey.mjs  The script. For each page it reads the Internet Archive's dated captures, takes the newest,
                  and compares it with the capture nearest to 7, 30, 90 and 180 days earlier.
drift-survey.json Per-page results. No page text is stored, only counts and a sample of numbers that disappeared.
analyse.mjs       Builds the summary.
evidence.json     The summary used in the article.

Method and limits
- A "block" is a run of text of 20 characters or more between HTML tags, after removing scripts, styles, navigation and footers.
- "Lost at least 5% of their text" means at least 5% of the older capture's blocks are not on the newer capture.
- "Lost at least one number" means a price, percentage or decimal value in the older capture is nowhere on the newer one.
- Windows: 7 days means 4 to 10, 30 means 22 to 38, 90 means 75 to 105, 180 means 150 to 210.
- Both sides of each comparison are from the archive's crawler, so origin is held constant.
- The list is hand-picked and small. 49 of 60 pages gave a usable comparison. A rewording counts as a change.
- Live checks: one request and one conditional repeat per page from a home connection in Ireland, client named as a survey.
- The run was resumed once after being made parallel, so the first five pages were measured minutes before the rest. The "started" time in the JSON is the resumed run.

Article: https://teneo-protocol.ai/blog/how-to-keep-rag-data-fresh