Key ideas

Röttger & Pierrehumbert (2021) · Temporal Adaptation of BERT

Temporal adaptation helps the language-modelling objective but not the downstream task. Continuing BERT on period text captures event-driven vocabulary, which turns out not to be what the classifier needed; adapting to domain alone did as well.

A negative result worth remembering: a model that “sounds like” a period is not thereby better at tasks in that period.

Loureiro et al. (2022) · TimeLMs

Ship the sequence with a router. Quarterly Twitter checkpoints plus a corresponding mode that scores each tweet with the checkpoint trained only up to that tweet's quarter. The paper also documents how perplexity on future tweets degrades and how continual updates repair it.

The routing idea, rather than the models, is the lasting contribution: a point-in-time sequence is only useful if every query is matched to the right cut-off automatically.

Qiu & Xu (2022) · HistBERT

Whether contextual embeddings detect semantic change depends on when their pretraining text was written. Decade models continued on COHA beat modern BERT on historical word-similarity and shift tasks.

A modest paper, but it is the earliest evidence that the temporal profile of pretraining data is a variable, not a nuisance.

Sarkar (2024) · StoriesLM

Sequentially expanding pretraining windows over a century of newspapers. Each of 64 yearly models continues the previous year's checkpoint on that year's American Stories text, so a historian can pick a model as of any year from 1900 to 1963.

The only released sequence covering the early twentieth century; it is what makes the “simulated historical informant” programme concrete.

He, Lv, Manela & Wu (2025) · Instruction Tuning Chronologically Consistent LMs

Instruction tuning need not reintroduce leakage if the instruction data is period-bounded too. The result is a chat model per cut-off whose forecasts are a conservative lower bound on the predictability that survives once leakage is gone.

Makes point-in-time models usable by people who prompt rather than fine-tune.

Fittschen, Li, Lippincott & Choshen (2025) · Pretraining for Diachronic Linguistic Change Discovery

Domain-restricted pretraining is the only guarantee of period-restricted inference, and it is cheaper than assumed. Small models pretrained on five 10-million-word time slices train faster than parameter-efficient fine-tunes of an 8B model and behave more faithfully to their period. Includes a date-attribution pipeline for undated text.

Argues against fine-tuning as a substitute for period pretraining in the humanities, on both cost and fidelity grounds.

Li et al. (2025) · TiC-LM

Continual pretraining with replay matches retraining from scratch at 2.6× less compute. A benchmark built from 114 Common Crawl dumps with time-stratified evaluation; an autoregressive meta-schedule plus fixed-ratio replay of old data holds held-out loss level. Replay matters for general web text, much less for specific domains.

The economics of producing a checkpoint sequence: you do not have to retrain every month if the update schedule is designed properly.

Faro, Fan, Alphaidze & Jaggi (2025) · TiMoE

One set of weights, every cut-off: mask experts by date. Experts are pretrained from scratch on disjoint two-year slices of 2013–2024; at inference, experts whose window ends after the query date are masked and the rest are merged in log-probability space. Causal validity comes from routing, not retraining. Also releases TSQA, whose answer options are labelled past, future, or irrelevant.

The most compute-efficient route to a point-in-time model so far proposed, at the cost of a coarser two-year cadence.

Pilchen, Fabre, Signe Talla & Pérez (2026) · Data Temporality in Pre-training

Order matters: pretraining on Common Crawl snapshots in chronological order gives fresher, more temporally precise knowledge than shuffled pretraining, with no loss of general ability. Shuffled training peaks on older facts. Comes with a 7,000-question benchmark that checks whether a fact is tied to the right period.

A cheap lever for temporal grounding that costs nothing in tokens.

Luo, Shinnick, Griesshaber & Wang (2026) · TypewriterLM

A 7B model trained only on pre-1913 English, with post-training that cannot leak. The novelties are the leakage-mitigation steps in building the 54B-token TypewriterCorpus, “lexically grounded” instruction tuning that forces answers to stay anchored in historical source documents, and a benchmark that tests for leakage as well as competence.

Shows that the chat-model recipe can be run inside a period without importing the present, at a scale where the model is genuinely useful.

Lopez-Lira & Tang (2023) · Can ChatGPT Forecast Stock Price Movements?

GPT-4 headline scores predict next-day returns, on post-cutoff headlines. The design was careful about the cut-off; the result is what made everyone else ask whether their in-sample results were.

The motivating paper for the whole finance thread, not because it leaked but because it showed how much was at stake if others did.

Rahimikia & Drinkall (2024) · Re(Visiting) Large Language Models in Finance

Re-run finance LLM results with point-in-time models in hand. Uses the Time Machine GPT sequence to separate what survives a clean cut-off from what does not. abstract not retrieved

The natural follow-up to TiMaGPT by the same group; details should be checked against the paper.

Ludwig, Mullainathan & Rambachan (2024) · An Applied Econometric Framework

“No training leakage” is an identification condition, like any other. For prediction uses of LLM output it must be enforced by model choice and research design; for estimation uses a validation sample is required, or seemingly innocuous choices of model and prompt swing the estimates.

Puts leakage where economists will take it seriously: in the list of assumptions a referee checks.

Crane, Karra & Soto (2025) · Total Recall? (Federal Reserve)

LLMs smooth across data vintages. They recall recent macro statistics precisely but blend first prints with later revisions, and blend past with future reference periods; on any given day the model believes it holds data that had not yet been released.

Vintage awareness is a separate failure from knowledge cutoff, and one that only real-time datasets can test.

Paleka, Goel, Geiping & Tramèr (2025) · Pitfalls in Evaluating LM Forecasters

A catalogue of the ways time leaks into forecaster evaluation. Question creation, retrieval, and resolution each admit leakage; combined with the difficulty of extrapolating from evaluation to deployment, claims of human-level forecasting are premature.

The checklist a forecasting paper should be read against.

Li et al. (2025) · DeepFund

The only leak-proof backtest is a live one. A benchmark that feeds LLM fund managers real-time market data published after each model's cutoff, so “time travel” is impossible.

Trades statistical power for validity; the complement to point-in-time models, which trade the other way.

Merchant & Levy (2025) · A Fast and Effective Solution to Look-ahead Bias

Logit arithmetic with two small models. Steer a large base model at inference using a small model fine-tuned on what to forget and another on what to keep. Removes both verbatim and semantic knowledge of the period without retraining the frontier model.

The practical answer for people who cannot retrain: unlearning by decoding rather than by weights.

Gao, Jiang & Yan (2025) · Detecting Lookahead Bias in LLM Forecasts

Lookahead Propensity: ask the model, from the date alone, what happened. The recall probability is high throughout the training window and collapses to zero after the cutoff. Its interaction with forecast accuracy in a regression is a test for contamination, and it fires on both headline–return and earnings-call–capex tasks.

A cheap diagnostic that needs no clean model, only a recall query per firm-date.

Eliseev & Seleznev (2026) · Fake Date Tests

Lie to the model about today's date and see if its in-sample forecast moves. Sensitivity to the fake date separates lookahead bias from context bias. No modern LLM passed.

A one-line test any macro forecaster can run before trusting a backtest.

Kong et al. (2026) · Evaluating LLMs in Finance Requires Explicit Bias Consideration

Five biases, and almost nobody checks for them. Look-ahead, survivorship, narrative, objective, and cost bias compound into an illusion of validity; of 164 papers from 2023–2025, no single bias is discussed in more than 28%. Proposes a structural-validity checklist.

A position paper, but the survey number is the contribution.

Zhang, Chen & Stadie (2026) · All Leaks Count, Some Count More

Weight each leaked claim by its effect on the decision. Decompose a rationale into atomic claims, attribute Shapley values to each, and report the fraction of decision-driving reasoning that is post-cutoff (Shapley-DCLR). TimeSPEC then interleaves date-filtered retrieval with claim-level supervision to force pre-cutoff grounding.

Moves leakage measurement from “did it mention a future fact” to “did that fact change the answer”.

Gao, Jiang & Yan (2026) · Debiasing LLMs by Fine-tuning

Extrapolation bias in LLM forecasts yields to LoRA fine-tuning on rational benchmark forecasts, where prompting did not. Holds out of sample in both controlled experiments and cross-sectional return prediction.

About a different bias than look-ahead, but the same lesson: parameter-level intervention succeeds where instructions fail.

Levy (2026) · Caution Ahead (Journal of Accounting Research)

Much of LLMs' apparent superhuman performance in accounting and finance is modelling artifact. Two culprits: very poor numerical reasoning, and look-ahead bias in commercial models that may account for a large share of their predictive ability.

The accounting literature's formal notice that its headline AI results need re-examination.

Zhang & Stadie (2026) · TEMPO

Temporal compliance is instance-specific, so unlearning cannot work. The same fact is legitimate evidence for one cutoff and a violation for another. Instead of erasing knowledge, train temporal discipline with reinforcement learning: a leakage mode that drives post-cutoff claims to zero is a hard prerequisite before a performance mode optimises accuracy.

Reframes the goal from forgetting to selecting, which is the only version that scales to many cut-offs.

Zhang, Li, Peng & Chen (2026) · When Alpha Disappears

Leakage lives in the evaluation protocol as well as the model. Toggle one convention at a time around a clean t+1-open reference and measure the inflation. Centred temporal features and same-day-open execution with post-open information inflate results a lot; global normalisation and same-day-close execution barely do.

A ranked list of which backtest sins actually matter.

Li, Wang & Ma (2026) · FinCAD

Find the prompt that wakes the model's memory of the outcome, then subtract that memory at decode time. An adversarial pipeline learns a model-specific memory-activating prompt; context-aware decoding attenuates its contribution, scaled per entity and date by a confidence signal. Largest in-sample return correction −67%; out-of-sample performance and general benchmarks essentially unchanged.

Names “parametric look-ahead bias” and offers the most surgical inference-time fix so far.

Merchant & Levy (2026) · Feature Steering

Time-awareness is a steerable feature; look-ahead bias is not. Sparse autoencoders find features for time-aware and for look-ahead-biased reasoning. Amplifying the time-awareness features cuts look-ahead bias across a different domain; steering the look-ahead features does nothing.

Suggests the bias is diffuse and the fix is to strengthen the model's sense of “when” rather than to delete what it knows.

Fonseca (2026) · Look-Ahead-Freedom as Temporal Non-Interference

Look-ahead-freedom is an information-flow property and can be type-checked. Separating a datum's availability time from its reference time, freedom from leakage is undecidable when availability depends on data values, but for the value-independent fragment (windowing, resampling, joins, point-in-time reads, agentic retrieval) a type-and-effect system decides it in linear time.

Moves part of the problem from empirical detection, which certifies nothing by its silence, to static verification.

Jia (2026) · HindsightBench

Audit parametric hindsight in any model at probe cost, with no backtest. A four-arm date-manipulation matrix (revealed, date-only, masked, transplanted) plus memory probes yields six metrics including a behaviourally effective cutoff. Across 15 models the date-trigger reflex tracks training recency, not scale.

A black-box protocol for buyers of models, who have neither log-probs nor corpus access.

Cai et al. (2026) · OpenPM

Point-in-time discipline for agents, with a certificate. Every record visible to a portfolio agent must be available at decision time; natural-language risk mandates become typed constraints enforced on the executed book; each run emits a contamination certificate and a cost-sensitivity curve.

Extends the cut-off idea from training data to the whole agent loop, including retrieval and execution.

Zhao, Brumbaugh, Wang, Hajishirzi & Smith (2024) · Set the Clock

Models have an internal “now” that lags their cutoff, and it can be moved. LLaMa2 with a 2022 cutoff answers as if it were 2019. Aligning it to 2022 improves answers by up to 62%; aligning to 2010 is also possible, with 2.8× the unaligned performance on that year's answers.

The positive case for post-hoc temporal alignment; later work shows its limits on causally downstream knowledge.

Park et al. (2025) · Temporal Heads

Specific attention heads carry time-specific knowledge. Ablating them removes recall of facts tied to a year while leaving time-invariant knowledge and general QA intact; they fire on textual aliases of dates as well as numbers, and their values can be edited.

The first mechanistic handle on where “as of when” lives inside a transformer.

Wongchamcharoen & Glasserman (2025) · Do LLMs Understand Chronology?

Frontier models keep local order but cannot hold a single global timeline. Exact-match ordering collapses as sequences lengthen even while rank correlation stays high; conditional sorting fails at the filtering step; anachronism detection is easiest but degrades with overlapping timelines. Extended reasoning budgets help.

Prompt-based defences against look-ahead presuppose a capability the models do not reliably have.

Pęzik et al. (2025) · LLMLagBench

Infer an undeclared training cutoff from what the model knows about recent events. A freshness benchmark that estimates the earliest probable temporal boundary, validated against disclosed cutoffs.

Necessary bookkeeping: you cannot reason about leakage in a model whose cutoff you do not know.

Li et al. (2026) · Simulated Ignorance Fails

Simulated ignorance leaves a 52% gap to true ignorance. Across 477 competition-grade forecasting questions and nine models, cutoff instructions do not suppress prior knowledge, chain-of-thought does not either even when the trace contains no post-cutoff reference, and reasoning-optimised models are worse at pretending. Retrospective forecasting on pre-cutoff events is therefore invalid.

The largest-scale demonstration that prompts cannot rewind a model.

Ding et al. (2026) · Temporal Critique Fine-Tuning

Ex-ante correctness is a relation between an answer and a cutoff, not a property of the answer, so supervised fine-tuning cannot teach it. Leakage is sensitive to prompt form (explicit cutoffs beat implicit framings; prefix constraints beat suffix), but prompting steers the frame without giving the model any way to check admissibility. TCFT trains a critic that judges temporal admissibility instead.

Same conclusion as TEMPO by a different route: the model needs a verifier, not a memory wipe.

Asai et al. (2026) · Recall-Based Prompting

Ask the model to restate its cutoff and to recall what it validly knows before answering. Self-Recall and Question-Recall beat direct answering and step-by-step reasoning, with the largest gains on counterfactual questions. A multi-cutoff benchmark shows performance varies with distance to the cutoff.

The best prompting can do; still not a substitute for a clean model.

van Adrichem, Bhaskar, Yang & Potts (2026) · Do LMs Consistently Encode the Current Year?

Two probes for “what year is it” use different mechanisms, and nothing updates both. An associative probe (infer the year from verb tense) tracks the pretraining cutoff within ten months for base models; a declarative probe asks outright. Prompting moves the declarative year; neither prompting, SFT, nor weight editing moves both together.

Explains why temporal alignment methods look successful on one test and fail on another.

Dhingra et al. (2021) · Time-Aware LMs as Temporal Knowledge Bases

Prefix each training document with its date, and one model learns to condition on time. Improves memorisation of period facts and calibration about the future, and allows refreshing with new data instead of retraining.

The single-model alternative to a checkpoint sequence; it does not prevent leakage, but it gives the model a clock.

Jang et al. (2021) · Continual Knowledge Learning

Updating world knowledge is its own continual-learning problem. Three things must happen at once: retain invariant knowledge, update outdated knowledge, acquire new knowledge. Standard CL methods fail; parameter expansion is needed.

Defines the metric that TemporalWiki and later benchmarks measure.

Jin et al. (2021) · Lifelong Pretraining

Distillation-based continual pretraining best retains earlier-domain performance while adapting to a chronological stream. Also improves temporal generalisation where train and test periods differ.

Method paper; the practical recipe for updating a sequence without forgetting.

Luu et al. (2021) · Time Waits for No One!

Temporal misalignment hurts more than previously reported, and continued pretraining barely helps. Across eight tasks over five-plus years, gains from temporal pretraining are small next to fine-tuning on target-period labels.

Tempers the hope that adapting the language model alone fixes downstream drift.

Agarwal & Nenkova (2021) · Temporal Effects on Pre-trained Models

Separate temporal deterioration from temporal domain adaptation. Deterioration is not always a problem; adaptation always helps. Self-labelling on new data beats even human annotation for NER adaptation.

Terminology the later literature adopted, plus a cheap adaptation trick.

Zhang & Choi (2023) · Discarding Outdated Facts

Predict how long a fact stays true, and discard the volatile ones. Fact-duration prediction improves calibration under misalignment and tells the system when to look something up.

A model that knows what it is likely to be wrong about is a step toward one that knows what it could not have known.

Shin et al. (2025) · TARDIS

Steer representations toward a target period without touching the weights. Steering vectors extracted from unlabelled data of the target time improve downstream performance, work when exact target-period data is missing, and do not need the query's date at inference.

Cheap adaptation for deployment drift; not a leakage remedy.

Elbadry et al. (2026) · The Geometry of Forgetting

Staleness is encoded orthogonally to correctness and uncertainty, so uncertainty-based detectors are blind to it by construction. A linear probe trained on drift labels reaches 0.83–0.95 AUROC while entropy and consistency methods stay near chance; the retrieval circuit behaves identically for stale recall and confabulation.

Why models are confidently out of date, and why fixing hallucination will not fix it.

Jang et al. (2022) · TemporalWiki

Use the diff between consecutive Wikipedia snapshots as the training set, and the Wikidata diff as the test. Training on diffs matches training on the whole snapshot at one twelfth the compute.

A renewable benchmark and the cheapest possible update schedule.

Liška et al. (2022) · StreamingQA

Questions carry the date they were asked, answered from 14 years of dated news. Quarterly evaluation shows parametric models can be updated without forgetting, and that a retrieval system with an outdated language model underperforms one with a retrained one.

Retrieval does not rescue a stale model; the parametric part matters too.

Kasai et al. (2022) · RealTime QA

Announce new questions weekly and evaluate live. GPT-3 updates its answers when retrieval provides them and returns stale answers when it does not.

The live-evaluation idea DeepFund later applied to finance.

Deng et al. (2024) · NewTerm

New vocabulary alone costs models over 20%, and moving the cutoff does not generalise to more distant new terms. Automatically constructed and refreshed annually.

Lexical drift is a separate, measurable channel of temporal failure.

Xian, Cui, Bauer & Abbasi-Asl (2025) · Date-Controlled Tool Use

Give an agent a search tool restricted to a date and watch its answers change. Tool temporality translates directly into agent performance; base-model choice and explicit reasoning instructions soften it.

For agents, the cutoff of the tool matters as much as the cutoff of the model.

Vladika, Dhaini & Matthes (2025) · Facts Fade Fast

512 medical questions where consensus changed, and all eight models tested answer with the old consensus. Built from systematic reviews; a general set of 16,501 pairs accompanies it.

The medical instance of outdated memorisation, with a dataset that isolates exactly the facts that flipped.

Yuan, Ding & Vlachos (2025) · Do LMs Update Their Forecasts?

Given post-cutoff evidence, models move their forecasts in the right direction but by far too little. They treat new information as retrieval context rather than as something that shifts a posterior, and their confidence is poorly calibrated against human forecasters.

The mirror image of leakage: models also fail to use legitimate new information.

Guan et al. (2026) · TempoMed-Bench

Medical knowledge in LLMs declines linearly with time rather than stopping at a cutoff, and models are worse at recalling superseded guidance than current guidance. Built from evolving clinical guidelines.

Parametric medical knowledge is not bounded by the stated cutoff, which complicates any point-in-time claim in medicine.

Zheng, Shao, Ritter & Xu (2026) · Synthetic Worlds

Evaluate knowledge updating on fictional but coherent future worlds, so contamination is impossible. ParallelEvents generates consistent event trajectories; Synapse uses model-generated data to insert them via mid-training and instruction tuning, beating prior methods by 14%.

A way to test temporal updating that never expires, unlike real-event benchmarks.

Dell et al. (2023) · American Stories

Layout-aware extraction turns 20 million newspaper scans into article-level text. Layout detection, legibility classification, custom OCR, and cross-box article association, built on mobile-scale architectures so it runs at that volume.

The dated corpus behind StoriesLM and much historical-LLM work; without article segmentation, newspaper text is unusable for pretraining.

Silcock, Arora, D'Amico-Wong & Dell (2024) · Newswire

Reconstruct the wire archive that never existed by deduplicating 138 million local-newspaper articles. 2.7 million unique public-domain newswire stories, 1878–1977, georeferenced, topic-tagged, and entity-linked to Wikipedia.

A century of nationally circulated, precisely dated news; arguably the best pretraining corpus for a mid-century point-in-time model.

Yang, Liu & Wang (2023) · FinGPT

A data-centric, open pipeline for financial text. Automatic curation plus low-rank adaptation as the open alternative to proprietary financial LLMs.

Not point-in-time; listed because its curation pipeline is a source of dated financial text.

This site · Point-in-Time LLMs Beyond Finance

The same construction is a controlled experiment on “what was knowable when” in medicine, law, history, journalism, policy, and social science. Each field gets concrete experiments that only a leakage-free model can run.

An agenda, not a result.