Key ideas
One note per paper: what is new, and why it matters for point-in-time modelling.
Most papers in this area restate the problem at length. This page skips that. Each entry names the single contribution a reader should remember, states it in plain terms, and adds one line on what follows from it. Notes are written from the papers' abstracts and, where available, full texts; they are summaries, not endorsements. Entries with a purple bar introduced an idea the rest of the field now builds on.
Grouped by thread, in the same order as the bibliography. Within a thread, roughly chronological.
What kind of contribution is it?
Every entry carries a topic and a contribution type. theory proves something. empirical measures something. method proposes a technique. resource releases models, data, or a benchmark. position argues for a practice without new results. Counts are computed from the entries below.
Red and amber blocks beneath an entry record what did not survive checking: a verified error, a methodological concern, or a limited contribution where the conclusions run wider than the evidence. Entries without such a block were not audited line by line.
Model sequences
Röttger & Pierrehumbert (2021) · Temporal Adaptation of BERT
Temporal adaptation helps the language-modelling objective but not the downstream task. Continuing BERT on period text captures event-driven vocabulary, which turns out not to be what the classifier needed; adapting to domain alone did as well.
A negative result worth remembering: a model that “sounds like” a period is not thereby better at tasks in that period.
Loureiro et al. (2022) · TimeLMs
Ship the sequence with a router. Quarterly Twitter checkpoints plus a corresponding mode that matches each tweet to its own quarter's checkpoint. Note what that does and does not give you: a quarter's model is trained on the whole quarter, so the paper's example of a 1 February 2021 tweet routed to the 2021-Q1 model scores it with a model that read late February and March. The paper also documents how perplexity on future tweets degrades and how continual updates repair it.
The routing idea, rather than the models, is the lasting contribution. But routing to the matching period is temporal adaptation; for a forecast you want the checkpoint whose window closes before the forecast origin, usually the preceding quarter.
Qiu & Xu (2022) · HistBERT
Whether contextual embeddings detect semantic change depends on when their pretraining text was written. Variants of BERT-base continued on COHA beat modern BERT on historical word-similarity and shift tasks. Not one model per decade: HistBERT-5 covers the 1910s–1950s collectively and HistBERT-10 the 1910s–2000s collectively, both continuing modern BERT.
Historical adaptation rather than a cut-off claim, and a useful demonstration that the temporal profile of pretraining data is a variable, not a nuisance. The paper itself discusses earlier work on the problem.
Drinkall, Rahimikia, Pierrehumbert & Zohren (2024) · Time Machine GPT
Train from scratch per year, and publish the exact training set. Earlier “temporal” models continued a modern base that had already read the future. TiMaGPT is nonprognosticative by construction, and because every year's corpus is released, leakage is auditable rather than asserted.
Sets the standard that later sequences follow, with two qualifications. Restarting from scratch every year is not required. Continuing a genuinely clean earlier checkpoint preserves consistency too. And open data makes a cut-off claim checkable rather than proven: someone still has to establish that the timestamps are right and the selection unbiased.
Sarkar (2024) · StoriesLM
Sequentially expanding pretraining windows over a century of newspapers. Each of 64 yearly models continues the previous year's checkpoint on that year's American Stories text, so a historian can pick a model as of any year from 1900 to 1963.
The sequence that makes the “simulated historical informant” programme concrete for the early twentieth century. We could not retrieve the full paper, so the claim that it is the only such released sequence is unverified here.
He, Lv, Manela & Wu (2025) · Chronologically Consistent LLMs
A chronologically restricted pipeline can be competitive. A clean model may understand language worse, but the prediction layer fitted on top compensates, so the clean pipeline's Sharpe ratio lands close to a much larger contaminated model's. What the difference between them is not is an identified estimate of look-ahead bias: the comparison also changes architecture, corpus, training budget, language competence, and the interaction with the downstream predictor, and offsetting effects can make two Sharpe ratios agree for unrelated reasons.
Gives you a useful control and a practical alternative to a contaminated model. Reserve quantitative leakage attribution for controlled comparisons, with the remaining assumptions stated. Also the first general-purpose, multi-decade sequence built on restricted pretraining.
He, Lv, Manela & Wu (2025) · Instruction Tuning Chronologically Consistent LMs
Modern instruction data, screened for anachronisms. The data are not historical documents: a from-scratch tutorial set, a GPT-3 self-instruct corpus, and Tulu-3's SFT mixture, each example then classified by GPT-4.1 and kept only when labelled “knowledge available pre-2000”. The screening is validated empirically, but classifier acceptance does not establish that retained examples contain only historically available information — the paper notes the classifier may misidentify pairs.
Makes point-in-time models usable by people who prompt rather than fine-tune. The lower-bound reading of their forecasts holds only as far as the screen did.
Fittschen, Li, Lippincott & Choshen (2025) · Pretraining for Diachronic Linguistic Change Discovery
Period pretraining beat fine-tuning on both cost and fidelity in the settings tested. Small models pretrained on five 10-million-word time slices train faster than parameter-efficient fine-tunes of an 8B model and behave more faithfully to their period. The cost comparison is specific to those architectures and setups. Includes a date-attribution pipeline for undated text.
Argues against fine-tuning as a substitute for period pretraining in the humanities. Restricting the pretraining corpus is the more direct route to period-restricted inference, which the paper demonstrates rather than proves.
Li et al. (2025) · TiC-LM
Continual pretraining with replay matches retraining from scratch at 2.6× less compute. A benchmark built from 114 Common Crawl dumps with time-stratified evaluation; an autoregressive meta-schedule plus fixed-ratio replay of old data holds held-out loss level. Replay matters for general web text, much less for specific domains.
The economics of producing a checkpoint sequence: you do not have to retrain every month if the update schedule is designed properly.
Faro, Fan, Alphaidze & Jaggi (2025) · TiMoE
A collection of separate experts, masked by date. Each expert is pretrained from scratch on a disjoint two-year slice of 2013–2024; at inference, experts whose window ends after the query date are masked and the rest merged in log-probability space. Fixed aggregation of the surviving clean experts has a clear chronological justification. Learned routing does not: the router is trained on the whole multi-period corpus, and the co-adaptation variant updates the router together with the selected expert, so an earlier query can get mixture weights shaped by later data even when later experts are masked. Also releases TSQA, whose answer options are labelled past, future, or irrelevant.
Serves every cut-off from one deployment at a coarser two-year cadence. Learned routing needs its own cut-off controls, co-adaptation included, before the guarantee covers the whole system.
Yan, Tang, Gao & Jiang (2026) · DatedGPT
Estimate the “lookahead premium”, carefully defined. Twelve 1.3B models with annual cut-offs 2013–2024, plus DatedInstruct, an instruction set grounded in each year's documents. The 26.4 basis points (t = 10.65) is not an extra portfolio return. it is the next-day return associated with one standard deviation of the difference between the latest-vintage and historically admissible sentiment signals, estimated with the admissible signal plus firm and date fixed effects in the regression. The portfolio comparison is separate, a Sharpe of 3.20 lookahead-free against 3.65 lookahead-biased.
The closest thing yet to a price on leakage in return prediction, using matched model vintages. The leakage reading is the authors' own, and it assumes matched vintages differ only in access to the future, when they also differ in language competence and corpus composition.
Kelly, Malamud, Schwab & Xu (2026) · Scaling Point-in-Time Language Models
Scale substantially narrows the measured benchmark gap. Decoder-only models up to 4B parameters on 1T chronologically filtered FineWeb tokens, at monthly cadence over 2013–2024, approach Gemma-3-4B and LLaMA-7B on common-sense and language-understanding benchmarks, for roughly 28,000 GPU-hours of pretraining. Full pipeline released. The instruction-tuned variants are a separate matter: their filter asks whether a fact “could become outdated”, explicitly keeping coding tutorials and stable scientific facts, which is a different question from whether the fact was available at the cut-off.
Removes the main excuse for using contaminated models at pretraining: a restricted model of comparable ability is attainable at academic scale, and the GPU-hour figure says how attainable. Treat the instruct variants' temporal validity as unestablished.
Pilchen, Fabre, Signe Talla & Pérez (2026) · Data Temporality in Pre-training
Order matters: pretraining on Common Crawl snapshots in chronological order gives fresher, more temporally precise knowledge than shuffled pretraining, with no loss of general ability. Shuffled training peaks on older facts. Comes with a 7,000-question benchmark that checks whether a fact is tied to the right period.
A cheap lever for temporal grounding that costs nothing in tokens.
Luo, Shinnick, Griesshaber & Wang (2026) · TypewriterLM
A 7B model trained only on pre-1913 English, with lexically constrained post-training. The novelties are the leakage-mitigation steps in building the 54B-token TypewriterCorpus, “lexically grounded” instruction tuning, which keeps a response only if every lexical token is derivable from the source passage plus a small function-word allowlist, human review of the small instruction set, and a benchmark that tests for leakage as well as competence. Instruction generation and judging still use modern LLMs.
Pushes the chat-model recipe furthest inside a period at useful scale. Source vocabulary, source-supported meaning and historical availability are three different things. The lexical constraint secures the first. Old words can still be rearranged, or passages chosen, to convey something learned later.
Look-ahead bias and leakage
Lopez-Lira & Tang (2023) · Can ChatGPT Forecast Stock Price Movements?
GPT-4 headline scores predict next-day returns, on post-cutoff headlines. The design was careful about the cut-off; the result is what made everyone else ask whether their in-sample results were.
The motivating paper for the whole finance thread, not because it leaked but because it showed how much was at stake if others did.
Glasserman & Lin (2023) · Assessing Look-Ahead Bias in GPT Sentiment
Two distinct biases, and one cheap intervention that improves on both counts at once. Look-ahead is knowing what the stock did; “distraction” is general knowledge of the company contaminating the sentiment read. Removing company identifiers from headlines improved in-sample performance.
Introduced the cheapest mitigation in the literature. The reading that distraction dominates look-ahead is an interpretation of one net improvement, which does not on its own separate the two channels or size them individually.
Sarkar & Vafa (2024) · Lookahead Bias in Pretrained Language Models
Test for leakage on events chosen to be hard to predict from the information available at the time. If a model predicts election winners from candidate biographies, or risk factors before they materialised, better than the information set allows, it has seen the future. The argument is that prompting does not fix this and restricted pretraining data does.
The identification argument behind every later detection statistic, and an early statement that point-in-time pretraining is the remedy. We could not retrieve the full paper. Candidate biographies alone do not make an election unpredictable, so the close-election design is the part to check before relying on the claim.
Rahimikia & Drinkall (2024) · Re(Visiting) Large Language Models in Finance
Re-runs finance LLM results using the Time Machine GPT sequence. Neither the abstract nor the full text could be retrieved, so this entry records what the title and surrounding literature indicate and no more. No findings are summarised. not retrieved
The natural follow-up to TiMaGPT by the same group. Treat any account of its results, here or elsewhere, as unverified until the paper is read.
Ludwig, Mullainathan & Rambachan (2024) · An Applied Econometric Framework
“No training leakage” is an identification condition, like any other. For prediction uses of LLM output it must be enforced by model choice and research design; for estimation uses a validation sample is required, or seemingly innocuous choices of model and prompt swing the estimates.
Puts leakage where economists will take it seriously: in the list of assumptions a referee checks.
Lopez-Lira, Tang & Zhu (2025) · The Memorization Problem
In-sample forecasting skill is non-identified. Once the model has seen realised values, any output is consistent with both skill and memorisation. LLMs recall exact economic values before their cutoff; instructions to respect the date fail, and masking fails because the model reconstructs entities and dates from minimal context. Memorisation extends to embeddings.
Closes the door on in-sample evaluation of forecasting LLMs, and on the two cheap fixes people reach for first.
Appendix C.3 claims finite real logits realise every distribution on a finite label set. Every softmax coordinate is strictly positive for finite logits, so boundary distributions such as (1, 0) are unreachable. Restricting the claim to the interior repairs it. The non-identification result does not use this step and survives, with its unrestricted-model assumptions attached. Full review.
Crane, Karra & Soto (2025) · Total Recall? (Federal Reserve)
LLMs smooth across data vintages. They recall recent macro statistics precisely but blend first prints with later revisions, and blend past with future reference periods; on any given day the model believes it holds data that had not yet been released.
Vintage awareness is a separate failure from knowledge cutoff, and one that real-time datasets are needed to test. The direct vintage and release-date findings are the strong part. The within-vintage leakage test in §4.2 leans on SPF forecasts capturing predictable future variation, and the authors call that evidence preliminary.
Paleka, Goel, Geiping & Tramèr (2025) · Pitfalls in Evaluating LM Forecasters
A catalogue of the ways time leaks into forecaster evaluation. Question creation, retrieval, and resolution each admit leakage; combined with the difficulty of extrapolating from evaluation to deployment, claims of human-level forecasting are premature.
The checklist a forecasting paper should be read against.
Li et al. (2025) · DeepFund
A prospective benchmark, where time travel is impossible by construction. LLM fund managers are fed real-time market data published after each model's cutoff. The sample is short and execution frictions are omitted.
Trades statistical power for validity, and complements point-in-time models, which trade the other way. Live testing is one way to rule out leakage rather than the only valid design: a matched clean model on historical data is the other.
Merchant & Levy (2025) · A Fast and Effective Solution to Look-ahead Bias
Logit arithmetic with two small models. Steer a large base model at inference using a small model fine-tuned on what to forget and another on what to keep. Suppresses both verbatim and semantic recall of the period under the elicitation methods tested, without retraining the frontier model.
The practical answer for people who cannot retrain: unlearning by decoding rather than by weights. The base weights are untouched and only the logits change, so this is measured suppression rather than a guarantee that the knowledge is beyond reach.
Gao, Jiang & Yan (2025) · Detecting Lookahead Bias in LLM Forecasts
Lookahead Propensity: ask the model, from the date alone, what happened. The recall probability is high throughout the training window and collapses to zero after the cutoff. Its interaction with forecast accuracy in a regression is a test for contamination, and it fires on both headline–return and earnings-call–capex tasks.
A cheap diagnostic that needs no clean model, only a recall query per firm-date. The detection result in Appendix B assumes exogeneity and conditional moment restrictions, with monotonicity for the positive-interaction claim. Absent those, recall propensity can track question difficulty, coverage or signal calibration instead of contamination.
Eliseev & Seleznev (2026) · Fake Date Tests
Lie to the model about today's date and see if its in-sample forecast moves. Sensitivity to the fake date separates lookahead bias from context bias. No modern LLM passed. Proposition 1 maintains a date-invariance assumption, which the paper states and discusses.
A one-line test any macro forecaster can run before trusting a backtest, read as a joint test: a failure can mean the date-invariance restriction broke rather than that the model leaked.
Benhenda (2026) · Look-Ahead-Bench
Measure bias as alpha decay, not as recall. Instead of asking whether the model knows post-cutoff facts, run it through realistic financial workflows across temporally distinct regimes and watch performance decay. Standard LLMs decay; point-in-time models do not, and improve with scale.
A standardised behavioural benchmark for the property, and a useful comparison across models. Decay on its own is not a causal leakage estimate: regime change, task difficulty and model staleness move it too.
Kong et al. (2026) · Evaluating LLMs in Finance Requires Explicit Bias Consideration
Five biases, and almost nobody checks for them. Look-ahead, survivorship, narrative, objective, and cost bias compound into an illusion of validity; of 164 papers from 2023–2025, no single bias is discussed in more than 28%. Proposes a structural-validity checklist.
A position paper, and the survey count is the contribution.
The five categories are a discretionary list rather than a derived taxonomy. Nothing fixes the boundaries between them or rules out a sixth, and the Structural Validity Checklist inherits that discretion. Two careful reviewers can disagree about whether a paper complies without either being wrong.
The survey records whether a paper mentions a bias. It does not test whether any evaluation actually failed, or by how much. From 26.8% acknowledgment of look-ahead bias the paper concludes that “many published numbers do not support the claims that readers and practitioners want to make from them”. That step needs a rate of real failures and their effect on reported results, which the survey does not measure. A paper that does not discuss a bias may still have avoided it. The rest of the audit is on the reviews page.
Zhang, Chen & Stadie (2026) · All Leaks Count, Some Count More
Weight each leaked claim by its effect on the decision. Decompose a rationale into atomic claims, attribute Shapley values to each, and report the fraction of decision-driving reasoning that is post-cutoff (Shapley-DCLR). TimeSPEC then interleaves date-filtered retrieval with claim-level supervision to force pre-cutoff grounding.
Moves leakage measurement from “did it mention a future fact” to “did that fact change the answer”.
The Shapley values are computed by re-prompting on subsets of the visible rationale, so they measure how much a stated rationale drives a later answer, not what caused the original one. A model recalling the outcome from its weights can be insensitive to every claim, giving a score of zero alongside total leakage. Weighting by decision impact is still an improvement on counting leaked claims equally. Full review.
Gao, Jiang & Yan (2026) · Debiasing LLMs by Fine-tuning
Extrapolation bias in LLM forecasts yields to LoRA fine-tuning on rational benchmark forecasts, where prompting did not. Holds out of sample in both controlled experiments and cross-sectional return prediction.
About a different bias than look-ahead, but the same lesson: parameter-level intervention succeeds where instructions fail.
Levy (2026) · Caution Ahead (Journal of Accounting Research)
Much of LLMs' apparent superhuman performance in accounting and finance is modelling artifact. Two culprits: very poor numerical reasoning, and look-ahead bias in commercial models that may account for a large share of their predictive ability. The numerical failures are not intrinsic: giving the model a code tool raises the statement-tallying and DuPont tasks to 100% accuracy, which matters for how the result should be read.
The accounting literature's formal notice that its headline AI results need re-examination. One technical slip in §2.3.1, which does not affect the accounting experiments: it treats decoding temperature and Concrete-relaxation temperature as the same knob. For softmax((z + Gumbel)/τ) the hard argmax does not depend on τ at all, whereas ordinary categorical sampling from softmax(z/T) does move with T.
Zhang & Stadie (2026) · TEMPO
Temporal compliance is instance-specific, so unlearning cannot work. The same fact is legitimate evidence for one cutoff and a violation for another. Instead of erasing knowledge, train temporal discipline with reinforcement learning: a leakage mode that drives post-cutoff claims to zero is a hard prerequisite before a performance mode optimises accuracy.
Reframes the goal from forgetting to selecting, which is the version that scales to many cut-offs.
The convergence argument rests on Proposition 1, that the leakage-mode gradient descends the leakage cost. Its proof (Appendix A.1) derives equation (8) correctly: S = ∑y π(y)2(f(y) − f̄)(c(y) − c̄), with f = exp(−0.5c) and both means taken under π. It then reads that sum as CovQ(f, c) under the tilted law Q(y) ∝ π(y)2 and signs it with Chebyshev. A covariance under Q centres under Q. Centring under π instead leaves a term the proof drops.
That term can outweigh the covariance. At leakage counts c = (0, 5, 39) and
π = (0.08, 0.82, 0.10), so Varπ(c) = 108.6 > 0 as the proposition
requires: CovQ = −0.0864, the dropped term is +0.1018, and the inner product
computed straight from the score functions is +0.0422. Inequality (5) needs it strictly negative.
The implied γ is −0.0026, so at this policy the leakage-mode gradient ascends the
leakage cost. Proposition 2, Theorem 1 and Corollary 1 all invoke Proposition 1.
Certificate: verify_tempo_prop1.py.
Full review, with a second independent witness
and the finite-sample gate.
The empirical results are unaffected. What fails is the claim that the method provably converges.
Definition 1 counts a claim as leaked when its declared source date, adjusted by a date-plausibility verifier, falls after the cutoff. That checks the dates of the evidence a model cites. A model can cite only pre-cutoff sources and still reach its prediction from what it knows happened later. A zero leakage count is therefore necessary for temporal compliance, not sufficient for it.
Zhang, Li, Peng & Chen (2026) · When Alpha Disappears
Leakage lives in the evaluation protocol as well as the model. Toggle one convention at a time around a clean t+1-open reference and measure the inflation. Centred temporal features and same-day-open execution with post-open information inflate results a lot; global normalisation and same-day-close execution barely do.
A ranked list of which backtest sins actually matter.
Li, Wang & Ma (2026) · FinCAD
Find the prompt that wakes the model's memory of the outcome, then subtract that memory at decode time. An adversarial pipeline learns a model-specific memory-activating prompt; context-aware decoding attenuates its contribution, scaled per entity and date by a confidence signal. The largest model-level mean in-sample return correction is −67.1%, which is a change in returns rather than a measured fraction of memory removed. The v2 abstract limits near-preservation of 2025 out-of-sample returns to three larger models, and the authors acknowledge that removal is incomplete.
Names “parametric look-ahead bias” and offers the most surgical inference-time fix so far.
Merchant & Levy (2026) · Feature Steering
Time-awareness steered well; the look-ahead features they found did not. Sparse autoencoders surface features for time-aware and for look-ahead-biased reasoning. Amplifying the time-awareness features cuts look-ahead bias across a different domain, while steering the look-ahead features does nothing. That is a result about the features this autoencoder surfaced, not a demonstration that no steerable look-ahead feature exists.
Suggests the bias is diffuse and the fix is to strengthen the model's sense of “when” rather than to delete what it knows.
Fonseca (2026) · Look-Ahead-Freedom as Temporal Non-Interference
Look-ahead-freedom is an information-flow property and can be type-checked. Separating a datum's availability time from its reference time, freedom from leakage is undecidable when availability depends on data values, but for the value-independent fragment (windowing, resampling, joins, point-in-time reads, agentic retrieval) a type-and-effect system decides it in linear time.
Moves part of the problem from empirical detection, which certifies nothing by its silence, to static verification.
The reduction in §7 does not gate the future value. Under the paper's own evaluation rules in
§6.2, stamp only relabels, val discards the label, and
decide emits without checking availability, so a future input still reaches the output
when the machine never halts. A separate construction shows that constant timestamps alone do not
make semantic non-interference decidable. Decidable type-checking remains a weaker, useful
guarantee. Full review.
Jia (2026) · HindsightBench
Audit parametric hindsight in any model at probe cost, with no backtest. A four-arm date-manipulation matrix (revealed, date-only, masked, transplanted) plus memory probes yields six metrics including a behaviourally effective cutoff. Across 15 models the date-trigger reflex tracks training recency, not scale.
A black-box protocol for buyers of models, who have neither log-probs nor corpus access.
Zhang & Stadie (2026) · Temporal Leakage in LLM Backtesting
The standard pre/post-cutoff comparison is uninformative, provably. Models legitimately know more about times near their cutoff, so recency mimics leakage, and no passive backtest can separate skill, recency, and leakage. Measurement needs outside information: a known cutoff identifies leakage at the boundary, a matched clean control identifies it globally and yields an adjusted score. Validated by planting leakage in twin models and recovering the dose.
Explains why so many earlier detection results are ambiguous, and why point-in-time models are needed as controls even if you never deploy them.
§2 defines leakage against the forecast as-of date, but Assumption 1 treats outcomes resolving after the training cutoff as clean. A forecast dated January, a model trained through June, an outcome resolving in December: March news can reveal a great deal about December while being unavailable in January. The prose after Corollary 10 also drops the corollary's zero-mean-shift condition. Full review.
Cai et al. (2026) · OpenPM
Point-in-time discipline for agents, with a certificate. Every record visible to a portfolio agent must be available at decision time; natural-language risk mandates become typed constraints enforced on the executed book; each run emits a contamination certificate and a cost-sensitivity curve.
Extends the cut-off idea from training data to the whole agent loop, including retrieval and execution. The certificate covers the availability and execution constraints the pipeline records and enforces. Checking timestamps on retrieved records says nothing about what an opaque base model absorbed during training.
Prompted cut-offs and the model's sense of time
Zhao, Brumbaugh, Wang, Hajishirzi & Smith (2024) · Set the Clock
Models have an internal “now” that lags their cutoff, and it can be moved. LLaMa2 with a 2022 cutoff answers as if it were 2019. Aligning it to 2022 improves answers by up to 62%; aligning to 2010 is also possible, with 2.8× the unaligned performance on that year's answers.
The positive case for post-hoc temporal alignment; later work shows its limits on causally downstream knowledge.
Cheng et al. (2024) · Dated Data
The effective cutoff differs from the reported one, resource by resource. Probing across versions of the same data shows where a model's knowledge actually stops. Two causes: Common Crawl dumps contain large amounts of old data, and deduplication misses semantic and lexical near-duplicates.
Effective recall boundaries vary by resource, so a point-in-time claim has to be made resource by resource. The date a corpus was actually frozen can still be a single number; what varies is how much of each resource survives in the model.
Park et al. (2025) · Temporal Heads
Specific attention heads carry time-specific knowledge. Ablating them removes recall of facts tied to a year while leaving time-invariant knowledge and general QA intact; they fire on textual aliases of dates as well as numbers, and their values can be edited.
The first mechanistic handle on where “as of when” lives inside a transformer.
Gao et al. (2025) · Can Prompts Rewind Time for LLMs?
Prompted cutoffs forget only what is directly asked. Three tests: direct facts, semantic shifts, causally related knowledge. Prompting works on the first, fails on the third: a model told to forget an event still uses its consequences.
The clean experimental refutation of “just tell it the date”.
Wongchamcharoen & Glasserman (2025) · Do LLMs Understand Chronology?
Frontier models keep local order but cannot hold a single global timeline. Exact-match ordering collapses as sequences lengthen even while rank correlation stays high; conditional sorting fails at the filtering step; anachronism detection is easiest but degrades with overlapping timelines. Extended reasoning budgets help.
Prompt-based defences against look-ahead presuppose a capability the models do not reliably have.
Pęzik et al. (2025) · LLMLagBench
Infer an undeclared training cutoff from what the model knows about recent events. A freshness benchmark that estimates the earliest probable temporal boundary, validated against disclosed cutoffs.
Necessary bookkeeping: you cannot reason about leakage in a model whose cutoff you do not know.
Li et al. (2026) · Simulated Ignorance Fails
Simulated ignorance leaves a 52% gap to true ignorance. Across 477 competition-grade forecasting questions and nine models, cutoff instructions do not suppress prior knowledge, chain-of-thought does not either even when the trace contains no post-cutoff reference, and reasoning-optimised models are worse at pretending. Retrospective forecasting on pre-cutoff events is therefore invalid.
The largest-scale demonstration that prompts cannot rewind a model.
The 52% is the share of a performance gap left after the prompting intervention, not a measured quantity of leakage. The §5.1 comparison between DeepSeek and GPT models changes architecture, training and task-specific capability as well as cutoff status. The evidence that the tested prompts fail to erase the advantage is unaffected. Full review.
Ding et al. (2026) · Temporal Critique Fine-Tuning
Ex-ante correctness is a relation between an answer and a cutoff, not a property of the answer, so supervised fine-tuning cannot teach it. Leakage is sensitive to prompt form (explicit cutoffs beat implicit framings; prefix constraints beat suffix), but prompting steers the frame without giving the model any way to check admissibility. TCFT trains a critic that judges temporal admissibility instead.
Same conclusion as TEMPO by a different route: the model needs a verifier, not a memory wipe.
§4.1 argues that supervised fine-tuning cannot teach temporal admissibility because admissibility depends on the cutoff. Supervised objectives condition on their inputs, and the cutoff can be one of them. The paper's Appendix C.1 writes that objective and §4.4 implements TCFT with SFT. Its QA leakage score also counts any correct post-cutoff answer as leakage with no utility check, so blanket refusal passes it. Full review.
Asai et al. (2026) · Recall-Based Prompting
Ask the model to restate its cutoff and to recall what it validly knows before answering. Self-Recall and Question-Recall beat direct answering and step-by-step reasoning, with the largest gains on counterfactual questions. A multi-cutoff benchmark shows performance varies with distance to the cutoff.
The strongest prompting result so far, and still not a substitute for a clean model. Nothing shows these are the best prompts available.
van Adrichem, Bhaskar, Yang & Potts (2026) · Do LMs Consistently Encode the Current Year?
Two probes for “what year is it” use different mechanisms, and nothing updates both. An associative probe (infer the year from verb tense) tracks the pretraining cutoff within ten months for base models; a declarative probe asks outright. Prompting moves the declarative year; neither prompting, SFT, nor weight editing moves both together.
Explains why temporal alignment methods look successful on one test and fail on another.
Temporal generalisation
Lazaridou et al. (2021) · Mind the Gap
Language models get worse on text from after their training period, and in the models tested scale did not fix it; continual updating did. Proposes time-stratified train/test splits as the realistic evaluation.
An early and influential statement of the problem. Much of the NLP thread follows its evaluation design.
Dhingra et al. (2021) · Time-Aware LMs as Temporal Knowledge Bases
Prefix each training document with its date, and one model learns to condition on time. Improves memorisation of period facts and calibration about the future, and allows refreshing with new data instead of retraining.
The single-model alternative to a checkpoint sequence; it does not prevent leakage, but it gives the model a clock.
Jang et al. (2021) · Continual Knowledge Learning
Updating world knowledge is its own continual-learning problem. Three things must happen at once: retain invariant knowledge, update outdated knowledge, acquire new knowledge. Standard continual-learning methods struggle, and parameter expansion did best among the methods tested, which is not the same as showing it is required.
Defines the metric that TemporalWiki and later benchmarks measure.
Jin et al. (2021) · Lifelong Pretraining
Distillation-based continual pretraining best retains earlier-domain performance while adapting to a chronological stream. Also improves temporal generalisation where train and test periods differ.
Method paper; the practical recipe for updating a sequence without forgetting.
Luu et al. (2021) · Time Waits for No One!
Temporal misalignment hurts more than previously reported, and continued pretraining barely helps. Across eight tasks over five-plus years, gains from temporal pretraining are small next to fine-tuning on target-period labels.
Tempers the hope that adapting the language model alone fixes downstream drift.
Agarwal & Nenkova (2021) · Temporal Effects on Pre-trained Models
Separate temporal deterioration from temporal domain adaptation. Deterioration is not always a problem, and adaptation helped in every setting they evaluated. Self-labelling on new data beats even human annotation for NER adaptation.
Terminology the later literature adopted, plus a cheap adaptation trick. The evaluated settings do not establish that adaptation helps for every task or distribution.
Zhang & Choi (2023) · Discarding Outdated Facts
Predict how long a fact stays true, and discard the volatile ones. Fact-duration prediction improves calibration under misalignment and tells the system when to look something up.
A model that knows what it is likely to be wrong about is a step toward one that knows what it could not have known.
Shin et al. (2025) · TARDIS
Steer representations toward a target period without touching the weights. Steering vectors extracted from unlabelled data of the target time improve downstream performance, work when exact target-period data is missing, and do not need the query's date at inference.
Cheap adaptation for deployment drift; not a leakage remedy.
Elbadry et al. (2026) · The Geometry of Forgetting
Staleness is encoded orthogonally to correctness and uncertainty, so uncertainty-based detectors are blind to it by construction. A linear probe trained on drift labels reaches 0.83–0.95 AUROC while entropy and consistency methods stay near chance; the retrieval circuit behaves identically for stale recall and confabulation.
Why models are confidently out of date, and why fixing hallucination will not fix it.
Near-orthogonal probe directions and the failure of the detectors actually tested do not establish that every uncertainty-based detector must fail. With U uniform on {−2, −1, 1, 2} and h = (U, U2), the two readouts are orthogonal and their scores have zero covariance, yet the second score and the drift label are exact functions of the first. The probe results stand as empirical evidence. Full review.
Temporal knowledge benchmarks
Jang et al. (2022) · TemporalWiki
Use the diff between consecutive Wikipedia snapshots as the training set, and the Wikidata diff as the test. Training on diffs matches training on the whole snapshot at one twelfth the compute.
A renewable benchmark, and a twelve-fold reported saving on update cost. Whether that is the cheapest schedule possible is not established.
Liška et al. (2022) · StreamingQA
A question-answering benchmark where every question is stamped with the date it was asked. StreamingQA builds 14 years of English news into a dated corpus, then asks questions that can only be answered from articles published by that date. Models are re-evaluated each quarter as the corpus grows. Two findings: a model whose weights are updated on new text keeps its old answers rather than forgetting them, and a retrieval system paired with an outdated language model does worse than the same retrieval paired with a retrained one.
A stale parametric model limits the retrieval system built on top of it, so retrieval alone does not make a model current. It does still improve one. The lesson for point-in-time work is that dated pretraining and a dated index are complements, not substitutes.
Kasai et al. (2022) · RealTime QA
Announce new questions weekly and evaluate live. GPT-3 updates its answers when retrieval provides them and returns stale answers when it does not.
The live-evaluation idea DeepFund later applied to finance.
Park et al. (2024) · ChroKnowledge
Separate knowledge that evolves from knowledge that is constant, across five domains, and test by year. Models show partial recall or a cutoff at temporal boundaries; the ability to elicit temporal knowledge depends on the training data's format; ChroKnowPrompt traverses adjacent years to recover it.
The benchmark that made “as of when” a first-class evaluation axis outside finance.
Deng et al. (2024) · NewTerm
New vocabulary alone costs models over 20%, and moving the cutoff does not generalise to more distant new terms. Automatically constructed and refreshed annually.
Lexical drift is a separate, measurable channel of temporal failure.
Xian, Cui, Bauer & Abbasi-Asl (2025) · Date-Controlled Tool Use
Give an agent a search tool restricted to a date and watch its answers change. Tool temporality translates directly into agent performance; base-model choice and explicit reasoning instructions soften it.
For agents, the cutoff of the tool matters as much as the cutoff of the model.
Vladika, Dhaini & Matthes (2025) · Facts Fade Fast
512 medical questions where consensus changed, and every one of the eight models tested gives the superseded answer on some of them. Built from systematic reviews; a general set of 16,501 pairs accompanies it.
The medical instance of outdated memorisation, with a dataset that isolates exactly the facts that flipped.
Yuan, Ding & Vlachos (2025) · Do LMs Update Their Forecasts?
Given post-cutoff evidence, models move their forecasts in the right direction but by far too little. They treat new information as retrieval context rather than as something that shifts a posterior, and their confidence is poorly calibrated against human forecasters.
The mirror image of leakage: models also fail to use legitimate new information.
Guan et al. (2026) · TempoMed-Bench
Medical knowledge in LLMs fades gradually rather than stopping at a cutoff, declining roughly linearly over the period measured, and models recall superseded guidance worse than current guidance. Built from evolving clinical guidelines.
Parametric medical knowledge is not bounded by the stated cutoff, which complicates any point-in-time claim in medicine. The trend is measured on these guidelines over this window, not a general law of how medical knowledge decays.
Zheng, Shao, Ritter & Xu (2026) · Synthetic Worlds
Evaluate knowledge updating on fictional but coherent future worlds, so contamination is impossible. ParallelEvents generates consistent event trajectories; Synapse uses model-generated data to insert them via mid-training and instruction tuning, beating prior methods by 14%.
A way to test temporal updating that does not expire the way real-event benchmarks do. Fresh instances cut the memorisation risk; published ones can still enter a later training run, and transfer from synthetic events to real ones needs validating.
Corpora
Dell et al. (2023) · American Stories
Layout-aware extraction turns 20 million newspaper scans into article-level text. Layout detection, legibility classification, custom OCR, and cross-box article association, built on mobile-scale architectures so it runs at that volume.
The dated corpus behind StoriesLM and much historical-LLM work. Article segmentation raises corpus quality substantially. Unsegmented scan text is worse for pretraining, not categorically unusable.
Silcock, Arora, D'Amico-Wong & Dell (2024) · Newswire
Reconstruct the wire archive that never existed by deduplicating 138 million local-newspaper articles. 2.7 million unique public-domain newswire stories, 1878–1977, georeferenced, topic-tagged, and entity-linked to Wikipedia.
A century of nationally circulated, precisely dated news; arguably the best pretraining corpus for a mid-century point-in-time model.
Yang, Liu & Wang (2023) · FinGPT
A data-centric, open pipeline for financial text. Automatic curation plus low-rank adaptation as the open alternative to proprietary financial LLMs.
Not point-in-time; listed because its curation pipeline is a source of dated financial text.
Beyond finance
Varnum, Baumard, Atari & Gray (2024) · Historical LLMs for Behavioral Science
Train on the texts of a past population and you get a simulated informant from it. Behavioural science cannot survey the dead; a model trained only on their writing is the closest available proxy, and early attempts reproduce period worldviews rather than modern ones.
The programme that gives StoriesLM and TypewriterLM a purpose outside finance. We could not retrieve the full paper. The obvious hazard is that surviving period writing is not a representative sample of the period's population, which bears directly on any behavioural inference drawn this way.
This site · Point-in-Time LLMs Beyond Finance
The same construction is a controlled experiment on “what was knowable when” in medicine, law, history, journalism, policy, and social science. Each field gets concrete experiments that only a leakage-free model can run.
An agenda, not a result.
What the field has actually established
Reading across the notes above, a small number of claims now have multiple independent supports:
- Leakage is real and large enough to matter. Direct tests (Sarkar & Vafa), recall probes (Lopez-Lira, Tang & Zhu; Gao, Jiang & Yan), and matched clean models (He et al.; Yan et al.) all find it, and DatedGPT prices it at tens of basis points per standard deviation in return prediction.
- Prompting does not remove it. Gao et al., Li et al., Lopez-Lira et al., and Wongchamcharoen & Glasserman show cutoff instructions fail on causally downstream knowledge and that models cannot hold a consistent timeline anyway. Recall-based prompting narrows the gap; nothing closes it.
- Scoring a model before and after its cutoff does not measure leakage. The obvious test is to check whether a model does better on pre-cutoff dates than post-cutoff ones. It does, but so would a clean model: any model knows more about periods near its cutoff than about later ones, so recency alone produces the gap that leakage is supposed to explain. Zhang & Stadie show no passive backtest separates the two. Something from outside the backtest is needed, either a cutoff known independently or a matched model trained without the disputed data.
- Erasing a fact is the wrong unit of intervention. Whether a fact is admissible depends on the cutoff being backtested, so the same fact is legitimate evidence for a 2020 query and a violation for a 2015 one. A method that removes it from the weights has to choose once, for every query. This argues for deciding at inference instead, which is what decoding-time filtering (Merchant & Levy; FinCAD) and feature steering do, though neither has been shown to hold up across many cut-offs. Ding et al. and TEMPO make the same case for a verifier over a memory wipe; TEMPO's convergence proof does not survive checking.
- Restricted models no longer have to be weak, at pretraining. Scale (Kelly et al.), routing (TiMoE), replay schedules (TiC-LM), and ordered data (Pilchen et al.) each cut the cost of restricting the pretraining corpus. Post-training is the weak point: every instruction-tuned variant in this literature relies on screening or grounding, and none of those procedures has been shown to be sound.
- “Cutoff” is a distribution. Effective cutoffs differ by resource (Cheng et al.), medical knowledge decays linearly rather than stopping (Guan et al.), and the model's internal year lags its data (Zhao et al.; van Adrichem et al.).
Disagree with a summary, or have a paper that belongs here? Open an issue on microprediction/pitllm. Every entry links to its source so the note can be checked against it.