Plans
Three costed ways to extend the open point-in-time model and corpus sequences on a small budget.
The existing sequences were built by a handful of academic groups. The gaps are known: no yearly coverage of 1964–1998, no monthly cadence before 2013, no non-English or domain-specific sequence, and no shared evaluation of temporal consistency outside finance. The documents below are working proposals for closing some of these gaps with roughly half a million dollars or less. They differ mainly in who does the work and who pays for the data.
| Plan | Budget | Data | Who does the work | Main deliverable |
|---|---|---|---|---|
| Focused programme | $500k, 12 months | Mixed: open sources plus licensed financial news | Small salaried team, 2–3 people | Extended TimeLMs and ChronoBERT sequences, a financial temporal benchmark, a backtesting tool |
| Open-source corpus programme | ~$550k, 12 months | 100% public domain or open licence | Small team, 4–6 part-time roles | Three flagship dated corpora and open temporal benchmarks |
| Bittensor subnet | $50k setup; work paid in emissions | 100% public domain or open licence | Incentivised miners, validated by the subnet | The same three corpora, produced by a decentralised network |
A focused $500k programme
Full document. The plan opens with a reality check: $500k buys six to twelve months of a two- or three-person team, not two years of infrastructure. It therefore rules out training large models from scratch, building comprehensive data infrastructure, or producing production applications, and commits to extending what exists.
- Phase 1, $200k — foundation. Extend TimeLMs by six to eight quarterly checkpoints, add two or three yearly ChronoBERT models with a financial-news emphasis, and build a unified loader and evaluation protocol. Extend ChroKnowBench to finance.
- Phase 2, $200k — models. Compute for the extensions, temporal-attention and fine-tuning experiments, and a financial point-in-time model with entity recognition and sentiment capabilities.
- Phase 3, $100k — proof of concept. A trading-strategy backtesting tool that routes each date to the right checkpoint, documentation, and an open-source release.
The financial-domain focus is a deliberate narrowing: it is where look-ahead bias has a measurable cost and where the evaluation literature already exists.
An open-source corpus programme
Full document. Rather than extending models, this plan builds the dated text they would need, using only public domain and openly licensed sources so that the outputs can be redistributed and maintained by the community.
- Three flagship corpora, $300k. A Twitter extension in the TimeLMs format ($120k), an open financial-news corpus from SEC EDGAR, FRED, and Common Crawl ($100k), and a historical-newspaper corpus from Chronicling America and the Internet Archive ($80k).
- Evaluation, $100k. Open temporal benchmarks and a quality-control framework, integrating ChroKnowledge for consistency validation.
- Team and infrastructure, $150k. A project lead, data engineer, research assistant, and part-time SRE, writer, and QA, plus cloud processing and a legal review of licensing.
Targets are stated as OKRs: 500GB–1TB of dated text across three domains, three to five years of added coverage per domain, and 98% temporal-consistency validation. The document's own team costing comes to more than the headline phase totals once overhead and contingency are added, which is worth knowing before quoting a single number.
A Bittensor subnet
Full document. The same three corpora and benchmarks as the open-source programme, but produced by miners on a Bittensor subnet and scored by validators, so the direct cost is the subnet setup rather than salaries. Miner tasks are temporal filtering, deduplication, quality scoring, and benchmark construction over the open sources; validators check temporal consistency and reward accordingly.
| Metric | Centralised | Open-source subnet |
|---|---|---|
| Set-up cost | ~$742k | $50k |
| Team | 6.5 FTE, one location | 50–100 miners, global |
| Incentive | Salaries | Token emissions |
| Data licensing | Mixed | 100% open |
| After month 12 | Project ends | Network continues |
The document's emissions figures assume a fixed TAO price and a fixed share of network emissions; both are volatile, and the “$0 direct cost” of phases 2 and 3 is cost borne by the network rather than cost that disappears. Treat the comparison as directional.
What would change the picture
- Scale helps. Kelly et al. (2026) show the gap between point-in-time and unconstrained models narrows at 4B parameters and 1T tokens. Any plan that trains models should budget for that scale or explicitly accept the gap.
- Data first. Time Machine GPT and StoriesLM both released their exact training sets. A dated, deduplicated, openly licensed corpus per month is more reusable than any single model trained on it.
- Evaluation is cheap and missing. Look-Ahead-Bench covers finance; TempoMed-Bench covers medicine. Nothing comparable exists for law, policy, or history, and the essay lists the experiments that would need one.
These are working documents. Suggestions and alternative costings are welcome on GitHub.