Demos

Notebooks that load a point-in-time checkpoint and ask it about its own present.

ChronoGPT: ask a 2023 model who is president

The ChronoGPT notebook in this repository lists every manelalab/chrono-gpt-v1-* checkpoint, downloads the model class shipped with the weights, and samples answers with nucleus sampling. Change the cut-off and the answers change with it.

from huggingface_hub import hf_hub_download, HfApi
import importlib.util, sys, torch
from tiktoken import get_encoding

# every available cut-off
api = HfApi()
print([m.modelId for m in api.list_models(author="manelalab") if "chrono-gpt-v1" in m.modelId])

repo_id = "manelalab/chrono-gpt-v1-20231231"          # pick your cut-off
path = hf_hub_download(repo_id=repo_id, filename="modeling_chronogpt.py")
spec = importlib.util.spec_from_file_location("chronogpt", path)
chronogpt = importlib.util.module_from_spec(spec); sys.modules["chronogpt"] = chronogpt
spec.loader.exec_module(chronogpt)

model = chronogpt.ChronoGPT.from_pretrained(repo_id).eval()
enc = get_encoding("gpt2")

prompt = "Q: Who was the president of the United States in 2000?\nA:"
ids = torch.tensor(enc.encode(prompt))[None]
# ... sample with model(ids) as in the notebook

Also available as a gist-backed Colab.

TimeLMs: route each tweet to the model that could have seen it

Cardiff NLP's TimeLMs notebook demonstrates the corresponding mode: perplexity of a tweet is computed with the quarterly checkpoint trained only up to that tweet's date, so later language never informs an earlier score.

from timelms import TimeLMs
tlms = TimeLMs(device="cuda:0")
tweets = [{"text": "Looking forward to watching Squid Game tonight!", "created_at": "2021-10-11T12:34:56Z"}]
tlms.get_pseudo_ppl(tweets, mode="corresponding")   # uses the sep2021 checkpoint

Time Machine GPT: a yearly model and its exact training set

Each Ti-Ma/TiMaGPT2-YYYY model on Hugging Face is paired with a dataset of the same name, so the “what did it see?” question has an exact answer. Standard transformers loading works:

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("Ti-Ma/TiMaGPT2-2015")
model = AutoModelForCausalLM.from_pretrained("Ti-Ma/TiMaGPT2-2015")
out = model.generate(**tok("The most promising new technology is", return_tensors="pt"), max_new_tokens=40)
print(tok.decode(out[0]))

A demo that runs the same prompt across every checkpoint of a sequence, and plots how the answer drifts, would be a natural addition. Contributions welcome on GitHub.