2.2M views · 3.8K reactions | "Gary Oldman gives the performance of the year." - Rolling Stone Darkest Hour is now nominated for a Golden Globe for Best Actor (Drama). See it in select theaters now, everywhere 12/22. | Focus Features | Facebook

A tiny GPT implementation as a comparison 312 lines. The model architecture, training loop, tokenization, etc, only differ in 19 lines of code. The main differences are contained within two functions (generate and get_batch). The reason to include the GPT implementation was to show how similar autoregressive LMs are to diffusion LMs on an architectural level. Only *1* line of code in the architecture needs to be modified to support masked language diffusion instead of next-token prediction (by d