Devlog 2

In here, I introduce you to the official generation pipeline.I used the TinyStories-Instruct-33M checkpoint as the progenitor model. It accepts a structured header as follows:

Words: verb, noun, adjective
Features: X, Y [optional]
Story:

The words were chosen from the original TS word list from TinyStories-Instruct-valid.txt. The features were derived from an optional feature set, sampled from the empirical distributions in the same TinyStories-Instruct data. Quality control was also enforced by rejecting incomplete stories and stories that were too long or too short. I wrote an .ipynb and began generating stories in batches of 1024 on a rented 4090 (CUDA) and my M1 MacBook Pro (MPS)! Should be ready in around 8 hours.