Learn · Markov Chain
Markov Chain Text Generator
A Markov chain predicts the next word using only the words immediately before it. Feed it a body of text, count which words tend to follow which, then roll the dice. The maths below is real — frequency counts and weighted sampling — but there is no meaning behind it, just statistics.
Training corpus
Controls
How many previous words form the key. n = 1 looks only at the last word (more random); n = 2 uses the last two words (more coherent, more repetitive).
How many words to generate before stopping. If the chain hits a dead-end (a key with no known continuation), it reseeds from a random starting point.
Generated text
Output is often grammatically loose or nonsensical — that is expected. The model only knows which words followed which in your corpus; it has no grasp of meaning, topic, or truth.
Explore the transition table
Pick a key from your corpus to see the words that can follow it, with the probability of each
— that probability is simply count(next word) / count(all continuations).
What is actually happening
The Markov property
A Markov chain is a system where the next state depends only on the current state, not on the full history of how you got there. Here the "state" is the last one or two words, and the "next state" is the word that follows.
Pure next-word statistics
Building the model is just counting: for every key, tally how often each word came next. Generation samples from those counts, weighted by frequency. There is no learning, no embedding, no understanding — only a lookup table of tallies.
How LLMs differ
A large language model does not look words up in a table. It learns a high-dimensional representation of context across thousands of tokens, so it can generalise to phrasings it never saw. See Temperature & Sampling for how those model probabilities become text.
This demo is a genuine word-level Markov chain — the counts and the weighted sampling are real — but it is deliberately tiny and shallow. Longer context, subword tokenisation, and learned parameters are what separate a toy chain from a modern model. For related demos, see Temperature & Sampling or the Learn page.