Summary of “Listening for Extraterrestrial Blah Blah”

One aspect of human linguistics that emerged from early statistical studies of letters, words, and phonemes is known as Zipf’s Law, after the Harvard University linguist George Zipf.
The same is true with the letters, words, or phonemes of a conversation in Japanese, German, Hindi, and dozens of other languages.
For English words, conditional probabilities can be specified up to about nine words in a row.
If you are missing one word, you can probably guess it by the context; if you are missing two words in a row, you can often still recover them from the context.
As a short example, take a sentence missing a word: “How are doing today?” We can easily fill in the missing word “You” from the rules we know about the English language.
For most human written language, the conditional dependencies disappear when one is missing about nine words in a row.
With 10 words missing, one really has no clue what these missing words could be.
In the language of information theory, human word entropy goes up to about ninth-order.

The orginal article.