Does AI Actually Understand Language?
This article was originally published by Quanta Magazine.
A picture may be worth a thousand words, but how many numbers is a word worth? The question may sound silly, but it happens to be the foundation that underlies large language models, or LLMs—and through them, many modern applications of artificial intelligence.
Every LLM has its own answer. In Meta’s open-source Llama 3 model, words are split into tokens represented by 4,096 numbers; for one version of GPT-3, it’s 12,288.