There are a few main reasons why large language models have output limitations:
- Computing resources - Generating text is computationally expensive, especially for very large models with billions of parameters. More text generation requires more computation, which costs money. Limits help manage this cost.
- Context Window - Maintaining context and coherence becomes more difficult as more text is generated. Limits help prevent the output from degrading or becoming repetitive/contradictory.
- Safety - Longer outputs increase the chances of generating harmful, biased, or nonsensical text. Limits help mitigate the risk.
To expand more on "Context Window":
Context Window in large language models has to do with the information captured in the model's internal representations, especially in the output layer.
In particular, as text generation progresses, information about the existing context must be retained in the model's neural network activations. The output layer contains the model's representation of the currently generated text. As more text is produced, earlier context begins to fade from the output layer activations as new text overwrites it.
This is because there is a limit to how much context information can be captured and propagated through the deep neural networks that make up these large models. The connections have a certain fixed capacity.