Why there aren't that many language models with a relatively small amount of parameters?

Question

E.g. language models with less than 3 billion parameters.

Answer 1

Why there aren't that many language models with a relatively small amount of parameters?

Because they don't perform as well as their larger counterparts. See https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard or https://arxiv.org/pdf/2001.08361:

More generally, here is a Survey of Small Language Models.