I am using large language models (LLMs) like GPT-3.5 and GPT-4, and others.

While going through various materials, I came across the term "knowledge base" which seems to be used in conjunction with these models. However, I am a bit puzzled about what exactly a knowledge base refers to.

From what I understand, a knowledge base in general terms refers to a collection of structured or unstructured data that can be leveraged for information retrieval or problem-solving.

However, when mentioned alongside LLMs, does it have a specific meaning or structure?

Does it refer to a continuously updated source of information that the model can access to provide more accurate or up-to-date responses?

Moreover, how does a knowledge base interact with a large language model, and what benefits does it provide in enhancing the model's capabilities?

Could someone shed some light on this or point me to relevant resources to better understand the concept of a knowledge base in the context of large language models?

  • In my understanding it's mainly about how to access those knowledge graphs: at training or at inference time, and in the case of the latter: for generating a response or just to cross- or fact-checking it. I can imagine all of this, but like you have no clear idea how it could be - or actually it is - done. Oct 29, 2023 at 6:00

1 Answer 1


One common use case of knowledge bases used with LLMs is for retrieval-augmented generative question-answering models, where the LLM generates an answer based on some fragments of the knowledge base that a retriever thought were relevant given a question.

See Huggingface - Transformers - Retrieval-augmented generation ("RAG").

  • "Thought were relevant" just means "found (dot-product?) similar to the question's text embedding", right? Without knowing why, by the way: text-embedding similarity is "blind". Or do you know of any post-processing of the texts found by text-embedding similarity, e.g. another more content-based sorting by relevance than just by the similarity measure? Nov 1, 2023 at 13:36
  • @Hans-PeterStricker text embeddings indeed or other techniques such as BM25. The solution can vary based on the use case eg we could look at user click data or source quality. But in most cases, we'd use some similarity measure. Nov 1, 2023 at 13:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.