Suppose I have a copy of a pre-trained transformer-based large language model like Google's T5 or Meta's Llama. Due to the pre-training, it contains a lot of knowledge.

However, I want to teach the model something new, knowledge it doesn't already contain about a specific domain. That way, when I ask it to do a task or answer a question about this domain, it can benefit from this specialized knowledge?

How would I go about teaching a pre-trained large language model new knowledge?

  • A similar/related question was asked as an example question on the SO for Teams, and the feedback given by multiple stakeholders (wizzwizz4 and Brian H.) was that it was too broad in asking about LLMs in general, and should be scoped to a specific LLM.
    – starball
    Commented Jul 19, 2023 at 19:01
  • 2
    I don't see how this is too broad. Transformer based models are very similar. The answer is the same, basically what SirBenet says, no matter which particular one you're interested in. I will admit, however, that it might be off topic. Commented Jul 24, 2023 at 14:06

3 Answers 3


Document retrieval

A vector database such as Chroma can store pre-computed embeddings of a large number of documents. On querying the LLM, a look-up for relevant documents is performed by closest embedding similarity, then a chunk of text is appended to the prompt for the LLM to draw from and summarise.

Several projects exist based around this idea, such as:

This approach has the advantage that it doesn't require training the model, and can work even with black-box models such as GPT-4. The model will also be able to draw from the provided text chunk more accurately than if it were just seen during training.

You could also go with something simpler, like running find-replace on the prompt to add definitions in brackets after occurrences of jargon words.

Fine-tuning the model

A number of parameter-efficient fine-tuning methods, in particular LoRA, allow tweaking a model with more reasonable hardware than what would be required for full training.

This repostory contains code for using LoRA on Meta's LLaMA with (high-end) consumer hardware: https://github.com/tloen/alpaca-lora

This approach has the advantage that the model is tuned to all of your provided data, rather than only one chunk. This can even be used to teach a LLM a new language, or turn a foundation model into a chat model.


I am unfamiliar with the specifics of T5, but in general, LLMs cannot easily learn "new" information after they have been trained/initialized with sample data. Adding new sample data could require a large degree of processing, possibly equal to the original amount of work to train the model.

Instead, new information is typically injected via each query's input (e.g. prompt). Various models have different input size limits (usually represented as tokens). Inquiries can be made about the new data by placing it alongside the query. For instance, if an LLM client has access to the web, it could convert a webpage into text, then send the text along with a user's query to the LLM. However, it would have to do this for each query or session. Once a session is complete (or the input size is reached), the LLM will either start "forgetting" the new information or will not allow any additional queries.

Some LLMs (like ChatGPT) have plugins available for automatically injecting an information source into each prompt or conversation, making the process more transparent to the user. But these still consume input/token space and the amount of information that can be given to the LLM is limited.

  • I did not think of that. So easy, you do not need another model on top. Say if you have an essay with a lot of chapters, and if you know which chapter hits the background of your question, you can just feed the model that chosen chapter and end that with a question on the text. Did i get that right? What is the best practice then if you want to make the two steps in one go (that is, if I am not in the online chat, but instead in an offline model). How to you make sure that the model understands which is the question and which is the context? Does it help to set the long text in quotes? Commented Jan 10 at 17:44

How do I "teach" a large language model new knowledge?

From the LIMA: Less Is More for Alignment paper:

These results strongly suggest that almost all knowledge in large language models is learned during pretraining, and only limited instruction tuning data is necessary to teach models to produce high quality output.

Remains prolonging pre-training and adding information in the prompt.

  • Does this mean that it is easy to fine-tune the big model further with your own small dataset? Since in that 5/2023 paper, they "fine-tuned with the standard supervised loss on only 1,000 carefully curated prompts and responses." ... "LIMA demonstrates remarkably strong performance, learning to follow specific response formats from only a handful of examples in the training data, including complex queries that range from planning trip itineraries to speculating about alternate history. Moreover, the model tends to generalize well to unseen tasks that did not appear in the training data." Commented Jan 10 at 17:50
  • In other words, if that is right, you should stress in your answer that it is about fine-tuning and that fine-tuning with a Question Answering model can work out very well with just a sample of 1000 questions and answers, and that it can even generalize from the input answers and answer other untrained questions (I hope this is the right now). Commented Jan 10 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.