Suppose I have a copy of a pre-trained transformer-based large language model like Google's T5 or Meta's Llama. Due to the pre-training, it contains a lot of knowledge.
However, I want to teach the model something new, knowledge it doesn't already contain about a specific domain. That way, when I ask it to do a task or answer a question about this domain, it can benefit from this specialized knowledge?
How would I go about teaching a pre-trained large language model new knowledge?
A vector database such as Chroma can store pre-computed embeddings of a large number of documents. On querying the LLM, a look-up for relevant documents is performed by closest embedding similarity, then a chunk of text is appended to the prompt for the LLM to draw from and summarise.
Several projects exist based around this idea, such as:
This approach has the advantage that it doesn't require training the model, and can work even with black-box models such as GPT-4. The model will also be able to draw from the provided text chunk more accurately than if it were just seen during training.
You could also go with something simpler, like running find-replace on the prompt to add definitions in brackets after occurrences of jargon words.
A number of parameter-efficient fine-tuning methods, in particular LoRA, allow tweaking a model with more reasonable hardware than what would be required for full training.
This repostory contains code for using LoRA on Meta's LLaMA with (high-end) consumer hardware: https://github.com/tloen/alpaca-lora
This approach has the advantage that the model is tuned to all of your provided data, rather than only one chunk. This can even be used to teach a LLM a new language, or turn a foundation model into a chat model.
I am unfamiliar with the specifics of T5, but in general, LLMs cannot easily learn "new" information after they have been trained/initialized with sample data. Adding new sample data could require a large degree of processing, possibly equal to the original amount of work to train the model.
Instead, new information is typically injected via each query's input (e.g. prompt). Various models have different input size limits (usually represented as tokens). Inquiries can be made about the new data by placing it alongside the query. For instance, if an LLM client has access to the web, it could convert a webpage into text, then send the text along with a user's query to the LLM. However, it would have to do this for each query or session. Once a session is complete (or the input size is reached), the LLM will either start "forgetting" the new information or will not allow any additional queries.
Some LLMs (like ChatGPT) have plugins available for automatically injecting an information source into each prompt or conversation, making the process more transparent to the user. But these still consume input/token space and the amount of information that can be given to the LLM is limited.
How do I "teach" a large language model new knowledge?
From the LIMA: Less Is More for Alignment paper:
These results strongly suggest that almost all knowledge in large language models is learned during pretraining, and only limited instruction tuning data is necessary to teach models to produce high quality output.
Remains prolonging pre-training and adding information in the prompt.
