0

I aim to develop a customer-centric LLM model capable of providing responses solely based on the context provided by the user. For instance, in a customer support application, when a user interacts with the LLM chat assistant, I want the LLM model to deliver responses strictly within the context of that particular user, without accessing information from other customers.

Is it technically viable to confine the LLM to a user-specific context?

3 Answers 3

3

Most LLMs have access only to the information they are trained on and the prompt. If you do not provide other customers' information in the training data or in the prompt, the LLM will not have access to it. It will, of course, be more than happy to make up that data, which may be just as bad. (For safety's sake, you should probably assume everything you put in the prompt and training data will at some point be leaked.)

2

Yes, it is possible to restrict the LLM's response to only address the context of the user's question and the data it has been trained on. This can be achieved by using the RAG methodology.

What is RAG? RAG (Retrieval-Augmented Generation) is an approach that integrates the power of retrieval (or searching) into LLM text generation. It combines a retriever system, which fetches relevant document snippets from a large corpus, with an LLM, which produces answers using the information from those snippets. In essence, RAG helps the model to “look up” external information to improve its responses.

In RAG, there is a parameter called "temperature" that influences how the LLM generates responses. If the temperature is set to 0, the LLM will respond strictly based on the data it has been trained on. As the temperature value increases, the responses become more varied and creative, potentially generating different answers each time the user asks a query.

1

You can make some engineering in the prompt telling it to only consider the context. A temperature of 0 in this case can be better to reduce hallucinations.

The concept of RAG (Retrieval Augmented Generation) today is based fully on this, give the LLM outside context and use only its natural language and comprehension capabilities, there is a lot of content about RAG online and advanced techniques for it.

But at the end of the day there will always be a small probability that it will hallucinate and deviate at some point, how small that probability is depends on the implementation of the RAG, techniques used and the model's own likeness to hallucinate.

Here is a benchmark of probability of hallucination called the Hallucination Index where you can see the latest versions of GPT-4 are the least likely of the tested LLMs to hallucinate: https://www.rungalileo.io/hallucinationindex

There are some meta-prompt approaches for RAG, where it calls it more than once and with validations to also reduce hallucination chances.

Also, when I am making my RAG prompt, besides telling it to always base the answer on the context I also instruct it what to do in negative cases like "If you cannot generate an answer based on the context given, always answer that you don't know".

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.