8

Toy example: Let's say I want to automate work off of my support team for my company's products, and try to replace some of their work with ChatGPT to answer support questions. However, ChatGPT has probably not been trained on information about my product, let's say- for example- that technical documentation and user manuals for my products are gated behind product registration. What challenges would I face in trying to do this? How feasible would it be?

Related on Stack Overflow: How to train ChatGPT with custom data

6
  • Heard of PEFT and LoRA?
    – Nav
    Jul 29, 2023 at 15:36
  • @Nav no. If that's answer material, feel free to write an answer. See also How to Answer.
    – starball
    Jul 29, 2023 at 18:57
  • 1
    Strictly ChatGPT (the app/website at chat.openai.com), or OpenAI's LLMs (accessible through their API)?
    – SirBenet
    Aug 2, 2023 at 15:16
  • @SirBenet strictly chatgpt. other models can have their own questions. I even worry that I should've made a question post per model version. (for focus/broadness reasons)
    – starball
    Aug 2, 2023 at 21:18
  • @starball "other models can have their own questions" - to clarify, you can access the same models (such as GPT-4, which is used by ChatGPT Plus) through OpenAI's API. Just double-checking whether you need to access them through ChatGPT (e.g: directing customers to chat.openai.com) as opposed to OpennAI's API (would be more expected for automating support).
    – SirBenet
    Aug 2, 2023 at 21:50

3 Answers 3

3

By default, assuming GPT-4/3.5 (the models used by ChatGPT) won't have been trained on your manuals, the models won't be able to provide useful information beyond what can be guessed by context of the prompt.

To make the models useful you'd likely have to set up document retrieval. A vector database would store embeddings for chunks of your technical documentation, pre-computed with OpenAI's embedding API. When a user asks a support question, you'd look-up relevant documentation by closest embedding similarity, then provide this as context to the LLM as part of the prompt. In implementing this, you may face challenges in data collation and the infrastructure involved.

Even with this, the model may struggle on tasks that require wider background knowledge of your products that can't be inferred from a limited number of chunks. For local models fine-tuning would be an option here, but that's not possible for GPT-3.5/4.


LLMs are known to "hallucinate" and give incorrect but confident-sounding responses. Moreover, even telling the model "don't answer if you don't know" won't necessarily prevent this, as LLMs lack sufficient introspection ability to know what they don't know.

Stating specifically what the LLM should stick to, such as only the content contained in the retrieved document chunk, should be more effective - but still won't entirely prevent hallucination. You could have the model provide documentation references for the user to check manually to verify accuracy.

In general, you'd likely have to work towards setting user expectations correctly not to expect the model to be entirely accurate.


Depending on how your support bot is exposed, you may have to handle misuse of the model by users. Paying for API usage could become costly if many users begin using it for long conversations unrelated to your product.

For example, after Slack's implementation of Claude, tools and tutorials popped up for using Claude for free, involving creating a Slack workspace as an intermediary.


Most likely you'd want to implement this using OpenAI's API. However, if you did in some way use the ChatGPT website/app, you'd also face the issue of data confidentiality for your gated documentation and user requests. For example, a user seeking support may copy-paste in a debug log containing IPs and names. By defualt ChatGPT trains on user input, whereas the OpenAI API accessing the same models does not.

-2

Or, you could add a statement to not respond if it does not know the answer to a specific question (based on your data). Log this entry into a 'Idea' or 'Product Backlog' and then add the product info to train the model further.

1
-3

What challenges would I face in trying to apply ChatGPT to answer questions about information it wasn't trained upon?

Most likely you'll get a very low answer quality and will need to train ChatGPT with custom data.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.