Questions tagged [fine-tuning]

Fine-tuning is the process of further training a pre-trained model on a typically smaller dataset in order to improve performance on a specific task or domain. Use this tag for questions about fine-tuning fundamentals, development of fine-tuning approach or implementation.

Filter by
Sorted by
Tagged with
5 votes
0 answers
56 views

How to fine-tune an LLM on a very small document

I am currently trying to fine-tune an LLM on a single document (about 1700 characters). I know that generally it is better to use prompt injection or something like a RAG system to provide specific ...
Roy Varon's user avatar
  • 151
1 vote
1 answer
30 views

Domain specific content generation - RAG or fine-tune?

I am exploring requirements for a project whose final output is a Markdown file where, the contents of the file are generated by a LLM. The contents will be similar to "Executive Summary" of ...
cogitoergosum's user avatar
0 votes
1 answer
40 views

RAG for sophisticated question-answering (Q&A) chatbots

I am about to start my first ever hobby project with LLMs. I hope to develop a Q&A chat bot. I am following the documentation from here. LLMs can reason about wide-ranging topics, but their ...
cogitoergosum's user avatar
1 vote
0 answers
30 views

Prompt-based image generative AI tool for editing specific details

I am trying to make some spritesheets using DALLE3, and while the initial generation of spritesheets by DALLE3 are fascinating, I have encountered these problems: Inconsistent art style(multiple ...
rasputin's user avatar
-2 votes
1 answer
267 views

BadRequestError: Error code: 400 - {'error': {'message': 'Invalid file format. Line 1, key "messages": ...one message must be from the assistant ...}}

I follow OpenAI Capabilities - Fine-tuning. This is myfile.jsonl: {"messages": [{"role": "system", "content": "You are a chatbot that wants to help ...
questionto42's user avatar
1 vote
0 answers
40 views

Freezing layers for finetuning

So i'm planning to fine tune a 7b parameter model for a research project. I understand the different steps of model fine tuning, namely Supervised fine tuning - where we train model on curated ...
kaiser's user avatar
  • 111
1 vote
0 answers
23 views

What could I do to further reduce my finetuning time of 67 hours with the following configuration?

I am taking a large text dataset and fitting them into 19k rows. Each row is exactly 4096 tokens (not characters), which is the maximum context length of llama 2 7B. GPU: 80GB My other configurations ...
Maximos's user avatar
  • 43
1 vote
1 answer
105 views

Comprehensive Guide for Finetuning

To learn about finetuning llms, I have read a number of online tutorials. I am about to teach a short course in this area, and looking for a book / paper / survey that discusses examples of different ...
Karl 17302's user avatar
3 votes
3 answers
165 views

How can I make sure that my private data is not in the future LLM training and there are no data leaks?

I have question about data leaks and security in LLMs. Suppose I want to build a chatbot based on an LLM like GPT. I use the pre-trained GPT as a base model and I use in-context learning to ground the ...
Nano's user avatar
  • 31
2 votes
1 answer
1k views

How to estimate GPU memory size and training time for fine tuning a LLM?

Here is what I observed for fine tuning Flan-T5-base: GPU: 1 Nvidia V100 with 16 GB memory. Flan-T5-base model size: 990 MB. Script: deep-learning-pytorch-huggingface. Samsum train dataset size: 370 ...
etang's user avatar
  • 203
-1 votes
1 answer
171 views

GPT-4 to Generate USMLE Exam Questions

I have a set of example multiple choice questions on the USMLE, a US medical licensing exam. I want to use the set of questions, their answers, and the explanations associated with the correct answers ...
2567655222's user avatar
2 votes
1 answer
188 views

How to generate the images either by Dall E model or Azure OpenAI services without getting bad or disformed human faces

I was trying to generate images using the Dall E model, the model is getting all the inputs of exactly from the prompt but the image generated is full of disformed faces, hands and legs after a few ...
Swesh's user avatar
  • 87
1 vote
1 answer
71 views

Programmatic fine-tune training in AWS

I'm trying to setup an automatic fine tuning training pipeline with DreamBooth. I've read this tutorial https://medium.com/@IAmxIAo/automating-fine-tuning-stable-diffusion-on-sagemaker-c340f65b24f2 ...
JJ Zh's user avatar
  • 29
17 votes
3 answers
8k views

Can an LLM be constrained to answer questions only about a specific dataset?

I've seen this, this, this, and this. But my question is different. Whether I use ChatGPT's API or any of the LLM's supported by GPT4All, when I deploy my GenAI app for people to use, I want it to ...
Nav's user avatar
  • 331
11 votes
3 answers
2k views

How do I "teach" a large language model new knowledge?

Suppose I have a copy of a pre-trained transformer-based large language model like Google's T5 or Meta's Llama. Due to the pre-training, it contains a lot of knowledge. However, I want to teach the ...
Ian Campbell's user avatar