Skip to main content

Questions tagged [llm]

LLM stands for Large Language Model. Use this tag for questions about the standard features of LLMs. Don't use questions about a specific LLM; instead, use a more specific tag.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
8 views

Having issues with RAG Application it's working but not providing me the most relevant answer from the documents i have provided there is no problems

Having issues with RAG Application it's working but not providing me the most relevant answer from the documents i have provided there is no problem to connect with LLm main problem is not getting ...
Zaid Ghaffar's user avatar
1 vote
1 answer
23 views

How to detect multiple tasks in a single command using LLM or any NLP model?

I am building a bot that takes inputs from users and performs action. Now I want it to differentiate between commands that contain single task and multiple tasks. How can I make this a reality? Should ...
Rishabh Tripathi's user avatar
4 votes
2 answers
81 views

How do people fine tune LLMs to only answer domain specific questions?

So I am new to LLMs and learning new things daily like RAG systems and stuff. I am right now trying to create a chatbot using OpenAI API that should answer questions specific to a particular format of ...
Yash Bajpai's user avatar
2 votes
2 answers
34 views

Efficiency of ReRanker in RAG system for private data retrieval

I'm building a Retrieval-Augmented Generation (RAG) system to process and answer queries on private data. I understand that the ReRanker component is crucial for filtering and ordering retrieved ...
CrazyCoder's user avatar
1 vote
0 answers
14 views

Separating information in prompt

I have multiple pieces of information to include in the prompt. There are various ways that I can separate them. I can separate them using XML tags, or putting the data in JSON format, or some ad-...
D.W.'s user avatar
  • 111
0 votes
1 answer
35 views

How do chatbots interface with LLMs?

I'm trying to understand how chatbots such as ChatGPT work on top of a LLM. I understand that at the basis chatbots have a LLM model that converts text to tokens and then continues the sequence of ...
jurez's user avatar
  • 101
3 votes
2 answers
78 views

Do LLMs “pause to think”?

Sometimes it seems like an LLM takes longer to respond, perhaps if the question is somehow more difficult. But maybe this is an illusion. Is there any reason that a response would take longer? Is ...
Julius Hamilton's user avatar
-1 votes
0 answers
31 views

How do I save and use a fine-tuned LLM model

This may seem like a very basic question, but I've just started learning about LLM's and how to fine tune them for particular tasks. So, I found this google collab notebook by UnSloth (https://colab....
L lawliet's user avatar
1 vote
0 answers
17 views

Which OpenAI GPT models, if any, allow specifying top_k?

I don't see top_k in the current doc but seems to be mentioned in the past. Which OpenAI GPT models, if any, allow specifying top_k?
Franck Dernoncourt's user avatar
1 vote
1 answer
37 views

Why isn't it generally recommended to alter both the temperature and `top_p` when calling GPT?

I read on platform.openai.com/docs/api-reference/chat/create: We generally recommend altering [the temperature] or top_p but not both. Why isn't it generally recommended to alter both the ...
Franck Dernoncourt's user avatar
-1 votes
1 answer
41 views

How Google's TextFX experiment works?

Link: https://textfx.withgoogle.com/ How can an LLM be trained to perform things like this experiment site does? (Visit the site and you will find out more about these tools.) This is different from ...
user56510's user avatar
2 votes
2 answers
54 views

How Can I Add Memory to My RAG-Based Chatbot for Better Responsiveness?

I have developed a chatbot utilizing the RAG methodology. However, there is an issue where it does not retain previous conversations. Does anyone suggest integrating memory into the chatbot to enhance ...
Kraftors Web Solutions Pvt Ltd's user avatar
2 votes
3 answers
119 views

What tasks are LLMs capable of doing?

Other than they being used to get answers when a question is prompted, and to write (source code, stories, etc), are there any tasks that they can do (specially when fine tuned)? Are there some ...
user56510's user avatar
1 vote
2 answers
69 views

prompt llm to generate an email with well structured line breaks

I'm using the latest llms (mixtral/gemini/..) I'm generating emails and I'm satisfied with the results except that the sentences are without proper line breaks. The greeting and signature parts are ok ...
kambi's user avatar
  • 131
4 votes
1 answer
203 views

LLMs, by their very nature, don't have a concept of "source" - is this true?

Summarize the problem LLMs, by their very nature, don't have a concept of "source". Attribution is pretty much impossible. Attribution only really works if you use language models as "...
user10186832's user avatar
2 votes
1 answer
29 views

How to find proper context in open book question answering?

I want to make an Open Book Question Answering / Retrieval Augmented Generation system. The major concern here is the proper context selection. There are some fundamental issues related to this. For ...
hafiz031's user avatar
  • 171
1 vote
1 answer
33 views

What's GPT-3.5-Turbo-0314? Same as GPT-3.5-Turbo-0301?

I see GPT-3.5-Turbo-0314 in https://chat.lmsys.org/?leaderboard: but I can't find GPT-3.5-Turbo-0314 in the OpenAI docs. The closest model (closest based on dates) I can see in https://platform....
Franck Dernoncourt's user avatar
3 votes
3 answers
84 views

Can GPT models be used for text prediction/autofill?

The title says it all. Can they be used in a way that is not for generation of text based on a prompt, but instead for word suggestions based on the context of previously written text?
user56510's user avatar
2 votes
0 answers
38 views

What are examples of statistically confirmed LLM functions?

I am looking for examples from research of highly statistically reliable LLM functions. So instead of using an LLM in a general purpose way, an exact, replicable LLM setup where it is guaranteed to do ...
Julius Hamilton's user avatar
1 vote
1 answer
83 views

How does Meta Llama-3 LLM compare against GPT 4, Claude Opus, and Mistral-Large on human evaluations?

https://ai.meta.com/blog/meta-llama-3 compared Meta Llama-3 LLM compare against GPT 3.5, Claude Sonnet and Mistral Medium on human evaluations: They conveniently avoided the state-of-art LLMs, namely ...
Franck Dernoncourt's user avatar
1 vote
1 answer
62 views

Why does RAG work?

Consider a corpus of answer sheets of History exam at high school level. Say it is 10 GiB in size. Case 1 Assume that these answer sheets were part of much larger corpus used to train an LLM such as ...
cogitoergosum's user avatar
1 vote
1 answer
56 views

Is it feasible to use LLM for natural language detection?

Huge models are trained on data in multiple languages, so I would like to use such model to detect the language of the input. I would extract paragraphs from a webpage, then have the AI analyze the ...
tpimh's user avatar
  • 159
5 votes
0 answers
99 views

How to fine-tune an LLM on a very small document

I am currently trying to fine-tune an LLM on a single document (about 1700 characters). I know that generally it is better to use prompt injection or something like a RAG system to provide specific ...
Roy Varon's user avatar
  • 151
1 vote
1 answer
66 views

Generalize knowledge across multiple documents

I created a chat via platform.openai.com. It uses GPT-4 turbo preview. I have uploaded multiple documents to it. The knowledge is scattered across different docs. The behavior I get is the assistant ...
Marcin's user avatar
  • 111
2 votes
2 answers
61 views

How to force the model to use different greeting in emails

I'm using llama2 model and I'm asking it to generate emails. it usually start the email with 'hi' or 'hey'. I would like it to be more creative and randomize the greeting. changing parameters like ...
kambi's user avatar
  • 131
0 votes
0 answers
17 views

Can I modify the weight of a particular section of text in an LLM training corpus?

I have a rather odd data format that I want to train a LLM on. I want it to focus on learning specific segments within one piece of training data. The problem is that I can't just delete the rest of ...
user avatar
4 votes
1 answer
71 views

How much RAM is required for model inferencing?

When I run ollama run mistral:text, I see a model of around 4GiB pulled. Is this 4GiB model the GGUF or GGML file? Once pulled, I see the ready prompt. At that point, is whole of the model loaded ...
cogitoergosum's user avatar
2 votes
1 answer
36 views

What is model inferencing - computation or look-up?

I am defining inferencing as querying the LLM. For example, if I goto ChatGPT and enter text in the text box, then I am inferencing the OpenAI LLM (GPT-x). From the UI's perspective, clearly it is an ...
cogitoergosum's user avatar
1 vote
3 answers
116 views

Domain specific content generation - RAG or fine-tune?

I am exploring requirements for a project whose final output is a Markdown file where, the contents of the file are generated by a LLM. The contents will be similar to "Executive Summary" of ...
cogitoergosum's user avatar
2 votes
1 answer
67 views

Can GenAI be Used to Generate Decision Trees from Text?

I'm exploring the capabilities of GenAI for text analysis and decision-making processes. I'm particularly interested in understanding whether GenAI can be leveraged to create decision trees directly ...
D3N3ON's user avatar
  • 31
0 votes
3 answers
92 views

Is it feasible to restrict a large language model (LLM) to respond exclusively within a user-specific context?

I aim to develop a customer-centric LLM model capable of providing responses solely based on the context provided by the user. For instance, in a customer support application, when a user interacts ...
Thalassophile's user avatar
2 votes
1 answer
52 views

How can we identify/detect hallucinations in Large Language Models (LLMs)?

Hallucinations in LLMs refer to instances where the model generates responses that are factually incorrect, nonsensical, or unrelated to the input prompt. These hallucinations stem from the ...
Thalassophile's user avatar
1 vote
1 answer
49 views

Does GPT 4 Turbo perform worse than GPT 4?

GPT 4 Turbo is cheaper and accepts a longer context than GPT 4 (128k vs. 8k or 32k). Does it mean GPT 4 Turbo perform worse than GPT 4? Prices from https://openai.com/pricing:
Franck Dernoncourt's user avatar
0 votes
1 answer
155 views

RAG for sophisticated question-answering (Q&A) chatbots

I am about to start my first ever hobby project with LLMs. I hope to develop a Q&A chat bot. I am following the documentation from here. LLMs can reason about wide-ranging topics, but their ...
cogitoergosum's user avatar
1 vote
1 answer
77 views

Why LLMs tend to use Greek mythology when asking them to write stories about deities?

Try using "Write a story about sea gods", "Write a story about underworld gods", or other phrases referencing gods as a prompt. They tend to use Greek mythology gods when a ...
user avatar
1 vote
1 answer
115 views

System prompts for RAG

How can i write effective system prompts for RAG ? I am using chatgpt-4 and i would like it to retrieve information based on documents provided. However it is answering questions such as : tell me ...
Maths12's user avatar
  • 119
4 votes
2 answers
277 views

How do I prevent Gemini from including my IP address location in its responses?

I use Google’s Gemini (formerly known as "Bard") to generate creative works, such as stories. However, I have noticed that Gemini often inserts details from my IP address location into its ...
galacticninja's user avatar
0 votes
0 answers
11 views

Is Per-User Retrieval supports open-source vectorstore chromadb?

From the langchain documentation - Per-User Retrieval When building a retrieval app, you often have to build it with multiple users in mind. This means that you may be storing data not just for one ...
Thalassophile's user avatar
0 votes
1 answer
324 views

Ollama Hallucinations for Simple Questions

Recently I installed Ollama and started to test its chatting skills. Unfortunately, so far, the results were very strange. Basically, I'm getting too much hallucinations even for simple questions like:...
Jeron Baffom's user avatar
3 votes
2 answers
4k views

Ollama GPU Support

I've just installed Ollama in my system and chatted with it a little. Unfortunately, the response time is very slow even for lightweight models like tinyllama. It seems that Ollama is in CPU-only mode ...
Jeron Baffom's user avatar
4 votes
0 answers
4k views

How to set ollama temperature from command line

I can run prompts from the command line like so: ollama run mixtral:latest 'Why is the sky blue?' But how do I change the temperature? I know that in the interactive mode (the REPL), I can run /set ...
Jo Liss's user avatar
  • 141
0 votes
2 answers
93 views

How to force LLM to output a variable list of items (such as strings)?

I want the output of an LLM to be a list of strings, that I can reliably parse to a Python list of strings. What prompt or completion generation framework should I use?
Elijas Dapšauskas's user avatar
2 votes
1 answer
79 views

Do LLMs also support weight modification like image generators?

For instance, using "(n:2.0)" on a image generator prompt causes it to prioritize that thing on generation. Is this supportes on LLMs as well?
user avatar
0 votes
1 answer
91 views

Architectures of LLMs

What does the architecture of a LLM look like? Which part stores the enormous parameters? I know the transformer model, but I don’t know how to scale it sensibly as LLMs do.
Zirui Wang's user avatar
0 votes
0 answers
51 views

Gemini Pro API's response ommits the "text" field when I ask to translate a document

I integrate Gemini Pro into one of our internal tools to allow users to "ask" the documents in our database. It all worked fine until I was conducting some tests today and noticed that the ...
ahmed's user avatar
  • 101
0 votes
0 answers
44 views

How to populate labels for dataset preprocessing before LLM fine tuning?

I am using vicgalle/alpaca-gpt4 dataset to fine tune a LLM. I am trying to figure out how to tokenize the dataset's output column into labels within a dataset preprocessor. Please see the below ...
Intrastellar Explorer's user avatar
4 votes
1 answer
91 views

Match LLM output to fixed ontology

We have a certain fixed vocabulary or ontology and would like to match the output of a LLM to this vocabulary. The LLM output could be either a single term, a list of terms, or free form text. For ...
jdm's user avatar
  • 141
3 votes
1 answer
1k views

What is the difference between in-context learning and few-shot prompting?

In-context learning is a prompt engineering technique where natural-language demonstrations are provided as part of the prompt (source). Few-shot prompting is about providing a few examples in the ...
Intrastellar Explorer's user avatar
3 votes
1 answer
64 views

What is the efficient way to tokenize a long string?

I have a really long string. How can I efficiently identify the boundaries of a fixed token length in the text? For example: text = "Quick silver brown fox jumped over the hedge" ...
Maximos's user avatar
  • 43
3 votes
1 answer
130 views

Which techstack and IDE are good to set up a RAG model? (=Retrieval-augmented generation models jointly fine-tune DPR and sequence-to-sequence models)

I'm new to this and would like to know the techstack to finetune an LLM and the techstack to create a RAG system. A good overview with the full code to set it up is at Huggingface - Transformers - RAG:...
okayytrish's user avatar