Questions tagged [llm]

LLM stands for Large Language Model. Use this tag for questions about the standard features of LLMs. Don't use questions about a specific LLM; instead, use a more specific tag.

Filter by
Sorted by
Tagged with
1 vote
0 answers
37 views

Ollama GPU Support

I've just installed Ollama in my system and chatted with it a little. Unfortunately, the response time is very slow even for lightweight models like tinyllama. It seems that Ollama is in CPU-only mode ...
Jeron Baffom's user avatar
0 votes
0 answers
27 views

How best can I explain the professional ethics of GenAI to students?

Recently, a decision was made to allow the removal of AI-generated content in response to a letter from Moderators at Stack Exchange. This type of AI-generated content is increasingly used in response ...
William Ledbetter's user avatar
1 vote
0 answers
242 views

How to set ollama temperature from command line

I can run prompts from the command line like so: ollama run mixtral:latest 'Why is the sky blue?' But how do I change the temperature? I know that in the interactive mode (the REPL), I can run /set ...
Jo Liss's user avatar
  • 111
-1 votes
0 answers
22 views

Adding inline links in LLM response

Background Bing Copilot's response is partially clickable. That means, some sentences are links. I'd like to have this feature in my LLM. In this discussion, it is recommended to lookup the source ...
Michael's user avatar
  • 99
0 votes
0 answers
31 views

How to force LLM to output a variable list of items (such as strings)?

I want the output of an LLM to be a list of strings, that I can reliably parse to a Python list of strings. What prompt or completion generation framework should I use?
Elijas Dapšauskas's user avatar
1 vote
1 answer
62 views

Do LLMs also support weight modification like image generators?

For instance, using "(n:2.0)" on a image generator prompt causes it to prioritize that thing on generation. Is this supportes on LLMs as well?
Poseidon of Milos's user avatar
0 votes
1 answer
66 views

Architectures of LLMs

What does the architecture of a LLM look like? Which part stores the enormous parameters? I know the transformer model, but I don’t know how to scale it sensibly as LLMs do.
Zirui Wang's user avatar
0 votes
0 answers
30 views

Gemini Pro API's response ommits the "text" field when I ask to translate a document

I integrate Gemini Pro into one of our internal tools to allow users to "ask" the documents in our database. It all worked fine until I was conducting some tests today and noticed that the ...
ahmed's user avatar
  • 101
0 votes
0 answers
35 views

How to populate labels for dataset preprocessing before LLM fine tuning?

I am using vicgalle/alpaca-gpt4 dataset to fine tune a LLM. I am trying to figure out how to tokenize the dataset's output column into labels within a dataset preprocessor. Please see the below ...
Intrastellar Explorer's user avatar
3 votes
1 answer
44 views

Match LLM output to fixed ontology

We have a certain fixed vocabulary or ontology and would like to match the output of a LLM to this vocabulary. The LLM output could be either a single term, a list of terms, or free form text. For ...
jdm's user avatar
  • 131
3 votes
1 answer
174 views

What is the difference between in-context learning and few-shot prompting?

In-context learning is a prompt engineering technique where natural-language demonstrations are provided as part of the prompt (source). Few-shot prompting is about providing a few examples in the ...
Intrastellar Explorer's user avatar
3 votes
1 answer
50 views

What is the efficient way to tokenize a long string?

I have a really long string. How can I efficiently identify the boundaries of a fixed token length in the text? For example: text = "Quick silver brown fox jumped over the hedge" ...
Maximos's user avatar
  • 43
1 vote
1 answer
87 views

Which techstack and IDE are good to set up a RAG model? (=Retrieval-augmented generation models jointly fine-tune DPR and sequence-to-sequence models)

I'm new to this and would like to know the techstack to finetune an LLM and the techstack to create a RAG system. A good overview with the full code to set it up is at Huggingface - Transformers - RAG:...
okayytrish's user avatar
0 votes
0 answers
27 views

Parallelised GPU training

When training/fine-tuning an LLM model it is necessary to have huge GPU resources (VRAM), as such I was wondering if there was a way in Linux to pool GPU resources if I have, say, 5 separate computers ...
mya205's user avatar
  • 9
2 votes
2 answers
89 views

How does corpus size affect an LLM? Would one trained on just a book still be able to grasp the whole language?

I'm trying to understand how various factors affect LLMs. Specifically the size of the dataset they're trained on. What would be the main difference between: A regular LLM (like ChatGPT) that's ...
laggingreflex's user avatar
0 votes
0 answers
10 views

Training a model to generating application specific configuration

I'm thinking of building a model which can generate application specific configuration file. one example can be a simple dhcp server config in JSON/XML format with user asking "need config file ...
sumeet kumar's user avatar
4 votes
2 answers
105 views

Why are language models bad at legal questions?

It seems that languge models are bad for law-related questions. For instance when you ask a LLM (assuming that it doesn't search the web for up-to-date information) about whether a thing that is ...
Poseidon of Milos's user avatar
0 votes
0 answers
12 views

Interpretation of Evaluation Values of Augmented SBERT Training with EmbeddingSimilarityEvaluator()

I train a BI-Encoder to get an Augmented SBERT and I get a final training result. How can I interpret the following output of the final training result? EmbeddingSimilarityEvaluator: Evaluating the ...
Christian01's user avatar
0 votes
0 answers
23 views

The using of golden dataset in Augmented SBERT Training

I use the training strategy of Augmented SBERT (Domain-Transfer) to train an augmented SBERT. In the code example they use the golden-dataset (STSb) for the training evaluator. Here two code snippets ...
Christian01's user avatar
1 vote
1 answer
77 views

Why LLMs are poor when recognizing weekdays for specific dates?

I asked the LLaMA V2 LLM using this site: https://llama2.ai/ - for the respective weekdays for February 29 for leap years since 2000. Here is the response: I'm happy to help! Here are the respective ...
Poseidon of Milos's user avatar
1 vote
0 answers
35 views

Are LLMs the quickest, most cost-effective way to assess affinity between two texts?

I would like to compare thousands of text descriptions with a specific 'master' input and receive boolean (true/false) results for each comparison, based on their affinity. The following is an example ...
Nic G's user avatar
  • 19
4 votes
1 answer
111 views

What causes a GPT model to output or leak its training data and how to avoid it?

As shown in the screenshot, the model outputs or leaks what appears to be some of the prompts used to train it. I couldn't reproduce the issue though. There were some other questions I had asked ...
Nav's user avatar
  • 291
0 votes
1 answer
105 views

Can I do a DPO training on a synthetic dataset?

I am currently following the fine-tuning methods for the Hugging Face model Zephyr 7B. They have implemented two fine-tuning methods, namely SFT and DPO, on a public dataset. Currently, I am fine-...
Mustafa Alahmid's user avatar
3 votes
1 answer
99 views

Are there any benchmarks for chatbot memory capabilities?

Are there any benchmarks for chatbot memory capabilities? For instance, memGPT uses an ‘LLM judge’ which is instructed to evaluate whether or not the generated response is consistent with the gold ...
Rexcirus's user avatar
  • 131
1 vote
1 answer
95 views

Comprehensive Guide for Finetuning

To learn about finetuning llms, I have read a number of online tutorials. I am about to teach a short course in this area, and looking for a book / paper / survey that discusses examples of different ...
Karl 17302's user avatar
3 votes
1 answer
221 views

Llama2 Vocab contents

since, Llama2 is multi lingual model and it supports multiple languages, including English, Spanish, French, German, Italian, Portuguese, and Dutch. Vocab Size of Llama is 32K. How to know out of 32k ...
Vinay Sharma's user avatar
3 votes
0 answers
195 views

Poor concurrency and perf on multiple GPUs VM (running CodeLlama locally using Ollama)

I am trying to figure out if self hosting CodeLlama on a sufficiently powerful multi-GPU machine can be cost effective for my specific product needs. However, when I go from 1 GPU to 4 GPU VM, I see ...
Shripad K's user avatar
-3 votes
1 answer
1k views

Best realistic story telling LLM?

I've been using WizardLM-SuperCOT30bUncensored for quite some time now, and I'm wondering if there is anything better these days for stories? Because apparently this model is cool, but I can see that ...
Marcin Jasny's user avatar
0 votes
0 answers
33 views

Reducing the occurrence of taboo terms in LLM output

What are some ways to reduce the occurrence of certain taboo or abusive terms in LLM responses? Assume I'm using Chat GPT, is there an established or empirically tested way to construct a preamble to ...
Chris's user avatar
  • 101
2 votes
3 answers
121 views

How can I make sure that my private data is not in the future LLM training and there are no data leaks?

I have question about data leaks and security in LLMs. Suppose I want to build a chatbot based on an LLM like GPT. I use the pre-trained GPT as a base model and I use in-context learning to ground the ...
Nano's user avatar
  • 21
0 votes
1 answer
79 views

How to fix my knowledge base?

I use Langchain.js with HNSWLib. I am building a chat bot for an automotive dealer. here's is part of knowledge base .txt file I use as input to the model. Q: Where is this inventory item/boat/RV/...
Adam S's user avatar
  • 1
1 vote
1 answer
50 views

How to compare the time cost of training the same model via different hardware architecture?

For example, I plan to compare the performance of training the same model via: 8 GPUs in one mode; vs two nodes with 8 GPUs in each node (equals 16 GPUs). How can I measure the time used the train ...
Dmitry J's user avatar
3 votes
1 answer
202 views

Is there a way I can get AI to generate Chinese reading materials, and track the number of characters I've read?

For language learners, LLMs can be a great way to practice reading. Recently, I've been getting ChatGPT to write articles at an appropriate level, and copy/paste them into a website called ...
Rebecca J. Stones's user avatar
1 vote
1 answer
48 views

how to do a quick pilot run when pre-training a large language model?

As I know, it will take several months to pre-train an LLM like chatGPT. I am wondering, what is the industry standard practice to do a pilot run to make sure that everything is correct (as it will be ...
Dmitry J's user avatar
3 votes
1 answer
84 views

Understanding the Concept of a Knowledge Base in the Context of Large Language

I am using large language models (LLMs) like GPT-3.5 and GPT-4, and others. While going through various materials, I came across the term "knowledge base" which seems to be used in ...
Exploring's user avatar
  • 131
1 vote
0 answers
156 views

Inference INT4 ONNX version of LLAMA-2 very slow on google colab

I am using the INT4 quantized version of Llama-2 13B to run inference on the T4 GPU in Google Colab. from optimum.onnxruntime import ORTModelForCausalLM from transformers import AutoTokenizer, ...
k-c's user avatar
  • 11
2 votes
1 answer
1k views

How to prompt Llama2 for text classification?

Here is my script: from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf") model = AutoModelForCausalLM....
etang's user avatar
  • 193
4 votes
3 answers
2k views

How to just get the answer from Llama-2 instead of repeating the whole prompt?

Here is my script: from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf") model = AutoModelForCausalLM....
etang's user avatar
  • 193
2 votes
1 answer
940 views

How to estimate GPU memory size and training time for fine tuning a LLM?

Here is what I observed for fine tuning Flan-T5-base: GPU: 1 Nvidia V100 with 16 GB memory. Flan-T5-base model size: 990 MB. Script: deep-learning-pytorch-huggingface. Samsum train dataset size: 370 ...
etang's user avatar
  • 193
3 votes
1 answer
299 views

Best prompt and model for fact-checking a text (disinformation/fake-news detection)

Given a short text <text_to_check>, I want the LLM to check whether there are some facts stated in the text which are NOT true. So I want to detect 'disinformation' / 'fake news'. And the LLM ...
user2454869's user avatar
1 vote
1 answer
1k views

How to prompt LLMs to get a concise answer for a question?

I use LLMs for QA tasks. The following prompt sent to Llama-2-13b-chat-hf: Give a precise answer to the question based on the context. Don't be verbose. CONTEXT: ..... QUESTION: what is the commission ...
etang's user avatar
  • 193
2 votes
2 answers
124 views

Embeddings: How can I find out, that I'm out-of-vocabulary?

I have a german dataset of financial/annual reports of companies. For example, I use the gensim package to embed my dataset with word2vec or use the huggingface package to embed. How can I find out, ...
Christian01's user avatar
1 vote
1 answer
248 views

Best-Practice in word-embeddings

In my project I follow the retrieval augmented generation (RAG) approach. I want to create embeddings for my own dataset and use it in combination with llama-2. In the dataset are german annual ...
Christian01's user avatar
-1 votes
1 answer
181 views

large language model for incident solving

we want to run a llm locally in our company and we want to give it the code, the documantation and the incidents so it can help to solve the incidents. We think about alpaca or llama for this task. ...
Christopher90's user avatar
4 votes
3 answers
1k views

What are negative prompts in LLMs?

In stable diffusion, a negative prompt can be used to specify elements that should not be part of the generated image. Example: Prompt: Portrait photo of a man Negative Prompt: mustache The negative ...
Turamarth's user avatar
  • 221
1 vote
2 answers
170 views

What works better: An LLM trained on better texts or an LLM with better prompts?

Assume you have a very large corpus of high-quality documents related to a given topic, and assume you have a pretrained large language model (the foundation model) with training data not containing ...
Hans-Peter Stricker's user avatar
3 votes
1 answer
392 views

Does the length of a token give LLMs a preference for words of certain lengths?

From the question How long is a "token"? we learn that tokens are commonly around 4 characters. So it seems plausible that LLMs might therefore prefer to have word boundaries coincide with ...
Rebecca J. Stones's user avatar
2 votes
1 answer
167 views

How to generate the images either by Dall E model or Azure OpenAI services without getting bad or disformed human faces

I was trying to generate images using the Dall E model, the model is getting all the inputs of exactly from the prompt but the image generated is full of disformed faces, hands and legs after a few ...
Swesh's user avatar
  • 87
5 votes
1 answer
378 views

To what extent do LLMs have grammar rules explicitly programmed?

Is there typically any explicit programming in LLMs that dictates how to form a sentence? Where does the "understanding" come to with regards to references like "rewrite last reply in 2 ...
Marek's user avatar
  • 153
1 vote
2 answers
234 views

Is there any automated system available that validate the accuracy of the data generated by GenAI? [closed]

For example, I prompt GenAI to write an essay on "world war I". GenAI generated a 200 words "world war I overview" essay. How a user or student will insure and trust the data ...
Rashid's user avatar
  • 169