Questions tagged [api]

Use this tag for questions about using generative artificial intelligence (GenAI) tools through an API. Questions focused on programming rather than using a GenAI tool are off-topic; these questions might be asked on htts://stackoverflow.com. When asking questions about a GenAI tool API, be specific, mention which API is being used, what programming platform and language is being used, and when it is relevant, add a minimal complete example.

Filter by
Sorted by
Tagged with
3 votes
1 answer
55 views

How much RAM is required for model inferencing?

When I run ollama run mistral:text, I see a model of around 4GiB pulled. Is this 4GiB model the GGUF or GGML file? Once pulled, I see the ready prompt. At that point, is whole of the model loaded ...
cogitoergosum's user avatar
2 votes
1 answer
33 views

What is model inferencing - computation or look-up?

I am defining inferencing as querying the LLM. For example, if I goto ChatGPT and enter text in the text box, then I am inferencing the OpenAI LLM (GPT-x). From the UI's perspective, clearly it is an ...
cogitoergosum's user avatar
0 votes
1 answer
56 views

Is there any AI apis that can remember your information?

As someone who is between the beginner and intermediate brackets in terms of programming, and fairly new AI apis, can someone help point me in the right direction? I want to program a chatbot/AI ...
EON's user avatar
  • 1
0 votes
0 answers
24 views

What's the difference between "Tokens per Minute Rate Limit (thousands)" and "Rate limit (Tokens per minute)" in Azure OpenAI?

I deployed GPT-4V on Azure OpenAI. I see: Tokens per Minute Rate Limit (thousands): 10 Rate limit (Tokens per minute): 30000 What's the difference between "Tokens per Minute Rate Limit (...
Franck Dernoncourt's user avatar
0 votes
1 answer
60 views

How can I open a licensed GPT chat like on https://chat.openai.com/ on my own user if I have an API key that is not from my user?

I can already ask questions against GPT4 with an API key that I got from a group license, see a question on my first tests at Stack Overflow How do I run a GPT API shell command in Jupyter Notebook ...
questionto42's user avatar
0 votes
0 answers
40 views

Gemini Pro API's response ommits the "text" field when I ask to translate a document

I integrate Gemini Pro into one of our internal tools to allow users to "ask" the documents in our database. It all worked fine until I was conducting some tests today and noticed that the ...
ahmed's user avatar
  • 101
3 votes
1 answer
107 views

Which techstack and IDE are good to set up a RAG model? (=Retrieval-augmented generation models jointly fine-tune DPR and sequence-to-sequence models)

I'm new to this and would like to know the techstack to finetune an LLM and the techstack to create a RAG system. A good overview with the full code to set it up is at Huggingface - Transformers - RAG:...
okayytrish's user avatar
6 votes
0 answers
415 views

I can't get my custom GPT to accept an API key - "error saving draft"

I'm trying to build a custom gpt with an action, basically the custom gpt can access an API if given proper instructions. I'm trying to get it to connect to plausible. I have built a simple code as a ...
Joe's user avatar
  • 161
2 votes
1 answer
199 views

Purpose of the "system" role in OpenAI chat completions API

I know that the "system" role can be used to set the behavior of the "assistant" (see here). But actually I can also set the "assistant" behavior with a "user" ...
MaDev's user avatar
  • 196
0 votes
1 answer
88 views

chatGPT messages not always in French, sometimes in English

When using a rather long prompt in French, I get messages sometimes back in English . All of my prompts are rather similar. Has anybody experienced a similar issue ? I am using the following code: ...
Bob Yoplait's user avatar
1 vote
1 answer
92 views

How can I get chatgpt to keep track of all the words in the list that I am providing?

I am trying to ask ChatGPT (gpt-3.5-turbo) using code to compare a text and a list of keywords and evaluate how well the keywords describe the text. However, ChatGPT seems confused about the list of ...
Akavall's user avatar
  • 111
5 votes
2 answers
465 views

How do I force ChatGPT API response to be brief while being brief myself?

For a hobby language learning tool, I use ChatGPT API to validate if a given sentence is grammatically correct. To save tokens, I want both the prompt and the response to be brief. But I can't make ...
ᴍᴇʜᴏᴠ's user avatar
17 votes
3 answers
8k views

Can an LLM be constrained to answer questions only about a specific dataset?

I've seen this, this, this, and this. But my question is different. Whether I use ChatGPT's API or any of the LLM's supported by GPT4All, when I deploy my GenAI app for people to use, I want it to ...
Nav's user avatar
  • 331
8 votes
2 answers
533 views

ChatGPT API Embedding Vectors - Are vectors standard across LLM's?

Can embeddings from one LLM's reused by others - any specific standardization exists on embeddings representation? reference - https://platform.openai.com/docs/guides/embeddings/what-are-embeddings
sumeet kumar's user avatar
0 votes
1 answer
197 views

Is it possible to create a reliable AI quotation tool using excel spreadsheets with ChatGPT API

I'm working on a project to create an AI Quotation tool using the ChatGPT API trained on my data. The data I have spans the last 10 years and is mostly in Excel and PDF format, but it's not ...
user avatar