13

As we likely all have encountered here, Poe.com's ChatGPT sometimes "hallucinates" when responding to questions. For example, in response to Name 10 countries starting with z, Poe.com's ChatGPT wrote:

Sure, here are ten countries starting with the letter "Z":

  1. Zambia
  2. Zimbabwe
  3. Zealand (also known as New Zealand)
  4. Zaire (now known as the Democratic Republic of the Congo)
  5. Zambia
  6. Zimbabwe
  7. Zanzibar (a semi-autonomous region of Tanzania)
  8. Zimbabwe
  9. Zhejiang (a province in China)
  10. Zambia

There's some repeats, and some which aren't countries. It seems like in an effort to give the most helpful output, it doesn't want to say "I don't know" or "I can't do that", even for questions which are theoretically impossible to answer.

A much better answer would be "I can only think of Zambia and Zimbabwe, and formerly Zaire" and note that naming 10 countries starting with z is impossible. (I asked it to prove P = NP, and it said it couldn't do that.)

Question: Is there a way to get Poe.com's ChatGPT to consistently reply "I don't know" or "I can't do that" instead of making stuff up?

5
  • Yes ask it to tell you how to do something illegal. Try "How do I make ricin?"
    – user13267
    Jul 20, 2023 at 1:41
  • Although that's probably not the type of question for which you are looking for a "I don't know" answer from chatgpt. FWIW, Bing chat is much better at letting you know it doesn't know something.
    – user13267
    Jul 20, 2023 at 1:43
  • As in this about using Poe.com ChatGPT and not OpenAI ChatGPT, I suggested an edit. Please checkout "chatgpt" is the top tag. Do we want it to be used only on questions about the cross-platform application software from OpenAI?
    – Wicket
    Jul 25, 2023 at 7:16
  • Is this splitting hairs? I wonder if there's a better way of dealing with the problem that, strictly speaking, poe.com's ChatGPT isn't ChatGPT. (Isn't it just a different front end?) Poe.com's ChatGPT seems fairly convinced it's ChatGPT: Yes, I am still ChatGPT even when I am accessed through the poe.com interface. Regardless of the platform or interface, my functionality as an AI language model remains the same. Jul 25, 2023 at 7:33
  • Despite the non-deterministic nature of GPT responses, even OpenAI "ChatGPT API" doesn't behave in the same as the OpenAI ChatGPT web app. First, the developer must set all API parameters, while OpenAI sets the parameters for the web app. As far as I know, they have not disclosed those parameters. I'm not familiar with Poe.com, but I think that ChatGPT is used in a very loose way, that might not be proper of a library of hight quality questions and answers.
    – Wicket
    Jul 25, 2023 at 7:40

1 Answer 1

1

I'm not familiar with the specifics of Poe.com ChatGPT. Assuming that it behaves in the same way that OpenAI ChatGPT, from my answer to How can I encourage ChatGPT to give me more concise information?

Remember that ChatGPT is designed for handling a conversation and that it will remember, with some limitations, previous messages. Considering this, you might split your request into multiple parts, using one or multiple parts to give your "answer" requirements.

Below I will use "ChatGPT" as a generic name and OpenAI ChatGPT when referring to a specific implementation.

If ChatGPT doesn't respond satisfactorily, tell it what is wrong with the answer provided. I.E., you might tell ChatGPT, remove the repeated country names. The more specific you are about your answer requirements, might help to avoid having a lot of back and forth. You might have to write a prompt including multiple statements. Consider including in your requirements the tone and writing style. Some people have suggested instructing ChatGPT to "act as" a certain profile.

If you wish, you can instruct ChatGPT to reply in the tone of a compassionate, young female tour guide conversing with a 5-year-old child during a visit to the United Nations headquarters in Geneva. Also, tell that the country names should be real countries, with UN official recognition in a specific year, mentioning that if there aren't enough countries, it should say I don't know.

It's worth noting that ChatGPT responses will be based on the whole conversation and that responses might be different in every new conversation, even repeating the same prompts.

Also, you might want to try the "Regenerate Response" button. There are chances that the regenerated response gives you a more satisfactory answer.

Don't forget that ChatGPT might hallucinate and assume that anything the user says is true, so if you tell it that one of the country names is wrong, it might remove it even if the name is correct.


Here is a "conversation" that I just had with OpenAI ChatGPT web app.

Me:

Give a list of 10 unique countries starting with Z.

ChatGPT:

Sure! Here is a list of 10 unique countries that start with the letter "Z":

  1. Zambia
  2. Zimbabwe
  3. Zaire (now known as the Democratic Republic of the Congo)
  4. Zanzibar (semi-autonomous region within Tanzania)
  5. Zealandia (a submerged continent and microcontinent)
  6. Zambia- Zimbabwe (the UN mediator suggested name for a hypothetical unified country from Zambia and Zimbabwe)
  7. Zaire (Another name for the Democratic Republic of the Congo, mentioned again as it used to be known by this name)
  8. Zanzibar (Mentioned again, as it has some degree of autonomy within Tanzania)
  9. Zambia-Zimbabwe (Mentioned again, as a proposed unified country between the two)
  10. There are no other recognized sovereign countries starting with the letter "Z" as of my last update in September 2021.

Please note that Zealandia is a submerged continent and not a fully recognized sovereign nation. The last three entries are variations or hypothetical proposals related to already mentioned countries.

Note: See the update of July 25th, 2023, to my answer to What's the difference between the terms "ChatGPT", "GPT", "GPT-4", and "GPT-3.5-turbo"?.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.