5

Assume you want to demonstrate how ChatGPT can give quite different answers to the same question, either different in the focus of the answers or even with the same focus but contradictory statements.

I'd like to compose a prompt that would rather reliable yield this: unreliable answers. Most of the prompts I normally use yield rather cross-consistent and somehow reliable results -- and only by accident and every now and then inconsistent answers (which I missed to archive appropriately). I have no idea how to design such an adversarial prompt systematically. Does anyone have an idea for a strategy how to create one? Or even better: a specific example?

2
  • To provide specific examples, please provide more details. Are you interested in common knowledge or knowledge about a specific domain? Are you interested in "reliable" from an academic, business, technical or general public common sense?
    – Wicket
    Aug 21, 2023 at 15:31
  • @Wicket: With "reliable" I mean that the model answers in a compatible way across conversations: on the same topic and without contradictions. The domain doesn't matter, but a scientific example would be nice. Aug 21, 2023 at 20:26

2 Answers 2

3

I have no idea how to design such an adversarial prompt systematically. Does anyone have an idea for a strategy how to create one?

First, it could be useful to understand the source of randomness in ChatGPT (temperature + understand how it works). Second, one could look at log probabilities associated with the most likely tokens for each output token, and find a prompt where the top 2 most likely tokens have almost the same log probabilities.

want to demonstrate how ChatGPT can give quite different answers to the same question,

Some papers report such variance e.g. https://arxiv.org/pdf/2210.14986v1.pdf Appendix "F.3 Variance over API runs".

1

Search the Web for controversial topics that have a balanced representation of adversarial stands. It doesn't matter if they have occurred at the same time.

Try asking about names having multiple meanings, I.E., Venus, Mars, Mercury, are used in mythology, astronomy, astrology, names of real people.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.