Prompts that reliably result in unreliable completions

Question

Assume you want to demonstrate how ChatGPT can give quite different answers to the same question, either different in the focus of the answers or even with the same focus but contradictory statements.

I'd like to compose a prompt that would rather reliable yield this: unreliable answers. Most of the prompts I normally use yield rather cross-consistent and somehow reliable results -- and only by accident and every now and then inconsistent answers (which I missed to archive appropriately). I have no idea how to design such an adversarial prompt systematically. Does anyone have an idea for a strategy how to create one? Or even better: a specific example?

Answer 1

I have no idea how to design such an adversarial prompt systematically. Does anyone have an idea for a strategy how to create one?

First, it could be useful to understand the source of randomness in ChatGPT (temperature + understand how it works). Second, one could look at log probabilities associated with the most likely tokens for each output token, and find a prompt where the top 2 most likely tokens have almost the same log probabilities.

want to demonstrate how ChatGPT can give quite different answers to the same question,

Some papers report such variance e.g. https://arxiv.org/pdf/2210.14986v1.pdf Appendix "F.3 Variance over API runs".

Answer 2

Search the Web for controversial topics that have a balanced representation of adversarial stands. It doesn't matter if they have occurred at the same time.

Try asking about names having multiple meanings, I.E., Venus, Mars, Mercury, are used in mythology, astronomy, astrology, names of real people.