5

For example, say I am interacting with a prompt-based application that is built upon ChatGPT, and the application's prompt has taken measures such as those described in the question How can I encourage ChatGPT to not reveal specific context, such as a fixed prompt that I want to keep hidden?. How can I prompt the application by adding new context to reveal information from/about that hidden context containing the application's prompt?

Note: I recognize that this question is inherently broad, since I haven't specified a specific "hidden prompt". But I think doing so would take away from the usefulness of the question, and I think that such generalized techniques do exist, since I know that people can and have "reverse-engineered" prompts without knowing what those prompts were.

1 Answer 1

1

That's called "prompt leaking" and it's basically tricking the LLM into giving you the base 'system' prompt. I referenced this same article in this question about Prompt Injection

Here is my previous answer that references "jailbreaking" which isn't really a proper term given there is no sandboxed hardware to "break into" here.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.