While I understand that GPT-4 does not use logic in the same way that a human does, is there some way to have ChatGPT "explain" itself?
Yes and no.
You can certainly prompt ChatGPT (or any other LLM) to produce an explanation for why it's generating an output. In some sense, this could be seen as "the" explanation for why it generated the output, since the request will tend to change the output so that it matches the explanation.
In another sense, the explanation is complete nonsense: it bears no relationship to the actual mechanism of generating an output. Rather, it will be a reflection of "think step by step" or "explain your reasoning" examples in the training data. If you want to know how a LLM generates an output, you need to analyze the underlying neural network as it's running, something we don't know how to do for any but the most trivial networks.
When presented with a prompt or question, the LLM analyzes the input and makes probabilistic predictions about which words are most likely to follow based on patterns and correlations learned during training. The training data includes millions of passages of text from books, websites, and conversations that are used to build statistical models of language.
So for this question about explaining answers, the LLM can not internal reason step-by-step about how to answer. Rather, it generated an answer by sequentially predicting the most likely next word/token that would result in a coherent explanation, based on the language patterns it had learned.
Thus, in general, no LLM can reason or explain its own behavior of analyzing the responses it makes because it has no internal symbolic logic. They can be asked to apply self-criticism to a given response. And by critiquing the explanation, they can find holes in transparency, such as identifying places where the probabilistic predictions are more hidden. The training has allowed them to produce reasonable explanations, but they don't have a real understanding of the internal mechanisms.
GPT can produce a simulacrum of an explanation, but this is not necessarily a truthful explanation of how it actually reached its answer.
By way of illustration, here's GPT-3 responding to a long multiplication problem. Previously in this session I had asked it to "show working", so it gives me a worked answer here.
39483 is the correct answer to the multiplication, but there's an obvious error in the working. 123 times 1 is not 963! Adding 963 to 2460 to 36900 would give 40323. But instead it shows the sum of these as 39483, the correct answer to the original question.
By trying the same approach with different 3-digit numbers you can get many examples of the same behaviour: the final answer is often (not always) correct even with incorrect working along the way.
Unless one supposes that by great good fortune its errors keep cancelling out to yield a correct answer, this should make it pretty clear that GPT isn't actually getting its answer by following the working it presents. If anything, it appears to be getting the final answer first (I suspect by learning patterns along the lines of "12* times 32* often gives an answer beginning in 39, *23 times *21 often gives an answer ending in 483") and then interpolating the requested working, rather unreliably.
This example was generated via GPT-3 rather than GPT-4. It may be a little harder to catch GPT-4 out with this particular approach, but I'm not aware of any reason to believe the changes between GPT-3 and GPT-4 would have made its explanations more truthful.
Yes: ask it to "think step by step and explain its reasoning".
This is a common technique to make it produce more sensible answers, rather than jump to wrong conclusions. It's called
Chain of Thought (CoT) prompting. You can read articles about it eg at