22

This is anecdotal, but in my use of ChatGPT (site and API) and Bard, in cases when I ask a generic coding question that does not specify the language, the answer is often provided in Python.

Is this my hallucination here? If this is a thing, then why is this happening?

5
  • 3
    Welcome to this fascinating experiment. Despite this question having a score of +10 it looks like it needs more work. I don't think this post also shows the tone and high quality that will help make this community attractive to subject matter experts. Rel. We are still looking to define the tone and quality standards of GenAI SE among other critical attributes
    – Wicket
    Commented Jul 28, 2023 at 1:46
  • The other two questions that use code are blatantly off-topic. Do you envision more questions on GenAI Stack Exchange having this tag? What do you think the tag excerpt for this tag should say?
    – Wicket
    Commented Aug 3, 2023 at 21:26
  • @Wicket that seems like more of a meta question to me. But in short, I would think that a code tag would be appropriate for questions for GenAI interactions related to coding. Especially when it comes to strategies for things like setting up prompts that will generate or evaluate code, fix or not fix logic errors, address some part of code (example: misspelled words in comments) while not addressing others (changing code logic) in code submitted wit a prompt. Defining expected code output formats, avoiding hallucinations related to code, etc. Commented Aug 4, 2023 at 13:18
  • Thanks Yaakov. I asked you here because it's looks that this post is the first using the code (and python) :) Because the some of the other questions using this tag are VLQ and off topic I was wondering if code should be "burninated" ( removed but not blocked).
    – Wicket
    Commented Aug 4, 2023 at 20:45
  • I asked Bard "what is 1234567890 plus 123567890?" Python code was given as part of the answer for verification.
    – MT1
    Commented Aug 5, 2023 at 6:15

5 Answers 5

10

The original OpenAI codex model was trained predominantly on Python code—that is almost certainly why by default it will produce Python. I imagine Bard was trained on similar code.

1
  • 2
    This answer can be improved by providing reference of the claim.
    – holydragon
    Commented Jul 31, 2023 at 3:02
10

There are a lot of different indices to measure code popularity/usage, none of which are perfect. Examples on Wikipedia.

Python most frequently rates top in these indices. Therefore, it's a reasonable choice for a chatbot to use if a preference is not specified.

5

Is it actual Python or Python-like?

It could be there is a lot of Python code out there or it could be that pseudocode can often look quite like Python as they both use indenting rather than lots of brackets. By not specifying a language, you could be left with what is common across languages rather than the presence of some language-specific constructs.

References

1
  • 1
    Almost every time that this has happened to me, the LLM response identifies python explicitly as the language in the answer. Commented Jul 28, 2023 at 11:32
2

Because Python is the English of coding? Possibly this particular Stochastic Parrot has been trained on Python code.

In this article there is a suggestion that many computational linguists act as if English were the standard language for linguistic studies.

In 2019, [Emily Bender] raised her hand at a conference and asked, “What language are you working with?” for every paper that didn’t specify, even though everyone knew it was English. (In linguistics, this is what’s called a “face-threatening question,” a term that comes from politeness studies. It means you’re rude and/or irritating, and your speech risks lowering the status of both the person you’re speaking to and yourself.) Carried inside the form of language is an intricate web of values. “Always name the language you’re working with” is now known as the Bender Rule.

0

Artificial Intelligence and more specifically GenAI tools that are based on Large Language Models like ChatGPT and Bard have shown biased responses.

This occurs because the sources are biased. While training might help handle certain bias it looks that there is still a lot to be developed on this matter.

Knowing about LLM bias is helpful to prevent get surprised by GenAI responses.

In this specific case the bias to Python might be explained because it's a very popular programming language having a large community collaborating around it on public spaces like Stack Overflow, GitHub among others.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.