3

In-context learning is a prompt engineering technique where natural-language demonstrations are provided as part of the prompt (source).

Few-shot prompting is about providing a few examples in the prompt (source).

These seem like the same concept to me. What is the difference between in-context learning and few-shot prompting?

Is few-shot prompting a subset of in-context learning, or are they just different names for the same thing?

1 Answer 1

2

In-context learning doesn't imply one only give a few examples, unlike few-shot prompting. That's the only difference I can see, and many people assume that in-context learning uses only a few examples anyway, e.g. see A Survey on In-context Learning:

Following the paper of GPT-3 (Brown et al., 2020), we provide a definition of in-context learning: Incontext learning is a paradigm that allows language models to learn tasks given only a few examples in the form of demonstration.

By the way, the same paper explains the difference between few-shot learning and in-context learning:

Few-shot learning is a general machine learning approach that uses parameter adaptation to learn the best model parameters for the task with a limited number of supervised examples (Wang and Yao, 2019). In contrast, ICL does not require parameter updates and is directly performed on pretrained LLMs.

2
  • 2
    Thanks for the answer! Per the second half on few-shot learning, note that few-shot learning is different than few-shot prompting Jan 11 at 19:55
  • @IntrastellarExplorer yes indeed, different Jan 11 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.