5

Meta released their open source LLM, Llama 2. Does Llama 2 have real time data or this was trained on data available till some date?

1
  • Welcome to GenAI. Should this question be posed to the community? Have you attempted to find the answer on the Meta (Meta Platforms Inc.) announcement or Llama 2 documentation?
    – Wicket
    Commented Aug 1, 2023 at 17:14

2 Answers 2

8

According to Llama 2's model card:

The pretraining data has a cutoff of September 2022, but some tuning data is more recent, up to July 2023

Llama 2 is provided as a downloadable model, rather than an online service, so the model won't be continuously updated with real-time data. There are however ways you could teach it new knowledge without re-training from scratch.

2
  • Could you please explain what does “tuning data” mean in this context? Commented Aug 3, 2023 at 16:29
  • 1
    @A.L LLMs are pre-trained with a huge quantity of arbitrary text on a self-supervised task (generally predicting the next token). To specialise it to a desired task, such as helpfully answering user questions, the model can then be "fine-tuned" by training on a smaller quantity of data (maybe labelled/curated) - in Llama 2's case "publicly available instruction datasets, as well as over one million new human-annotated examples". Tuning data would be that smaller set of task-specific data.
    – SirBenet
    Commented Aug 3, 2023 at 16:37
2

Most likely pre-trained however, some interfaces allow input from web searches e.g. https://huggingface.co/chat/ allows to search web with your input as well as utilise it's pre-trained data.

I also found this: https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md

Which states: "Data Freshness The pretraining data has a cutoff of September 2022, but some tuning data is more recent, up to July 2023."

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.