It seems that languge models are bad for law-related questions. For instance when you ask a LLM (assuming that it doesn't search the web for up-to-date information) about whether a thing that is obviously in the public domain can be used for any purpose, it will (sometimes) answer that it is not necessarily free to use.
-
The limitations in addressing legal questions stem from the fact that a language model requires specialized training from legal databases to grasp the intricacies of the law fully. While it can leverage the context provided in a question to some extent, the complexities and nuanced interpretations inherent in legal matters often surpass the current training level of language models. It's unlikely that LLM AI will replace attorneys in the near future, given the intricate nature of legal expertise and the ongoing challenges in training models to encompass such complexity.– IT Thug NinjaCommented Dec 25, 2023 at 1:06
Add a comment
|
2 Answers
I don't think that LLMs are inherently "bad" at answering legal questions. If you prompt with enough context for your question, you'll often get better (ie: more specific and accurate) answers. I believe that some models,Like ChatGPT, are trained to be extremely careful with answering legal questions for plenty of good reasons. Laws differ by state and country. Laws change daily. Arguments can be made one way or another based on the nuances of the case. LLMs have no legal authority to give advice. They can hallucinate and give incorrect information.
You will always see some "hedging" from any LLM particularly from a public company. If it appears the LLM is giving legal advice, that might cause someone to rely on that advice even if it is wrong. So, you're never going to get a definitive answer. Further, it's difficult to answer any legal question without tons of context about the jurisdiction (local, state, federal in the US for example).
If you want to work more closely with an LLM that will give a direct answer, I'd say host one your own and do some custom training on your particular jurisdiction.
