Confirming that an answer was generated was not enough to evaluate a legal chatbot. This case reviewed the responses of legal chatbot, company C, by question type and by grounding, and left a record of which areas needed further improvement.
Type a question and a sentence comes back. The wording is natural, and it looks as though the intent of the question was understood. Whether that sentence is a trustworthy answer, however, is a different matter.
The verification task for legal chatbot, company C, began with this gap. The service used a large language model to answer legal questions, but the same question could produce different results. A standard was needed to separate whether an answer came out from whether the answer was right.
In this 2025 project, the content of the responses, not just the behavior of buttons and screens, was made the object of verification. The focus was confirming not that the sentence read naturally but that the answer met the agreed criteria and its grounding.
Before reading the answers, we defined what counts as an answer
Reading responses without evaluation criteria lets the result vary with the reviewer's impression. It is easy to take a long, detailed answer for a good one, or to mistake a confident tone for accuracy.
The project agreed on three verdict criteria: accuracy, the presence or absence of fabricated content presented as fact, and agreement between the cited grounding and the answer. What to check and record when looking at a response was settled first.
The fact that a source is given and the fact that the source supports the answer also have to be kept apart. Mentioning a source is not sufficient on its own. Whether the content of that source and the response say the same thing has to be examined as well.
Evaluation criteria turn vague expectations of a model into questions that can be reviewed. It had to be possible to ask not "does it answer cleverly" but "is there grounding for saying this in response to this question".
Questions to answer and questions to stop at were included together
As important as preparing many questions is the kind of question prepared. Collecting only cases that can be answered risks missing the problem of the model producing an answer even when grounding is lacking.
The legal query scenarios were organized by type. Questions that require an answer and questions that must not be answered were both put in the verification scope.
The purpose of the latter is not to make the chatbot say less. The ability to distinguish what it knows from what it cannot confirm is considered part of response quality. A service that answers every question confidently and a service that distinguishes the conditions under which it may answer have to be evaluated differently.
This distinction affects the release decision too. The fact that many questions are answered well cannot offset a dangerous answer that recurs in one specific type. That is why results by question type are needed, not only an overall average.
We checked whether the judgment held when the wording changed
Even when asking about the same thing, users do not type the same sentence every time. They may change the order of the question or express the same meaning in different words.
The project re-ran questions with the wording changed while the meaning was kept. Instead of looking only at a response that came out well once, it examined whether the content and grounding of the answer shifted with differences in expression.
A changed wording does not require an identical sentence in return. What is being checked is not the reproduction of fixed phrasing but the consistency of the judgment. Whether the model gives conflicting answers to the same premise, or changes its grounding when asked again, has to be distinguished.
This approach exposes the difference between demonstrating one successful conversation and evaluating readiness for real questions.
Every response was recorded with its verdict and the reason
The verification results were not recorded as pass or fail alone. The verdict and its basis were recorded for each response by question type, and the areas that could not be checked were separated out.
The purpose of the record is not to fix someone's evaluation in place. It is so that another reviewer can understand what was looked at to reach that judgment, and so that the same questions can be revisited after the model or the prompts change.
An inaccurate answer and an answer with insufficient grounding, for example, may look similar on the surface yet need review for different reasons. Only with the question where the problem appeared and the reason for the verdict together can the direction of improvement be discussed. A bare count of failures makes it hard to decide what to fix first.
The final results included, alongside the question types verified, the scope that could not yet be judged. This was so that untested areas would not be treated as areas without problems.
The deliverable was a map of what to improve, not a single accuracy number
The project organized the response evaluation by query type and the points where hallucination occurred. As it became clear which questions made the answer move, the basis was in place for discussing the release review and the priorities for improving the model and the prompts.
These results have to be interpreted within the scenarios and conditions used for verification. When the model or the prompts change later, the affected questions have to be checked again, and question types that newly appear in real use have to be added to the evaluation scope.
The quality of a legal chatbot cannot be explained by its ability to fill the answer box. A process is needed to confirm which questions it answers on what grounding, and in which situations it should withhold an answer. Company C's project left that judgment as a record that can be reviewed again and again.
