-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Description
Describe the bug
A clear and concise description of what the bug is.
On calling LLM critic validator, sometimes we get responses like -
{
"success": false,
"data": {
"response_id": "11c2d6b7-6f2e-4d09-a78a-23b3d40aed6b",
"rephrase_needed": false
},
"error": "The response failed the following metrics: ['query']."
}
And sometimes like this -
{
"success": false,
"data": {
"response_id": "4985fe71-39fb-42d4-977c-d2d27a16f2d6",
"rephrase_needed": false
},
"error": "The response is missing or has invalid evaluations for the following metrics: ['query']."
}
Investigate why that happens and override the errors for the validator
To Reproduce
Steps to reproduce the behavior:
- Go to '...'
- Click on '....'
- Scroll down to '....'
- See error
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels