Implement verifiers env for K-QA
Implement a verifiers environment for the K-QA datasets. Use the authors’ question, system, and LLM-as-judge prompts if provided. Only create a training dataset if a training split exists.
Paper: https://arxiv.org/abs/2401.14493
Dataset / Project: https://github.com/Itaymanes/K-QA
Format & Env
- Single-turn open-ended clinical Q&A:
- Use
vf.SingleTurnEnv + FActScore +vf.JudgeRubric()
Implement verifiers env for K-QA
Implement a verifiers environment for the K-QA datasets. Use the authors’ question, system, and LLM-as-judge prompts if provided. Only create a training dataset if a training split exists.
Paper: https://arxiv.org/abs/2401.14493
Dataset / Project: https://github.com/Itaymanes/K-QA
Format & Env
vf.SingleTurnEnv+ FActScore +vf.JudgeRubric()