Official Codebase for "METAL: Towards Multilingual Meta-Evaluation" (published in Findings of ACL: NAACL 2024)
-
Updated
Jun 18, 2024 - Python
#
meta-evaluation
Here are 6 public repositories matching this topic...
Automatic validation of grammatical-error-correction metrics using MAEGE lattices (ACL 2018).
-
Updated
Jul 3, 2023 - Python
A local LLM benchmarking framework designed to evaluate model performance across multiple backends (LM Studio, Ollama, etc.), including metrics for speed, quality, and instruction adherence. Supports structured test runs, result analysis, and reproducible evaluations.
docker benchmarking model-evaluation model-evaluation-metrics llm prompt-testing local-llm llm-tools ollama llm-benchmarking meta-evaluation prompt-test openai-compatible llm-benchmark llm-benchmarks
-
Updated
May 3, 2026 - HTML
A meta-eval that grades the quality of other evals: discrimination, contamination, difficulty spread, statistical power
-
Updated
Jul 7, 2026 - Python
Meta-evaluation system that detects bias, inconsistency, and hallucinated rationales in LLM judges using PubMedQA
-
Updated
Apr 6, 2026 - Python
Cross-Family LLM-Judge Agreement for Institutional RAG: 5 families, 9 judges. Validated on TREC RAG 2024 (kappa=0.4941) + BEIR scifact.
information-retrieval cohens-kappa inter-rater-agreement beir retrieval-augmented-generation llm-as-judge meta-evaluation trec-rag institutional-search
-
Updated
May 17, 2026 - Python
Add this topic to your repo
To associate your repository with the meta-evaluation topic, visit your repo's landing page and select "manage topics."