Context Relevance
LLM Evaluation frameworks like RAGAS depend heavily on off-the-shelf LLMs (zero, single, or few shot methods). This technique suffers from variance, inconsistency, subjectiveness, and cost inefficiency.
Read more about pros and cons of LLM Judges here.
AIMon has built purpose-built and customizable relevance graders that outshine large off-the-shelf LLMs and enable you to use them inline or for offline evaluations.
We are currently running them in beta. Please reach out if you are interested in trying them.