Hallucination

Given a context, a generated text and optionally a user query, we are able to detect 2 different types of model hallucinations: intrinsic and extrinsic. Intrinsic Hallucinations are hallucinations where the models manipulate original information and slightly misrepresent true facts. For example, “Eiffel Tower is the tallest building in France'' or “The first man on the moon was Yuri Gagarin”. Extrinsic Hallucinations are the ones where models completely fabricate new false information. This can’t be inferred from or traced back to original sources. For example, “Cristiano Ronaldo is a Cricket player”. The "is_hallucinated" field indicates whether the "generated_text" (passed in the input) is hallucinated. A top level "score" field indicates if the entire paragraph contained any hallucinations. The score is a probability measure of how hallucinated the text is compared to the context. A score >= 0.5 can be classified as a hallucination. We also provide sentence level scores to help with explainability.

Example Request

[
  {
    "context": "Paul Graham is an English-born computer scientist, entrepreneur, venture capitalist, author, and essayist. He is best known for his work on Lisp, his former startup Viaweb (later renamed Yahoo! Store), co-founding the influential startup accelerator and seed capital firm Y Combinator, his blog, and Hacker News.",
    "generated_text": "Paul Graham has worked in several key areas throughout his career: IBM 1401: He began programming on the IBM 1401 during his school years, specifically in 9th grade. In addition, he has also been involved in writing essays and sharing his thoughts on technology, startups, and programming.",
    "config": {
      "hallucination": {
        "detector_name": "default"
      }
    }
  }
]

Example Response

[
  {
    "hallucination": {
      "is_hallucinated": "True",
      "score": 0.7407,
      "sentences": [
        {
          "score": 0.7407,
          "text": "Paul Graham has worked in several key areas throughout his career: IBM 1401: He began programming on the IBM 1401 during his school years, specifically in 9th grade."
        },
        {
          "score": 0.03326,
          "text": "In addition, he has also been involved in writing essays and sharing his thoughts on technology, startups, and programming."
        }
      ]
    }
  }
]

Example (Synchronous detection)

The below example demonstrates how to use the hallucination detector in a synchronous manner. We also support asynchronous computation. Refer to the evaluation and monitoring section for details.

Python

from aimon import Detect

# See analyze_prod for the asynchronous version
# that can be used for continuous monitoring
detect = Detect(values_returned=['context', 'generated_text'], config={"hallucination": {"detector_name": "default"}})

@detect
def my_llm_app(context, query):
    generated_text = my_llm_model(context, query)
    return context, generated_text

TypeScript

import Client from "aimon";

const aimon_client = new Client({
  authHeader: `Bearer API_KEY`,
});

const detectMetrics: any = async (sourceText: any) => {
  // Split the source text
  const textSplitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000 });
  const docs = await textSplitter.createDocuments([sourceText]);
  const contextDocs = docs.map((doc) => doc.pageContent);

  // Summarize the texts
  const llm = new OpenAI({ temperature: 0, openAIApiKey: openaiApiKey });
  const chain = loadSummarizationChain(llm, { type: "map_reduce" });
  const output = await chain.invoke({
    input_documents: docs,
  });

  const hall_config = {
      hallucination: {
      detector_name: "default"
    }
  };

  // Detect quality of the generated output using AIMon
  const detectParams: Client.InferenceDetectParams.Body[] = [
    {
      context: contextDocs,
      generated_text: output.text,
      config: hall_config
    },
  ];

  // Call the API
  const aimonResponse: Client.InferenceDetectResponse =
    await aimon_client.inference.detect(detectParams);
};