Skip to main content

Datadog

AIMon integrates with Datadog to forward evaluation results and logs, helping you monitor your LLM application quality directly from your observability stack.

At the moment, AIMon sends:

  • Evaluation Metrics (e.g., hallucination score, instruction adherence, etc.)
  • One Log Entry per Evaluation (with metadata like user query, generated output etc.)

This setup is ideal for teams that want lightweight visibility into model behavior with minimal configuration.

How to Connect to the Datadog Platform​

  1. Log in to your account on the AIMon app.
  2. Navigate to the Integrations tab.
  3. Toggle the Datadog integration on.
  4. When prompted, complete the Datadog login to authorize the connection.

Once connected, evaluation results will be forwarded automatically to your Datadog account.

Organization & Access Control​

  • Every AIMon user belongs to a company, which determines how integrations and access are shared.
  • Users can set or change their company from the My Account section.
  • Only the first user from a company to enable the Datadog integration can create a new Datadog API key (as required by Datadog).
  • All other users in the same AIMon company will automatically inherit access to that API key.
  • Access to the key on the Datadog side is controlled by your organization's permission settings, including which individual users or roles are allowed to view and manage it.

This ensures a secure, shared integration across your team — with no need for each user to configure credentials individually.

How to Disconnect from the Datadog Platform​

  1. Log in to your account on the AIMon app.
  2. Navigate to the Integrations tab.
  3. Toggle the Datadog integration off.

This will immediately stop the flow of data to your Datadog account.

Have questions about company access or Datadog integration setup? Contact AIMon Support at info@aimon.ai.