# Evaluate and investigate AI failures

Editorial revision: 2026-10-28. Product documentation dates and access limits are shown on individual listings.

> Which tools help trace, evaluate and red-team an AI application?

Tracing, evaluation and adversarial testing solve different jobs. Choose tools that expose the evidence you need, then define what failure looks like for your actual application.

A hand-picked shortlist from the Agent Search Engine index. Every pick is a real listing; the repository, pricing-category and activity figures are recorded catalogue fields, not performance tests. Browse on the web: https://agentsearchengine.app/use/evaluate-and-secure-ai

## Editorial starting points

### Traces and evaluation datasets: LangSmith
- Developer allowance; paid seats, usage and service entitlements · Infrastructure · License not checked
- **Setup:** Developer — application instrumentation
- **Why:** LangSmith combines application traces, datasets and evaluations. Cloud, hybrid and self-hosted entitlements differ; inspect trace retention, model calls and usage billing.
- [Capability source](https://docs.langchain.com/langsmith/observability)
- [listing](https://agentsearchengine.app/agents/langsmith) · [official](https://www.langchain.com/langsmith)

### Self-hostable observability: Arize Phoenix
- Self-host software; infrastructure and evaluator/model costs · Infrastructure · 12k★ · Python · Source available · recorded license: Elastic-2.0 · repository push 2 days ago
- **Setup:** Developer — runtime / storage setup
- **Why:** Phoenix provides tracing, datasets and evaluator workflows. Its current server terms are ELv2, while infrastructure and evaluator/model calls carry separate costs and data routes.
- [Capability source](https://arize.com/docs/phoenix)
- [listing](https://agentsearchengine.app/agents/arize-phoenix) · [official](https://arize.com/docs/phoenix)

### Configured evaluation and red teaming: Promptfoo
- Community software; model/target costs; paid enterprise options · Infrastructure · Open source · recorded license: MIT
- **Setup:** Developer — target / provider configuration
- **Why:** Promptfoo tests configured prompts and application targets. A locally running scanner can still use remote attack or target models; review authorization, model costs and report contents.
- [Capability source](https://www.promptfoo.dev/docs/red-team/quickstart/)
- [listing](https://agentsearchengine.app/agents/promptfoo) · [official](https://www.promptfoo.dev/)

## What to watch for
Start with representative successes, failures and adversarial cases. Use explicit assertions and human review for consequential outcomes. Observing a trace is not blocking an action, and a passing test set is not a security certification. Red-team only targets you are authorized to test.

---
Source: [Agent Search Engine](https://agentsearchengine.app) — more task-first guides: https://agentsearchengine.app/use