Analysis Overview
Deep-dive into your AI agent performance with advanced analysis tools.
Analysis Capabilities
TuringPulse provides powerful analysis tools to help you understand why your AI agents behave the way they do and how to improve them.
Root Cause Analysis
When anomalies or drift are detected, TuringPulse automatically performs root cause analysis to identify potential causes:
What's Analyzed
- Fingerprint Changes - Prompt, config, or structure changes
- Deployments - Recent code deployments
- Input Distribution - Changes in input patterns
- External Dependencies - Third-party API changes
- Resource Utilization - Memory, CPU, rate limits
Attribution Scores
Each potential cause is assigned an attribution score (0-100%) indicating how likely it is to be the root cause.
💡
Automatic Correlation
TuringPulse automatically correlates anomalies with recent changes. No manual investigation required for common issues.
Correlation Insights
Discover hidden relationships between metrics and behaviors:
Types of Correlations
- Metric Correlations - How metrics relate to each other
- Input-Output Correlations - How inputs affect outputs
- Temporal Correlations - Time-based patterns
- Cross-Workflow Correlations - Dependencies between workflows
Use Cases
- Identify which input types cause high latency
- Find correlations between token usage and accuracy
- Discover time-of-day performance patterns
- Understand cascading failures across workflows
Incident Management
Track and resolve production issues systematically:
Incident Lifecycle
- Detection - Automatic or manual incident creation
- Triage - Assign severity and owner
- Investigation - Root cause analysis and correlation
- Resolution - Fix and verify
- Post-mortem - Document learnings
Incident Sources
- KPI threshold breaches
- Drift detection alerts
- Anomaly clusters
- Error rate spikes
- Manual creation
Evaluations
Score and compare agent outputs using various metrics:
Evaluation Types
- Heuristic Metrics - ROUGE, BLEU, JSON validity, etc.
- LLM-as-a-Judge - Use LLMs to score outputs
- Custom Metrics - Define your own scoring functions
Evaluation Modes
- Online - Evaluate traces in real-time
- Offline - Batch evaluate historical traces
- A/B Testing - Compare different configurations
Navigation
Access Analysis from the main sidebar:
- Metrics Explorer - Query and visualize metrics
- Root Cause Analysis - View RCA reports
- Correlation Insights - Explore correlations
- Evaluations - Run and view evaluations
Next Steps
- Root Cause Analysis - Deep dive into RCA
- Correlation Insights - Discover patterns
- Incident Management - Handle issues
- Evaluations - Score outputs