Drift Detection

What is Drift?

Drift occurs when your AI agent's behavior changes over time. This can happen due to:

• Model updates or version changes
• Changes in input data distribution
• Prompt modifications
• API or dependency updates

TuringPulse continuously monitors your agents and alerts you when significant behavioral changes are detected.

How It Works

TuringPulse establishes baselines for your key metrics and monitors for deviations:

1. Baseline Creation - Compute statistical profiles from historical data
2. Continuous Monitoring - Compare new runs against baselines
3. Anomaly Detection - Flag significant deviations
4. Alerting - Notify when drift exceeds thresholds

Configuration

# Drift detection is configured via the TuringPulse UI or API.
# Navigate to Controls > Drift Rules to create rules.
#
# You can also configure drift rules via the REST API:
#
# POST https://api.turingpulse.ai/api/v1/config/drift-rules
# {
#   "name": "Latency drift monitor",
#   "metric_name": "latency_ms",
#   "detection_methods": ["jsd", "psi", "ks_test"],
#   "reference_window_size": 500,
#   "detection_window_size": 50,
#   "jsd_threshold": 0.1,
#   "severity": "warning"
# }
#
# Drift detection runs automatically on instrumented workflows.
# No SDK-side configuration is needed beyond standard instrumentation.

from turingpulse_sdk import instrument

@instrument(name="my-workflow")
def my_agent(query: str):
    return process(query)

Monitored Metrics

Severity Levels

• Info - Minor deviation, no action needed
• Warning - Significant change, investigate recommended
• Critical - Major drift, immediate attention required

Best Practices

• Start with default sensitivity and adjust based on your use case
• Ensure adequate baseline data before enabling drift detection
• Review drift events regularly to tune thresholds
• Set up alert channels for critical drift events
• Correlate drift with deployments and model updates