Anomaly Detection in CSV Data: Let AI Find the Outliers

Anomaly detection sounds like a machine learning problem. And it can be — if you want to spend two weeks tuning an Isolation Forest or building a custom autoencoder for your specific data distribution. But most of the time, the question is simpler: "Is anything weird in this data?"

A spike in error rates. A customer with 10x the average order value. A region that suddenly stopped reporting. A negative number in a column that should only be positive. These are the anomalies that matter to most teams, and they don't require a PhD to find. They require someone — or something — to actually look.

This article shows how to use the DataStoryBot API to detect anomalies in CSV data: upload your file, steer the analysis toward outliers, and get back a narrative explaining what's abnormal and why it might matter.

What Counts as an Anomaly

Before throwing data at an API, it helps to know what you're looking for. Anomalies in tabular data fall into a few categories:

Point anomalies — individual values that are far outside the expected range. A transaction of $847,000 when the average is $2,300. A server response time of 45 seconds when the p99 is 800ms. These are the classic outliers.

Contextual anomalies — values that are normal in one context but abnormal in another. $50,000 in revenue is fine for Q4. It's alarming for a random Tuesday in February. Context is usually temporal (time of day, day of week, season) or categorical (region, product line, customer segment).

Collective anomalies — groups of data points that are individually normal but collectively unusual. Five servers each running at 75% CPU is fine. All five spiking to 75% at the same time when they usually alternate — that's a pattern worth investigating.

Structural anomalies — data quality issues that indicate something broke upstream. Null values in a column that's always populated. Duplicate rows. Date formats that suddenly change. These aren't statistical outliers — they're pipeline failures masquerading as data.

DataStoryBot's Code Interpreter can detect all four types, because it runs actual Python code against your data — computing z-scores, IQR ranges, time-series decomposition, and correlation analysis inside the container. The key is steering it toward anomalies rather than general trends.

The Anomaly Detection Workflow

The API flow is the same three steps as any DataStoryBot analysis, but the steering prompt makes all the difference.

Step 1: Upload Your Data

curl -X POST https://datastory.bot/api/upload \
  -F "file=@server_metrics.csv"

{
  "containerId": "ctr_abc123",
  "fileId": "file-xyz789",
  "metadata": {
    "fileName": "server_metrics.csv",
    "rowCount": 8640,
    "columnCount": 7,
    "columns": ["timestamp", "server_id", "cpu_pct", "memory_pct", "request_count", "error_count", "response_time_ms"]
  }
}

Step 2: Steer the Analysis Toward Anomalies

This is where the approach diverges from general analysis. Pass a steeringPrompt that explicitly asks for anomaly detection:

curl -X POST https://datastory.bot/api/analyze \
  -H "Content-Type: application/json" \
  -d '{
    "containerId": "ctr_abc123",
    "steeringPrompt": "Focus on anomaly detection. Identify outlier values, unexpected spikes or drops, and any rows or time periods that deviate significantly from normal patterns. Flag data quality issues if present."
  }'

The response surfaces anomaly-focused story angles instead of general trends:

[
  {
    "id": 1,
    "title": "Server 7 Error Rate Spikes 12x on March 15-16",
    "summary": "Server 7 averaged 2.3 errors per hour for the prior 30 days. On March 15-16, it logged 28 errors per hour — a 12x increase. No other servers were affected, suggesting a localized issue rather than a traffic surge.",
    "chartFileId": "file-chart001"
  },
  {
    "id": 2,
    "title": "Three Servers Show Correlated Memory Spikes at 02:00 UTC",
    "summary": "Servers 3, 5, and 9 experience simultaneous memory usage spikes to 94-97% every night at 02:00 UTC. This pattern started on March 8 and was not present in earlier data, suggesting a new scheduled job or batch process.",
    "chartFileId": "file-chart002"
  },
  {
    "id": 3,
    "title": "Response Time Distribution Has 47 Extreme Outliers Above 10s",
    "summary": "While the median response time is 180ms and the p99 is 820ms, 47 requests exceeded 10 seconds. These outliers cluster on Mondays between 09:00-10:00 UTC and affect servers 2 and 7 disproportionately.",
    "chartFileId": "file-chart003"
  }
]

Without the steering prompt, DataStoryBot might have surfaced general trends ("average CPU usage increased 8% over the month") instead of the anomalies. The steering doesn't fabricate findings — it tells the Code Interpreter which statistical tests to prioritize.

Step 3: Get the Full Anomaly Report

curl -X POST https://datastory.bot/api/refine \
  -H "Content-Type: application/json" \
  -d '{
    "containerId": "ctr_abc123",
    "selectedStoryTitle": "Server 7 Error Rate Spikes 12x on March 15-16"
  }'

The narrative output for anomaly stories is particularly useful because it includes the baseline comparison — not just "errors spiked" but "errors spiked from X to Y, which is Z standard deviations above normal":

{
  "narrative": "## Server 7 Error Rate Spikes 12x on March 15-16\n\n**Server 7 logged 672 errors over a 24-hour period starting March 15 at 14:00 UTC**, compared to a 30-day baseline of 55 errors per day (mean) with a standard deviation of 12. The spike represents a **51-sigma event** — effectively impossible under normal operating conditions.\n\n> The error spike was isolated to Server 7. All other servers maintained error rates within 1.5 standard deviations of their baselines during the same period.\n\nThe errors were not uniformly distributed across the spike window...",
  "charts": [
    { "fileId": "file-chart010", "caption": "Server 7 hourly error count with 30-day baseline and 3-sigma threshold" },
    { "fileId": "file-chart011", "caption": "Error rate comparison across all servers, March 15-16" }
  ],
  "resultDataset": {
    "fileId": "file-data001",
    "fileName": "server7_error_spike_rows.csv",
    "rowCount": 672,
    "colCount": 7
  }
}

The filtered dataset contains only the anomalous rows — the 672 error events from Server 7 during the spike window. This is immediately useful for debugging: download it, grep the logs for the corresponding timestamps, and you have your investigation starting point.

Complete Python Example

Here's the full anomaly detection pipeline:

import requests

BASE_URL = "https://datastory.bot/api"

def detect_anomalies(csv_path, context=None):
    """Upload a CSV and run anomaly-focused analysis."""

    # Upload
    with open(csv_path, "rb") as f:
        upload = requests.post(f"{BASE_URL}/upload", files={"file": f})
        upload.raise_for_status()
    container_id = upload.json()["containerId"]

    # Analyze with anomaly-focused steering
    steering = (
        "Focus on anomaly detection. Identify outlier values, unexpected "
        "spikes or drops, and any rows or time periods that deviate "
        "significantly from normal patterns. Use z-scores, IQR analysis, "
        "and time-series decomposition where appropriate."
    )
    if context:
        steering += f" Additional context: {context}"

    stories = requests.post(f"{BASE_URL}/analyze", json={
        "containerId": container_id,
        "steeringPrompt": steering
    })
    stories.raise_for_status()
    anomalies = stories.json()

    print(f"Detected {len(anomalies)} anomaly patterns:")
    for a in anomalies:
        print(f"\n  [{a['id']}] {a['title']}")
        print(f"      {a['summary']}")

    # Refine the most significant anomaly
    report = requests.post(f"{BASE_URL}/refine", json={
        "containerId": container_id,
        "selectedStoryTitle": anomalies[0]["title"]
    })
    report.raise_for_status()
    result = report.json()

    # Save narrative
    with open("anomaly_report.md", "w") as f:
        f.write(result["narrative"])

    # Download charts
    for i, chart in enumerate(result["charts"]):
        img = requests.get(
            f"{BASE_URL}/files/{container_id}/{chart['fileId']}"
        )
        with open(f"anomaly_chart_{i+1}.png", "wb") as f:
            f.write(img.content)
        print(f"Chart saved: {chart['caption']}")

    # Download the anomalous rows
    ds = result["resultDataset"]
    ds_resp = requests.get(f"{BASE_URL}/files/{container_id}/{ds['fileId']}")
    with open(ds["fileName"], "wb") as f:
        f.write(ds_resp.content)
    print(f"Anomalous rows saved: {ds['fileName']} ({ds['rowCount']} rows)")

    return result

# Run it
result = detect_anomalies(
    "server_metrics.csv",
    context="This is infrastructure monitoring data. Servers 1-10 are production. Normal error rate is under 5 per hour."
)

The context parameter is important. If you know the normal baseline for your data, tell the API. "Normal error rate is under 5 per hour" helps the Code Interpreter calibrate what counts as anomalous, rather than relying purely on statistical thresholds computed from the dataset.

Steering Prompts for Different Anomaly Types

The steering prompt is your primary control over what kinds of anomalies get surfaced. Here are patterns for common use cases:

Infrastructure monitoring:

steering = (
    "Detect anomalies in server/infrastructure metrics. Look for: "
    "sudden spikes in error rates or latency, correlated failures across "
    "multiple servers, gradual resource exhaustion trends, and any values "
    "outside the normal operating range."
)

Financial transaction data:

steering = (
    "Identify unusual transactions and patterns. Look for: "
    "individual transactions with amounts significantly above the norm, "
    "unusual frequency patterns (too many or too few transactions in a period), "
    "and any category or account showing behavior that deviates from its "
    "historical baseline."
)

E-commerce / product metrics:

steering = (
    "Find anomalies in product and sales data. Look for: "
    "sudden drops in conversion rates, products with abnormal return rates, "
    "traffic spikes that didn't convert, and any segments where metrics "
    "diverged sharply from the overall trend."
)

Data quality audit:

steering = (
    "Focus on data quality anomalies. Look for: "
    "unexpected null or missing values, duplicate rows, values that violate "
    "expected constraints (negatives in positive-only columns, dates out of range), "
    "and any columns where the distribution shifted unexpectedly."
)

How DataStoryBot Detects Anomalies Under the Hood

DataStoryBot doesn't use a single anomaly detection algorithm. The Code Interpreter running inside the container writes and executes Python code tailored to your specific dataset. Depending on the data shape, it might use:

• Z-score analysis for normally distributed numeric columns — flagging values beyond 3 standard deviations.
• IQR (interquartile range) for skewed distributions — the classic box-plot method that's robust to non-normal data.
• Time-series decomposition for temporal data — separating trend, seasonality, and residuals, then flagging residuals that exceed expected bounds.
• Rolling statistics for detecting regime changes — when the mean or variance of a metric shifts to a new level.
• Cross-column correlation breaks — when two metrics that normally move together suddenly diverge.

The advantage of running actual code (versus a fixed ML model) is adaptability. A pre-trained anomaly detection model needs to be retrained when your data changes. Code Interpreter inspects the data, decides which techniques apply, writes the analysis code, executes it, and interprets the results — all in one pass.

The disadvantage is non-determinism. Run the same dataset twice and you might get slightly different anomaly rankings. For production alerting systems that need deterministic thresholds, you're better off with a dedicated tool like Prometheus rules or custom Python scripts. DataStoryBot is strongest for exploratory anomaly detection — the "what should I be worried about?" question.

Anomaly Detection vs. Trend Analysis

These are complementary but distinct analyses. Trends tell you where things are heading. Anomalies tell you where things broke.

A metric can be trending upward and still contain anomalies — a spike that's 5x the trend line on a single day. Conversely, a flat metric with no trend can still harbor anomalies — a subtle shift in its variance or a handful of outlier rows.

If you've already read how to find trends in data automatically, think of anomaly detection as the inverse lens. Trend analysis asks "what's the pattern?" Anomaly detection asks "what doesn't fit the pattern?"

For the most thorough analysis, run both. Use a general steering prompt first to find trends, then run a second analysis with anomaly-specific steering. The container lives 20 minutes, which is more than enough time for multiple analyze-refine cycles on the same uploaded data.

When AI Anomaly Detection Falls Short

Be honest about the limitations:

No domain knowledge by default. DataStoryBot doesn't know that your e-commerce site runs a flash sale every first Friday of the month. That 300% traffic spike looks like an anomaly unless you tell it otherwise via the steering prompt. Always provide context for known events.

Statistical, not causal. The API can tell you that an anomaly occurred and quantify its severity. It cannot tell you why it happened. The narrative will sometimes suggest possible causes based on correlations in the data, but these are hypotheses, not diagnoses.

Sensitive to data quality. If your CSV has systematic data quality issues — missing rows, inconsistent formats, timezone mismatches — the "anomalies" detected might be artifacts of bad data rather than real events. Clean your data first, or use the data quality steering prompt to let the API flag quality issues separately.

Not real-time. DataStoryBot processes uploaded files. It's not a streaming anomaly detection system. For real-time alerting, use purpose-built tools (Prometheus, Datadog, custom streaming pipelines) and reserve DataStoryBot for periodic deeper analysis — daily or weekly anomaly sweeps where the narrative context adds value.

A Practical Example: E-Commerce Order Data

Here's a realistic scenario. You have a CSV of daily e-commerce orders:

result = detect_anomalies(
    "orders_march_2026.csv",
    context=(
        "E-commerce order data. Typical daily order count is 800-1200. "
        "Average order value is $85. We ran a 20% off promotion on March 10-12. "
        "Ignore that period as a known event."
    )
)

By providing the promotion dates as context, you prevent the API from flagging the March 10-12 spike as an anomaly. Instead, it focuses on genuinely unexpected patterns — maybe a drop in orders from a specific region, or an unusual shift in product mix that happened after the promotion ended.

This is the practical advantage of AI-driven anomaly detection over static threshold rules: you can express context in natural language rather than encoding it as a complex set of exclusion conditions.

What to Read Next

For trend analysis rather than anomaly detection, see how to find trends in your data automatically — the complementary analysis that shows where your metrics are heading.

To understand the full DataStoryBot analysis pipeline and how steering prompts influence results, read how to use AI to analyze data.

Or upload your own dataset to the DataStoryBot playground and try an anomaly-focused analysis interactively — pass a steering prompt like "find outliers and anomalies" to see what it surfaces.