andre/crewAI
1
0
Fork 0
mirror of https://github.com/crewAIInc/crewAI.git synced 2026-07-06 15:39:24 +00:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity

docs(enterprise): add structured JSON logs guide + Datadog dashboard

Browse Source 
Documents the structured-logs work shipped in crewAI-enterprise
PR #1195 and ships the customer-facing Datadog dashboard the CON-249
self-hosted observability ask called out for.

- docs/edge/en/enterprise/guides/structured_logs.mdx: schema v1
  reference, opt-in env var (CREWAI_LOG_FORMAT=json), before/after
  JSON example, compatibility contract. Backend-agnostic — usable
  for Splunk, Loki, ELK, CloudWatch as well.

- docs/edge/en/enterprise/guides/datadog_dashboard.mdx: two ingestion
  paths (Datadog Agent stdout vs Datadog OTLP intake) for self-hosted
  customers to pick from, facet-promotion prerequisites, 3-step
  dashboard import, dashboard tour, customization tips, troubleshooting.

- docs/edge/en/enterprise/guides/datadog_dashboard.json: the importable
  dashboard artifact itself — 4 sections (Header / Throughput / Errors /
  Cost) with template variables wired to @automation_name,
  @crewai_version, and service.

- docs/edge/en/enterprise/guides/capture_telemetry_logs.mdx: clarify
  that the default Datadog OTel template ships traces only and link to
  the new log-export options (Structured Logs + Datadog Dashboard).

- docs/docs.json: register both new pages in the edge/en sidebar
  alongside capture_telemetry_logs. Version snapshots (v1.x.x) and
  non-English locales deliberately untouched — new content lives only
  on the edge channel; translation stubs land in a follow-up PR.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Lucas Gomide
2026-06-17 17:24:42 -03:00
parent 431100ddca
commit eb18db13b3
5 changed files with 865 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/docs.json
View File

@@ -515,6 +515,8 @@
"edge/en/enterprise/guides/update-crew",
"edge/en/enterprise/guides/enable-crew-studio",
"edge/en/enterprise/guides/capture_telemetry_logs",
"edge/en/enterprise/guides/structured_logs",
"edge/en/enterprise/guides/datadog_dashboard",
"edge/en/enterprise/guides/azure-openai-setup",
"edge/en/enterprise/guides/vertex-ai-workload-identity-setup",
"edge/en/enterprise/guides/tool-repository",

4
docs/edge/en/enterprise/guides/capture_telemetry_logs.mdx
View File

@@ -49,7 +49,9 @@ Telemetry data follows the [OpenTelemetry GenAI semantic conventions](https://op
- `otlp.ap1.datadoghq.com` (AP1)
- **API Key** — Your Datadog API key. See [how to create one](https://docs.datadoghq.com/account_management/api-app-keys/#api-keys).
The Datadog integration exports **traces**.
The default Datadog template ships **traces** to the `/v1/traces` path. To export **logs** via OTLP instead, add an **OpenTelemetry Logs** collector pointed at the same Datadog OTLP host with the path set to `/v1/logs` — both signals can run side by side.
For stdout-based log shipping (the Datadog Agent path) rather than OTLP, see [Structured JSON Logs](/en/enterprise/guides/structured_logs) and [Datadog Dashboard for crewAI](/en/enterprise/guides/datadog_dashboard).
<Frame>![Datadog collector configuration](/images/crewai-otel-collector-datadog.png)</Frame>
</Tab>

582
docs/edge/en/enterprise/guides/datadog_dashboard.json Normal file
View File

@@ -0,0 +1,582 @@
{
"title": "crewAI -- Operations",
"description": "Monitoring dashboard for self-hosted crewAI deployments running structured JSON logs. Tracks executions, errors, token usage, and automation health.",
"widgets": [
{
"id": 8810001,
"definition": {
"title": "Header",
"background_color": "vivid_blue",
"show_title": true,
"type": "group",
"layout_type": "ordered",
"widgets": [
{
"id": 9910001,
"definition": {
"title": "Total Executions",
"time": {
"live_span": "1h"
},
"type": "query_value",
"requests": [
{
"response_format": "scalar",
"queries": [
{
"data_source": "logs",
"name": "query1",
"search": {
"query": "$automation $version $service"
},
"compute": {
"aggregation": "cardinality",
"metric": "@execution_id"
},
"indexes": [
"*"
]
}
],
"formulas": [
{
"formula": "query1"
}
]
}
],
"autoscale": true,
"precision": 0
},
"layout": {
"x": 0,
"y": 0,
"width": 3,
"height": 2
}
},
{
"id": 9910002,
"definition": {
"title": "Error Rate (%)",
"time": {
"live_span": "1h"
},
"type": "query_value",
"requests": [
{
"response_format": "scalar",
"queries": [
{
"data_source": "logs",
"name": "query1",
"search": {
"query": "status:error $automation $version $service"
},
"compute": {
"aggregation": "count"
},
"indexes": [
"*"
]
},
{
"data_source": "logs",
"name": "query2",
"search": {
"query": "$automation $version $service"
},
"compute": {
"aggregation": "cardinality",
"metric": "@execution_id"
},
"indexes": [
"*"
]
}
],
"formulas": [
{
"formula": "query1 / query2 * 100"
}
],
"conditional_formats": [
{
"comparator": ">",
"value": 10,
"palette": "white_on_red"
},
{
"comparator": ">",
"value": 5,
"palette": "white_on_yellow"
},
{
"comparator": ">=",
"value": 0,
"palette": "white_on_green"
}
]
}
],
"autoscale": false,
"custom_unit": "%",
"precision": 2
},
"layout": {
"x": 3,
"y": 0,
"width": 3,
"height": 2
}
},
{
"id": 9910003,
"definition": {
"title": "Active Automations",
"time": {
"live_span": "1h"
},
"type": "query_value",
"requests": [
{
"response_format": "scalar",
"queries": [
{
"data_source": "logs",
"name": "query1",
"search": {
"query": "$automation $version $service"
},
"compute": {
"aggregation": "cardinality",
"metric": "@automation_id"
},
"indexes": [
"*"
]
}
],
"formulas": [
{
"formula": "query1"
}
]
}
],
"autoscale": true,
"precision": 0
},
"layout": {
"x": 6,
"y": 0,
"width": 3,
"height": 2
}
},
{
"id": 9910004,
"definition": {
"title": "CrewAI Versions in Use",
"time": {
"live_span": "1h"
},
"type": "query_value",
"requests": [
{
"response_format": "scalar",
"queries": [
{
"data_source": "logs",
"name": "query1",
"search": {
"query": "$automation $version $service"
},
"compute": {
"aggregation": "cardinality",
"metric": "@crewai_version"
},
"indexes": [
"*"
]
}
],
"formulas": [
{
"formula": "query1"
}
]
}
],
"autoscale": true,
"precision": 0
},
"layout": {
"x": 9,
"y": 0,
"width": 3,
"height": 2
}
}
]
},
"layout": {
"x": 0,
"y": 0,
"width": 12,
"height": 3
}
},
{
"id": 8820001,
"definition": {
"title": "Throughput",
"background_color": "vivid_green",
"show_title": true,
"type": "group",
"layout_type": "ordered",
"widgets": [
{
"id": 9920001,
"definition": {
"title": "Executions per Hour by Automation (top 10)",
"show_legend": false,
"type": "timeseries",
"requests": [
{
"response_format": "timeseries",
"queries": [
{
"data_source": "logs",
"name": "query1",
"search": {
"query": "$automation $version $service"
},
"compute": {
"aggregation": "cardinality",
"metric": "@execution_id",
"interval": 3600000
},
"group_by": [
{
"facet": "@automation_name",
"limit": 10,
"sort": {
"aggregation": "cardinality",
"metric": "@execution_id",
"order": "desc"
}
}
],
"indexes": [
"*"
]
}
],
"formulas": [
{
"formula": "query1"
}
],
"style": {
"palette": "semantic"
},
"display_type": "bars"
}
]
},
"layout": {
"x": 0,
"y": 0,
"width": 12,
"height": 3
}
}
]
},
"layout": {
"x": 0,
"y": 3,
"width": 12,
"height": 4
}
},
{
"id": 8830001,
"definition": {
"title": "Errors",
"background_color": "vivid_orange",
"show_title": true,
"type": "group",
"layout_type": "ordered",
"widgets": [
{
"id": 9930001,
"definition": {
"title": "Errors by Exception Type (top 5)",
"show_legend": false,
"type": "timeseries",
"requests": [
{
"response_format": "timeseries",
"queries": [
{
"data_source": "logs",
"name": "query1",
"search": {
"query": "status:error $automation $version $service"
},
"compute": {
"aggregation": "count"
},
"group_by": [
{
"facet": "@exception.type",
"limit": 5,
"sort": {
"aggregation": "count",
"order": "desc"
}
}
],
"indexes": [
"*"
]
}
],
"formulas": [
{
"formula": "query1"
}
],
"style": {
"palette": "warm"
},
"display_type": "bars"
}
]
},
"layout": {
"x": 0,
"y": 0,
"width": 6,
"height": 3
}
},
{
"id": 9930002,
"definition": {
"title": "Top Exception Types by Count",
"type": "toplist",
"requests": [
{
"response_format": "scalar",
"queries": [
{
"data_source": "logs",
"name": "query1",
"search": {
"query": "status:error $automation $version $service"
},
"compute": {
"aggregation": "count"
},
"group_by": [
{
"facet": "@exception.type",
"limit": 10,
"sort": {
"aggregation": "count",
"order": "desc"
}
}
],
"indexes": [
"*"
]
}
],
"formulas": [
{
"formula": "query1"
}
],
"sort": {
"count": 10,
"order_by": [
{
"type": "formula",
"index": 0,
"order": "desc"
}
]
}
}
],
"style": {
"palette": "datadog16"
}
},
"layout": {
"x": 6,
"y": 0,
"width": 6,
"height": 3
}
}
]
},
"layout": {
"x": 0,
"y": 7,
"width": 12,
"height": 4
}
},
{
"id": 8840001,
"definition": {
"title": "Cost",
"background_color": "vivid_purple",
"show_title": true,
"type": "group",
"layout_type": "ordered",
"widgets": [
{
"id": 9940001,
"definition": {
"title": "Total Tokens per Hour by Model (input + output)",
"show_legend": false,
"type": "timeseries",
"requests": [
{
"response_format": "timeseries",
"queries": [
{
"data_source": "logs",
"name": "query1",
"search": {
"query": "$automation $version $service"
},
"compute": {
"aggregation": "sum",
"metric": "@gen_ai.usage.input_tokens",
"interval": 3600000
},
"group_by": [
{
"facet": "@gen_ai.request.model",
"limit": 10,
"sort": {
"aggregation": "sum",
"metric": "@gen_ai.usage.input_tokens",
"order": "desc"
}
}
],
"indexes": [
"*"
]
},
{
"data_source": "logs",
"name": "query2",
"search": {
"query": "$automation $version $service"
},
"compute": {
"aggregation": "sum",
"metric": "@gen_ai.usage.output_tokens",
"interval": 3600000
},
"group_by": [
{
"facet": "@gen_ai.request.model",
"limit": 10,
"sort": {
"aggregation": "sum",
"metric": "@gen_ai.usage.output_tokens",
"order": "desc"
}
}
],
"indexes": [
"*"
]
}
],
"formulas": [
{
"formula": "query1 + query2",
"alias": "Total Tokens"
}
],
"style": {
"palette": "cool"
},
"display_type": "area"
}
]
},
"layout": {
"x": 0,
"y": 0,
"width": 12,
"height": 3
}
}
]
},
"layout": {
"x": 0,
"y": 11,
"width": 12,
"height": 4
}
},
{
"id": 8850002,
"definition": {
"title": "Drill-Down",
"background_color": "gray",
"show_title": true,
"type": "group",
"layout_type": "ordered",
"widgets": []
},
"layout": {
"x": 0,
"y": 15,
"width": 12,
"height": 1
}
}
],
"template_variables": [
{
"name": "automation",
"prefix": "@automation_name",
"available_values": [],
"default": "*"
},
{
"name": "version",
"prefix": "@crewai_version",
"available_values": [],
"default": "*"
},
{
"name": "service",
"prefix": "service",
"available_values": [],
"default": "*"
}
],
"layout_type": "ordered",
"notify_list": [],
"pause_auto_refresh": false,
"reflow_type": "fixed",
"tags": [
"ai:created_with_ai"
]
}

136
docs/edge/en/enterprise/guides/datadog_dashboard.mdx Normal file
View File

@@ -0,0 +1,136 @@
---
title: "Datadog Dashboard for crewAI"
description: "Import a ready-made Datadog dashboard for monitoring self-hosted CrewAI AMP deployments — executions, errors, token cost, and version distribution. Works with both the Datadog Agent and Datadog's OTLP intake."
icon: "dog"
mode: "wide"
---
CrewAI ships a ready-made Datadog dashboard for self-hosted AMP deployments. Once your logs are flowing into Datadog, you can import the dashboard JSON and have an operations view live in your account in under five minutes.
The dashboard works with either of Datadog's two log-ingestion paths — pick whichever fits your infrastructure:
<Tabs>
<Tab title="Datadog Agent (stdout)">
The Datadog Agent runs alongside your CrewAI containers (typically as a DaemonSet on Kubernetes) and tails their stdout. This path requires enabling [Structured JSON Logs](/en/enterprise/guides/structured_logs) so each log event is a single billable line instead of a multi-line traceback.
**Setup:**
1. Set `CREWAI_LOG_FORMAT=json` on every CrewAI container — see [Structured JSON Logs](/en/enterprise/guides/structured_logs) for full details.
2. Install the Datadog Agent in your cluster following [Datadog's Kubernetes setup guide](https://docs.datadoghq.com/containers/kubernetes/installation/). Enable log collection (`logs_enabled: true`) and container log collection (`logs_config.container_collect_all: true`).
3. Confirm logs are landing in Datadog by searching `service:crewai*` in the [Logs Explorer](https://app.datadoghq.com/logs).
**When to pick this path:** you already run the Datadog Agent for infrastructure metrics, or you want logs without configuring an OTel collector in AMP.
</Tab>
<Tab title="Datadog OTLP intake (no agent)">
Datadog accepts OTLP traffic directly at its intake endpoint, no agent required. Configure CrewAI AMP's built-in OTel collector to point at Datadog's OTLP host.
**Setup:**
1. In CrewAI AMP: **Settings → OpenTelemetry Collectors → Add Collector → Datadog**. See [OpenTelemetry Export](/en/enterprise/guides/capture_telemetry_logs) for the full collector setup.
2. The default Datadog template ships **traces** to `/v1/traces`. For log export, switch the endpoint path to `/v1/logs` on the OpenTelemetry Logs collector (use the same Datadog OTLP host).
3. Confirm logs are landing by searching `source:otlp service:crewai*` in the [Logs Explorer](https://app.datadoghq.com/logs).
**When to pick this path:** you can't or don't want to run the Datadog Agent, or you're already using OTLP for traces and want a single export pipeline.
</Tab>
</Tabs>
Either path lands the same structured facets in Datadog (`@automation_id`, `@kickoff_id`, `@execution_id`, `@automation_name`, `@crewai_version`, `@exception.type`, `@gen_ai.*`), so the dashboard works identically with either choice.
## Prerequisite: promote facets
Datadog auto-discovers fields the first time it sees them but doesn't make them queryable in widgets until they're promoted to **facets**. This is a one-time setup in your Datadog account.
<Steps>
<Step title="Search for a CrewAI log">
Open [Logs Explorer](https://app.datadoghq.com/logs) and search `service:crewai*`. You should see at least one log event.
</Step>
<Step title="Promote each field">
Click any log entry to open the right-hand details panel. For each field below, hover the field name → click the gear icon → **Create facet**.
- `automation_id`, `automation_name`, `execution_id`, `kickoff_id`, `task_id`
- `crewai_version`, `model_id`
- `exception.type`, `exception.message`
Skip any field that already shows a star icon next to its name — that means it's already a facet. The `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, and `gen_ai.request.model` facets are typically promoted automatically by Datadog's LLM Observability auto-discovery, but verify they exist before importing the dashboard.
</Step>
</Steps>
## Import the dashboard
<Steps>
<Step title="Download the dashboard JSON">
Save [`datadog_dashboard.json`](https://raw.githubusercontent.com/crewAIInc/crewAI/main/docs/edge/en/enterprise/guides/datadog_dashboard.json) to your machine.
</Step>
<Step title="Open the import dialog in Datadog">
Navigate to **Dashboards → New Dashboard**. Click the **gear icon** in the top right of the empty dashboard and select **Import Dashboard JSON**.
</Step>
<Step title="Paste or upload the JSON">
Paste the contents of `datadog_dashboard.json` into the import dialog (or drag the file in). Click **Import**.
Datadog creates the dashboard immediately and lands you on it. The first load may show empty widgets for a few seconds while queries execute against the time range.
</Step>
</Steps>
<Tip>
Datadog's [Dashboard API](https://docs.datadoghq.com/api/latest/dashboards/#create-a-new-dashboard) accepts the same JSON via `POST /api/v1/dashboard`. Use it if you manage dashboards through Terraform, Pulumi, or CI.
</Tip>
## What you get
The dashboard is organized into four sections plus a placeholder for a custom drill-down widget:
| Section | Widgets | Useful for |
|---------|---------|------------|
| **Header** | Total Executions · Error Rate (%) · Active Automations · CrewAI Versions in Use | At-a-glance health for the last hour. Error Rate is conditionally formatted (green ≤ 5%, yellow ≤ 10%, red > 10%). |
| **Throughput** | Executions per Hour by Automation (top 10, stacked bars) | Spotting traffic shifts, surfacing busy automations, validating that a rollout didn't change baseline volume. |
| **Errors** | Errors by Exception Type (top 5, stacked bars) · Top Exception Types by Count (toplist) | Triaging failures — which exception types are spiking, which automations they're hitting. |
| **Cost** | Total Tokens per Hour by Model (input + output, stacked area) | Tracking LLM token spend by model. Useful for catching cost regressions when an automation switches model or starts looping. |
| **Drill-Down** | _(empty placeholder)_ | See [Customization](#customization) for adding a recent-errors log stream here. |
Three template variables at the top of the dashboard re-scope every widget at once:
- **`$automation`** — filter to a single automation by name.
- **`$version`** — filter to a single `crewai` SDK version (useful for comparing pre- and post-upgrade behavior).
- **`$service`** — filter to a specific Datadog `service` tag (useful when multiple CrewAI deployments share one Datadog account).
## Customization
The dashboard ships with deliberate gaps so you can extend it without uninstalling and re-importing.
### Add a Recent Errors log stream
The **Drill-Down** section is intentionally empty. Add a Log Stream widget to it for an inline view of recent failures:
1. Edit the dashboard and click **+ Add Widgets** inside the Drill-Down group.
2. Drag in a **Log Stream** widget.
3. Set the filter query to `status:error $automation $version $service`.
4. Choose columns: `@timestamp`, `@automation_name`, `@exception.type`, `@exception.message`, `@execution_id`.
5. Sort by most recent, limit to 25 entries.
Clicking any row jumps to Logs Explorer with the same filter pre-applied.
### Add p95 latency
Logs don't include execution duration by default. Two ways to add a latency widget:
- **From APM traces** — if you also export OTLP traces to Datadog, add a Timeseries widget with data source **Traces**, query `service:crewai*`, aggregation `p95 of @duration`. Datadog APM auto-tracks span duration.
- **From metric extraction** — extract a `flow.duration_ms` metric from logs via [Datadog's log-to-metric pipeline](https://docs.datadoghq.com/logs/log_configuration/logs_to_metrics/), then chart it like any other metric. Useful if you don't run APM.
### Re-scope to multiple deployments
The `$service` template variable defaults to `*` and will catch every CrewAI deployment in your Datadog account. Change the default to a specific service name in **Configure → Template Variables** if you want the dashboard to focus on one deployment by default.
## Troubleshooting
| Symptom | Likely cause | Fix |
|---------|--------------|-----|
| All widgets show "No data" | Facets aren't promoted | Re-do the [Promote facets](#prerequisite-promote-facets) step. Datadog won't query against an un-promoted field. |
| Error Rate widget shows `NaN` | No executions in the time window | Either no traffic, or `@execution_id` isn't faceted. Expand the time range and re-check facets. |
| Throughput chart is flat at the same value | Logs aren't reaching Datadog | Search `service:crewai*` in Logs Explorer. If nothing shows, verify the Datadog Agent is running (Agent path) or the OTel collector endpoint is correct (OTLP path). |
| `crewai_version` shows fewer values than expected | Some containers predate the structured-logs work | The `crewai_version` field was added alongside JSON output. Older deployments running text mode (or older AMP builds) won't emit it. Upgrade those deployments to pick up the field. |
| Template variables don't filter widgets | The widget's filter line doesn't reference the template variable | Edit the widget and confirm the search includes `$automation $version $service`. |
## References
- [Structured JSON Logs](/en/enterprise/guides/structured_logs) — the underlying log format the dashboard queries against.
- [OpenTelemetry Export](/en/enterprise/guides/capture_telemetry_logs) — set up the OTLP path if you're not using the Datadog Agent.
- [Datadog Log Search Syntax](https://docs.datadoghq.com/logs/explorer/search_syntax/) — reference for customizing widget queries.
- [Datadog Dashboard JSON Schema](https://docs.datadoghq.com/dashboards/graphing_json/) — full reference for the dashboard file format if you want to script changes.

142
docs/edge/en/enterprise/guides/structured_logs.mdx Normal file
View File

@@ -0,0 +1,142 @@
---
title: "Structured JSON Logs"
description: "Emit single-line JSON log events from CrewAI AMP deployments for cheaper, structured ingestion in Datadog, Splunk, Loki, and other log backends."
icon: "brackets-curly"
mode: "wide"
---
CrewAI AMP can emit one JSON object per log event on stdout instead of the default multi-line text format. Each event ships with typed context fields (automation, kickoff, execution, trace IDs, exception details), making logs cheaper to index, easier to search, and trivially correlatable with traces.
This page describes the JSON schema, how to enable it, and how to verify it's working. For a ready-made Datadog dashboard built on top of these fields, see [Datadog Dashboard for crewAI](/en/enterprise/guides/datadog_dashboard).
## Why use JSON output
<CardGroup cols={2}>
<Card title="Lower ingestion cost" icon="dollar-sign">
Most managed log backends bill per event. A Python traceback in text format is counted as one event per line — 30+ events for a single error. JSON output collapses each traceback into a single event with the stack trace as an escaped string field.
</Card>
<Card title="Structured search" icon="magnifying-glass">
Search by `@automation_id`, `@exception.type`, `@kickoff_id` instead of grepping free-text. Build dashboards on typed facets without parser configuration.
</Card>
<Card title="APM ↔ logs correlation" icon="link">
Every event carries `trace_id` and `span_id` when fired inside a recording span, so backends auto-link logs to traces.
</Card>
<Card title="Backend agnostic" icon="server">
The format is plain JSON — Datadog, Splunk, Loki, Elasticsearch, and CloudWatch all parse it natively without custom log pipelines.
</Card>
</CardGroup>
## Enabling JSON output
Set the `CREWAI_LOG_FORMAT` environment variable to `json` on every container that runs your deployment (API + workers).
```shell
CREWAI_LOG_FORMAT=json
```
Restart the deployment to pick up the change. Every log line on stdout from that point on is a single JSON object.
<Note>
The default value is `text`, which preserves the legacy human-readable line format byte-for-byte. Setting any value other than `json` falls back to text mode. There is no migration step — the variable is read at process start and the format switches immediately.
</Note>
## What a log event looks like
A single info-level log inside an active automation kickoff:
```json
{
"schema": "v1",
"ts": "2026-06-17T16:14:23.482914Z",
"level": "INFO",
"logger": "crewai_enterprise.utilities.pii_redaction",
"crewai_version": "1.14.7",
"msg": "PII tracking state reset (engines preserved)",
"automation_id": "12",
"task_id": "0843a930-b306-464b-89c8-bfafa78cc711",
"kickoff_id": "0843a930-b306-464b-89c8-bfafa78cc711",
"execution_id": "0843a930-b306-464b-89c8-bfafa78cc711",
"automation_name": "research_flow"
}
```
An error with a Python exception is collapsed into a single event with the traceback as a string:
```json
{
"schema": "v1",
"ts": "2026-06-17T16:14:31.218450Z",
"level": "ERROR",
"logger": "api.tasks.flow_run_task",
"crewai_version": "1.14.7",
"msg": "Flow execution failed",
"automation_id": "12",
"kickoff_id": "0843a930-b306-464b-89c8-bfafa78cc711",
"execution_id": "0843a930-b306-464b-89c8-bfafa78cc711",
"automation_name": "research_flow",
"exception": {
"type": "ValueError",
"message": "Topic cannot be empty",
"stacktrace": "Traceback (most recent call last):\n File \"/app/flow.py\", line 42, in summarize\n ...\nValueError: Topic cannot be empty\n"
}
}
```
The same error in legacy text mode would have produced ~25 separate log events (one per traceback line) — all of which the backend would bill and index individually.
## Schema v1 field reference
Within the `v1` schema, fields are only added, never renamed or removed. New fields will appear as soon as a deployment is upgraded.
| Field | Type | Always present | Source |
|-------|------|----------------|--------|
| `schema` | string | Yes | Constant `"v1"`. Increment indicates a breaking schema change. |
| `ts` | string (ISO-8601 UTC, microseconds) | Yes | Record creation time, e.g. `2026-06-17T16:14:23.482914Z`. |
| `level` | string | Yes | Python log level name: `DEBUG` / `INFO` / `WARNING` / `ERROR` / `CRITICAL`. |
| `logger` | string | Yes | Dotted logger name, e.g. `api.tasks.flow_run_task`. |
| `crewai_version` | string | Yes (when `crewai` package metadata is resolvable) | Installed `crewai` package version, e.g. `"1.14.7"`. |
| `msg` | string | Yes | Rendered log message (after `%`-formatting / `{}`-formatting). |
| `automation_id` | string | When `CREWAI_PLUS_ID` env var is set | Numeric deployment ID (AMP provisions this on every container). |
| `task_id` | string | On Celery worker logs | Celery task UUID, or `"no-task"` for non-task contexts. |
| `kickoff_id` | string | Inside an automation kickoff | UUID of the current kickoff. |
| `execution_id` | string | Inside an automation kickoff | UUID of the current sub-execution. Equal to `kickoff_id` at the top level; differs for nested flow methods that spawn sub-executions. |
| `automation_name` | string | Inside an automation kickoff | Human-readable automation/flow name, e.g. `"research_flow"`. |
| `trace_id` | string (32-hex) | Inside a recording OpenTelemetry span | Hex trace ID. Omitted when no span is active. |
| `span_id` | string (16-hex) | Inside a recording OpenTelemetry span | Hex span ID. Omitted when no span is active. |
| `exception` | object | When the log record has `exc_info` | `{type, message, stacktrace}` — full traceback as a single escaped string. |
<Tip>
Any additional `extra={...}` kwargs passed to a logger call appear as top-level JSON fields verbatim. Reserved field names above always win to keep the schema stable.
</Tip>
## Verifying it's working
After enabling the env var and restarting, fetch the latest container logs and confirm each line is a single JSON object:
```shell
# Example: docker logs <api-container> --tail 10
docker logs $(docker ps -qf name=crewai-api) --tail 10 | jq -r '.msg'
```
If the output is JSON, each line will parse successfully and `jq` will print only the `msg` field. If you see "parse error", the env var didn't take effect — confirm it's set in the running container and that the deployment was restarted after the change.
## Compatibility and versioning
The `schema` field declares the contract. Within `v1`, CrewAI commits to:
- **Never removing a field** that customers may have built queries or dashboards against.
- **Never renaming a field** in place — renames happen via a schema bump (e.g. `v2`), with the old name kept as a deprecated alias for at least one release cycle.
- **Adding new fields** at any time. Consumers should ignore unknown top-level keys.
When a `v2` is introduced, both the `schema` field and the migration guide will be published in advance, and `v1` will continue to be emitted for one release cycle so dashboards and queries have time to migrate.
## What's next
<CardGroup cols={2}>
<Card title="Datadog Dashboard for crewAI" icon="dog" href="/en/enterprise/guides/datadog_dashboard">
Import a ready-made operations dashboard built on these facets — executions, errors, token cost, version distribution. Works with both the Datadog Agent and Datadog's OTLP intake.
</Card>
<Card title="OpenTelemetry Export" icon="magnifying-glass-chart" href="/en/enterprise/guides/capture_telemetry_logs">
Ship logs and traces to your own OTel collector or directly to a backend's OTLP intake. The same context fields land as OTLP attributes, so the dashboard works regardless of which path you use.
</Card>
</CardGroup>