andre/crewAI
1
0
Fork 0
mirror of https://github.com/crewAIInc/crewAI.git synced 2026-01-09 08:08:32 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Merge branch 'main' into main

Browse Source
This commit is contained in:
Lorenze Jay
2025-04-01 10:35:26 -07:00
committed by GitHub
parent b6c32b014c b0f9637662
commit 3f92e217f9
84 changed files with 5962 additions and 971 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/how-to/agentops-observability.mdx
View File

@@ -1,5 +1,5 @@
---
title: Agent Monitoring with AgentOps
title: AgentOps Integration
description: Understanding and logging your agent performance with AgentOps.
icon: paperclip
---

3
docs/how-to/kickoff-for-each.mdx
View File

@@ -39,8 +39,7 @@ analysis_crew = Crew(
agents=[coding_agent],
tasks=[data_analysis_task],
verbose=True,
memory=False,
respect_context_window=True # enable by default
memory=False
)
datasets = [

4
docs/how-to/langfuse-observability.mdx
View File

@@ -1,7 +1,7 @@
---
title: Agent Monitoring with Langfuse
title: Langfuse Integration
description: Learn how to integrate Langfuse with CrewAI via OpenTelemetry using OpenLit
icon: magnifying-glass-chart
icon: vials
---
# Integrate Langfuse with CrewAI

2
docs/how-to/langtrace-observability.mdx
View File

@@ -1,5 +1,5 @@
---
title: Agent Monitoring with Langtrace
title: Langtrace Integration
description: How to monitor cost, latency, and performance of CrewAI Agents using Langtrace, an external observability tool.
icon: chart-line
---

2
docs/how-to/mlflow-observability.mdx
View File

@@ -1,5 +1,5 @@
---
title: Agent Monitoring with MLflow
title: MLflow Integration
description: Quickly start monitoring your Agents with MLflow.
icon: bars-staggered
---

2
docs/how-to/openlit-observability.mdx
View File

@@ -1,5 +1,5 @@
---
title: Agent Monitoring with OpenLIT
title: OpenLIT Integration
description: Quickly start monitoring your Agents in just a single line of code with OpenTelemetry.
icon: magnifying-glass-chart
---

129
docs/how-to/opik-observability.mdx Normal file
View File

@@ -0,0 +1,129 @@
---
title: Opik Integration
description: Learn how to use Comet Opik to debug, evaluate, and monitor your CrewAI applications with comprehensive tracing, automated evaluations, and production-ready dashboards.
icon: meteor
---
# Opik Overview
With [Comet Opik](https://www.comet.com/docs/opik/), debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
<Frame caption="Opik Agent Dashboard">
<img src="/images/opik-crewai-dashboard.png" alt="Opik agent monitoring example with CrewAI" />
</Frame>
Opik provides comprehensive support for every stage of your CrewAI application development:
- **Log Traces and Spans**: Automatically track LLM calls and application logic to debug and analyze development and production systems. Manually or programmatically annotate, view, and compare responses across projects.
- **Evaluate Your LLM Application's Performance**: Evaluate against a custom test set and run built-in evaluation metrics or define your own metrics in the SDK or UI.
- **Test Within Your CI/CD Pipeline**: Establish reliable performance baselines with Opik's LLM unit tests, built on PyTest. Run online evaluations for continuous monitoring in production.
- **Monitor & Analyze Production Data**: Understand your models' performance on unseen data in production and generate datasets for new dev iterations.
## Setup
Comet provides a hosted version of the Opik platform, or you can run the platform locally.
To use the hosted version, simply [create a free Comet account](https://www.comet.com/signup?utm_medium=github&utm_source=crewai_docs) and grab you API Key.
To run the Opik platform locally, see our [installation guide](https://www.comet.com/docs/opik/self-host/overview/) for more information.
For this guide we will use CrewAI’s quickstart example.
<Steps>
<Step title="Install required packages">
```shell
pip install crewai crewai-tools opik --upgrade
```
</Step>
<Step title="Configure Opik">
```python
import opik
opik.configure(use_local=False)
```
</Step>
<Step title="Prepare environment">
First, we set up our API keys for our LLM-provider as environment variables:
```python
import os
import getpass
if "OPENAI_API_KEY" not in os.environ:
os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter your OpenAI API key: ")
```
</Step>
<Step title="Using CrewAI">
The first step is to create our project. We will use an example from CrewAI’s documentation:
```python
from crewai import Agent, Crew, Task, Process
class YourCrewName:
def agent_one(self) -> Agent:
return Agent(
role="Data Analyst",
goal="Analyze data trends in the market",
backstory="An experienced data analyst with a background in economics",
verbose=True,
)
def agent_two(self) -> Agent:
return Agent(
role="Market Researcher",
goal="Gather information on market dynamics",
backstory="A diligent researcher with a keen eye for detail",
verbose=True,
)
def task_one(self) -> Task:
return Task(
name="Collect Data Task",
description="Collect recent market data and identify trends.",
expected_output="A report summarizing key trends in the market.",
agent=self.agent_one(),
)
def task_two(self) -> Task:
return Task(
name="Market Research Task",
description="Research factors affecting market dynamics.",
expected_output="An analysis of factors influencing the market.",
agent=self.agent_two(),
)
def crew(self) -> Crew:
return Crew(
agents=[self.agent_one(), self.agent_two()],
tasks=[self.task_one(), self.task_two()],
process=Process.sequential,
verbose=True,
)
```
Now we can import Opik’s tracker and run our crew:
```python
from opik.integrations.crewai import track_crewai
track_crewai(project_name="crewai-integration-demo")
my_crew = YourCrewName().crew()
result = my_crew.kickoff()
print(result)
```
After running your CrewAI application, visit the Opik app to view:
- LLM traces, spans, and their metadata
- Agent interactions and task execution flow
- Performance metrics like latency and token usage
- Evaluation metrics (built-in or custom)
</Step>
</Steps>
## Resources
- [🦉 Opik Documentation](https://www.comet.com/docs/opik/)
- [👉 Opik + CrewAI Colab](https://colab.research.google.com/github/comet-ml/opik/blob/main/apps/opik-documentation/documentation/docs/cookbook/crewai.ipynb)
- [🐦 X](https://x.com/cometml)
- [💬 Slack](https://slack.comet.com/)

2
docs/how-to/portkey-observability.mdx
View File

@@ -1,5 +1,5 @@
---
title: Agent Monitoring with Portkey
title: Portkey Integration
description: How to use Portkey with CrewAI
icon: key
---

124
docs/how-to/weave-integration.mdx Normal file
View File

@@ -0,0 +1,124 @@
---
title: Weave Integration
description: Learn how to use Weights & Biases (W&B) Weave to track, experiment with, evaluate, and improve your CrewAI applications.
icon: radar
---
# Weave Overview
[Weights & Biases (W&B) Weave](https://weave-docs.wandb.ai/) is a framework for tracking, experimenting with, evaluating, deploying, and improving LLM-based applications.
![Overview of W&B Weave CrewAI tracing usage](/images/weave-tracing.gif)
Weave provides comprehensive support for every stage of your CrewAI application development:
- **Tracing & Monitoring**: Automatically track LLM calls and application logic to debug and analyze production systems
- **Systematic Iteration**: Refine and iterate on prompts, datasets, and models
- **Evaluation**: Use custom or pre-built scorers to systematically assess and enhance agent performance
- **Guardrails**: Protect your agents with pre- and post-safeguards for content moderation and prompt safety
Weave automatically captures traces for your CrewAI applications, enabling you to monitor and analyze your agents' performance, interactions, and execution flow. This helps you build better evaluation datasets and optimize your agent workflows.
## Setup Instructions
<Steps>
<Step title="Install required packages">
```shell
pip install crewai weave
```
</Step>
<Step title="Set up W&B Account">
Sign up for a [Weights & Biases account](https://wandb.ai) if you haven't already. You'll need this to view your traces and metrics.
</Step>
<Step title="Initialize Weave in Your Application">
Add the following code to your application:
```python
import weave
# Initialize Weave with your project name
weave.init(project_name="crewai_demo")
```
After initialization, Weave will provide a URL where you can view your traces and metrics.
</Step>
<Step title="Create your Crews/Flows">
```python
from crewai import Agent, Task, Crew, LLM, Process
# Create an LLM with a temperature of 0 to ensure deterministic outputs
llm = LLM(model="gpt-4o", temperature=0)
# Create agents
researcher = Agent(
role='Research Analyst',
goal='Find and analyze the best investment opportunities',
backstory='Expert in financial analysis and market research',
llm=llm,
verbose=True,
allow_delegation=False,
)
writer = Agent(
role='Report Writer',
goal='Write clear and concise investment reports',
backstory='Experienced in creating detailed financial reports',
llm=llm,
verbose=True,
allow_delegation=False,
)
# Create tasks
research_task = Task(
description='Deep research on the {topic}',
expected_output='Comprehensive market data including key players, market size, and growth trends.',
agent=researcher
)
writing_task = Task(
description='Write a detailed report based on the research',
expected_output='The report should be easy to read and understand. Use bullet points where applicable.',
agent=writer
)
# Create a crew
crew = Crew(
agents=[researcher, writer],
tasks=[research_task, writing_task],
verbose=True,
process=Process.sequential,
)
# Run the crew
result = crew.kickoff(inputs={"topic": "AI in material science"})
print(result)
```
</Step>
<Step title="View Traces in Weave">
After running your CrewAI application, visit the Weave URL provided during initialization to view:
- LLM calls and their metadata
- Agent interactions and task execution flow
- Performance metrics like latency and token usage
- Any errors or issues that occurred during execution
<Frame caption="Weave Tracing Dashboard">
<img src="/images/weave-tracing.png" alt="Weave tracing example with CrewAI" />
</Frame>
</Step>
</Steps>
## Features
- Weave automatically captures all CrewAI operations: agent interactions and task executions; LLM calls with metadata and token usage; tool usage and results.
- The integration supports all CrewAI execution methods: `kickoff()`, `kickoff_for_each()`, `kickoff_async()`, and `kickoff_for_each_async()`.
- Automatic tracing of all [crewAI-tools](https://github.com/crewAIInc/crewAI-tools).
- Flow feature support with decorator patching (`@start`, `@listen`, `@router`, `@or_`, `@and_`).
- Track custom guardrails passed to CrewAI `Task` with `@weave.op()`.
For detailed information on what's supported, visit the [Weave CrewAI documentation](https://weave-docs.wandb.ai/guides/integrations/crewai/#getting-started-with-flow).
## Resources
- [📘 Weave Documentation](https://weave-docs.wandb.ai)
- [📊 Example Weave x CrewAI dashboard](https://wandb.ai/ayut/crewai_demo/weave/traces?cols=%7B%22wb_run_id%22%3Afalse%2C%22attributes.weave.client_version%22%3Afalse%2C%22attributes.weave.os_name%22%3Afalse%2C%22attributes.weave.os_release%22%3Afalse%2C%22attributes.weave.os_version%22%3Afalse%2C%22attributes.weave.source%22%3Afalse%2C%22attributes.weave.sys_version%22%3Afalse%7D&peekPath=%2Fayut%2Fcrewai_demo%2Fcalls%2F0195c838-38cb-71a2-8a15-651ecddf9d89)
- [🐦 X](https://x.com/weave_wb)