Incorrect documentation link for AgentOps (#458)

* remove .md

* made language more clear

* update images and documentation for spelling

* update typos and links

* update repo placement

* update wording

* clarify

* update wording

* Added clearer features

---------

Co-authored-by: João Moura <joaomdmoura@gmail.com>
This commit is contained in:
Alex Reibman
2024-04-16 04:24:30 -07:00
committed by GitHub
parent 3d862538d2
commit 2ee6ab6332
6 changed files with 43 additions and 27 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 272 KiB

After

Width:  |  Height:  |  Size: 288 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 190 KiB

After

Width:  |  Height:  |  Size: 419 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 176 KiB

After

Width:  |  Height:  |  Size: 263 KiB

View File

@@ -1,51 +1,52 @@
--- ---
title: (AgentOps) Observability using AgentOps title: Agent Monitoring with AgentOps
description: Understanding and logging your agent performance with AgentOps. description: Understanding and logging your agent performance with AgentOps.
--- ---
# Intro # Intro
Observability is a key aspect of developing and deploying conversational AI agents. It allows developers to understand how the agent is performing, how users are interacting with the agent, and how the agent is responding to user inputs. Observability is a key aspect of developing and deploying conversational AI agents. It allows developers to understand how their agents are performing, how their agents are interacting with users, and how their agents use external tools and APIs. AgentOps is a product independent of CrewAI that provides a comprehensive observability solution for agents.
AgentOps is a product, idependent of crewAI that provides a comprehensive observability solution for agents.
This notebook will provide an overview of AgentOps and how to use it with crewAI.
## AgentOps ## AgentOps
[AgentOps](https://agentops.ai) provides session replays, metrics, and monitoring for agents. [AgentOps](https://agentops.ai/?=crew) provides session replays, metrics, and monitoring for agents.
[AgentOps Repo](https://github.com/AgentOps-AI/agentops)
At a high level, AgentOps gives you the ability to monitor cost, token usage, latency, agent failures, session-wide statistics, and more. For more info, check out the [AgentOps Repo](https://github.com/AgentOps-AI/agentops).
### Overview ### Overview
AgentOps provides monotoring for agents in development and production. It provides a dashboard for monitoring agent performance, session replays, and custom reporting. AgentOps provides monitoring for agents in development and production. It provides a dashboard for tracking agent performance, session replays, and custom reporting.
![agentops-overview.png](..%2Fassets%2Fagentops-overview.png) Additionally, AgentOps provides session drilldowns for viewing Crew agent interactions, LLM calls, and tool usage in real-time. This feature is useful for debugging and understanding how agents interact with users as well as other agents.
Additionally, AgentOps provides session drilldowns that allows users to view the agent's interactions with users in real-time. This feature is useful for debugging and understanding how the agent interacts with users. ![Overview of a select series of agent session runs](..%2Fassets%2Fagentops-overview.png)
![Overview of session drilldowns for examining agent runs](..%2Fassets%2Fagentops-session.png)
![agentops-session.png](..%2Fassets%2Fagentops-session.png) ![Viewing a step-by-step agent replay execution graph](..%2Fassets%2Fagentops-replay.png)
![agentops-replay.png](..%2Fassets%2Fagentops-replay.png)
### Features ### Features
- LLM Cost management and tracking - **LLM Cost Management and Tracking**: Track spend with foundation model providers
- Replay Analytics - **Replay Analytics**: Watch step-by-step agent execution graphs
- Recursive thought detection - **Recursive Thought Detection**: Identify when agents fall into infinite loops
- Custom Reporting - **Custom Reporting**: Create custom analytics on agent performance
- Analytics Dashboard - **Analytics Dashboard**: Monitor high level statistics about agents in development and production
- Public Model Testing - **Public Model Testing**: Test your agents against benchmarks and leaderboards
- Custom Tests - **Custom Tests**: Run your agents against domain specific tests
- Time Travel Debugging - **Time Travel Debugging**: Restart your sessions from checkpoints
- Compliance and Security - **Compliance and Security**: Create audit logs and detect potential threats such as profanity and PII leaks
- **Prompt Injection Detection**: Identify potential code injection and secret leaks
### Using AgentOps ### Using AgentOps
Create a user API key here: app.agentops.ai/account 1. **Create an API Key:**
Create a user API key here: [Create API Key](app.agentops.ai/account)
2. **Configure Your Environment:**
Add your API key to your environment variables Add your API key to your environment variables
``` ```
AGENTOPS_API_KEY=<YOUR_AGENTOPS_API_KEY> AGENTOPS_API_KEY=<YOUR_AGENTOPS_API_KEY>
``` ```
3. **Install AgentOps:**
Install AgentOps with: Install AgentOps with:
``` ```
pip install crewai[agentops] pip install crewai[agentops]
@@ -62,11 +63,26 @@ import agentops
agentops.init() agentops.init()
``` ```
This will initiate an AgentOps session as well as automatically track Crew agents. For further info on how to outfit more complex agentic systems, check out the [AgentOps documentation](https://docs.agentops.ai) or join the [Discord](https://discord.gg/j4f3KbeH).
### Crew + AgentOps Examples ### Crew + AgentOps Examples
- [Job Posting](https://github.com/joaomdmoura/crewAI-examples/tree/main/job-posting) - [Job Posting](https://github.com/joaomdmoura/crewAI-examples/tree/main/job-posting)
- [Markdown Validator](https://github.com/joaomdmoura/crewAI-examples/tree/main/markdown_validator) - [Markdown Validator](https://github.com/joaomdmoura/crewAI-examples/tree/main/markdown_validator)
- [Instagram Post](https://github.com/joaomdmoura/crewAI-examples/tree/main/instagram_post) - [Instagram Post](https://github.com/joaomdmoura/crewAI-examples/tree/main/instagram_post)
### Futher Information ### Further Information
To implement more features and better observability, please see the [AgentOps Repo](https://github.com/AgentOps-AI/agentops)
To get started, create an [AgentOps account](https://agentops.ai/?=crew).
For feature requests or bug reports, please reach out to the AgentOps team on the [AgentOps Repo](https://github.com/AgentOps-AI/agentops).
#### Extra links
<a href="https://twitter.com/agentopsai/">🐦 Twitter</a>
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
<a href="https://discord.gg/JHPt4C7r">📢 Discord</a>
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
<a href="https://app.agentops.ai/?=crew">🖇️ AgentOps Dashboard</a>
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
<a href="https://docs.agentops.ai/introduction">📙 Documentation</a>

View File

@@ -80,7 +80,7 @@ Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By
</li> </li>
<li> <li>
<a href="./how-to/AgentOps-Observability"> <a href="./how-to/AgentOps-Observability">
Agent Observability using AgentOps Agent Monitoring with AgentOps
</a> </a>
</li> </li>
</ul> </ul>

View File

@@ -135,7 +135,7 @@ nav:
- Connecting to any LLM: 'how-to/LLM-Connections.md' - Connecting to any LLM: 'how-to/LLM-Connections.md'
- Customizing Agents: 'how-to/Customizing-Agents.md' - Customizing Agents: 'how-to/Customizing-Agents.md'
- Human Input on Execution: 'how-to/Human-Input-on-Execution.md' - Human Input on Execution: 'how-to/Human-Input-on-Execution.md'
- Agent Observability using AgentOps: 'how-to/AgentOps-Observability.md' - Agent Monitoring with AgentOps: 'how-to/AgentOps-Observability.md'
- Tools Docs: - Tools Docs:
- Google Serper Search: 'tools/SerperDevTool.md' - Google Serper Search: 'tools/SerperDevTool.md'
- Scrape Website: 'tools/ScrapeWebsiteTool.md' - Scrape Website: 'tools/ScrapeWebsiteTool.md'