Merge remote-tracking branch 'upstream/main'

# Conflicts:
#	src/crewai/crew.py

README.md

@@ -30,7 +30,6 @@
 - [Connecting Your Crew to a Model](#connecting-your-crew-to-a-model)
 - [How CrewAI Compares](#how-crewai-compares)
 - [Contribution](#contribution)
-- [Hire CrewAI](#hire-crewai)
 - [Telemetry](#telemetry)
 - [License](#license)
 
@@ -247,11 +246,6 @@ poetry build
 pip install dist/*.tar.gz
 ```
 
-## Hire CrewAI
-
-We're a company developing crewAI and crewAI Enterprise. We, for a limited time, are offering consulting with selected customers; to get them early access to our enterprise solution.
-If you are interested in having access to it, and hiring weekly hours with our team, feel free to email us at [joao@crewai.com](mailto:joao@crewai.com).
-
 ## Telemetry
 
 CrewAI uses anonymous telemetry to collect usage data with the main purpose of helping us improve the library by focusing our efforts on the most used features, integrations and tools.

docs/core-concepts/Tools.md

@@ -107,7 +107,7 @@ Here is a list of the available tools and their descriptions:
 | **DirectoryReadTool**       | Facilitates reading and processing of directory structures and their contents.                |
 | **FileReadTool**            | Enables reading and extracting data from files, supporting various file formats.              |
 | **GithubSearchTool**        | A RAG tool for searching within GitHub repositories, useful for code and documentation search.|
-| **SeperDevTool**            | A specialized tool for development purposes, with specific functionalities under development. |
+| **SerperDevTool**            | A specialized tool for development purposes, with specific functionalities under development. |
 | **TXTSearchTool**           | A RAG tool focused on searching within text (.txt) files, suitable for unstructured data.     |
 | **JSONSearchTool**          | A RAG tool designed for searching within JSON files, catering to structured data handling.     |
 | **MDXSearchTool**           | A RAG tool tailored for searching within Markdown (MDX) files, useful for documentation.      |
@@ -221,4 +221,4 @@ agent = Agent(
 ```
 
 ## Conclusion
-Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively. When building solutions with CrewAI, leverage both custom and existing tools to empower your agents and enhance the AI ecosystem. Consider utilizing error handling, caching mechanisms, and the flexibility of tool arguments to optimize your agents' performance and capabilities.
+Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively. When building solutions with CrewAI, leverage both custom and existing tools to empower your agents and enhance the AI ecosystem. Consider utilizing error handling, caching mechanisms, and the flexibility of tool arguments to optimize your agents' performance and capabilities.

docs/how-to/AgentOps-Observability.md

@@ -1,51 +1,52 @@
 ---
-title: (AgentOps) Observability using AgentOps
+title: Agent Monitoring with AgentOps
 description: Understanding and logging your agent performance with AgentOps.
 ---
 
 # Intro
-Observability is a key aspect of developing and deploying conversational AI agents. It allows developers to understand how the agent is performing, how users are interacting with the agent, and how the agent is responding to user inputs. 
+Observability is a key aspect of developing and deploying conversational AI agents. It allows developers to understand how their agents are performing, how their agents are interacting with users, and how their agents use external tools and APIs. AgentOps is a product independent of CrewAI that provides a comprehensive observability solution for agents. 
 
-AgentOps is a product, idependent of crewAI that provides a comprehensive observability solution for agents. 
-
-This notebook will provide an overview of AgentOps and how to use it with crewAI.
 
 ## AgentOps
 
-[AgentOps](https://agentops.ai) provides session replays, metrics, and monitoring for agents.
-[AgentOps Repo](https://github.com/AgentOps-AI/agentops)
+[AgentOps](https://agentops.ai/?=crew) provides session replays, metrics, and monitoring for agents.
+
+At a high level, AgentOps gives you the ability to monitor cost, token usage, latency, agent failures, session-wide statistics, and more. For more info, check out the [AgentOps Repo](https://github.com/AgentOps-AI/agentops).
 
 ### Overview
-AgentOps provides monotoring for agents in development and production. It provides a dashboard for monitoring agent performance, session replays, and custom reporting.
+AgentOps provides monitoring for agents in development and production. It provides a dashboard for tracking agent performance, session replays, and custom reporting.
 
-![agentops-overview.png](..%2Fassets%2Fagentops-overview.png)
+Additionally, AgentOps provides session drilldowns for viewing Crew agent interactions, LLM calls, and tool usage in real-time. This feature is useful for debugging and understanding how agents interact with users as well as other agents.
 
-Additionally, AgentOps provides session drilldowns that allows users to view the agent's interactions with users in real-time. This feature is useful for debugging and understanding how the agent interacts with users.
-
-![agentops-session.png](..%2Fassets%2Fagentops-session.png)
-![agentops-replay.png](..%2Fassets%2Fagentops-replay.png)
+![Overview of a select series of agent session runs](..%2Fassets%2Fagentops-overview.png)
+![Overview of session drilldowns for examining agent runs](..%2Fassets%2Fagentops-session.png)
+![Viewing a step-by-step agent replay execution graph](..%2Fassets%2Fagentops-replay.png)
 
 ### Features
-- LLM Cost management and tracking
-- Replay Analytics
-- Recursive thought detection
-- Custom Reporting
-- Analytics Dashboard
-- Public Model Testing
-- Custom Tests
-- Time Travel Debugging
-- Compliance and Security
+- **LLM Cost Management and Tracking**: Track spend with foundation model providers
+- **Replay Analytics**: Watch step-by-step agent execution graphs
+- **Recursive Thought Detection**: Identify when agents fall into infinite loops
+- **Custom Reporting**: Create custom analytics on agent performance
+- **Analytics Dashboard**: Monitor high level statistics about agents in development and production
+- **Public Model Testing**: Test your agents against benchmarks and leaderboards
+- **Custom Tests**: Run your agents against domain specific tests
+- **Time Travel Debugging**: Restart your sessions from checkpoints
+- **Compliance and Security**: Create audit logs and detect potential threats such as profanity and PII leaks
+- **Prompt Injection Detection**: Identify potential code injection and secret leaks
 
 ### Using AgentOps
 
-Create a user API key here: app.agentops.ai/account
+1. **Create an API Key:**
+Create a user API key here: [Create API Key](app.agentops.ai/account)
 
+2. **Configure Your Environment:**
 Add your API key to your environment variables
 
 ```
 AGENTOPS_API_KEY=<YOUR_AGENTOPS_API_KEY>
 ```
 
+3. **Install AgentOps:**
 Install AgentOps with:
 ```
 pip install crewai[agentops]
@@ -62,11 +63,26 @@ import agentops
 agentops.init()
 ```
 
+This will initiate an AgentOps session as well as automatically track Crew agents. For further info on how to outfit more complex agentic systems, check out the [AgentOps documentation](https://docs.agentops.ai) or join the [Discord](https://discord.gg/j4f3KbeH).
+
 ### Crew + AgentOps Examples
 - [Job Posting](https://github.com/joaomdmoura/crewAI-examples/tree/main/job-posting)
 - [Markdown Validator](https://github.com/joaomdmoura/crewAI-examples/tree/main/markdown_validator)
 - [Instagram Post](https://github.com/joaomdmoura/crewAI-examples/tree/main/instagram_post)
 
 
-### Futher Information
-To implement more features and better observability, please see the [AgentOps Repo](https://github.com/AgentOps-AI/agentops)
+### Further Information
+
+To get started, create an [AgentOps account](https://agentops.ai/?=crew).
+
+For feature requests or bug reports, please reach out to the AgentOps team on the [AgentOps Repo](https://github.com/AgentOps-AI/agentops).
+
+#### Extra links
+
+<a href="https://twitter.com/agentopsai/">🐦 Twitter</a>
+<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
+<a href="https://discord.gg/JHPt4C7r">📢 Discord</a>
+<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
+<a href="https://app.agentops.ai/?=crew">🖇️ AgentOps Dashboard</a>
+<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
+<a href="https://docs.agentops.ai/introduction">📙 Documentation</a>

docs/how-to/Installing-CrewAI.md

@@ -0,0 +1,21 @@
+---
+title: Installing crewAI
+description: A comprehensive guide to installing crewAI and its dependencies, including the latest updates and installation methods.
+---
+
+# Installing crewAI
+
+Welcome to crewAI! This guide will walk you through the installation process for crewAI and its dependencies. crewAI is a flexible and powerful AI framework that enables you to create and manage AI agents, tools, and tasks efficiently. Let's get started!
+
+## Installation
+
+To install crewAI, you need to have Python >=3.10 and <=3.13 installed on your system:
+
+```shell
+# Install the mains crewAI package
+pip install crewai
+
+# Install the main crewAI package and the tools package
+# that includes a series of helpful tools for your agents
+pip install 'crewai[tools]'
+```

docs/how-to/LLM-Connections.md

@@ -97,10 +97,10 @@ OPENAI_API_KEY=NA
 ```
 
 #### LM Studio
+Launch [LM Studio](https://lmstudio.ai) and go to the Server tab. Then select a model from the dropdown menu then wait for it to load. Once it's loaded, click the green Start Server button and use the URL, port, and API key that's shown (you can modify them). Below is an example of the default settings as of LM Studio 0.2.19:
 ```sh
-OPENAI_API_BASE="http://localhost:8000/v1"
-OPENAI_MODEL_NAME=NA
-OPENAI_API_KEY=NA
+OPENAI_API_BASE="http://localhost:1234/v1"
+OPENAI_API_KEY="lm-studio"
 ```
 
 #### Mistral API

docs/index.md

@@ -43,6 +43,11 @@ Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By
     <div style="width:30%">
         <h2>How-To Guides</h2>
         <ul>
+            <li>
+                <a href="./how-to/Installing-CrewAI">
+                    Installing crewAI
+                </a>
+            </li>
             <li>
                 <a href="./how-to/Creating-a-Crew-and-kick-it-off">
                     Getting Started
@@ -79,8 +84,8 @@ Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By
                 </a>
             </li>
             <li>
-                <a href="./how-to/AgentOps-Observability.md">
-                    Agent Observability using AgentOps
+                <a href="./how-to/AgentOps-Observability">
+                    Agent Monitoring with AgentOps
                 </a>
             </li>
         </ul>

mkdocs.yml

@@ -128,6 +128,7 @@ nav:
     - Collaboration: 'core-concepts/Collaboration.md'
     - Memory: 'core-concepts/Memory.md'
   - How to Guides:
+    - Installing CrewAI: 'how-to/Installing-CrewAI.md'
     - Getting Started: 'how-to/Creating-a-Crew-and-kick-it-off.md'
     - Create Custom Tools: 'how-to/Create-Custom-Tools.md'
     - Using Sequential Process: 'how-to/Sequential.md'
@@ -135,7 +136,7 @@ nav:
     - Connecting to any LLM: 'how-to/LLM-Connections.md'
     - Customizing Agents: 'how-to/Customizing-Agents.md'
     - Human Input on Execution: 'how-to/Human-Input-on-Execution.md'
-    - Agent Observability using AgentOps: 'how-to/AgentOps-Observability.md'
+    - Agent Monitoring with AgentOps: 'how-to/AgentOps-Observability.md'
   - Tools Docs:
     - Google Serper Search: 'tools/SerperDevTool.md'
     - Scrape Website: 'tools/ScrapeWebsiteTool.md'

src/crewai/agent.py

@@ -172,10 +172,14 @@ class Agent(BaseModel):
         """set agent executor is set."""
         if hasattr(self.llm, "model_name"):
             token_handler = TokenCalcHandler(self.llm.model_name, self._token_process)
-            if isinstance(self.llm.callbacks, list):
+
+            # Ensure self.llm.callbacks is a list
+            if not isinstance(self.llm.callbacks, list):
+                self.llm.callbacks = []
+
+            # Check if an instance of TokenCalcHandler already exists in the list
+            if not any(isinstance(handler, TokenCalcHandler) for handler in self.llm.callbacks):
                 self.llm.callbacks.append(token_handler)
-            else:
-                self.llm.callbacks = [token_handler]
 
         if not self.agent_executor:
             if not self.cache_handler:

src/crewai/crew.py

@@ -25,7 +25,7 @@ from crewai.process import Process
 from crewai.task import Task
 from crewai.telemetry import Telemetry
 from crewai.tools.agent_tools import AgentTools
-from crewai.utilities import I18N, Logger, RPMController, FileHandler
+from crewai.utilities import I18N, FileHandler, Logger, RPMController
 try:
   import agentops
 except ImportError:
@@ -90,6 +90,9 @@ class Crew(BaseModel):
     manager_llm: Optional[Any] = Field(
         description="Language model that will run the agent.", default=None
     )
+    manager_agent: Optional[Any] = Field(
+        description="Custom agent that will be used as manager.", default=None
+    )
     manager_callbacks: Optional[List[InstanceOf[BaseCallbackHandler]]] = Field(
         default=None,
         description="A list of callback handlers to be executed by the manager agent when hierarchical process is used",
@@ -174,7 +177,9 @@ class Crew(BaseModel):
     @model_validator(mode="after")
     def check_manager_llm(self):
         """Validates that the language model is set when using hierarchical process."""
-        if self.process == Process.hierarchical and not self.manager_llm:
+        if self.process == Process.hierarchical and (
+            not self.manager_llm and not self.manager_agent
+        ):
             raise PydanticCustomError(
                 "missing_manager_llm",
                 "Attribute `manager_llm` is required when using hierarchical process.",
@@ -311,14 +316,20 @@ class Crew(BaseModel):
         """Creates and assigns a manager agent to make sure the crew completes the tasks."""
 
         i18n = I18N(language=self.language, language_file=self.language_file)
-        manager = Agent(
-            role=i18n.retrieve("hierarchical_manager_agent", "role"),
-            goal=i18n.retrieve("hierarchical_manager_agent", "goal"),
-            backstory=i18n.retrieve("hierarchical_manager_agent", "backstory"),
-            tools=AgentTools(agents=self.agents).tools(),
-            llm=self.manager_llm,
-            verbose=True,
-        )
+        try:
+            self.manager_agent.allow_delegation = (
+                True  # Forcing Allow delegation to the manager
+            )
+            manager = self.manager_agent
+        except:
+            manager = Agent(
+                role=i18n.retrieve("hierarchical_manager_agent", "role"),
+                goal=i18n.retrieve("hierarchical_manager_agent", "goal"),
+                backstory=i18n.retrieve("hierarchical_manager_agent", "backstory"),
+                tools=AgentTools(agents=self.agents).tools(),
+                llm=self.manager_llm,
+                verbose=True,
+            )
 
         task_output = ""
         for task in self.tasks:
@@ -347,7 +358,7 @@ class Crew(BaseModel):
     def _set_tasks_callbacks(self) -> str:
         """Sets callback for every task suing task_callback"""
         for task in self.tasks:
-            task.callback = self.task_callback
+            self.task_callback = task.callback
 
     def _interpolate_inputs(self, inputs: Dict[str, Any]) -> str:
         """Interpolates the inputs in the tasks and agents."""

src/crewai/memory/long_term/long_term_memory.py

@@ -28,5 +28,5 @@ class LongTermMemory(Memory):
             datetime=item.datetime,
         )
 
-    def search(self, task: str, latest_n: int) -> Dict[str, Any]:
+    def search(self, task: str, latest_n: int = 3) -> Dict[str, Any]:
         return self.storage.load(task, latest_n)

tests/crew_test.py

@@ -912,3 +912,35 @@ def test_crew_log_file_output(tmp_path):
     crew = Crew(agents=[researcher], tasks=tasks, output_log_file=str(test_file))
     crew.kickoff()
     assert test_file.exists()
+
+
+@pytest.mark.vcr(filter_headers=["authorization"])
+def test_manager_agent():
+    from unittest.mock import patch
+
+    from langchain_openai import ChatOpenAI
+
+    task = Task(
+        description="Come up with a list of 5 interesting ideas to explore for an article, then write one amazing paragraph highlight for each idea that showcases how good an article about this topic could be. Return the list of ideas with their paragraph and your notes.",
+        expected_output="5 bullet points with a paragraph for each idea.",
+    )
+
+    manager = Agent(
+        role="Manager",
+        goal="Manage the crew and ensure the tasks are completed efficiently.",
+        backstory="You're an experienced manager, skilled in overseeing complex projects and guiding teams to success. Your role is to coordinate the efforts of the crew members, ensuring that each task is completed on time and to the highest standard.",
+        allow_delegation=False,
+        llm=ChatOpenAI(temperature=0, model="gpt-4"),
+    )
+
+    crew = Crew(
+        agents=[researcher, writer],
+        process=Process.hierarchical,
+        manager_agent=manager,
+        tasks=[task],
+    )
+
+    with patch.object(Task, "execute") as execute:
+        crew.kickoff()
+        assert manager.allow_delegation == True
+        execute.assert_called()