preapring to cut new version

* removed hyphen in co-workers

* Fix issue with AgentTool agent selection. The LLM included double quotes in the agent name which messed up the string comparison. Added additional types. Cleaned up error messaging.

* Remove duplicate import

* Improve explanation

* Revert poetry.lock changes

* Fix missing line in poetry.lock

---------

Co-authored-by: madmag77 <goncharov.artemv@gmail.com>

.editorconfig

@@ -0,0 +1,14 @@
+# .editorconfig
+root = true
+
+# All files
+[*]
+charset = utf-8
+end_of_line = lf
+insert_final_newline = true
+trim_trailing_whitespace = true
+
+# Python files
+[*.py]
+indent_style = space
+indent_size = 2

.github/workflows/black.yml

@@ -1,10 +0,0 @@
-name: Lint
-
-on: [push, pull_request]
-
-jobs:
-  lint:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v3
-      - uses: psf/black@stable

.github/workflows/linter.yml

@@ -0,0 +1,16 @@
+name: Lint
+
+on: [pull_request]
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Install Requirements
+        run: |
+          pip install ruff
+
+      - name: Run Ruff Linter
+        run: ruff check --exclude "templates","__init__.py"

.github/workflows/mkdocs.yml

@@ -1,7 +1,7 @@
 name: Deploy MkDocs
 
 on:
-  workflow_dispatch:    
+  workflow_dispatch:
   push:
     branches:
       - main
@@ -22,11 +22,23 @@ jobs:
         with:
           python-version: '3.10'
 
+      - name: Calculate requirements hash
+        id: req-hash
+        run: echo "::set-output name=hash::$(sha256sum requirements-doc.txt | awk '{print $1}')"
+
+      - name: Setup cache
+        uses: actions/cache@v3
+        with:
+          key: mkdocs-material-${{ steps.req-hash.outputs.hash }}
+          path: .cache
+          restore-keys: |
+            mkdocs-material-
+
       - name: Install Requirements
         run: |
           sudo apt-get update &&
           sudo apt-get install pngquant &&
-          pip install mkdocs-material
+          pip install mkdocs-material mkdocs-material-extensions pillow cairosvg
 
         env:
           GH_TOKEN: ${{ secrets.GH_TOKEN }}

.github/workflows/tests.yml

@@ -1,6 +1,6 @@
 name: Run Tests
 
-on: [push, pull_request]
+on: [pull_request]
 
 permissions:
   contents: write
@@ -14,19 +14,18 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
 
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
-          python-version: '3.10'
+          python-version: "3.10"
 
       - name: Install Requirements
         run: |
-          sudo apt-get update &&
-          pip install poetry &&
+          pip install poetry
           poetry lock &&
           poetry install
 
       - name: Run tests
-        run: poetry run pytest
+        run: poetry run pytest tests

.github/workflows/type-checker.yml

@@ -0,0 +1,26 @@
+name: Run Type Checks
+
+on: [pull_request]
+
+permissions:
+  contents: write
+
+jobs:
+  type-checker:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.10"
+
+      - name: Install Requirements
+        run: |
+          pip install mypy
+
+      - name: Run type checks
+        run: mypy src

.gitignore

@@ -5,4 +5,10 @@ dist/
 .env
 assets/*
 .idea
-test.py
+test/
+docs_crew/
+chroma.sqlite3
+old_en.json
+db/
+test.py
+rc-tests/*

.pre-commit-config.yaml

@@ -1,21 +1,9 @@
 repos:
-
-  - repo: https://github.com/psf/black-pre-commit-mirror
-    rev: 23.12.1
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.4.4
     hooks:
-      - id: black
-        language_version: python3.11
-        files: \.(py)$
-
-  - repo: https://github.com/pycqa/isort
-    rev: 5.13.2
-    hooks:
-      - id: isort
-        name: isort (python)
-        args: ["--profile", "black", "--filter-files"]
-
-  - repo: https://github.com/PyCQA/autoflake
-    rev: v2.2.1
-    hooks:
-      - id: autoflake
-        args: ['--in-place', '--remove-all-unused-imports', '--remove-unused-variables', '--ignore-init-module-imports']
+      - id: ruff
+        args: ["--fix"]
+        exclude: "templates"
+      - id: ruff-format
+        exclude: "templates"

README.md

@@ -1,71 +1,87 @@
-# crewAI
+<div align="center">
 
-![Logo of crewAI, tow people rowing on a boat](./docs/crewai_logo.png)
+![Logo of crewAI, two people rowing on a boat](./docs/crewai_logo.png)
 
-🤖 Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
+# **crewAI**
 
-- [crewAI](#crewai)
-  - [Why CrewAI?](#why-crewai)
-  - [Getting Started](#getting-started)
-  - [Key Features](#key-features)
-  - [Examples](#examples)
-    - [Code](#code)
-    - [Video](#video)
-      - [Quick Tutorial](#quick-tutorial)
-      - [Trip Planner](#trip-planner)
-      - [Stock Analysis](#stock-analysis)
-  - [Connecting Your Crew to a Model](#connecting-your-crew-to-a-model)
-  - [How CrewAI Compares](#how-crewai-compares)
-  - [Contribution](#contribution)
-    - [Installing Dependencies](#installing-dependencies)
-    - [Virtual Env](#virtual-env)
-    - [Pre-commit hooks](#pre-commit-hooks)
-    - [Running Tests](#running-tests)
-    - [Packaging](#packaging)
-    - [Installing Locally](#installing-locally)
-  - [Hire CrewAI](#hire-crewai)
-  - [License](#license)
+🤖 **crewAI**: Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
+
+<h3>
+
+[Homepage](https://www.crewai.io/) | [Documentation](https://docs.crewai.com/) | [Chat with Docs](https://chatg.pt/DWjSBZn) | [Examples](https://github.com/joaomdmoura/crewai-examples) | [Discord](https://discord.com/invite/X4JWnZnxPb)
+
+</h3>
+
+[![GitHub Repo stars](https://img.shields.io/github/stars/joaomdmoura/crewAI)](https://github.com/joaomdmoura/crewAI)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+
+</div>
+
+## Table of contents
+
+- [Why CrewAI?](#why-crewai)
+- [Getting Started](#getting-started)
+- [Key Features](#key-features)
+- [Examples](#examples)
+  - [Quick Tutorial](#quick-tutorial)
+  - [Write Job Descriptions](#write-job-descriptions)
+  - [Trip Planner](#trip-planner)
+  - [Stock Analysis](#stock-analysis)
+- [Connecting Your Crew to a Model](#connecting-your-crew-to-a-model)
+- [How CrewAI Compares](#how-crewai-compares)
+- [Contribution](#contribution)
+- [Telemetry](#telemetry)
+- [License](#license)
 
 ## Why CrewAI?
 
 The power of AI collaboration has too much to offer.
 CrewAI is designed to enable AI agents to assume roles, share goals, and operate in a cohesive unit - much like a well-oiled crew. Whether you're building a smart assistant platform, an automated customer service ensemble, or a multi-agent research team, CrewAI provides the backbone for sophisticated multi-agent interactions.
 
-- 🤖 [Talk with the Docs](https://chatg.pt/DWjSBZn)
-- 📄 [Documentation Wiki](https://joaomdmoura.github.io/crewAI/)
-
 ## Getting Started
 
 To get started with CrewAI, follow these simple steps:
 
-1. **Installation**:
+### 1. Installation
 
 ```shell
 pip install crewai
 ```
 
-The example below also uses duckduckgo, so also install that
+If you want to install the 'crewai' package along with its optional features that include additional tools for agents, you can do so by using the following command: pip install 'crewai[tools]'. This command installs the basic package and also adds extra components which require more dependencies to function."
+
 ```shell
-pip install duckduckgo-search
+pip install 'crewai[tools]'
 ```
 
-2. **Setting Up Your Crew**:
+### 2. Setting Up Your Crew
 
 ```python
 import os
 from crewai import Agent, Task, Crew, Process
+from crewai_tools import SerperDevTool
 
-os.environ["OPENAI_API_KEY"] = "YOUR KEY"
+os.environ["OPENAI_API_KEY"] = "YOUR_API_KEY"
+os.environ["SERPER_API_KEY"] = "Your Key" # serper.dev API key
 
-# You can choose to use a local model through Ollama for example. See ./docs/llm-connections.md for more information.
-# from langchain.llms import Ollama
-# ollama_llm = Ollama(model="openhermes")
+# You can choose to use a local model through Ollama for example. See https://docs.crewai.com/how-to/LLM-Connections/ for more information.
 
-# Install duckduckgo-search for this example:
-# !pip install -U duckduckgo-search
+# os.environ["OPENAI_API_BASE"] = 'http://localhost:11434/v1'
+# os.environ["OPENAI_MODEL_NAME"] ='openhermes'  # Adjust based on available model
+# os.environ["OPENAI_API_KEY"] ='sk-111111111111111111111111111111111111111111111111'
 
-from langchain.tools import DuckDuckGoSearchRun
-search_tool = DuckDuckGoSearchRun()
+# You can pass an optional llm attribute specifying what model you wanna use.
+# It can be a local model through Ollama / LM Studio or a remote
+# model like OpenAI, Mistral, Antrophic or others (https://docs.crewai.com/how-to/LLM-Connections/)
+#
+# import os
+# os.environ['OPENAI_MODEL_NAME'] = 'gpt-3.5-turbo'
+#
+# OR
+#
+# from langchain_openai import ChatOpenAI
+
+search_tool = SerperDevTool()
 
 # Define your agents with roles and goals
 researcher = Agent(
@@ -73,37 +89,27 @@ researcher = Agent(
   goal='Uncover cutting-edge developments in AI and data science',
   backstory="""You work at a leading tech think tank.
   Your expertise lies in identifying emerging trends.
-  You have a knack for dissecting complex data and presenting
-  actionable insights.""",
+  You have a knack for dissecting complex data and presenting actionable insights.""",
   verbose=True,
   allow_delegation=False,
+  # You can pass an optional llm attribute specifying what model you wanna use.
+  # llm=ChatOpenAI(model_name="gpt-3.5", temperature=0.7),
   tools=[search_tool]
-  # You can pass an optional llm attribute specifying what mode you wanna use.
-  # It can be a local model through Ollama / LM Studio or a remote
-  # model like OpenAI, Mistral, Antrophic or others (https://python.langchain.com/docs/integrations/llms/)
-  #
-  # Examples:
-  # llm=ollama_llm # was defined above in the file
-  # llm=OpenAI(model_name="gpt-3.5", temperature=0.7)
-  # For the OpenAI model you would need to import
-  # from langchain_openai import OpenAI
 )
 writer = Agent(
   role='Tech Content Strategist',
   goal='Craft compelling content on tech advancements',
-  backstory="""You are a renowned Content Strategist, known for
-  your insightful and engaging articles.
+  backstory="""You are a renowned Content Strategist, known for your insightful and engaging articles.
   You transform complex concepts into compelling narratives.""",
   verbose=True,
-  allow_delegation=True,
-  # (optional) llm=ollama_llm
+  allow_delegation=True
 )
 
 # Create tasks for your agents
 task1 = Task(
   description="""Conduct a comprehensive analysis of the latest advancements in AI in 2024.
-  Identify key trends, breakthrough technologies, and potential industry impacts.
-  Your final answer MUST be a full analysis report""",
+  Identify key trends, breakthrough technologies, and potential industry impacts.""",
+  expected_output="Full analysis report in bullet points",
   agent=researcher
 )
 
@@ -111,8 +117,8 @@ task2 = Task(
   description="""Using the insights provided, develop an engaging blog
   post that highlights the most significant AI advancements.
   Your post should be informative yet accessible, catering to a tech-savvy audience.
-  Make it sound cool, avoid complex words so it doesn't sound like AI.
-  Your final answer MUST be the full blog post of at least 4 paragraphs.""",
+  Make it sound cool, avoid complex words so it doesn't sound like AI.""",
+  expected_output="Full blog post of at least 4 paragraphs",
   agent=writer
 )
 
@@ -130,47 +136,60 @@ print("######################")
 print(result)
 ```
 
-Currently the only supported process is `Process.sequential`, where one task is executed after the other and the outcome of one is passed as extra content into this next.
-
+In addition to the sequential process, you can use the hierarchical process, which automatically assigns a manager to the defined crew to properly coordinate the planning and execution of tasks through delegation and validation of results. [See more about the processes here](https://docs.crewai.com/core-concepts/Processes/).
 
 ## Key Features
 
 - **Role-Based Agent Design**: Customize agents with specific roles, goals, and tools.
 - **Autonomous Inter-Agent Delegation**: Agents can autonomously delegate tasks and inquire amongst themselves, enhancing problem-solving efficiency.
 - **Flexible Task Management**: Define tasks with customizable tools and assign them to agents dynamically.
-- **Processes Driven**: Currently only supports `sequential` task execution but more complex processes like consensual and hierarchical being worked on.
-- **Works with Open Source Models**: Run your crew using Open AI or open source models refer to the [Connect crewAI to LLMs](./docs/llm-connections.md) page for details on configuring you agents' connections to models, even ones running locally!
+- **Processes Driven**: Currently only supports `sequential` task execution and `hierarchical` processes, but more complex processes like consensual and autonomous are being worked on.
+- **Save output as file**: Save the output of individual tasks as a file, so you can use it later.
+- **Parse output as Pydantic or Json**: Parse the output of individual tasks as a Pydantic model or as a Json if you want to.
+- **Works with Open Source Models**: Run your crew using Open AI or open source models refer to the [Connect crewAI to LLMs](https://docs.crewai.com/how-to/LLM-Connections/) page for details on configuring your agents' connections to models, even ones running locally!
 
 ![CrewAI Mind Map](./docs/crewAI-mindmap.png "CrewAI Mind Map")
 
 ## Examples
-You can test different real life examples of AI crews [in the examples repo](https://github.com/joaomdmoura/crewAI-examples?tab=readme-ov-file)
 
-### Code
+You can test different real life examples of AI crews in the [crewAI-examples repo](https://github.com/joaomdmoura/crewAI-examples?tab=readme-ov-file):
+
+- [Landing Page Generator](https://github.com/joaomdmoura/crewAI-examples/tree/main/landing_page_generator)
+- [Having Human input on the execution](https://docs.crewai.com/how-to/Human-Input-on-Execution)
 - [Trip Planner](https://github.com/joaomdmoura/crewAI-examples/tree/main/trip_planner)
 - [Stock Analysis](https://github.com/joaomdmoura/crewAI-examples/tree/main/stock_analysis)
-- [Landing Page Generator](https://github.com/joaomdmoura/crewAI-examples/tree/main/landing_page_generator)
-- [Having Human input on the execution](./docs/how-to/Human-Input-on-Execution.md)
 
-### Video
-#### Quick Tutorial
-[![CrewAI Tutorial](https://img.youtube.com/vi/tnejrr-0a94/0.jpg)](https://www.youtube.com/watch?v=tnejrr-0a94 "CrewAI Tutorial")
+### Quick Tutorial
 
-#### Trip Planner
-[![Trip Planner](https://img.youtube.com/vi/xis7rWp-hjs/0.jpg)](https://www.youtube.com/watch?v=xis7rWp-hjs "Trip Planner")
+[![CrewAI Tutorial](https://img.youtube.com/vi/tnejrr-0a94/maxresdefault.jpg)](https://www.youtube.com/watch?v=tnejrr-0a94 "CrewAI Tutorial")
 
-#### Stock Analysis
-[![Stock Analysis](https://img.youtube.com/vi/e0Uj4yWdaAg/0.jpg)](https://www.youtube.com/watch?v=e0Uj4yWdaAg "Stock Analysis")
+### Write Job Descriptions
+
+[Check out code for this example](https://github.com/joaomdmoura/crewAI-examples/tree/main/job-posting) or watch a video below:
+
+[![Jobs postings](https://img.youtube.com/vi/u98wEMz-9to/maxresdefault.jpg)](https://www.youtube.com/watch?v=u98wEMz-9to "Jobs postings")
+
+### Trip Planner
+
+[Check out code for this example](https://github.com/joaomdmoura/crewAI-examples/tree/main/trip_planner) or watch a video below:
+
+[![Trip Planner](https://img.youtube.com/vi/xis7rWp-hjs/maxresdefault.jpg)](https://www.youtube.com/watch?v=xis7rWp-hjs "Trip Planner")
+
+### Stock Analysis
+
+[Check out code for this example](https://github.com/joaomdmoura/crewAI-examples/tree/main/stock_analysis) or watch a video below:
+
+[![Stock Analysis](https://img.youtube.com/vi/e0Uj4yWdaAg/maxresdefault.jpg)](https://www.youtube.com/watch?v=e0Uj4yWdaAg "Stock Analysis")
 
 ## Connecting Your Crew to a Model
 
 crewAI supports using various LLMs through a variety of connection options. By default your agents will use the OpenAI API when querying the model. However, there are several other ways to allow your agents to connect to models. For example, you can configure your agents to use a local model via the Ollama tool.
 
-Please refer to the [Connect crewAI to LLMs](./docs/how-to/llm-connections.md) page for details on configuring you agents' connections to models.
+Please refer to the [Connect crewAI to LLMs](https://docs.crewai.com/how-to/LLM-Connections/) page for details on configuring you agents' connections to models.
 
 ## How CrewAI Compares
 
-- **Autogen**: While Autogen excels in creating conversational agents capable of working together, it lacks an inherent concept of process. In Autogen, orchestrating agents' interactions requires additional programming, which can become complex and cumbersome as the scale of tasks grows.
+- **Autogen**: While Autogen does good in creating conversational agents capable of working together, it lacks an inherent concept of process. In Autogen, orchestrating agents' interactions requires additional programming, which can become complex and cumbersome as the scale of tasks grows.
 
 - **ChatDev**: ChatDev introduced the idea of processes into the realm of AI agents, but its implementation is quite rigid. Customizations in ChatDev are limited and not geared towards production environments, which can hinder scalability and flexibility in real-world applications.
 
@@ -187,12 +206,14 @@ CrewAI is open-source and we welcome contributions. If you're looking to contrib
 - We appreciate your input!
 
 ### Installing Dependencies
+
 ```bash
 poetry lock
 poetry install
 ```
 
 ### Virtual Env
+
 ```bash
 poetry shell
 ```
@@ -204,25 +225,60 @@ pre-commit install
 ```
 
 ### Running Tests
+
 ```bash
 poetry run pytest
 ```
 
+### Running static type checks
+
+```bash
+poetry run mypy
+```
+
 ### Packaging
+
 ```bash
 poetry build
 ```
 
 ### Installing Locally
+
 ```bash
 pip install dist/*.tar.gz
 ```
 
-## Hire CrewAI
-We're a company developing crewAI and crewAI Enterprise, we for a limited time are offer consulting with selected customers, to get them early access to our enterprise solution
-If you are interested on having access to it and hiring weekly hours with our team, feel free to email us at [sales@crewai.io](mailto:sales@crewai.io)
+## Telemetry
+
+CrewAI uses anonymous telemetry to collect usage data with the main purpose of helping us improve the library by focusing our efforts on the most used features, integrations and tools.
+
+There is NO data being collected on the prompts, tasks descriptions agents backstories or goals nor tools usage, no API calls, nor responses nor any data that is being processed by the agents, nor any secrets and env vars.
+
+Data collected includes:
+
+- Version of crewAI
+  - So we can understand how many users are using the latest version
+- Version of Python
+  - So we can decide on what versions to better support
+- General OS (e.g. number of CPUs, macOS/Windows/Linux)
+  - So we know what OS we should focus on and if we could build specific OS related features
+- Number of agents and tasks in a crew
+  - So we make sure we are testing internally with similar use cases and educate people on the best practices
+- Crew Process being used
+  - Understand where we should focus our efforts
+- If Agents are using memory or allowing delegation
+  - Understand if we improved the features or maybe even drop them
+- If Tasks are being executed in parallel or sequentially
+  - Understand if we should focus more on parallel execution
+- Language model being used
+  - Improved support on most used languages
+- Roles of agents in a crew
+  - Understand high level use cases so we can build better tools, integrations and examples about it
+- Tools names available
+  - Understand out of the publically available tools, which ones are being used the most so we can improve them
+
+Users can opt-in sharing the complete telemetry data by setting the `share_crew` attribute to `True` on their Crews.
 
 ## License
-CrewAI is released under the MIT License
-
 
+CrewAI is released under the MIT License.

crewAI.excalidraw

@@ -1463,11 +1463,11 @@
       "locked": false,
       "fontSize": 20,
       "fontFamily": 3,
-      "text": "Agents have the inert ability of\nreach out to another to delegate\nwork or ask questions.",
+      "text": "Agents have the innate ability of\nreach out to another to delegate\nwork or ask questions.",
       "textAlign": "right",
       "verticalAlign": "top",
       "containerId": null,
-      "originalText": "Agents have the inert ability of\nreach out to another to delegate\nwork or ask questions.",
+      "originalText": "Agents have the innate ability of\nreach out to another to delegate\nwork or ask questions.",
       "lineHeight": 1.2,
       "baseline": 68
     },
@@ -1734,4 +1734,4 @@
     "viewBackgroundColor": "#ffffff"
   },
   "files": {}
-}
+}

docs/CNAME

@@ -0,0 +1 @@
+docs.crewai.com

docs/core-concepts/Agent-Tools.md

@@ -1,90 +0,0 @@
-# What is a Tool?
-
-A tool in CrewAI is a function or capability that an agent can utilize to perform actions, gather information, or interact with external systems, behind the scenes tools are [LangChain Tools](https://python.langchain.com/docs/modules/agents/tools/).
-These tools can be as straightforward as a search function or as sophisticated as integrations with other chains or APIs.
-
-## Key Characteristics of Tools
-
-- **Utility**: Tools are designed to serve specific purposes, such as searching the web, analyzing data, or generating content.
-- **Integration**: Tools can be integrated into agents to extend their capabilities beyond their basic functions.
-- **Customizability**: Developers can create custom tools tailored to the specific needs of their agents or use pre-built LangChain ones available in the ecosystem.
-
-# Creating your own Tools
-
-You can easily create your own tool using [LangChain Tool Custom Tool Creation](https://python.langchain.com/docs/modules/agents/tools/custom_tools).
-
-Example:
-```python
-import json
-import requests
-
-from crewai import Agent
-from langchain.tools import tool
-from unstructured.partition.html import partition_html
-
-class BrowserTools():
-  @tool("Scrape website content")
-  def scrape_website(website):
-    """Useful to scrape a website content"""
-    url = f"https://chrome.browserless.io/content?token={config('BROWSERLESS_API_KEY')}"
-    payload = json.dumps({"url": website})
-    headers = {
-      'cache-control': 'no-cache',
-      'content-type': 'application/json'
-    }
-    response = requests.request("POST", url, headers=headers, data=payload)
-    elements = partition_html(text=response.text)
-    content = "\n\n".join([str(el) for el in elements])
-
-    # Return only the first 5k characters
-    return content[:5000]
-
-
-# Create an agent and assign the scrapping tool
-agent = Agent(
-  role='Research Analyst',
-  goal='Provide up-to-date market analysis',
-  backstory='An expert analyst with a keen eye for market trends.',
-  tools=[BrowserTools().scrape_website]
-)
-```
-
-# Using Existing Tools
-
-Check [LangChain Integration](https://python.langchain.com/docs/integrations/tools/) for a set of useful existing tools.
-To assign a tool to an agent, you'd provide it as part of the agent's properties during initialization.
-
-```python
-from crewai import Agent
-from langchain.agents import Tool
-from langchain.utilities import GoogleSerperAPIWrapper
-
-# Initialize SerpAPI tool with your API key
-os.environ["OPENAI_API_KEY"] = "Your Key"
-os.environ["SERPER_API_KEY"] = "Your Key"
-
-search = GoogleSerperAPIWrapper()
-
-# Create tool to be used by agent
-serper_tool = Tool(
-  name="Intermediate Answer",
-  func=search.run,
-  description="useful for when you need to ask with search",
-)
-
-# Create an agent and assign the search tool
-agent = Agent(
-  role='Research Analyst',
-  goal='Provide up-to-date market analysis',
-  backstory='An expert analyst with a keen eye for market trends.',
-  tools=[serper_tool]
-)
-```
-
-# Tool Interaction
-
-Tools enhance an agent's ability to perform tasks autonomously or in collaboration with other agents. For instance, an agent might use a search tool to gather information, then pass that data to another agent specialized in analysis.
-
-# Conclusion
-
-Tools are vital components that expand the functionality of agents within the CrewAI framework. They enable agents to perform a wide range of actions and collaborate effectively with one another. As you build with CrewAI, consider the array of tools you can leverage to empower your agents and how they can be interwoven to create a robust AI ecosystem.

docs/core-concepts/Agents.md

@@ -0,0 +1,100 @@
+---
+title: crewAI Agents
+description: What are crewAI Agents and how to use them.
+---
+
+## What is an Agent?
+!!! note "What is an Agent?"
+    An agent is an **autonomous unit** programmed to:
+    <ul>
+      <li class='leading-3'>Perform tasks</li>
+      <li class='leading-3'>Make decisions</li>
+      <li class='leading-3'>Communicate with other agents</li>
+    </ul>
+      <br/>
+    Think of an agent as a member of a team, with specific skills and a particular job to do. Agents can have different roles like 'Researcher', 'Writer', or 'Customer Support', each contributing to the overall goal of the crew.
+
+## Agent Attributes
+
+| Attribute                  | Description                                                                                                                                                                                                                                    |
+| :------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Role**                   | Defines the agent's function within the crew. It determines the kind of tasks the agent is best suited for.                                                                                                                                    |
+| **Goal**                   | The individual objective that the agent aims to achieve. It guides the agent's decision-making process.                                                                                                                                        |
+| **Backstory**              | Provides context to the agent's role and goal, enriching the interaction and collaboration dynamics.                                                                                                                                           |
+| **LLM** *(optional)*       | Represents the language model that will run the agent. It dynamically fetches the model name from the `OPENAI_MODEL_NAME` environment variable, defaulting to "gpt-4" if not specified.                                                         |
+| **Tools** *(optional)*     | Set of capabilities or functions that the agent can use to perform tasks. Expected to be instances of custom classes compatible with the agent's execution environment. Tools are initialized with a default value of an empty list.             |
+| **Function Calling LLM** *(optional)* | Specifies the language model that will handle the tool calling for this agent, overriding the crew function calling LLM if passed. Default is `None`.                                                                                          |
+| **Max Iter** *(optional)*  | `max_iter` is the maximum number of iterations the agent can perform before being forced to give its best answer. Default is `25`.                                                                                                                           |
+| **Max RPM** *(optional)*   | `max_rpm` is Tte maximum number of requests per minute the agent can perform to avoid rate limits. It's optional and can be left unspecified, with a default value of `None`.                                                                               |
+| **Max Execution Time** *(optional)*   | `max_execution_time` is the Maximum execution time for an agent to execute a task. It's optional and can be left unspecified, with a default value of `None`, meaning no max execution time.                                                                     |
+| **Verbose** *(optional)*   | Setting this to `True` configures the internal logger to provide detailed execution logs, aiding in debugging and monitoring. Default is `False`.                                                                                              |
+| **Allow Delegation** *(optional)* | Agents can delegate tasks or questions to one another, ensuring that each task is handled by the most suitable agent. Default is `True`.                                                                                                       |
+| **Step Callback** *(optional)* | A function that is called after each step of the agent. This can be used to log the agent's actions or to perform other operations. It will overwrite the crew `step_callback`.                                                               |
+| **Cache** *(optional)*     | Indicates if the agent should use a cache for tool usage. Default is `True`.                                                                                                                                                                  |
+| **System Template** *(optional)*     | Specifies the system format for the agent. Default is `None`.                                                                                                                                                                  |
+| **Prompt Template** *(optional)*     | Specifies the prompt format for the agent. Default is `None`.                                                                                                                                                                  |
+| **Response Template** *(optional)*     | Specifies the response format for the agent. Default is `None`.                                                                                                                                                                  |
+## Creating an Agent
+
+!!! note "Agent Interaction"
+    Agents can interact with each other using crewAI's built-in delegation and communication mechanisms. This allows for dynamic task management and problem-solving within the crew.
+
+To create an agent, you would typically initialize an instance of the `Agent` class with the desired properties. Here's a conceptual example including all attributes:
+
+```python
+# Example: Creating an agent with all attributes
+from crewai import Agent
+
+agent = Agent(
+  role='Data Analyst',
+  goal='Extract actionable insights',
+  backstory="""You're a data analyst at a large company.
+  You're responsible for analyzing data and providing insights
+  to the business.
+  You're currently working on a project to analyze the
+  performance of our marketing campaigns.""",
+  tools=[my_tool1, my_tool2],  # Optional, defaults to an empty list
+  llm=my_llm,  # Optional
+  function_calling_llm=my_llm,  # Optional
+  max_iter=15,  # Optional
+  max_rpm=None, # Optional
+  max_execution_time=None, # Optional
+  verbose=True,  # Optional
+  allow_delegation=True,  # Optional
+  step_callback=my_intermediate_step_callback,  # Optional
+  cache=True,  # Optional
+  system_template=my_system_template,  # Optional
+  prompt_template=my_prompt_template,  # Optional
+  response_template=my_response_template,  # Optional
+  config=my_config,  # Optional
+  crew=my_crew,  # Optional
+  tools_handler=my_tools_handler,  # Optional
+  cache_handler=my_cache_handler,  # Optional
+  callbacks=[callback1, callback2],  # Optional
+  agent_executor=my_agent_executor  # Optional
+)
+```
+
+## Setting prompt templates
+
+Prompt templates are used to format the prompt for the agent. You can use to update the system, regular and response templates for the agent. Here's an example of how to set prompt templates:
+
+```python
+agent = Agent(
+        role="{topic} specialist",
+        goal="Figure {goal} out",
+        backstory="I am the master of {role}",
+        system_template="""<|start_header_id|>system<|end_header_id|>
+
+{{ .System }}<|eot_id|>""",
+        prompt_template="""<|start_header_id|>user<|end_header_id|>
+
+{{ .Prompt }}<|eot_id|>""",
+        response_template="""<|start_header_id|>assistant<|end_header_id|>
+
+{{ .Response }}<|eot_id|>""",
+    )
+```
+
+## Conclusion
+Agents are the building blocks of the CrewAI framework. By understanding how to define and interact with agents, you can create sophisticated AI systems that leverage the power of collaborative intelligence.

docs/core-concepts/Collaboration.md

@@ -0,0 +1,42 @@
+---
+title: How Agents Collaborate in CrewAI
+description: Exploring the dynamics of agent collaboration within the CrewAI framework, focusing on the newly integrated features for enhanced functionality.
+---
+
+## Collaboration Fundamentals
+!!! note "Core of Agent Interaction"
+    Collaboration in CrewAI is fundamental, enabling agents to combine their skills, share information, and assist each other in task execution, embodying a truly cooperative ecosystem.
+
+- **Information Sharing**: Ensures all agents are well-informed and can contribute effectively by sharing data and findings.
+- **Task Assistance**: Allows agents to seek help from peers with the required expertise for specific tasks.
+- **Resource Allocation**: Optimizes task execution through the efficient distribution and sharing of resources among agents.
+
+## Enhanced Attributes for Improved Collaboration
+The `Crew` class has been enriched with several attributes to support advanced functionalities:
+
+- **Language Model Management (`manager_llm`, `function_calling_llm`)**: Manages language models for executing tasks and tools, facilitating sophisticated agent-tool interactions. Note that while `manager_llm` is mandatory for hierarchical processes to ensure proper execution flow, `function_calling_llm` is optional, with a default value provided for streamlined tool interaction.
+- **Custom Manager Agent (`manager_agent`)**: Allows specifying a custom agent as the manager instead of using the default manager provided by CrewAI.
+- **Process Flow (`process`)**: Defines the execution logic (e.g., sequential, hierarchical) to streamline task distribution and execution.
+- **Verbose Logging (`verbose`)**: Offers detailed logging capabilities for monitoring and debugging purposes. It supports both integer and boolean types to indicate the verbosity level. For example, setting `verbose` to 1 might enable basic logging, whereas setting it to True enables more detailed logs.
+- **Rate Limiting (`max_rpm`)**: Ensures efficient utilization of resources by limiting requests per minute. Guidelines for setting `max_rpm` should consider the complexity of tasks and the expected load on resources.
+- **Internationalization / Customization Support (`language`, `prompt_file`)**: Facilitates full customization of the inner prompts, enhancing global usability. Supported languages and the process for utilizing the `prompt_file` attribute for customization should be clearly documented. [Example of file](https://github.com/joaomdmoura/crewAI/blob/main/src/crewai/translations/en.json)
+- **Execution and Output Handling (`full_output`)**: Distinguishes between full and final outputs for nuanced control over task results. Examples showcasing the difference in outputs can aid in understanding the practical implications of this attribute.
+- **Callback and Telemetry (`step_callback`, `task_callback`)**: Integrates callbacks for step-wise and task-level execution monitoring, alongside telemetry for performance analytics. The purpose and usage of `task_callback` alongside `step_callback` for granular monitoring should be clearly explained.
+- **Crew Sharing (`share_crew`)**: Enables sharing of crew information with CrewAI for continuous improvement and training models. The privacy implications and benefits of this feature, including how it contributes to model improvement, should be outlined.
+- **Usage Metrics (`usage_metrics`)**: Stores all metrics for the language model (LLM) usage during all tasks' execution, providing insights into operational efficiency and areas for improvement. Detailed information on accessing and interpreting these metrics for performance analysis should be provided.
+- **Memory Usage (`memory`)**: Indicates whether the crew should use memory to store memories of its execution, enhancing task execution and agent learning.
+- **Embedder Configuration (`embedder`)**: Specifies the configuration for the embedder to be used by the crew for understanding and generating language. This attribute supports customization of the language model provider.
+- **Cache Management (`cache`)**: Determines whether the crew should use a cache to store the results of tool executions, optimizing performance.
+- **Output Logging (`output_log_file`)**: Specifies the file path for logging the output of the crew execution.
+
+## Delegation: Dividing to Conquer
+Delegation enhances functionality by allowing agents to intelligently assign tasks or seek help, thereby amplifying the crew's overall capability.
+
+## Implementing Collaboration and Delegation
+Setting up a crew involves defining the roles and capabilities of each agent. CrewAI seamlessly manages their interactions, ensuring efficient collaboration and delegation, with enhanced customization and monitoring features to adapt to various operational needs.
+
+## Example Scenario
+Consider a crew with a researcher agent tasked with data gathering and a writer agent responsible for compiling reports. The integration of advanced language model management and process flow attributes allows for more sophisticated interactions, such as the writer delegating complex research tasks to the researcher or querying specific information, thereby facilitating a seamless workflow.
+
+## Conclusion
+The integration of advanced attributes and functionalities into the CrewAI framework significantly enriches the agent collaboration ecosystem. These enhancements not only simplify interactions but also offer unprecedented flexibility and control, paving the way for sophisticated AI-driven solutions capable of tackling complex tasks through intelligent collaboration and delegation.

docs/core-concepts/Creating-Tasks.md

@@ -1,50 +0,0 @@
-# What is a Task?
-
-A Task in CrewAI is essentially a job or an assignment that an AI agent needs to complete. It's defined by what needs to be done and can include additional information like which agent should do it and what tools they might need.
-
-# Task Properties
-
-- **Description**: A clear, concise statement of what the task entails.
-- **Agent**: Optionally, you can specify which agent is responsible for the task. If not, the crew's process will determine who takes it on.
-- **Tools**: These are the functions or capabilities the agent can utilize to perform the task. They can be anything from simple actions like 'search' to more complex interactions with other agents or APIs.
-
-# Integrating Tools with Tasks
-
-In CrewAI, tools are functions from the `langchain` toolkit that agents can use to interact with the world. These can be generic utilities or specialized functions designed for specific actions. When you assign tools to a task, they empower the agent to perform its duties more effectively.
-
-## Example of Creating a Task with Tools
-
-```python
-from crewai import Task
-from langchain.agents import Tool
-from langchain.utilities import GoogleSerperAPIWrapper
-
-# Initialize SerpAPI tool with your API key
-os.environ["OPENAI_API_KEY"] = "Your Key"
-os.environ["SERPER_API_KEY"] = "Your Key"
-
-search = GoogleSerperAPIWrapper()
-
-# Create tool to be used by agent
-serper_tool = Tool(
-  name="Intermediate Answer",
-  func=search.run,
-  description="useful for when you need to ask with search",
-)
-
-# Create a task with a description and the search tool
-task = Task(
-  description='Find and summarize the latest and most relevant news on AI',
-  tools=[serper_tool]
-)
-```
-
-When the task is executed by an agent, the tools specified in the task will override the agent's default tools. This means that for the duration of this task, the agent will use the search tool provided, even if it has other tools assigned to it.
-
-# Tool Override Mechanism
-
-The ability to override an agent's tools with those specified in a task allows for greater flexibility. An agent might generally use a set of standard tools, but for certain tasks, you may want it to use a particular tool that is more suited to the task at hand.
-
-# Conclusion
-
-Creating tasks with the right tools is crucial in CrewAI. It ensures that your agents are not only aware of what they need to do but are also equipped with the right functions to do it effectively. This feature underlines the flexibility and power of the CrewAI system, where tasks can be tailored with specific tools to achieve the best outcome.

docs/core-concepts/Crews.md

@@ -0,0 +1,155 @@
+---
+title: crewAI Crews
+description: Understanding and utilizing crews in the crewAI framework with comprehensive attributes and functionalities.
+---
+
+## What is a Crew?
+A crew in crewAI represents a collaborative group of agents working together to achieve a set of tasks. Each crew defines the strategy for task execution, agent collaboration, and the overall workflow.
+
+## Crew Attributes
+
+| Attribute                   | Description                                                  |
+| :-------------------------- | :----------------------------------------------------------- |
+| **Tasks**                   | A list of tasks assigned to the crew.                        |
+| **Agents**                  | A list of agents that are part of the crew.                  |
+| **Process** *(optional)*    | The process flow (e.g., sequential, hierarchical) the crew follows. |
+| **Verbose** *(optional)*    | The verbosity level for logging during execution.            |
+| **Manager LLM** *(optional)*| The language model used by the manager agent in a hierarchical process. **Required when using a hierarchical process.** |
+| **Function Calling LLM** *(optional)* | If passed, the crew will use this LLM to do function calling for tools for all agents in the crew. Each agent can have its own LLM, which overrides the crew's LLM for function calling. |
+| **Config** *(optional)*     | Optional configuration settings for the crew, in `Json` or `Dict[str, Any]` format. |
+| **Max RPM** *(optional)*    | Maximum requests per minute the crew adheres to during execution. |
+| **Language**  *(optional)*  | Language used for the crew, defaults to English.             |
+| **Language File** *(optional)* | Path to the language file to be used for the crew.          |
+| **Memory** *(optional)*     | Utilized for storing execution memories (short-term, long-term, entity memory). |
+| **Cache** *(optional)*      | Specifies whether to use a cache for storing the results of tools' execution. |
+| **Embedder** *(optional)*   | Configuration for the embedder to be used by the crew. Mostly used by memory for now.       |
+| **Full Output** *(optional)*| Whether the crew should return the full output with all tasks outputs or just the final output. |
+| **Step Callback** *(optional)* | A function that is called after each step of every agent. This can be used to log the agent's actions or to perform other operations; it won't override the agent-specific `step_callback`. |
+| **Task Callback** *(optional)* | A function that is called after the completion of each task. Useful for monitoring or additional operations post-task execution. |
+| **Share Crew** *(optional)* | Whether you want to share the complete crew information and execution with the crewAI team to make the library better, and allow us to train models. |
+| **Output Log File** *(optional)* | Whether you want to have a file with the complete crew output and execution. You can set it using True and it will default to the folder you are currently in and it will be called logs.txt or passing a string with the full path and name of the file. |
+| **Manager Agent** *(optional)* | `manager` sets a ustom agent that will be used as a manager. |
+| **Manager Callbacks** *(optional)* | `manager_callbacks` takes a list of callback handlers to be executed by the manager agent when a hierarchical process is used. |
+| **Prompt File** *(optional)* | Path to the prompt JSON file to be used for the crew. |
+
+!!! note "Crew Max RPM"
+    The `max_rpm` attribute sets the maximum number of requests per minute the crew can perform to avoid rate limits and will override individual agents' `max_rpm` settings if you set it.
+
+## Creating a Crew
+
+When assembling a crew, you combine agents with complementary roles and tools, assign tasks, and select a process that dictates their execution order and interaction.
+
+### Example: Assembling a Crew
+
+```python
+from crewai import Crew, Agent, Task, Process
+from langchain_community.tools import DuckDuckGoSearchRun
+
+# Define agents with specific roles and tools
+researcher = Agent(
+    role='Senior Research Analyst',
+    goal='Discover innovative AI technologies',
+    backstory="""You're a senior research analyst at a large company.
+        You're responsible for analyzing data and providing insights
+        to the business.
+        You're currently working on a project to analyze the
+        trends and innovations in the space of artificial intelligence.""",
+    tools=[DuckDuckGoSearchRun()]
+)
+
+writer = Agent(
+    role='Content Writer',
+    goal='Write engaging articles on AI discoveries',
+    backstory="""You're a senior writer at a large company.
+        You're responsible for creating content to the business.
+        You're currently working on a project to write about trends
+        and innovations in the space of AI for your next meeting.""",
+    verbose=True
+)
+
+# Create tasks for the agents
+research_task = Task(
+    description='Identify breakthrough AI technologies',
+    agent=researcher,
+    expected_output='A bullet list summary of the top 5 most important AI news'
+)
+write_article_task = Task(
+    description='Draft an article on the latest AI technologies',
+    agent=writer,
+    expected_output='3 paragraph blog post on the latest AI technologies'
+)
+
+# Assemble the crew with a sequential process
+my_crew = Crew(
+    agents=[researcher, writer],
+    tasks=[research_task, write_article_task],
+    process=Process.sequential,
+    full_output=True,
+    verbose=True,
+)
+```
+
+## Memory Utilization
+
+Crews can utilize memory (short-term, long-term, and entity memory) to enhance their execution and learning over time. This feature allows crews to store and recall execution memories, aiding in decision-making and task execution strategies.
+
+## Cache Utilization
+
+Caches can be employed to store the results of tools' execution, making the process more efficient by reducing the need to re-execute identical tasks.
+
+## Crew Usage Metrics
+
+After the crew execution, you can access the `usage_metrics` attribute to view the language model (LLM) usage metrics for all tasks executed by the crew. This provides insights into operational efficiency and areas for improvement.
+
+```python
+# Access the crew's usage metrics
+crew = Crew(agents=[agent1, agent2], tasks=[task1, task2])
+crew.kickoff()
+print(crew.usage_metrics)
+```
+
+## Crew Execution Process
+
+- **Sequential Process**: Tasks are executed one after another, allowing for a linear flow of work.
+- **Hierarchical Process**: A manager agent coordinates the crew, delegating tasks and validating outcomes before proceeding. **Note**: A `manager_llm` or `manager_agent` is required for this process and it's essential for validating the process flow.
+
+### Kicking Off a Crew
+
+Once your crew is assembled, initiate the workflow with the `kickoff()` method. This starts the execution process according to the defined process flow.
+
+```python
+# Start the crew's task execution
+result = my_crew.kickoff()
+print(result)
+```
+
+### Kicking Off a Crew
+
+Once your crew is assembled, initiate the workflow with the appropriate kickoff method. CrewAI provides several methods for better control over the kickoff process: `kickoff()`, `kickoff_for_each()`, `kickoff_async()`, and `kickoff_for_each_async()`.
+
+`kickoff()`: Starts the execution process according to the defined process flow.
+`kickoff_for_each()`: Executes tasks for each agent individually.
+`kickoff_async()`: Initiates the workflow asynchronously.
+`kickoff_for_each_async()`: Executes tasks for each agent individually in an asynchronous manner.
+
+```python
+# Start the crew's task execution
+result = my_crew.kickoff()
+print(result)
+
+# Example of using kickoff_for_each
+results = my_crew.kickoff_for_each()
+for result in results:
+    print(result)
+
+# Example of using kickoff_async
+async_result = my_crew.kickoff_async()
+print(async_result)
+
+# Example of using kickoff_for_each_async
+async_results = my_crew.kickoff_for_each_async()
+for async_result in async_results:
+    print(async_result)
+```
+
+These methods provide flexibility in how you manage and execute tasks within your crew, allowing for both synchronous and asynchronous workflows tailored to your needs

docs/core-concepts/Defining-Tasks.md

@@ -1,42 +0,0 @@
-# Overview of a Task
-
-In the CrewAI framework, tasks are the individual assignments that agents are responsible for completing. They are the fundamental units of work that your AI crew will undertake. Understanding how to define and manage tasks is key to leveraging the full potential of CrewAI.
-
-A task in CrewAI encapsulates all the information needed for an agent to execute it, including a description, the agent assigned to it, and any specific tools required. Tasks are designed to be flexible, allowing for both simple and complex actions depending on your needs.
-
-# Properties of a Task
-
-Every task in CrewAI has several properties:
-
-- **Description**: A clear and concise statement of what needs to be done.
-- **Agent**: The agent assigned to the task (optional). If no agent is specified, the task can be picked up by any agent based on the process defined.
-- **Tools**: A list of tools (optional) that the agent can use to complete the task. These can override the agent's default tools if necessary.
-
-# Creating a Task
-
-Creating a task is straightforward. You define what needs to be done and, optionally, who should do it and what tools they should use. Here’s a conceptual guide:
-
-```python
-from crewai import Task
-
-# Define a task with a designated agent and specific tools
-task = Task(description='Generate monthly sales report', agent=sales_agent, tools=[reporting_tool])
-```
-
-# Task Assignment
-
-Tasks can be assigned to agents in several ways:
-
-- Directly, by specifying the agent when creating the task.
-- [WIP] Through the Crew's process, which can assign tasks based on agent roles, availability, or other criteria.
-
-# Task Execution
-
-Once a task has been defined and assigned, it's ready to be executed. Execution is typically handled by the Crew object, which manages the workflow and ensures that tasks are completed according to the defined process.
-
-# Task Collaboration
-
-Tasks in CrewAI can be designed to require collaboration between agents. For example, one agent might gather data while another analyzes it. This collaborative approach can be defined within the task properties and managed by the Crew's process.
-
-# Conclusion
-Tasks are the driving force behind the actions of agents in CrewAI. By properly defining tasks, you set the stage for your AI agents to work effectively, either independently or as a collaborative unit. In the following sections, we will explore how tasks fit into the larger picture of processes and crew management.

docs/core-concepts/Delegation-and-Collaboration.md

@@ -1,26 +0,0 @@
-# How Agents Collaborate:
-
-In CrewAI, collaboration is the cornerstone of agent interaction. Agents are designed to work together by sharing information, requesting assistance, and combining their skills to complete tasks more efficiently.
-
-- **Information Sharing**: Agents can share findings and data amongst themselves to ensure all members are informed and can contribute effectively.
-- **Task Assistance**: If an agent encounters a task that requires additional expertise, it can seek the help of another agent with the necessary skill set.
-- **Resource Allocation**: Agents can share or allocate resources such as tools or processing power to optimize task execution.
-
-Collaboration is embedded in the DNA of CrewAI, enabling a dynamic and adaptive approach to problem-solving.
-
-# Delegation: Dividing to Conquer
-
-Delegation is the process by which an agent assigns a task to another agent, or just ask another agent, it's an intelligent decision-making process that enhances the crew's functionality.
-By default all agents can delegate work and ask questions, so if you want an agent to work alone make sure to set that option when initializing an Agent, this is useful to prevent deviations if the task is supposed to be straightforward.
-
-## Implementing Collaboration and Delegation
-
-When setting up your crew, you'll define the roles and capabilities of each agent. CrewAI's infrastructure takes care of the rest, managing the complex interplay of agents as they work together.
-
-## Example Scenario:
-
-Imagine a scenario where you have a researcher agent that gathers data and a writer agent that compiles reports. The writer can autonomously ask question or delegate more in depth research work depending on its needs as it tries to complete its task.
-
-# Conclusion
-
-Collaboration and delegation are what transform a collection of AI agents into a unified, intelligent crew. With CrewAI, you have a framework that not only simplifies these interactions but also makes them more effective, paving the way for sophisticated AI systems that can tackle complex, multi-dimensional tasks.

docs/core-concepts/Managing-Processes.md

@@ -1,49 +0,0 @@
-# Managing Processes in CrewAI
-
-Processes are the heart of CrewAI's workflow management, akin to the way a human team organizes its work. In CrewAI, processes define the sequence and manner in which tasks are executed by agents, mirroring the coordination you'd expect in a well-functioning team of people.
-
-## Understanding Processes
-
-A process in CrewAI can be thought of as the game plan for how your AI agents will handle their workload. Just as a project manager assigns tasks to team members based on their skills and the project timeline, CrewAI processes assign tasks to agents to ensure efficient workflow.
-
-## Process Implementations
-
-- **Sequential (Supported)**: This is the only process currently implemented in CrewAI. It ensures tasks are handled one at a time, in a given order, much like a relay race where one runner passes the baton to the next.
-- **Consensual (WIP)**: Envisioned for a future update, the consensual process will enable agents to make joint decisions on task execution, similar to a team consensus in a meeting before proceeding.
-- **Hierarchical (WIP)**: Also in the pipeline, this process will introduce a chain of command to task execution, where some agents may have the authority to prioritize tasks or delegate them, akin to a traditional corporate hierarchy.
-These additional processes, once implemented, will offer more nuanced and sophisticated ways for agents to interact and complete tasks, much like teams in complex organizational structures.
-
-
-## Defining a Sequential Process
-
-Creating a sequential process in CrewAI is straightforward and reflects the simplicity of coordinating a team's efforts step by step. In this process the outcome of the previous task is sent into the next one as context that I should use to accomplish it's task
-
-```python
-from crewai import Process
-
-# Define a sequential process
-sequential_process = Process.sequential
-```
-
-# The Magic of Sequential Processes
-
-The sequential process is where much of CrewAI's magic happens. It ensures that tasks are approached with the same thoughtful progression that a human team would use, fostering a natural and logical flow of work while passing on task outcome into the next.
-
-## Assigning Processes to a Crew
-
-To assign a process to a crew, simply set it during the crew's creation. The process will dictate the crew's approach to task execution.
-
-```python
-from crewai import Crew
-
-# Create a crew with a sequential process
-crew = Crew(agents=my_agents, tasks=my_tasks, process=sequential_process)
-```
-
-## The Role of Processes in Teamwork
-
-The process you choose for your crew is critical. It's what transforms a group of individual agents into a cohesive unit that can tackle complex projects with the precision and harmony you'd find in a team of skilled humans.
-
-## Conclusion
-
-Processes bring structure and order to the CrewAI ecosystem, allowing agents to collaborate effectively and accomplish goals systematically. As CrewAI evolves, additional process types will be introduced to enhance the framework's versatility, much like a team that grows and adapts over time.

docs/core-concepts/Memory.md

@@ -0,0 +1,170 @@
+---
+title: crewAI Memory Systems
+description: Leveraging memory systems in the crewAI framework to enhance agent capabilities.
+---
+
+## Introduction to Memory Systems in crewAI
+!!! note "Enhancing Agent Intelligence"
+    The crewAI framework introduces a sophisticated memory system designed to significantly enhance the capabilities of AI agents. This system comprises short-term memory, long-term memory, entity memory, and contextual memory, each serving a unique purpose in aiding agents to remember, reason, and learn from past interactions.
+
+## Memory System Components
+
+| Component            | Description                                                  |
+| :------------------- | :----------------------------------------------------------- |
+| **Short-Term Memory**| Temporarily stores recent interactions and outcomes, enabling agents to recall and utilize information relevant to their current context during the current executions. |
+| **Long-Term Memory** | Preserves valuable insights and learnings from past executions, allowing agents to build and refine their knowledge over time. So Agents can remeber what they did right and wrong across multiple executions |
+| **Entity Memory**    | Captures and organizes information about entities (people, places, concepts) encountered during tasks, facilitating deeper understanding and relationship mapping. |
+| **Contextual Memory**| Maintains the context of interactions by combining `ShortTermMemory`, `LongTermMemory`, and `EntityMemory`, aiding in the coherence and relevance of agent responses over a sequence of tasks or a conversation. |
+
+## How Memory Systems Empower Agents
+
+1. **Contextual Awareness**: With short-term and contextual memory, agents gain the ability to maintain context over a conversation or task sequence, leading to more coherent and relevant responses.
+
+2. **Experience Accumulation**: Long-term memory allows agents to accumulate experiences, learning from past actions to improve future decision-making and problem-solving.
+
+3. **Entity Understanding**: By maintaining entity memory, agents can recognize and remember key entities, enhancing their ability to process and interact with complex information.
+
+## Implementing Memory in Your Crew
+
+When configuring a crew, you can enable and customize each memory component to suit the crew's objectives and the nature of tasks it will perform.
+By default, the memory system is disabled, and you can ensure it is active by setting `memory=True` in the crew configuration. The memory will use OpenAI Embeddings by default, but you can change it by setting `embedder` to a different model.
+
+### Example: Configuring Memory for a Crew
+
+```python
+from crewai import Crew, Agent, Task, Process
+
+# Assemble your crew with memory capabilities
+my_crew = Crew(
+    agents=[...],
+    tasks=[...],
+    process=Process.sequential,
+    memory=True,
+    verbose=True
+)
+```
+
+## Additional Embedding Providers
+
+### Using OpenAI embeddings (already default)
+```python
+from crewai import Crew, Agent, Task, Process
+
+my_crew = Crew(
+		agents=[...],
+		tasks=[...],
+		process=Process.sequential,
+		memory=True,
+		verbose=True,
+		embedder={
+				"provider": "openai",
+				"config":{
+						"model": 'text-embedding-3-small'
+				}
+		}
+)
+```
+
+### Using Google AI embeddings
+```python
+from crewai import Crew, Agent, Task, Process
+
+my_crew = Crew(
+		agents=[...],
+		tasks=[...],
+		process=Process.sequential,
+		memory=True,
+		verbose=True,
+		embedder={
+			"provider": "google",
+			"config":{
+				"model": 'models/embedding-001',
+				"task_type": "retrieval_document",
+				"title": "Embeddings for Embedchain"
+			}
+		}
+)
+```
+
+### Using Azure OpenAI embeddings
+```python
+from crewai import Crew, Agent, Task, Process
+
+my_crew = Crew(
+		agents=[...],
+		tasks=[...],
+		process=Process.sequential,
+		memory=True,
+		verbose=True,
+		embedder={
+			"provider": "azure_openai",
+			"config":{
+				"model": 'text-embedding-ada-002',
+				"deployment_name": "you_embedding_model_deployment_name"
+			}
+		}
+)
+```
+
+### Using GPT4ALL embeddings
+```python
+from crewai import Crew, Agent, Task, Process
+
+my_crew = Crew(
+		agents=[...],
+		tasks=[...],
+		process=Process.sequential,
+		memory=True,
+		verbose=True,
+		embedder={
+			"provider": "gpt4all"
+		}
+)
+```
+
+### Using Vertex AI embeddings
+```python
+from crewai import Crew, Agent, Task, Process
+
+my_crew = Crew(
+		agents=[...],
+		tasks=[...],
+		process=Process.sequential,
+		memory=True,
+		verbose=True,
+		embedder={
+			"provider": "vertexai",
+			"config":{
+				"model": 'textembedding-gecko'
+			}
+		}
+)
+```
+
+### Using Cohere embeddings
+```python
+from crewai import Crew, Agent, Task, Process
+
+my_crew = Crew(
+		agents=[...],
+		tasks=[...],
+		process=Process.sequential,
+		memory=True,
+		verbose=True,
+		embedder={
+			"provider": "cohere",
+			"config":{
+				"model": "embed-english-v3.0"
+    		"vector_dimension": 1024
+			}
+		}
+)
+```
+
+## Benefits of Using crewAI's Memory System
+- **Adaptive Learning:** Crews become more efficient over time, adapting to new information and refining their approach to tasks.
+- **Enhanced Personalization:** Memory enables agents to remember user preferences and historical interactions, leading to personalized experiences.
+- **Improved Problem Solving:** Access to a rich memory store aids agents in making more informed decisions, drawing on past learnings and contextual insights.
+
+## Getting Started
+Integrating crewAI's memory system into your projects is straightforward. By leveraging the provided memory components and configurations, you can quickly empower your agents with the ability to remember, reason, and learn from their interactions, unlocking new levels of intelligence and capability.

docs/core-concepts/Processes.md

@@ -0,0 +1,64 @@
+---
+title: Managing Processes in CrewAI
+description: Detailed guide on workflow management through processes in CrewAI, with updated implementation details.
+---
+
+## Understanding Processes
+!!! note "Core Concept"
+    In CrewAI, processes orchestrate the execution of tasks by agents, akin to project management in human teams. These processes ensure tasks are distributed and executed efficiently, in alignment with a predefined strategy.
+
+## Process Implementations
+
+- **Sequential**: Executes tasks sequentially, ensuring tasks are completed in an orderly progression.
+- **Hierarchical**: Organizes tasks in a managerial hierarchy, where tasks are delegated and executed based on a structured chain of command. A manager language model (`manager_llm`) or a custom manager agent (`manager_agent`) must be specified in the crew to enable the hierarchical process, facilitating the creation and management of tasks by the manager.
+- **Consensual Process (Planned)**: Aiming for collaborative decision-making among agents on task execution, this process type introduces a democratic approach to task management within CrewAI. It is planned for future development and is not currently implemented in the codebase.
+
+## The Role of Processes in Teamwork
+Processes enable individual agents to operate as a cohesive unit, streamlining their efforts to achieve common objectives with efficiency and coherence.
+
+## Assigning Processes to a Crew
+To assign a process to a crew, specify the process type upon crew creation to set the execution strategy. For a hierarchical process, ensure to define `manager_llm` or `manager_agent` for the manager agent.
+
+```python
+from crewai import Crew
+from crewai.process import Process
+from langchain_openai import ChatOpenAI
+
+# Example: Creating a crew with a sequential process
+crew = Crew(
+    agents=my_agents,
+    tasks=my_tasks,
+    process=Process.sequential
+)
+
+# Example: Creating a crew with a hierarchical process
+# Ensure to provide a manager_llm or manager_agent
+crew = Crew(
+    agents=my_agents,
+    tasks=my_tasks,
+    process=Process.hierarchical,
+    manager_llm=ChatOpenAI(model="gpt-4")
+    # or
+    # manager_agent=my_manager_agent
+)
+```
+**Note:** Ensure `my_agents` and `my_tasks` are defined prior to creating a `Crew` object, and for the hierarchical process, either `manager_llm` or `manager_agent` is also required.
+
+## Sequential Process
+This method mirrors dynamic team workflows, progressing through tasks in a thoughtful and systematic manner. Task execution follows the predefined order in the task list, with the output of one task serving as context for the next.
+
+To customize task context, utilize the `context` parameter in the `Task` class to specify outputs that should be used as context for subsequent tasks.
+
+## Hierarchical Process
+Emulates a corporate hierarchy, CrewAI allows specifying a custom manager agent or automatically creates one, requiring the specification of a manager language model (`manager_llm`). This agent oversees task execution, including planning, delegation, and validation. Tasks are not pre-assigned; the manager allocates tasks to agents based on their capabilities, reviews outputs, and assesses task completion.
+
+## Process Class: Detailed Overview
+The `Process` class is implemented as an enumeration (`Enum`), ensuring type safety and restricting process values to the defined types (`sequential`, `hierarchical`). The consensual process is planned for future inclusion, emphasizing our commitment to continuous development and innovation.
+
+## Additional Task Features
+- **Asynchronous Execution**: Tasks can now be executed asynchronously, allowing for parallel processing and efficiency improvements. This feature is designed to enable tasks to be carried out concurrently, enhancing the overall productivity of the crew.
+- **Human Input Review**: An optional feature that enables the review of task outputs by humans to ensure quality and accuracy before finalization. This additional step introduces a layer of oversight, providing an opportunity for human intervention and validation.
+- **Output Customization**: Tasks support various output formats, including JSON (`output_json`), Pydantic models (`output_pydantic`), and file outputs (`output_file`), providing flexibility in how task results are captured and utilized. This allows for a wide range of output possibilities, catering to different needs and requirements.
+
+## Conclusion
+The structured collaboration facilitated by processes within CrewAI is crucial for enabling systematic teamwork among agents. This documentation has been updated to reflect the latest features, enhancements, and the planned integration of the Consensual Process, ensuring users have access to the most current and comprehensive information.

docs/core-concepts/Tasks.md

@@ -0,0 +1,249 @@
+---
+title: crewAI Tasks
+description: Detailed guide on managing and creating tasks within the crewAI framework, reflecting the latest codebase updates.
+---
+
+## Overview of a Task
+!!! note "What is a Task?"
+    In the crewAI framework, tasks are specific assignments completed by agents. They provide all necessary details for execution, such as a description, the agent responsible, required tools, and more, facilitating a wide range of action complexities.
+
+Tasks within crewAI can be collaborative, requiring multiple agents to work together. This is managed through the task properties and orchestrated by the Crew's process, enhancing teamwork and efficiency.
+
+## Task Attributes
+
+| Attribute              | Description                                                                                   |
+| :----------------------| :-------------------------------------------------------------------------------------------- |
+| **Description**        | A clear, concise statement of what the task entails.                                          |
+| **Agent**              | The agent responsible for the task, assigned either directly or by the crew's process.        |
+| **Expected Output**    | A detailed description of what the task's completion looks like.                              |
+| **Tools** *(optional)* | The functions or capabilities the agent can utilize to perform the task.                      |
+| **Async Execution** *(optional)* | If set, the task executes asynchronously, allowing progression without waiting for completion.|
+| **Context**  *(optional)* | Specifies tasks whose outputs are used as context for this task.                              |
+| **Config** *(optional)* | Additional configuration details for the agent executing the task, allowing further customization. |
+| **Output JSON**  *(optional)* | Outputs a JSON object, requiring an OpenAI client. Only one output format can be set.         |
+| **Output Pydantic**  *(optional)* | Outputs a Pydantic model object, requiring an OpenAI client. Only one output format can be set. |
+| **Output File**  *(optional)* | Saves the task output to a file. If used with `Output JSON` or `Output Pydantic`, specifies how the output is saved. |
+| **Callback**  *(optional)* | A Python callable that is executed with the task's output upon completion.                    |
+| **Human Input** *(optional)* | Indicates if the task requires human feedback at the end, useful for tasks needing human oversight. |
+
+## Creating a Task
+
+Creating a task involves defining its scope, responsible agent, and any additional attributes for flexibility:
+
+```python
+from crewai import Task
+
+task = Task(
+    description='Find and summarize the latest and most relevant news on AI',
+    agent=sales_agent
+)
+```
+
+!!! note "Task Assignment"
+    Directly specify an `agent` for assignment or let the `hierarchical` CrewAI's process decide based on roles, availability, etc.
+
+## Integrating Tools with Tasks
+
+Leverage tools from the [crewAI Toolkit](https://github.com/joaomdmoura/crewai-tools) and [LangChain Tools](https://python.langchain.com/docs/integrations/tools) for enhanced task performance and agent interaction.
+
+## Creating a Task with Tools
+
+```python
+import os
+os.environ["OPENAI_API_KEY"] = "Your Key"
+os.environ["SERPER_API_KEY"] = "Your Key" # serper.dev API key
+
+from crewai import Agent, Task, Crew
+from crewai_tools import SerperDevTool
+
+research_agent = Agent(
+  role='Researcher',
+  goal='Find and summarize the latest AI news',
+  backstory="""You're a researcher at a large company.
+  You're responsible for analyzing data and providing insights
+  to the business.""",
+  verbose=True
+)
+
+search_tool = SerperDevTool()
+
+task = Task(
+  description='Find and summarize the latest AI news',
+  expected_output='A bullet list summary of the top 5 most important AI news',
+  agent=research_agent,
+  tools=[search_tool]
+)
+
+crew = Crew(
+    agents=[research_agent],
+    tasks=[task],
+    verbose=2
+)
+
+result = crew.kickoff()
+print(result)
+```
+
+This demonstrates how tasks with specific tools can override an agent's default set for tailored task execution.
+
+## Referring to Other Tasks
+
+In crewAI, the output of one task is automatically relayed into the next one, but you can specifically define what tasks' output, including multiple, should be used as context for another task.
+
+This is useful when you have a task that depends on the output of another task that is not performed immediately after it. This is done through the `context` attribute of the task:
+
+```python
+# ...
+
+research_ai_task = Task(
+    description='Find and summarize the latest AI news',
+    expected_output='A bullet list summary of the top 5 most important AI news',
+    async_execution=True,
+    agent=research_agent,
+    tools=[search_tool]
+)
+
+research_ops_task = Task(
+    description='Find and summarize the latest AI Ops news',
+    expected_output='A bullet list summary of the top 5 most important AI Ops news',
+    async_execution=True,
+    agent=research_agent,
+    tools=[search_tool]
+)
+
+write_blog_task = Task(
+    description="Write a full blog post about the importance of AI and its latest news",
+    expected_output='Full blog post that is 4 paragraphs long',
+    agent=writer_agent,
+    context=[research_ai_task, research_ops_task]
+)
+
+#...
+```
+
+## Asynchronous Execution
+
+You can define a task to be executed asynchronously. This means that the crew will not wait for it to be completed to continue with the next task. This is useful for tasks that take a long time to be completed, or that are not crucial for the next tasks to be performed.
+
+You can then use the `context` attribute to define in a future task that it should wait for the output of the asynchronous task to be completed.
+
+```python
+#...
+
+list_ideas = Task(
+    description="List of 5 interesting ideas to explore for an article about AI.",
+    expected_output="Bullet point list of 5 ideas for an article.",
+    agent=researcher,
+    async_execution=True # Will be executed asynchronously
+)
+
+list_important_history = Task(
+    description="Research the history of AI and give me the 5 most important events.",
+    expected_output="Bullet point list of 5 important events.",
+    agent=researcher,
+    async_execution=True # Will be executed asynchronously
+)
+
+write_article = Task(
+    description="Write an article about AI, its history, and interesting ideas.",
+    expected_output="A 4 paragraph article about AI.",
+    agent=writer,
+    context=[list_ideas, list_important_history] # Will wait for the output of the two tasks to be completed
+)
+
+#...
+```
+
+## Callback Mechanism
+
+The callback function is executed after the task is completed, allowing for actions or notifications to be triggered based on the task's outcome.
+
+```python
+# ...
+
+def callback_function(output: TaskOutput):
+    # Do something after the task is completed
+    # Example: Send an email to the manager
+    print(f"""
+        Task completed!
+        Task: {output.description}
+        Output: {output.raw_output}
+    """)
+
+research_task = Task(
+    description='Find and summarize the latest AI news',
+    expected_output='A bullet list summary of the top 5 most important AI news',
+    agent=research_agent,
+    tools=[search_tool],
+    callback=callback_function
+)
+
+#...
+```
+
+## Accessing a Specific Task Output
+
+Once a crew finishes running, you can access the output of a specific task by using the `output` attribute of the task object:
+
+```python
+# ...
+task1 = Task(
+    description='Find and summarize the latest AI news',
+    expected_output='A bullet list summary of the top 5 most important AI news',
+    agent=research_agent,
+    tools=[search_tool]
+)
+
+#...
+
+crew = Crew(
+    agents=[research_agent],
+    tasks=[task1, task2, task3],
+    verbose=2
+)
+
+result = crew.kickoff()
+
+# Returns a TaskOutput object with the description and results of the task
+print(f"""
+    Task completed!
+    Task: {task1.output.description}
+    Output: {task1.output.raw_output}
+""")
+```
+
+## Tool Override Mechanism
+
+Specifying tools in a task allows for dynamic adaptation of agent capabilities, emphasizing CrewAI's flexibility.
+
+## Error Handling and Validation Mechanisms
+
+While creating and executing tasks, certain validation mechanisms are in place to ensure the robustness and reliability of task attributes. These include but are not limited to:
+
+- Ensuring only one output type is set per task to maintain clear output expectations.
+- Preventing the manual assignment of the `id` attribute to uphold the integrity of the unique identifier system.
+
+These validations help in maintaining the consistency and reliability of task executions within the crewAI framework.
+
+## Creating Directories when Saving Files
+
+You can now specify if a task should create directories when saving its output to a file. This is particularly useful for organizing outputs and ensuring that file paths are correctly structured.
+
+```python
+# ...
+
+save_output_task = Task(
+    description='Save the summarized AI news to a file',
+    expected_output='File saved successfully',
+    agent=research_agent,
+    tools=[file_save_tool],
+    output_file='outputs/ai_news_summary.txt',
+    create_directory=True
+)
+
+#...
+```
+
+## Conclusion
+
+Tasks are the driving force behind the actions of agents in crewAI. By properly defining tasks and their outcomes, you set the stage for your AI agents to work effectively, either independently or as a collaborative unit. Equipping tasks with appropriate tools, understanding the execution process, and following robust validation practices are crucial for maximizing CrewAI's potential, ensuring agents are effectively prepared for their assignments and that tasks are executed as intended.

docs/core-concepts/Tools.md

@@ -0,0 +1,194 @@
+---
+title: crewAI Tools
+description: Understanding and leveraging tools within the crewAI framework for agent collaboration and task execution.
+---
+
+## Introduction
+CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among coworkers. This documentation outlines how to create, integrate, and leverage these tools within the CrewAI framework, including a new focus on collaboration tools.
+
+## What is a Tool?
+!!! note "Definition"
+    A tool in CrewAI is a skill or function that agents can utilize to perform various actions. This includes tools from the [crewAI Toolkit](https://github.com/joaomdmoura/crewai-tools) and [LangChain Tools](https://python.langchain.com/docs/integrations/tools), enabling everything from simple searches to complex interactions and effective teamwork among agents.
+
+## Key Characteristics of Tools
+
+- **Utility**: Crafted for tasks such as web searching, data analysis, content generation, and agent collaboration.
+- **Integration**: Boosts agent capabilities by seamlessly integrating tools into their workflow.
+- **Customizability**: Provides the flexibility to develop custom tools or utilize existing ones, catering to the specific needs of agents.
+- **Error Handling**: Incorporates robust error handling mechanisms to ensure smooth operation.
+- **Caching Mechanism**: Features intelligent caching to optimize performance and reduce redundant operations.
+
+## Using crewAI Tools
+
+To enhance your agents' capabilities with crewAI tools, begin by installing our extra tools package:
+
+```bash
+pip install 'crewai[tools]'
+```
+
+Here's an example demonstrating their use:
+
+```python
+import os
+from crewai import Agent, Task, Crew
+# Importing crewAI tools
+from crewai_tools import (
+    DirectoryReadTool,
+    FileReadTool,
+    SerperDevTool,
+    WebsiteSearchTool
+)
+
+# Set up API keys
+os.environ["SERPER_API_KEY"] = "Your Key" # serper.dev API key
+os.environ["OPENAI_API_KEY"] = "Your Key"
+
+# Instantiate tools
+docs_tool = DirectoryReadTool(directory='./blog-posts')
+file_tool = FileReadTool()
+search_tool = SerperDevTool()
+web_rag_tool = WebsiteSearchTool()
+
+# Create agents
+researcher = Agent(
+    role='Market Research Analyst',
+    goal='Provide up-to-date market analysis of the AI industry',
+    backstory='An expert analyst with a keen eye for market trends.',
+    tools=[search_tool, web_rag_tool],
+    verbose=True
+)
+
+writer = Agent(
+    role='Content Writer',
+    goal='Craft engaging blog posts about the AI industry',
+    backstory='A skilled writer with a passion for technology.',
+    tools=[docs_tool, file_tool],
+    verbose=True
+)
+
+# Define tasks
+research = Task(
+    description='Research the latest trends in the AI industry and provide a summary.',
+    expected_output='A summary of the top 3 trending developments in the AI industry with a unique perspective on their significance.',
+    agent=researcher
+)
+
+write = Task(
+    description='Write an engaging blog post about the AI industry, based on the research analyst’s summary. Draw inspiration from the latest blog posts in the directory.',
+    expected_output='A 4-paragraph blog post formatted in markdown with engaging, informative, and accessible content, avoiding complex jargon.',
+    agent=writer,
+    output_file='blog-posts/new_post.md'  # The final blog post will be saved here
+)
+
+# Assemble a crew
+crew = Crew(
+    agents=[researcher, writer],
+    tasks=[research, write],
+    verbose=2
+)
+
+# Execute tasks
+crew.kickoff()
+```
+
+## Available crewAI Tools
+
+- **Error Handling**: All tools are built with error handling capabilities, allowing agents to gracefully manage exceptions and continue their tasks.
+- **Caching Mechanism**: All tools support caching, enabling agents to efficiently reuse previously obtained results, reducing the load on external resources and speeding up the execution time. You can also define finer control over the caching mechanism using the `cache_function` attribute on the tool.
+
+Here is a list of the available tools and their descriptions:
+
+| Tool                        | Description                                                                                   |
+| :-------------------------- | :-------------------------------------------------------------------------------------------- |
+| **CodeDocsSearchTool**      | A RAG tool optimized for searching through code documentation and related technical documents. |
+| **CSVSearchTool**           | A RAG tool designed for searching within CSV files, tailored to handle structured data.       |
+| **DirectorySearchTool**     | A RAG tool for searching within directories, useful for navigating through file systems.      |
+| **DOCXSearchTool**          | A RAG tool aimed at searching within DOCX documents, ideal for processing Word files.         |
+| **DirectoryReadTool**       | Facilitates reading and processing of directory structures and their contents.                |
+| **FileReadTool**            | Enables reading and extracting data from files, supporting various file formats.              |
+| **GithubSearchTool**        | A RAG tool for searching within GitHub repositories, useful for code and documentation search.|
+| **SerperDevTool**           | A specialized tool for development purposes, with specific functionalities under development. |
+| **TXTSearchTool**           | A RAG tool focused on searching within text (.txt) files, suitable for unstructured data.     |
+| **JSONSearchTool**          | A RAG tool designed for searching within JSON files, catering to structured data handling.     |
+| **MDXSearchTool**           | A RAG tool tailored for searching within Markdown (MDX) files, useful for documentation.      |
+| **PDFSearchTool**           | A RAG tool aimed at searching within PDF documents, ideal for processing scanned documents.    |
+| **PGSearchTool**            | A RAG tool optimized for searching within PostgreSQL databases, suitable for database queries. |
+| **RagTool**                 | A general-purpose RAG tool capable of handling various data sources and types.                 |
+| **ScrapeElementFromWebsiteTool** | Enables scraping specific elements from websites, useful for targeted data extraction.     |
+| **ScrapeWebsiteTool**       | Facilitates scraping entire websites, ideal for comprehensive data collection.                 |
+| **WebsiteSearchTool**       | A RAG tool for searching website content, optimized for web data extraction.                   |
+| **XMLSearchTool**           | A RAG tool designed for searching within XML files, suitable for structured data formats.      |
+| **YoutubeChannelSearchTool**| A RAG tool for searching within YouTube channels, useful for video content analysis.           |
+| **YoutubeVideoSearchTool**  | A RAG tool aimed at searching within YouTube videos, ideal for video data extraction.          |
+| **BrowserbaseTool**         | A tool for interacting with and extracting data from web browsers.                            |
+| **ExaSearchTool**           | A tool designed for performing exhaustive searches across various data sources.               |
+
+## Creating your own Tools
+
+!!! example "Custom Tool Creation"
+    Developers can craft custom tools tailored for their agent’s needs or utilize pre-built options:
+
+To create your own crewAI tools you will need to install our extra tools package:
+
+```bash
+pip install 'crewai[tools]'
+```
+
+Once you do that there are two main ways for one to create a crewAI tool:
+### Subclassing `BaseTool`
+
+```python
+from crewai_tools import BaseTool
+
+class MyCustomTool(BaseTool):
+    name: str = "Name of my tool"
+    description: str = "Clear description for what this tool is useful for, your agent will need this information to use it."
+
+    def _run(self, argument: str) -> str:
+        # Implementation goes here
+        return "Result from custom tool"
+```
+
+### Utilizing the `tool` Decorator
+
+```python
+from crewai_tools import tool
+@tool("Name of my tool")
+def my_tool(question: str) -> str:
+    """Clear description for what this tool is useful for, your agent will need this information to use it."""
+    # Function logic here
+    return "Result from your custom tool"
+```
+
+### Custom Caching Mechanism
+!!! note "Caching"
+    Tools can optionally implement a `cache_function` to fine-tune caching behavior. This function determines when to cache results based on specific conditions, offering granular control over caching logic.
+
+```python
+from crewai_tools import tool
+
+@tool
+def multiplication_tool(first_number: int, second_number: int) -> str:
+    """Useful for when you need to multiply two numbers together."""
+    return first_number * second_number
+
+def cache_func(args, result):
+    # In this case, we only cache the result if it's a multiple of 2
+    cache = result % 2 == 0
+    return cache
+
+multiplication_tool.cache_function = cache_func
+
+writer1 = Agent(
+        role="Writer",
+        goal="You write lessons of math for kids.",
+        backstory="You're an expert in writing and you love to teach kids but you know nothing of math.",
+        tools=[multiplication_tool],
+        allow_delegation=False,
+    )
+    #...
+```
+
+
+## Conclusion
+Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively. When building solutions with CrewAI, leverage both custom and existing tools to empower your agents and enhance the AI ecosystem. Consider utilizing error handling, caching mechanisms, and the flexibility of tool arguments to optimize your agents' performance and capabilities.

docs/core-concepts/Understanding-Agents.md

@@ -1,41 +0,0 @@
-# What is an Agent?
-
-In CrewAI, an agent is an autonomous unit programmed to perform tasks, make decisions, and communicate with other agents. Think of an agent as a member of a team, with specific skills and a particular job to do. Agents can have different roles like 'Researcher', 'Writer', or 'Customer Support', each contributing to the overall goal of the crew.
-
-# Key Properties of an Agent
-
-- **Role**: Defines the agent's function within the crew. It determines the kind of tasks the agent is best suited for.
-- **Goal**: The individual objective that the agent aims to achieve. It guides the agent's decision-making process.
-- **Backstory**: Provides context to the agent's role and goal, enriching the interaction and collaboration dynamics.
-- **Tools**: A set of capabilities or functions that the agent can use to perform tasks. Tools can be shared or exclusive to specific agents.
-- **Verbose**: This allow you to actually see what is going on during the Crew execution.
-- **Allow Delegation**: Agents can delegate tasks or questions to one another, ensuring that each task is handled by the most suitable agent.
-
-# Agent Lifecycle
-
-1. **Initialization**: An agent is created with a defined role, goal, backstory, and set of tools.
-2. **Task Assignment**: The agent is assigned tasks either directly or through the crew's process management.
-3. **Execution**: The agent performs the task using its available tools and in accordance with its role and goal.
-4. **Collaboration**: Throughout the execution, the agent can communicate with other agents to delegate, inquire, or assist.
-
-# Creating an Agent
-
-To create an agent, you would typically initialize an instance of the `Agent` class with the desired properties. Here's a conceptual example:
-
-```python
-from crewai import Agent
-
-# Create an agent with a role and a goal
-agent = Agent(
-  role='Data Analyst',
-  goal='Extract actionable insights',
-  verbose=True,
-  backstory="You'er a data analyst at a large company. I am responsible for analyzing data and providing insights to the business. I am currently working on a project to analyze the performance of our marketing campaigns. I have been asked to provide insights on how to improve the performance of our marketing campaigns."
-)
-```
-
-# Agent Interaction
-Agents can interact with each other using the CrewAI's built-in delegation and communication mechanisms. This allows for dynamic task management and problem-solving within the crew.
-
-# Conclusion
-Agents are the building blocks of the CrewAI framework. By understanding how to define and interact with agents, you can create sophisticated AI systems that leverage the power of collaborative intelligence.

docs/core-concepts/Using-LangChain-Tools.md

@@ -0,0 +1,38 @@
+---
+title: Using LangChain Tools
+description: Learn how to integrate LangChain tools with CrewAI agents to enhance search-based queries and more.
+---
+
+## Using LangChain Tools
+!!! info "LangChain Integration"
+    CrewAI seamlessly integrates with LangChain’s comprehensive toolkit for search-based queries and more, here are the available built-in tools that are offered by Langchain [LangChain Toolkit](https://python.langchain.com/docs/integrations/tools/)
+
+```python
+from crewai import Agent
+from langchain.agents import Tool
+from langchain.utilities import GoogleSerperAPIWrapper
+
+# Setup API keys
+os.environ["SERPER_API_KEY"] = "Your Key"
+
+search = GoogleSerperAPIWrapper()
+
+# Create and assign the search tool to an agent
+serper_tool = Tool(
+  name="Intermediate Answer",
+  func=search.run,
+  description="Useful for search-based queries",
+)
+
+agent = Agent(
+  role='Research Analyst',
+  goal='Provide up-to-date market analysis',
+  backstory='An expert analyst with a keen eye for market trends.',
+  tools=[serper_tool]
+)
+
+# rest of the code ...
+```
+
+## Conclusion
+Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively. When building solutions with CrewAI, leverage both custom and existing tools to empower your agents and enhance the AI ecosystem. Consider utilizing error handling, caching mechanisms, and the flexibility of tool arguments to optimize your agents' performance and capabilities.

docs/core-concepts/Using-LlamaIndex-Tools.md

@@ -0,0 +1,57 @@
+---
+title: Using LlamaIndex Tools
+description: Learn how to integrate LlamaIndex tools with CrewAI agents to enhance search-based queries and more.
+---
+
+## Using LlamaIndex Tools
+
+!!! info "LlamaIndex Integration"
+    CrewAI seamlessly integrates with LlamaIndex’s comprehensive toolkit for RAG (Retrieval-Augmented Generation) and agentic pipelines, enabling advanced search-based queries and more. Here are the available built-in tools offered by LlamaIndex.
+
+```python
+from crewai import Agent
+from crewai_tools import LlamaIndexTool
+
+# Example 1: Initialize from FunctionTool
+from llama_index.core.tools import FunctionTool
+
+your_python_function = lambda ...: ...
+og_tool = FunctionTool.from_defaults(your_python_function, name="<name>", description='<description>')
+tool = LlamaIndexTool.from_tool(og_tool)
+
+# Example 2: Initialize from LlamaHub Tools
+from llama_index.tools.wolfram_alpha import WolframAlphaToolSpec
+wolfram_spec = WolframAlphaToolSpec(app_id="<app_id>")
+wolfram_tools = wolfram_spec.to_tool_list()
+tools = [LlamaIndexTool.from_tool(t) for t in wolfram_tools]
+
+# Example 3: Initialize Tool from a LlamaIndex Query Engine
+query_engine = index.as_query_engine()
+query_tool = LlamaIndexTool.from_query_engine(
+    query_engine,
+    name="Uber 2019 10K Query Tool",
+    description="Use this tool to lookup the 2019 Uber 10K Annual Report"
+)
+
+# Create and assign the tools to an agent
+agent = Agent(
+  role='Research Analyst',
+  goal='Provide up-to-date market analysis',
+  backstory='An expert analyst with a keen eye for market trends.',
+  tools=[tool, *tools, query_tool]
+)
+
+# rest of the code ...
+```
+
+## Steps to Get Started
+
+To effectively use the LlamaIndexTool, follow these steps:
+
+1. **Package Installation**: Confirm that the `crewai[tools]` package is installed in your Python environment.
+
+    ```shell
+    pip install 'crewai[tools]'
+    ```
+
+2. **Install and Use LlamaIndex**: Follow LlamaIndex documentation [LlamaIndex Documentation](https://docs.llamaindex.ai/) to set up a RAG/agent pipeline.

docs/getting-started.md

@@ -1,95 +0,0 @@
-# Getting Started
-
-To get started with CrewAI, follow these simple steps:
-
-1. **Installation**:
-
-```shell
-pip install crewai
-```
-
-The example below also uses duckduckgo, so also install that
-```shell
-pip install duckduckgo-search
-```
-
-2. **Setting Up Your Crew**:
-
-```python
-import os
-from crewai import Agent, Task, Crew, Process
-
-os.environ["OPENAI_API_KEY"] = "YOUR KEY"
-
-# You can choose to use a local model through Ollama for example. See ./docs/llm-connections.md for more information.
-# from langchain.llms import Ollama
-# ollama_llm = Ollama(model="openhermes")
-
-# Install duckduckgo-search for this example:
-# !pip install -U duckduckgo-search
-
-from langchain.tools import DuckDuckGoSearchRun
-search_tool = DuckDuckGoSearchRun()
-
-# Define your agents with roles and goals
-researcher = Agent(
-  role='Senior Research Analyst',
-  goal='Uncover cutting-edge developments in AI and data science',
-  backstory="""You work at a leading tech think tank.
-  Your expertise lies in identifying emerging trends.
-  You have a knack for dissecting complex data and presenting
-  actionable insights.""",
-  verbose=True,
-  allow_delegation=False,
-  tools=[search_tool]
-  # You can pass an optional llm attribute specifying what mode you wanna use.
-  # It can be a local model through Ollama / LM Studio or a remote
-  # model like OpenAI, Mistral, Antrophic of others (https://python.langchain.com/docs/integrations/llms/)
-  #
-  # Examples:
-  # llm=ollama_llm # was defined above in the file
-  # llm=ChatOpenAI(model_name="gpt-3.5", temperature=0.7)
-)
-writer = Agent(
-  role='Tech Content Strategist',
-  goal='Craft compelling content on tech advancements',
-  backstory="""You are a renowned Content Strategist, known for
-  your insightful and engaging articles.
-  You transform complex concepts into compelling narratives.""",
-  verbose=True,
-  allow_delegation=True,
-  # (optional) llm=ollama_llm
-)
-
-# Create tasks for your agents
-task1 = Task(
-  description="""Conduct a comprehensive analysis of the latest advancements in AI in 2024.
-  Identify key trends, breakthrough technologies, and potential industry impacts.
-  Your final answer MUST be a full analysis report""",
-  agent=researcher
-)
-
-task2 = Task(
-  description="""Using the insights provided, develop an engaging blog
-  post that highlights the most significant AI advancements.
-  Your post should be informative yet accessible, catering to a tech-savvy audience.
-  Make it sound cool, avoid complex words so it doesn't sound like AI.
-  Your final answer MUST be the full blog post of at least 4 paragraphs.""",
-  agent=writer
-)
-
-# Instantiate your crew with a sequential process
-crew = Crew(
-  agents=[researcher, writer],
-  tasks=[task1, task2],
-  verbose=2, # You can set it to 1 or 2 to different logging levels
-)
-
-# Get your crew to work!
-result = crew.kickoff()
-
-print("######################")
-print(result)
-```
-
-Currently the only supported process is `Process.sequential`, where one task is executed after the other and the outcome of one is passed as extra content into this next.

docs/how-to/AgentOps-Observability.md

@@ -0,0 +1,86 @@
+---
+title: Agent Monitoring with AgentOps
+description: Understanding and logging your agent performance with AgentOps.
+---
+
+# Intro
+Observability is a key aspect of developing and deploying conversational AI agents. It allows developers to understand how their agents are performing, how their agents are interacting with users, and how their agents use external tools and APIs. AgentOps is a product independent of CrewAI that provides a comprehensive observability solution for agents.
+
+## AgentOps
+
+[AgentOps](https://agentops.ai/?=crew) provides session replays, metrics, and monitoring for agents.
+
+At a high level, AgentOps gives you the ability to monitor cost, token usage, latency, agent failures, session-wide statistics, and more. For more info, check out the [AgentOps Repo](https://github.com/AgentOps-AI/agentops).
+
+### Overview
+AgentOps provides monitoring for agents in development and production. It provides a dashboard for tracking agent performance, session replays, and custom reporting.
+
+Additionally, AgentOps provides session drilldowns for viewing Crew agent interactions, LLM calls, and tool usage in real-time. This feature is useful for debugging and understanding how agents interact with users as well as other agents.
+
+![Overview of a select series of agent session runs](..%2Fassets%2Fagentops-overview.png)
+![Overview of session drilldowns for examining agent runs](..%2Fassets%2Fagentops-session.png)
+![Viewing a step-by-step agent replay execution graph](..%2Fassets%2Fagentops-replay.png)
+
+### Features
+- **LLM Cost Management and Tracking**: Track spend with foundation model providers.
+- **Replay Analytics**: Watch step-by-step agent execution graphs.
+- **Recursive Thought Detection**: Identify when agents fall into infinite loops.
+- **Custom Reporting**: Create custom analytics on agent performance.
+- **Analytics Dashboard**: Monitor high-level statistics about agents in development and production.
+- **Public Model Testing**: Test your agents against benchmarks and leaderboards.
+- **Custom Tests**: Run your agents against domain-specific tests.
+- **Time Travel Debugging**: Restart your sessions from checkpoints.
+- **Compliance and Security**: Create audit logs and detect potential threats such as profanity and PII leaks.
+- **Prompt Injection Detection**: Identify potential code injection and secret leaks.
+
+### Using AgentOps
+
+1. **Create an API Key:**
+   Create a user API key here: [Create API Key](app.agentops.ai/account)
+
+2. **Configure Your Environment:**
+   Add your API key to your environment variables
+
+   ```bash
+   AGENTOPS_API_KEY=<YOUR_AGENTOPS_API_KEY>
+   ```
+
+3. **Install AgentOps:**
+   Install AgentOps with:
+   ```bash
+   pip install crewai[agentops]
+   ```
+   or
+   ```bash
+   pip install agentops
+   ```
+
+   Before using `Crew` in your script, include these lines:
+
+   ```python
+   import agentops
+   agentops.init()
+   ```
+
+   This will initiate an AgentOps session as well as automatically track Crew agents. For further info on how to outfit more complex agentic systems, check out the [AgentOps documentation](https://docs.agentops.ai) or join the [Discord](https://discord.gg/j4f3KbeH).
+
+### Crew + AgentOps Examples
+- [Job Posting](https://github.com/joaomdmoura/crewAI-examples/tree/main/job-posting)
+- [Markdown Validator](https://github.com/joaomdmoura/crewAI-examples/tree/main/markdown_validator)
+- [Instagram Post](https://github.com/joaomdmoura/crewAI-examples/tree/main/instagram_post)
+
+### Further Information
+
+To get started, create an [AgentOps account](https://agentops.ai/?=crew).
+
+For feature requests or bug reports, please reach out to the AgentOps team on the [AgentOps Repo](https://github.com/AgentOps-AI/agentops).
+
+#### Extra links
+
+<a href="https://twitter.com/agentopsai/">🐦 Twitter</a>
+<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
+<a href="https://discord.gg/JHPt4C7r">📢 Discord</a>
+<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
+<a href="https://app.agentops.ai/?=crew">🖇️ AgentOps Dashboard</a>
+<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
+<a href="https://docs.agentops.ai/introduction">📙 Documentation</a>

docs/how-to/Create-Custom-Tools.md

@@ -0,0 +1,63 @@
+---
+title: Creating and Utilizing Tools in crewAI
+description: Comprehensive guide on crafting, using, and managing custom tools within the crewAI framework, including new functionalities and error handling.
+---
+
+## Creating and Utilizing Tools in crewAI
+This guide provides detailed instructions on creating custom tools for the crewAI framework and how to efficiently manage and utilize these tools, incorporating the latest functionalities such as tool delegation, error handling, and dynamic tool calling. It also highlights the importance of collaboration tools, enabling agents to perform a wide range of actions.
+
+### Prerequisites
+Before creating your own tools, ensure you have the crewAI extra tools package installed:
+
+```bash
+pip install 'crewai[tools]'
+```
+
+### Subclassing `BaseTool`
+
+To create a personalized tool, inherit from `BaseTool` and define the necessary attributes and the `_run` method.
+
+```python
+from crewai_tools import BaseTool
+
+class MyCustomTool(BaseTool):
+    name: str = "Name of my tool"
+    description: str = "What this tool does. It's vital for effective utilization."
+
+    def _run(self, argument: str) -> str:
+        # Your tool's logic here
+        return "Tool's result"
+```
+
+### Using the `tool` Decorator
+
+Alternatively, use the `tool` decorator for a direct approach to create tools. This requires specifying attributes and the tool's logic within a function.
+
+```python
+from crewai_tools import tool
+
+@tool("Tool Name")
+def my_simple_tool(question: str) -> str:
+    """Tool description for clarity."""
+    # Tool logic here
+    return "Tool output"
+```
+
+### Defining a Cache Function for the Tool
+
+To optimize tool performance with caching, define custom caching strategies using the `cache_function` attribute.
+
+```python
+@tool("Tool with Caching")
+def cached_tool(argument: str) -> str:
+    """Tool functionality description."""
+    return "Cachable result"
+
+def my_cache_strategy(arguments: dict, result: str) -> bool:
+    # Define custom caching logic
+    return True if some_condition else False
+
+cached_tool.cache_function = my_cache_strategy
+```
+
+By adhering to these guidelines and incorporating new functionalities and collaboration tools into your tool creation and management processes, you can leverage the full capabilities of the crewAI framework, enhancing both the development experience and the efficiency of your AI agents.

docs/how-to/Creating-a-Crew-and-kick-it-off.md

@@ -1,75 +1,136 @@
-# Get a crew working
+---
+title: Assembling and Activating Your CrewAI Team
+description: A comprehensive guide to creating a dynamic CrewAI team for your projects, with updated functionalities including verbose mode, memory capabilities, asynchronous execution, output customization, language model configuration, and more.
 
-Assembling a Crew in CrewAI is like casting characters for a play. Each agent you create is a cast member with a unique part to play. When your crew is assembled, you'll give the signal, and they'll spring into action, each performing their role in the grand scheme of your project.
+---
 
-# Step 1: Assemble Your Agents
+## Introduction
+Embark on your CrewAI journey by setting up your environment and initiating your AI crew with the latest features. This guide ensures a smooth start, incorporating all recent updates for an enhanced experience.
 
-Start by creating your agents, each with its own role and backstory. These backstories add depth to the agents, influencing how they approach their tasks and interact with one another.
+## Step 0: Installation
+Install CrewAI and any necessary packages for your project. CrewAI is compatible with Python >=3.10,<=3.13.
+
+```shell
+pip install crewai
+pip install 'crewai[tools]'
+```
+
+## Step 1: Assemble Your Agents
+Define your agents with distinct roles, backstories, and enhanced capabilities like verbose mode, memory usage, and the ability to set specific agents as managers. These elements add depth and guide their task execution and interaction within the crew.
 
 ```python
-from crewai import Agent
+import os
+os.environ["SERPER_API_KEY"] = "Your Key"  # serper.dev API key
+os.environ["OPENAI_API_KEY"] = "Your Key"
 
-# Create a researcher agent
+from crewai import Agent
+from crewai_tools import SerperDevTool, BrowserbaseTool, ExaSearchTool
+search_tool = SerperDevTool()
+browser_tool = BrowserbaseTool()
+exa_search_tool = ExaSearchTool()
+
+# Creating a senior researcher agent with memory and verbose mode
 researcher = Agent(
   role='Senior Researcher',
-  goal='Discover groundbreaking technologies',
+  goal='Uncover groundbreaking technologies in {topic}',
   verbose=True,
-  backstory='A curious mind fascinated by cutting-edge innovation and the potential to change the world, you know everything about tech.'
+  memory=True,
+  backstory=(
+    "Driven by curiosity, you're at the forefront of"
+    "innovation, eager to explore and share knowledge that could change"
+    "the world."
+  ),
+  tools=[search_tool, browser_tool],
 )
 
-# Create a writer agent
+# Creating a writer agent with custom tools and delegation capability
 writer = Agent(
   role='Writer',
-  goal='Craft compelling stories about tech discoveries',
+  goal='Narrate compelling tech stories about {topic}',
   verbose=True,
-  backstory='A creative soul who translates complex tech jargon into engaging narratives for the masses, you write using simple words in a friendly and inviting tone that does not sounds like AI.'
+  memory=True,
+  backstory=(
+    "With a flair for simplifying complex topics, you craft"
+    "engaging narratives that captivate and educate, bringing new"
+    "discoveries to light in an accessible manner."
+  ),
+  tools=[exa_search_tool],
+  allow_delegation=False
+)
+
+# Setting a specific manager agent
+manager = Agent(
+  role='Manager',
+  goal='Ensure the smooth operation and coordination of the team',
+  verbose=True,
+  backstory=(
+    "As a seasoned project manager, you excel in organizing"
+    "tasks, managing timelines, and ensuring the team stays on track."
+  )
 )
 ```
 
-# Step 2: Define the Tasks
-
-Outline the tasks that your agents need to tackle. These tasks are their missions, the specific objectives they need to achieve.
+## Step 2: Define the Tasks
+Detail the specific objectives for your agents, including new features for asynchronous execution and output customization. These tasks ensure a targeted approach to their roles.
 
 ```python
 from crewai import Task
 
-# Task for the researcher
+# Research task
 research_task = Task(
-  description='Identify the next big trend in AI',
-  agent=researcher  # Assigning the task to the researcher
+  description=(
+    "Identify the next big trend in {topic}."
+    "Focus on identifying pros and cons and the overall narrative."
+    "Your final report should clearly articulate the key points,"
+    "its market opportunities, and potential risks."
+  ),
+  expected_output='A comprehensive 3 paragraphs long report on the latest AI trends.',
+  tools=[search_tool],
+  agent=researcher,
+  callback="research_callback",  # Example of task callback
+  human_input=True
 )
 
-# Task for the writer
+# Writing task with language model configuration
 write_task = Task(
-  description='Write an article on AI advancements leveraging the research made.',
-  agent=writer  # Assigning the task to the writer
+  description=(
+    "Compose an insightful article on {topic}."
+    "Focus on the latest trends and how it's impacting the industry."
+    "This article should be easy to understand, engaging, and positive."
+  ),
+  expected_output='A 4 paragraph article on {topic} advancements formatted as markdown.',
+  tools=[exa_search_tool],
+  agent=writer,
+  output_file='new-blog-post.md',  # Example of output customization
 )
 ```
 
-# Step 3: Form the Crew
-
-Bring your agents together into a crew. This is where you define the process they'll follow to complete their tasks.
+## Step 3: Form the Crew
+Combine your agents into a crew, setting the workflow process they'll follow to accomplish the tasks. Now with options to configure language models for enhanced interaction and additional configurations for optimizing performance, such as creating directories when saving files.
 
 ```python
 from crewai import Crew, Process
 
-# Instantiate your crew
-tech_crew = Crew(
+# Forming the tech-focused crew with some enhanced configurations
+crew = Crew(
   agents=[researcher, writer],
   tasks=[research_task, write_task],
-  process=Process.sequential  # Tasks will be executed one after the other
+  process=Process.sequential,  # Optional: Sequential task execution is default
+  memory=True,
+  cache=True,
+  max_rpm=100,
+  manager_agent=manager
 )
 ```
 
-# Step 4: Kick It Off
-
-With the crew formed and the stage set, it's time to start the show. Kick off the process and watch as your agents collaborate to achieve their goals.
+## Step 4: Kick It Off
+Initiate the process with your enhanced crew ready. Observe as your agents collaborate, leveraging their new capabilities for a successful project outcome. Input variables will be interpolated into the agents and tasks for a personalized approach.
 
 ```python
-# Begin the task execution
-tech_crew.kickoff()
+# Starting the task execution process with enhanced feedback
+result = crew.kickoff(inputs={'topic': 'AI in healthcare'})
+print(result)
 ```
 
-# Conclusion
-
-Creating a crew and setting it into motion is a straightforward process in CrewAI. With each agent playing their part and a clear set of tasks, your AI ensemble is ready to take on any challenge. Remember, the richness of their backstories and the clarity of their goals will greatly enhance their performance and the outcomes of their collaboration.
+## Conclusion
+Building and activating a crew in CrewAI has evolved with new functionalities. By incorporating verbose mode, memory capabilities, asynchronous task execution, output customization, language model configuration, and enhanced crew configurations, your AI team is more equipped than ever to tackle challenges efficiently. The depth of agent backstories and the precision of their objectives enrich collaboration, leading to successful project outcomes. This guide aims to provide you with a clear and detailed understanding of setting up and utilizing the CrewAI framework to its full potential.

docs/how-to/Customize-Prompts.md

@@ -0,0 +1,94 @@
+---
+title: Initial Support to Bring Your Own Prompts in CrewAI
+description: Enhancing customization and internationalization by allowing users to bring their own prompts in CrewAI.
+
+---
+
+# Initial Support to Bring Your Own Prompts in CrewAI
+
+CrewAI now supports the ability to bring your own prompts, enabling extensive customization and internationalization. This feature allows users to tailor the inner workings of their agents to better suit specific needs, including support for multiple languages.
+
+## Internationalization and Customization Support
+
+### Custom Prompts with `prompt_file`
+
+The `prompt_file` attribute facilitates full customization of the agent prompts, enhancing the global usability of CrewAI. Users can specify their prompt templates, ensuring that the agents communicate in a manner that aligns with specific project requirements or language preferences.
+
+#### Example of a Custom Prompt File
+
+The custom prompts can be defined in a JSON file, similar to the example provided [here](https://github.com/joaomdmoura/crewAI/blob/main/src/crewai/translations/en.json).
+
+### Supported Languages
+
+CrewAI's custom prompt support includes internationalization, allowing prompts to be written in different languages. This is particularly useful for global teams or projects that require multilingual support.
+
+## How to Use the `prompt_file` Attribute
+
+To utilize the `prompt_file` attribute, include it in your crew definition. Below is an example demonstrating how to set up agents and tasks with custom prompts.
+
+### Example
+
+```python
+import os
+from crewai import Agent, Task, Crew
+
+# Define your agents
+researcher = Agent(
+    role="Researcher",
+    goal="Make the best research and analysis on content about AI and AI agents",
+    backstory="You're an expert researcher, specialized in technology, software engineering, AI and startups. You work as a freelancer and is now working on doing research and analysis for a new customer.",
+    allow_delegation=False,
+)
+
+writer = Agent(
+    role="Senior Writer",
+    goal="Write the best content about AI and AI agents.",
+    backstory="You're a senior writer, specialized in technology, software engineering, AI and startups. You work as a freelancer and are now working on writing content for a new customer.",
+    allow_delegation=False,
+)
+
+# Define your tasks
+tasks = [
+    Task(
+        description="Say Hi",
+        expected_output="The word: Hi",
+        agent=researcher,
+    )
+]
+
+# Instantiate your crew with custom prompts
+crew = Crew(
+    agents=[researcher],
+    tasks=tasks,
+    prompt_file="prompt.json",  # Path to your custom prompt file
+)
+
+# Get your crew to work!
+crew.kickoff()
+```
+
+## Advanced Customization Features
+
+### `language` Attribute
+
+In addition to `prompt_file`, the `language` attribute can be used to specify the language for the agent's prompts. This ensures that the prompts are generated in the desired language, further enhancing the internationalization capabilities of CrewAI.
+
+### Creating Custom Prompt Files
+
+Custom prompt files should be structured in JSON format and include all necessary prompt templates. Below is a simplified example of a prompt JSON file:
+
+```json
+{
+    "system": "You are a system template.",
+    "prompt": "Here is your prompt template.",
+    "response": "Here is your response template."
+}
+```
+
+### Benefits of Custom Prompts
+
+- **Enhanced Flexibility**: Tailor agent communication to specific project needs.
+- **Improved Usability**: Supports multiple languages, making it suitable for global projects.
+- **Consistency**: Ensures uniform prompt structures across different agents and tasks.
+
+By incorporating these updates, CrewAI provides users with the ability to fully customize and internationalize their agent prompts, making the platform more versatile and user-friendly.

docs/how-to/Customizing-Agents.md

@@ -1,66 +1,88 @@
-# Customizable Attributes
+---
+title: Customizing Agents in CrewAI
+description: A comprehensive guide to tailoring agents for specific roles, tasks, and advanced customizations within the CrewAI framework.
+---
 
-Customizing your AI agents is a cornerstone of creating an effective CrewAI team. Each agent can be tailored to fit the unique needs of your project, allowing for a dynamic and versatile AI workforce.
+## Customizable Attributes
+Crafting an efficient CrewAI team hinges on the ability to dynamically tailor your AI agents to meet the unique requirements of any project. This section covers the foundational attributes you can customize.
 
-When you initialize an Agent, you can set various attributes that define its behavior and role within the Crew:
+### Key Attributes for Customization
+- **Role**: Specifies the agent's job within the crew, such as 'Analyst' or 'Customer Service Rep'.
+- **Goal**: Defines what the agent aims to achieve, in alignment with its role and the overarching objectives of the crew.
+- **Backstory**: Provides depth to the agent's persona, enriching its motivations and engagements within the crew.
+- **Tools** *(Optional)*: Represents the capabilities or methods the agent uses to perform tasks, from simple functions to intricate integrations.
+- **Cache** *(Optional)*: Determines whether the agent should use a cache for tool usage.
+- **Max RPM**: Sets the maximum number of requests per minute (`max_rpm`). This attribute is optional and can be set to `None` for no limit, allowing for unlimited queries to external services if needed.
+- **Verbose** *(Optional)*: Enables detailed logging of an agent's actions, useful for debugging and optimization. Specifically, it provides insights into agent execution processes, aiding in the optimization of performance.
+- **Allow Delegation** *(Optional)*: `allow_delegation` controls whether the agent is allowed to delegate tasks to other agents.
+- **Max Iter** *(Optional)*: The `max_iter` attribute allows users to define the maximum number of iterations an agent can perform for a single task, preventing infinite loops or excessively long executions. The default value is set to 25, providing a balance between thoroughness and efficiency. Once the agent approaches this number, it will try its best to give a good answer.
+- **Max Execution Time** *(Optional)*: `max_execution_time` Sets the maximum execution time for an agent to complete a task.
+- **System Template** *(Optional)*: `system_template` defines the system format for the agent.
+- **Prompt Template** *(Optional)*: `prompt_template` defines the prompt format for the agent.
+- **Response Template** *(Optional)*: `response_template` defines the response format for the agent.
 
-- **Role**: The job title or function of the agent within your crew. This can be anything from 'Analyst' to 'Customer Service Rep'.
-- **Goal**: What the agent is aiming to achieve. Goals should be aligned with the agent's role and the overall objectives of the crew.
-- **Backstory**: A narrative that provides depth to the agent's character. This could include previous experience, motivations, or anything that adds context to their role.
-- **Tools**: The abilities or methods the agent uses to complete tasks. This could be as simple as a 'search' function or as complex as a custom-built analysis tool.
+## Advanced Customization Options
+Beyond the basic attributes, CrewAI allows for deeper customization to enhance an agent's behavior and capabilities significantly.
 
-# Understanding Tools in CrewAI
+### Language Model Customization
+Agents can be customized with specific language models (`llm`) and function-calling language models (`function_calling_llm`), offering advanced control over their processing and decision-making abilities. It's important to note that setting the `function_calling_llm` allows for overriding the default crew function-calling language model, providing a greater degree of customization.
 
-Tools in CrewAI are functions that empower agents to interact with the world around them. These can range from generic utilities like a search function to more complex ones like integrating with an external API. The integration with LangChain allows you to utilize a suite of ready-to-use tools such as [Google Serper](https://python.langchain.com/docs/integrations/tools/google_serper), which enables agents to perform web searches and gather data.
+## Performance and Debugging Settings
+Adjusting an agent's performance and monitoring its operations are crucial for efficient task execution.
 
-# Customizing Agents and Tools
+### Verbose Mode and RPM Limit
+- **Verbose Mode**: Enables detailed logging of an agent's actions, useful for debugging and optimization. Specifically, it provides insights into agent execution processes, aiding in the optimization of performance.
+- **RPM Limit**: Sets the maximum number of requests per minute (`max_rpm`). This attribute is optional and can be set to `None` for no limit, allowing for unlimited queries to external services if needed.
 
-You can customize an agent by passing parameters when creating an instance. Each parameter tweaks how the agent behaves and interacts within the crew.
+### Maximum Iterations for Task Execution
+The `max_iter` attribute allows users to define the maximum number of iterations an agent can perform for a single task, preventing infinite loops or excessively long executions. The default value is set to 25, providing a balance between thoroughness and efficiency. Once the agent approaches this number, it will try its best to give a good answer.
 
-Customizing an agent's tools is particularly important. Tools define what an agent can do and how it interacts with tasks. For instance, if a task requires data analysis, assigning an agent with data-related tools would be optimal.
+## Customizing Agents and Tools
+Agents are customized by defining their attributes and tools during initialization. Tools are critical for an agent's functionality, enabling them to perform specialized tasks. The `tools` attribute should be an array of tools the agent can utilize, and it's initialized as an empty list by default. Tools can be added or modified post-agent initialization to adapt to new requirements.
 
-When initializing your agents, you can equip them with a set of tools that enable them to perform their roles more effectively:
+```shell
+pip install 'crewai[tools]'
+```
 
+### Example: Assigning Tools to an Agent
 ```python
+import os
 from crewai import Agent
-from langchain.agents import Tool
-from langchain.utilities import GoogleSerperAPIWrapper
+from crewai_tools import SerperDevTool
 
-# Initialize SerpAPI tool with your API key
+# Set API keys for tool initialization
 os.environ["OPENAI_API_KEY"] = "Your Key"
 os.environ["SERPER_API_KEY"] = "Your Key"
 
-search = GoogleSerperAPIWrapper()
+# Initialize a search tool
+search_tool = SerperDevTool()
 
-# Create tool to be used by agent
-serper_tool = Tool(
-  name="Intermediate Answer",
-  func=search.run,
-  description="useful for when you need to ask with search",
-)
-
-# Create an agent and assign the search tool
+# Initialize the agent with advanced options
 agent = Agent(
   role='Research Analyst',
   goal='Provide up-to-date market analysis',
   backstory='An expert analyst with a keen eye for market trends.',
-  tools=[serper_tool]
-)
-```
-
-## Delegation and Autonomy
-
-One of the most powerful aspects of CrewAI agents is their ability to delegate tasks to one another. Each agent by default can delegate work or ask question to anyone in the crew, but you can disable that by setting `allow_delegation` to `false`, this is particularly useful for straightforward agents that should execute their tasks in isolation.
-
-```python
-agent = Agent(
-  role='Content Writer',
-  goal='Write the most amazing content related to market trends an business.',
-  backstory='An expert writer with many years of experience in market trends, stocks and all business related things.',
+  tools=[search_tool],
+  memory=True, # Enable memory
+  verbose=True,
+  max_rpm=None, # No limit on requests per minute
+  max_iter=25, # Default value for maximum iterations
   allow_delegation=False
 )
 ```
 
-## Conclusion
+## Delegation and Autonomy
+Controlling an agent's ability to delegate tasks or ask questions is vital for tailoring its autonomy and collaborative dynamics within the CrewAI framework. By default, the `allow_delegation` attribute is set to `True`, enabling agents to seek assistance or delegate tasks as needed. This default behavior promotes collaborative problem-solving and efficiency within the CrewAI ecosystem. If needed, delegation can be disabled to suit specific operational requirements.
 
-Customization is what makes CrewAI powerful. By adjusting the attributes of each agent, you can ensure that your AI team is well-equipped to handle the challenges you set for them. Remember, the more thought you put into your agents' roles, goals, backstories, and tools, the more nuanced and effective their interactions and task execution will be.
+### Example: Disabling Delegation for an Agent
+```python
+agent = Agent(
+  role='Content Writer',
+  goal='Write engaging content on market trends',
+  backstory='A seasoned writer with expertise in market analysis.',
+  allow_delegation=False # Disabling delegation
+)
+```
+
+## Conclusion
+Customizing agents in CrewAI by setting their roles, goals, backstories, and tools, alongside advanced options like language model customization, memory, performance settings, and delegation preferences, equips a nuanced and capable AI team ready for complex challenges.