Merge branch 'main' into feature/procedure_v2

docs/core-concepts/Agents.md

@@ -114,7 +114,7 @@ from langchain.agents import load_tools
 langchain_tools = load_tools(["google-serper"], llm=llm)
 
 agent1 = CustomAgent(
-    role="backstory agent",
+    role="agent role",
     goal="who is {input}?",
     backstory="agent backstory",
     verbose=True,
@@ -127,7 +127,7 @@ task1 = Task(
 )
 
 agent2 = Agent(
-    role="bio agent",
+    role="agent role",
     goal="summarize the short bio for {input} and if needed do more research",
     backstory="agent backstory",
     verbose=True,

docs/core-concepts/Crews.md

@@ -33,6 +33,7 @@ A crew in crewAI represents a collaborative group of agents working together to
 | **Manager Callbacks** _(optional)_    | `manager_callbacks`    | `manager_callbacks` takes a list of callback handlers to be executed by the manager agent when a hierarchical process is used.                                                                                                                            |
 | **Prompt File** _(optional)_          | `prompt_file`          | Path to the prompt JSON file to be used for the crew.                                                                                                                                                                                                     |
 | **Planning** *(optional)*             | `planning`             |  Adds planning ability to the Crew. When activated before each Crew iteration, all Crew data is sent to an AgentPlanner that will plan the tasks and this plan will be added to each task description.
+| **Planning LLM** *(optional)*         | `planning_llm`         | The language model used by the AgentPlanner in a planning process. |
 
 !!! note "Crew Max RPM"
 The `max_rpm` attribute sets the maximum number of requests per minute the crew can perform to avoid rate limits and will override individual agents' `max_rpm` settings if you set it.
@@ -136,7 +137,7 @@ crew = Crew(
     verbose=2
 )
 
-result = crew.kickoff()
+crew_output = crew.kickoff()
 
 # Accessing the crew output
 print(f"Raw Output: {crew_output.raw}")

docs/core-concepts/Planning.md

@@ -23,6 +23,25 @@ my_crew = Crew(
 
 From this point on, your crew will have planning enabled, and the tasks will be planned before each iteration.
 
+#### Planning LLM
+
+Now you can define the LLM that will be used to plan the tasks. You can use any ChatOpenAI LLM model available.
+
+```python
+from crewai import Crew, Agent, Task, Process
+from langchain_openai import ChatOpenAI
+
+# Assemble your crew with planning capabilities and custom LLM
+my_crew = Crew(
+    agents=self.agents,
+    tasks=self.tasks,
+    process=Process.sequential,
+    planning=True,
+    planning_llm=ChatOpenAI(model="gpt-4o")
+)
+```
+
+
 ### Example
 
 When running the base case example, you will see something like the following output, which represents the output of the AgentPlanner responsible for creating the step-by-step logic to add to the Agents tasks.

docs/core-concepts/Testing.md

@@ -0,0 +1,41 @@
+---
+title: crewAI Testing
+description: Learn how to test your crewAI Crew and evaluate their performance.
+---
+
+## Introduction
+
+Testing is a crucial part of the development process, and it is essential to ensure that your crew is performing as expected. And with crewAI, you can easily test your crew and evaluate its performance using the built-in testing capabilities.
+
+### Using the Testing Feature
+
+We added the CLI command `crewai test` to make it easy to test your crew. This command will run your crew for a specified number of iterations and provide detailed performance metrics.
+The parameters are `n_iterations` and `model` which are optional and default to 2 and `gpt-4o-mini` respectively. For now the only provider available is OpenAI.
+
+```bash
+crewai test
+```
+
+If you want to run more iterations or use a different model, you can specify the parameters like this:
+
+```bash
+crewai test --n_iterations 5 --model gpt-4o
+```
+
+What happens when you run the `crewai test` command is that the crew will be executed for the specified number of iterations, and the performance metrics will be displayed at the end of the run.
+
+A table of scores at the end will show the performance of the crew in terms of the following metrics:
+```
+                Task Scores
+          (1-10 Higher is better)
+┏━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┓
+┃ Tasks/Crew ┃ Run 1 ┃ Run 2 ┃ Avg. Total ┃
+┡━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━┩
+│ Task 1     │ 10.0  │ 9.0   │ 9.5        │
+│ Task 2     │ 9.0   │ 9.0   │ 9.0        │
+│ Crew       │ 9.5   │ 9.0   │ 9.2        │
+└────────────┴───────┴───────┴────────────┘
+```
+
+The example above shows the test results for two runs of the crew with two tasks, with the average total score for each task and the crew as a whole.
+