mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-08-13 01:38:41 +00:00
11890e6701
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
* Standardize CLI flags to kebab-case with deprecated snake_case aliases. Unify active long-option naming across create, train, test, and replay while keeping hidden backward-compatible aliases and documenting the migration in edge docs and AGENTS.md. * Emit deprecation warnings when snake_case CLI flag aliases are used. Route hidden legacy flags through separate internal params so warnings fire only when the alias is supplied, and extend CLI tests for create, replay, and --help coverage. * Merge CLI deprecation warn helpers into warn_deprecated(kind=...). Replace warn_deprecated_command and warn_deprecated_flag with one helper that accepts kind="command" or kind="flag".
55 lines
2.8 KiB
Plaintext
55 lines
2.8 KiB
Plaintext
---
|
|
title: Testing
|
|
description: Learn how to test your CrewAI Crew and evaluate their performance.
|
|
icon: vial
|
|
mode: "wide"
|
|
---
|
|
|
|
## Overview
|
|
|
|
Testing is a crucial part of the development process, and it is essential to ensure that your crew is performing as expected. With crewAI, you can easily test your crew and evaluate its performance using the built-in testing capabilities.
|
|
|
|
### Using the Testing Feature
|
|
|
|
We added the CLI command `crewai test` to make it easy to test your crew. This command will run your crew for a specified number of iterations and provide detailed performance metrics. The parameters are `n_iterations` and `model`, which are optional and default to 2 and `gpt-4o-mini` respectively. For now, the only provider available is OpenAI.
|
|
|
|
```bash
|
|
crewai test
|
|
```
|
|
|
|
If you want to run more iterations or use a different model, you can specify the parameters like this:
|
|
|
|
```bash
|
|
crewai test --n-iterations 5 --model gpt-4o
|
|
```
|
|
|
|
or using the short forms:
|
|
|
|
```bash
|
|
crewai test -n 5 -m gpt-4o
|
|
```
|
|
|
|
<Note>
|
|
The older `--n_iterations` flag still works but is deprecated and hidden from
|
|
`--help`. Use `--n-iterations` (or `-n`) instead.
|
|
</Note>
|
|
|
|
When you run the `crewai test` command, the crew will be executed for the specified number of iterations, and the performance metrics will be displayed at the end of the run.
|
|
|
|
A table of scores at the end will show the performance of the crew in terms of the following metrics:
|
|
|
|
<center>**Tasks Scores (1-10 Higher is better)**</center>
|
|
|
|
| Tasks/Crew/Agents | Run 1 | Run 2 | Avg. Total | Agents | Additional Info |
|
|
|:------------------|:-----:|:-----:|:----------:|:------------------------------:|:---------------------------------|
|
|
| Task 1 | 9.0 | 9.5 | **9.2** | Professional Insights | |
|
|
| | | | | Researcher | |
|
|
| Task 2 | 9.0 | 10.0 | **9.5** | Company Profile Investigator | |
|
|
| Task 3 | 9.0 | 9.0 | **9.0** | Automation Insights | |
|
|
| | | | | Specialist | |
|
|
| Task 4 | 9.0 | 9.0 | **9.0** | Final Report Compiler | Automation Insights Specialist |
|
|
| Crew | 9.00 | 9.38 | **9.2** | | |
|
|
| Execution Time (s) | 126 | 145 | **135** | | |
|
|
|
|
The example above shows the test results for two runs of the crew with two tasks, with the average total score for each task and the crew as a whole.
|