Commit Graph

1575 Commits

Author SHA1 Message Date
Lucas Gomide
de01b66f07 Merge branch 'main' into lg-agent-experiments 2025-07-14 09:17:07 -03:00
devin-ai-integration[bot]
3ada4053bd Fix #3149: Add missing create_directory parameter to Task class (#3150)
* Fix #3149: Add missing create_directory parameter to Task class

- Add create_directory field with default value True for backward compatibility
- Update _save_file method to respect create_directory parameter
- Add comprehensive tests covering all scenarios
- Maintain existing behavior when create_directory=True (default)

The create_directory parameter was documented but missing from implementation.
Users can now control directory creation behavior:
- create_directory=True (default): Creates directories if they don't exist
- create_directory=False: Raises RuntimeError if directory doesn't exist

Fixes issue where users got TypeError when trying to use the documented
create_directory parameter.

Co-Authored-By: Jo\u00E3o <joao@crewai.com>

* Fix lint: Remove unused import os from test_create_directory_true

- Removes F401 lint error: 'os' imported but unused
- All lint checks should now pass

Co-Authored-By: Jo\u00E3o <joao@crewai.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Jo\u00E3o <joao@crewai.com>
2025-07-14 08:15:41 -04:00
Lucas Gomide
69df70f4a3 Merge branch 'main' into lg-agent-experiments 2025-07-14 09:09:38 -03:00
Lucas Gomide
0f52071706 test: move tests to experimental folder 2025-07-14 09:07:49 -03:00
Vidit Ostwal
e7a5747c6b Comparing BaseLLM class instead of LLM (#3120)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* Compaing BaseLLM class instead of LLM

* Fixed test cases

* Fixed Linting Issues

* removed last line

---------

Co-authored-by: Lucas Gomide <lucaslg200@gmail.com>
2025-07-11 20:50:36 -04:00
Lucas Gomide
2ab33ac549 Merge remote-tracking branch 'origin/main' into lg-agent-experiments 2025-07-11 14:58:16 -03:00
Vidit Ostwal
eec1262d4f Fix agent knowledge (#2831)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
* Added add_sources()

* Fixed the agent knowledge querying

* Added test cases

* Fixed linting issue

* Fixed logic

* Seems like a falky test case

* Minor changes

* Added knowledge attriute to the crew documentation

* Flaky test

* fixed spaces

* Flaky Test Case

* Seems like a flaky test case

---------

Co-authored-by: Lucas Gomide <lucaslg200@gmail.com>
2025-07-11 13:52:26 -04:00
Lucas Gomide
4439755b2d feat: add experimental folder for beta features (#3141) 2025-07-11 13:43:29 -04:00
Tony Kipkemboi
c6caa763d7 docs: Add guardrail attribute documentation and examples (#3139)
- Document string-based guardrails in tasks
- Add guardrail examples to YAML configuration
- Fix Python code formatting in PT-BR CLI docs
2025-07-11 13:32:59 -04:00
Lucas Gomide
08fa3797ca Introducing Agent evaluation (#3130)
* feat: add exchanged messages in LLMCallCompletedEvent

* feat: add GoalAlignment metric for Agent evaluation

* feat: add SemanticQuality metric for Agent evaluation

* feat: add Tool Metrics for Agent evaluation

* feat: add Reasoning Metrics for Agent evaluation, still in progress

* feat: add AgentEvaluator class

This class will evaluate Agent' results and report to user

* fix: do not evaluate Agent by default

This is a experimental feature we still need refine it further

* test: add Agent eval tests

* fix: render all feedback per iteration

* style: resolve linter issues

* style: fix mypy issues

* fix: allow messages be empty on LLMCallCompletedEvent
2025-07-11 13:18:03 -04:00
Lucas Gomide
e3b044c044 style: resolve linter issues 2025-07-11 14:09:22 -03:00
Lucas Gomide
ee490a19fb fix: encode string before hashing 2025-07-11 14:09:22 -03:00
Lucas Gomide
de425450d4 style: fix linter issues 2025-07-11 14:09:22 -03:00
Lucas Gomide
3fee619798 test: add tests for Evaluator Experiment 2025-07-11 14:09:22 -03:00
Lucas Gomide
bb2a0f95ba fix: use time-aware to define experiment result 2025-07-11 14:09:22 -03:00
Lucas Gomide
22d46999fe fix: make crew required to run an experiment 2025-07-11 14:09:22 -03:00
Lucas Gomide
9f00760437 refactor: isolate Console print in a dedicated class 2025-07-11 14:09:22 -03:00
Lucas Gomide
ffab51ce2c refactor: remove unused method 2025-07-11 14:09:22 -03:00
Lucas Gomide
007f396f4d chore: split Experimental evaluation classes 2025-07-11 14:09:22 -03:00
Lucas Gomide
cf91084f21 fix: fix track of new test cases 2025-07-11 14:09:22 -03:00
Lucas Gomide
ccea8870d7 fix: reset evaluator for each experiement iteraction 2025-07-11 14:09:22 -03:00
Lucas Gomide
15f1912c2e feat: add Experiment evaluation framework with baseline comparison 2025-07-11 14:09:22 -03:00
Lucas Gomide
064997464e fix: allow messages be empty on LLMCallCompletedEvent 2025-07-11 14:05:25 -03:00
Lucas Gomide
6f0ed6642b style: fix mypy issues 2025-07-11 13:02:34 -03:00
Lucas Gomide
43f339fa84 style: resolve linter issues 2025-07-11 13:02:34 -03:00
Lucas Gomide
5ea221e54e fix: render all feedback per iteration 2025-07-11 13:02:34 -03:00
Lucas Gomide
d4c15ec25f test: add Agent eval tests 2025-07-11 13:02:34 -03:00
Lucas Gomide
37cfbe7389 fix: do not evaluate Agent by default
This is a experimental feature we still need refine it further
2025-07-11 13:02:34 -03:00
Lucas Gomide
6d7c7d940e feat: add AgentEvaluator class
This class will evaluate Agent' results and report to user
2025-07-11 13:02:34 -03:00
Lucas Gomide
80bd23a8a9 feat: add Reasoning Metrics for Agent evaluation, still in progress 2025-07-11 13:02:34 -03:00
Lucas Gomide
50593d1485 feat: add Tool Metrics for Agent evaluation 2025-07-11 13:02:34 -03:00
Lucas Gomide
60084af745 feat: add SemanticQuality metric for Agent evaluation 2025-07-11 13:02:34 -03:00
Lucas Gomide
be4ade8c45 feat: add GoalAlignment metric for Agent evaluation 2025-07-11 13:02:34 -03:00
Lucas Gomide
6a49a24810 feat: add exchanged messages in LLMCallCompletedEvent 2025-07-11 13:02:34 -03:00
Greyson LaLonde
bf8fa3232b Add SQLite FTS5 support to test workflow (#3140)
* Add SQLite FTS5 support to test workflow

* Add explanatory comment for SQLite FTS5 workaround
2025-07-11 12:01:25 -04:00
Heitor Carvalho
a6e60a5d42 fix: use production workos environment id (#3129)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
2025-07-09 17:09:01 -04:00
Lorenze Jay
7b0f3aabd9 chore: update crewAI and dependencies to version 0.141.0 and 0.51.0 (#3128)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
- Bump crewAI version to 0.141.0 in __init__.py for alignment with updated dependencies.
- Update `crewai-tools` dependency version to 0.51.0 in pyproject.toml and related template files.
- Add new testing dependencies: pytest-split and pytest-xdist for improved test execution.
- Ensure compatibility with the latest package versions in uv.lock and template files.
0.141.0
2025-07-09 10:37:06 -07:00
Lucas Gomide
f071966951 docs: add docs about Agent.kickoff usage (#3121)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Co-authored-by: Tony Kipkemboi <iamtonykipkemboi@gmail.com>
2025-07-08 16:15:40 -04:00
Lucas Gomide
318310bb7a docs: add docs about Agent repository (#3122) 2025-07-08 15:56:08 -04:00
Greyson LaLonde
34a03f882c feat: add crew context tracking for LLM guardrail events (#3111)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Add crew context tracking using OpenTelemetry baggage for thread-safe propagation. Context is set during kickoff and cleaned up in finally block. Added thread safety tests with mocked agent execution.
2025-07-07 16:33:07 -04:00
Greyson LaLonde
a0fcc0c8d1 Speed up GitHub Actions tests with parallelization (#3107)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
- Add pytest-xdist and pytest-split to dev dependencies for parallel test execution
- Split tests into 8 parallel groups per Python version for better distribution
- Enable CPU-level parallelization with -n auto to maximize resource usage
- Add fail-fast strategy and maxfail=3 to stop early on failures
- Add job name to match branch protection rules
- Reduce test timeout from default to 30s for faster failure detection
- Remove redundant cache configuration
2025-07-03 21:08:00 -04:00
Lorenze Jay
748c25451c Lorenze/new version 0.140.0 (#3106)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* fix: clean up whitespace and update dependencies

* Removed unnecessary whitespace in multiple files for consistency.
* Updated `crewai-tools` dependency version to `0.49.0` in `pyproject.toml` and related template files.
* Bumped CrewAI version to `0.140.0` in `__init__.py` for alignment with updated dependencies.

* chore: update pyproject.toml to exclude documentation from build targets

* Added exclusions for the `docs` directory in both wheel and sdist build targets to streamline the build process and reduce unnecessary file inclusion.

* chore: update uv.lock for dependency resolution and Python version compatibility

* Incremented revision to 2.
* Updated resolution markers to include support for Python 3.13 and adjusted platform checks for better compatibility.
* Added new wheel URLs for zstandard version 0.23.0 to ensure availability across various platforms.

* chore: pin json-repair dependency version in pyproject.toml and uv.lock

* Updated json-repair dependency from a range to a specific version (0.25.2) for consistency and to avoid potential compatibility issues.
* Adjusted related entries in uv.lock to reflect the pinned version, ensuring alignment across project files.

* chore: pin agentops dependency version in pyproject.toml and uv.lock

* Updated agentops dependency from a range to a specific version (0.3.18) for consistency and to avoid potential compatibility issues.
* Adjusted related entries in uv.lock to reflect the pinned version, ensuring alignment across project files.

* test: enhance cache call assertions in crew tests

* Improved the test for cache hitting between agents by filtering mock calls to ensure they include the expected 'tool' and 'input' keywords.
* Added assertions to verify the number of cache calls and their expected arguments, enhancing the reliability of the test.
* Cleaned up whitespace and improved readability in various test cases for better maintainability.
0.140.0
2025-07-02 15:22:18 -07:00
Heitor Carvalho
a77dcdd419 feat: add multiple provider support (#3089)
* Remove `crewai signup` command, update docs

* Add `Settings.clear()` and clear settings before each login

* Add pyjwt

* Remove print statement from ToolCommand.login()

* Remove auth0 dependency

* Update docs
2025-07-02 16:44:47 -04:00
Greyson LaLonde
68f5bdf0d9 feat: add console logging for LLM guardrail events (#3105)
* feat: add console logging for memory events

* fix: emit guardrail events in correct order and handle exceptions

* fix: remove unreachable elif conditions in guardrail event listener

* fix: resolve mypy type errors in guardrail event handler
2025-07-02 16:19:22 -04:00
Irineu Brito
7f83947020 fix: correct code example language inconsistency in pt-BR docs (#3088)
* fix: correct code example language inconsistency in pt-BR docs

* fix: fix: fully standardize code example language and naming in pt-BR docs

* fix: fix: fully standardize code example language and naming in pt-BR docs fixed variables

* fix: fix: fully standardize code example language and naming in pt-BR docs fixed params

---------

Co-authored-by: Lucas Gomide <lucaslg200@gmail.com>
2025-07-02 12:18:32 -04:00
Lucas Gomide
ceb310bcde docs: add docs about Memory Events (#3104)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
2025-07-02 11:10:45 -04:00
Lucas Gomide
ae57e5723c feat: add console logging for memory system usage (#3103) 2025-07-02 11:00:26 -04:00
Lucas Gomide
ab39753a75 Introduce MemoryEvents to monitor their usage (#3098)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* feat: emit events about memory usage

* test: add tests about memory events usage

* fixed linter issues

* test: use scoped_handlers while listener Memory events
2025-07-01 22:50:39 -04:00
Tony Kipkemboi
640e1a7bc2 Add docs redirects and development tools (#3096)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
* Add Reo.dev tracking script to documentation

* Comprehensive docs improvements and development tools

- Add comprehensive .cursorrules with CrewAI and Flow development patterns
- Add redirect rules for old doc links without /en/ prefix
- Replace changelog pages with direct GitHub releases links
- Fix installation page directory tree rendering issue
- Fix broken Visual Studio Build Tools link formatting
- Remove obsolete changelog files to reduce maintenance overhead

These changes improve developer experience and ensure all old documentation links continue working.
2025-07-01 14:41:34 -04:00
Lorenze Jay
e544ff8ba3 refactor: streamline collection handling in RAGStorage (#3097)
Replaced the try-except block for collection retrieval with a single call to get_or_create_collection, simplifying the code and improving readability. Added logging to confirm whether the collection was found or created.
2025-07-01 10:14:39 -07:00