Commit Graph

68 Commits

Author SHA1 Message Date
Bowen Liang
0b0f2d30ab sort imports with isort rules by ruff linter (#1730)
* sort imports

* update

---------

Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com>
Co-authored-by: Eduardo Chiarotti <dudumelgaco@hotmail.com>
2024-12-11 10:46:53 -05:00
Frieda Huang
bc2a86d66a Fixed output_file not respecting system path (#1726)
Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com>
2024-12-09 10:05:54 -05:00
Brandon Hancock (bhancock_ai)
4069b621d5 Improve typed task outputs (#1651)
* V1 working

* clean up imports and prints

* more clean up and add tests

* fixing tests

* fix test

* fix linting

* Fix tests

* Fix linting

* add doc string as requested by eduardo
2024-11-26 09:41:14 -05:00
Gui Vieira
8f5f67de41 Fix threading 2024-11-21 15:33:20 -03:00
C0deZ
e66a135d5d refactor: Move BaseTool to main package and centralize tool description generation (#1514)
* move base_tool to main package and consolidate tool desscription generation

* update import path

* update tests

* update doc

* add base_tool test

* migrate agent delegation tools to use BaseTool

* update tests

* update import path for tool

* fix lint

* update param signature

* add from_langchain to BaseTool for backwards support of langchain tools

* fix the case where StructuredTool doesn't have func

---------

Co-authored-by: c0dez <li@vitablehealth.com>
Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com>
2024-11-01 12:30:48 -04:00
Brandon Hancock (bhancock_ai)
02718e291b fix task cloning error (#1416) 2024-10-10 12:00:28 -04:00
João Moura
e77442cf34 Removing LangChain and Rebuilding Executor (#1322)
* rebuilding executor

* removing langchain

* Making all tests good

* fixing types and adding ability for nor using system prompts

* improving types

* pleasing the types gods

* pleasing the types gods

* fixing parser, tools and executor

* making sure all tests pass

* final pass

* fixing type

* Updating Docs

* preparing to cut new version
2024-09-16 14:14:04 -03:00
Brandon Hancock (bhancock_ai)
26d9af8367 Brandon/cre 252 add agent to crewai test (#1308)
* Update config typecheck to accept agents

* Clean up prints

* Adding agents to crew evaluator output table

* Properly generating table now

* Update tests
2024-09-07 03:53:23 -03:00
Eduardo Chiarotti
f777c1c2e0 fix: All files pre commit (#1249) 2024-08-23 10:52:36 -03:00
Brandon Hancock (bhancock_ai)
8119edb495 Brandon/cre 211 fix agent and task config for yaml based projects (#1211)
* Fixed agents. Now need to fix tasks.

* Add type fixes and fix task decorator

* Clean up logs

* fix more type errors

* Revert back to required

* Undo changes.

* Remove default none for properties that cannot be none

* Clean up comments

* Implement all of Guis feedback
2024-08-20 09:31:02 -04:00
Brandon Hancock (bhancock_ai)
bf7372fefa Adding Autocomplete to OSS (#1198)
* Cleaned up model_config

* Fix pydantic issues

* 99% done with autocomplete

* fixed test issues

* Fix type checking issues
2024-08-16 15:04:21 -04:00
Vini Brasil
dbf2570353 Add name and expected_output to TaskOutput (#1199)
* Add name and expected_output to TaskOutput

This commit adds task information to the TaskOutput class. This is
useful to provide extra context to callbacks.

* Populate task name from function names

This commit populates task name from function names when using
annotations.
2024-08-15 22:24:41 +01:00
Eduardo Chiarotti
d0707fac91 feat: Add bandit ci pipeline (#1200)
* feat: Add bandit ci pipeline

* feat: add useforsecurty false for bandit pipeline

* feat: Add report only for High severity issues
2024-08-15 18:19:57 -03:00
Lorenze Jay
62f5b2fb2e Brandon/cre 130 pipeline project structure (#1066)
* WIP. Procedure appears to be working well. Working on mocking properly for tests

* All tests are passing now

* rshift working

* Add back in Gui's tool_usage fix

* WIP

* Going to start refactoring for pipeline_output

* Update terminology

* new pipeline flow with traces and usage metrics working. need to add more tests and make sure PipelineOutput behaves likew CrewOutput

* Fix pipelineoutput to look more like crewoutput and taskoutput

* Implemented additional tests for pipeline. One test is failing. Need team support

* Update docs for pipeline

* Update pipeline to properly process input and ouput dictionary

* Update Pipeline docs

* Add back in commentary at top of pipeline file

* Starting to work on router

* Drop router for now. will add in separately

* In the middle of fixing router. A ton of circular dependencies. Moving over to a new design.

* WIP.

* Fix circular dependencies and updated PipelineRouter

* Add in Eduardo feedback. Still need to add in more commentary describing the design decisions for pipeline

* Add developer notes to explain what is going on in pipelines.

* Add doc strings

* Fix missing rag datatype

* WIP. Converting usage metrics from a dict to an object

* Fix tests that were checking usage metrics

* Drop todo

* Fix 1 type error in pipeline

* Update pipeline to use UsageMetric

* Add missing doc string

* WIP.

* Change names

* Rename variables based on joaos feedback

* Fix critical circular dependency issues. Now needing to fix trace issue.

* Tests working now!

* Add more tests which showed underlying issue with traces

* Fix tests

* Remove overly complicated test

* Add router example to docs

* Clean up end of docs

* Clean up docs

* Working on creating Crew templates and pipeline templates

* WIP.

* WIP

* Fix poetry install from templates

* WIP

* Restructure

* changes for lorenze

* more todos

* WIP: create pipelines cli working

* wrapped up router

* ignore mypy src on templates

* ignored signature of copy

* fix all verbose

* rm print statements

* brought back correct folders

* fixes missing folders and then rm print statements

* fixed tests

* fixed broken test

* fixed type checker

* fixed type ignore

* ignore types for templates

* needed

* revert

* exclude only required

* rm type errors on templates

* rm excluding type checks for template files on github action

* fixed missing quotes

---------

Co-authored-by: Brandon Hancock <brandon@brandonhancock.io>
2024-08-09 14:13:29 -07:00
Eduardo Chiarotti
d824db82a3 feat: Add execution time to both task and testing feature (#1031)
* feat: Add execution time to both task and testing feature

* feat: Remove unused functions

* feat: change test_crew to evalaute_crew to avoid issues with testing libs

* feat: fix tests
2024-07-29 23:17:07 -03:00
Brandon Hancock (bhancock_ai)
fa4393d77e Add in missing triple quote and execution time to resume agent functionality. (#1025)
* Add in missing triple quote and execution time to resume agent functionality

* Fixing broken kwargs and other issues causing our tests to fail
2024-07-29 14:39:02 -03:00
Taleb
0a23e1dc13 Performed spell check across the rest of code base, and enahnced the yaml paraser code a little (#895)
* Performed spell check across the entire documentation

Thank you once again!

* Performed spell check across the most of code base
Folders been checked:
- agents
- cli
- memory
- project
- tasks
- telemetry
- tools
- translations

* Trying to add a max_token for the agents, so they limited by number of tokens.

* Performed spell check across the rest of code base, and enahnced the yaml paraser code a little

* Small change in the main agent doc

* Improve _save_file method to handle both dict and str inputs

- Add check for dict type input
- Use json.dump for dict serialization
- Convert non-dict inputs to string
- Remove type ignore comments

---------

Co-authored-by: João Moura <joaomdmoura@gmail.com>
2024-07-28 15:39:54 -03:00
Taleb
6b4710a8d1 Improve _save_file method to handle both dict and str inputs (#1011)
- Add check for dict type input
- Use json.dump for dict serialization
- Convert non-dict inputs to string
- Remove type ignore comments
2024-07-28 15:03:18 -03:00
Brandon Hancock (bhancock_ai)
da7d8256eb Json Task Output Truncation with Escape Characters (#1009)
* Fixed special character issue when converting json to models. Added numerous tests to ensure thigns work properly.

* Fix linting error and cleaned up tests

* Fix customer_converter_cls test failure

* Fixed tests. Thank you lorenze for pointing that out. added a few more to ensure converter creation works properly

* Address lorenze feedback

* Fix linting issues
2024-07-26 17:27:01 -04:00
Eduardo Chiarotti
f9288295e6 fix: agent missing fix (#966) 2024-07-19 13:15:33 -03:00
Eduardo Chiarotti
61a196394b feat: Add planning feature to crew (#919)
* feat: add planning feature to crew

* feat: add test to planning handler and change to execute_async method

* docs: add planning parameter to the Core documentation

* docs: add planning docs

* fix: fix type checking issue

* fix: test and logic
2024-07-18 13:15:08 -03:00
Brandon Hancock
a7a2dfd296 Fix logging 2024-07-17 10:10:34 -04:00
Gui Vieira
dd8a199e99 Introduce structure keys (#902)
* Introduce structure keys

* Add agent key to tasks

* Rebasing is hard

* Rename task output telemetry

* Feedback
2024-07-15 19:37:07 -03:00
Lorenze Jay
67b04b30bf Replay feat using db (#930)
* Cleaned up task execution to now have separate paths for async and sync execution. Updating all kickoff functions to return CrewOutput. WIP. Waiting for Joao feedback on async task execution with task_output

* Consistently storing async and sync output for context

* outline tests I need to create going forward

* Major rehaul of TaskOutput and CrewOutput. Updated all tests to work with new change. Need to add in a few final tricky async tests and add a few more to verify output types on TaskOutput and CrewOutput.

* Encountering issues with callback. Need to test on main. WIP

* working on tests. WIP

* WIP. Figuring out disconnect issue.

* Cleaned up logs now that I've isolated the issue to the LLM

* more wip.

* WIP. It looks like usage metrics has always been broken for async

* Update parent crew who is managing for_each loop

* Merge in main to bugfix/kickoff-for-each-usage-metrics

* Clean up code for review

* Add new tests

* Final cleanup. Ready for review.

* Moving copy functionality from Agent to BaseAgent

* Fix renaming issue

* Fix linting errors

* use BaseAgent instead of Agent where applicable

* Fixing missing function. Working on tests.

* WIP. Needing team to review change

* Fixing issues brought about by merge

* WIP: need to fix json encoder

* WIP need to fix encoder

* WIP

* WIP: replay working with async. need to add tests

* Implement major fixes from yesterdays group conversation. Now working on tests.

* The majority of tasks are working now. Need to fix converter class

* Fix final failing test

* Fix linting and type-checker issues

* Add more tests to fully test CrewOutput and TaskOutput changes

* Add in validation for async cannot depend on other async tasks.

* WIP: working replay feat fixing inputs, need tests

* WIP: core logic of seq and heir for executing tasks added into one

* Update validators and tests

* better logic for seq and hier

* replay working for both seq and hier just need tests

* fixed context

* added cli command + code cleanup TODO: need better refactoring

* refactoring for cleaner code

* added better tests

* removed todo comments and fixed some tests

* fix logging now all tests should pass

* cleaner code

* ensure replay is delcared when replaying specific tasks

* ensure hierarchical works

* better typing for stored_outputs and separated task_output_handler

* added better tests

* added replay feature to crew docs

* easier cli command name

* fixing changes

* using sqllite instead of .json file for logging previous task_outputs

* tools fix

* added to docs and fixed tests

* fixed .db

* fixed docs and removed unneeded comments

* separating ltm and replay db

* fixed printing colors

* added how to doc

---------

Co-authored-by: Brandon Hancock <brandon@brandonhancock.io>
2024-07-15 17:14:10 -03:00
Brandon Hancock (bhancock_ai)
7b53457ef3 Feature/kickoff consistent output (#847)
* Cleaned up task execution to now have separate paths for async and sync execution. Updating all kickoff functions to return CrewOutput. WIP. Waiting for Joao feedback on async task execution with task_output

* Consistently storing async and sync output for context

* outline tests I need to create going forward

* Major rehaul of TaskOutput and CrewOutput. Updated all tests to work with new change. Need to add in a few final tricky async tests and add a few more to verify output types on TaskOutput and CrewOutput.

* Encountering issues with callback. Need to test on main. WIP

* working on tests. WIP

* WIP. Figuring out disconnect issue.

* Cleaned up logs now that I've isolated the issue to the LLM

* more wip.

* WIP. It looks like usage metrics has always been broken for async

* Update parent crew who is managing for_each loop

* Merge in main to bugfix/kickoff-for-each-usage-metrics

* Clean up code for review

* Add new tests

* Final cleanup. Ready for review.

* Moving copy functionality from Agent to BaseAgent

* Fix renaming issue

* Fix linting errors

* use BaseAgent instead of Agent where applicable

* Fixing missing function. Working on tests.

* WIP. Needing team to review change

* Fixing issues brought about by merge

* WIP

* Implement major fixes from yesterdays group conversation. Now working on tests.

* The majority of tasks are working now. Need to fix converter class

* Fix final failing test

* Fix linting and type-checker issues

* Add more tests to fully test CrewOutput and TaskOutput changes

* Add in validation for async cannot depend on other async tasks.

* Update validators and tests
2024-07-11 00:35:02 -03:00
WellyngtonF
47f9d026dd passing cloned agents when copying context (#885) 2024-07-07 12:58:38 -03:00
Gui Vieira
b75b0b5552 Emit task created (#875)
* Emit task created

* Limit data to shared crews
2024-07-07 12:58:24 -03:00
Eelke van den Bos
7edacf6e24 Add converter_cls option to Task (#800)
* Add converter_cls option to Task

Fixes #799

* Update task_test.py

* Update task.py

* Update task.py

* Update task_test.py

* Update task.py

* Update task.py

* Update task.py

* Update task.py

---------

Co-authored-by: João Moura <joaomdmoura@gmail.com>
2024-07-06 02:01:39 -03:00
Eduardo Chiarotti
a41bd18599 Fix/async tasks (#877)
* fix: async tasks calls

* fix: some issue along with some type check errors

* fix: some issue along with some type check errors

* fix: async test
2024-07-06 01:30:07 -03:00
Eduardo Chiarotti
bb64c80964 fix: Fix tests (#873)
* fix: call asserts

* fix: test_increment_tool_errors

* fix: test_increment_delegations_for_sequential_process

* fix: test_increment_delegations_for_hierarchical_process

* fix: test_code_execution_flag_adds_code_tool_upon_kickoff

* fix: test_tool_usage_information_is_appended_to_agent

* fix: try to fix test_crew_full_output

* fix: try to fix test_crew_full_output

* fix: test remove vcr to test crew_test test

* fix: comment test to see if ci passes

* fix: comment test to see if ci passes

* fix: test changing prompt tokens to get error on CI

* fix: test changing prompt tokens to get error on CI

* fix: test changing prompt tokens to get error on CI

* fix: test changing prompt tokens to get error on CI

* fix: test new approach

* fix: comment funciont not working in CI

* fix: github python version

* fix: remove need of vcr

* fix: fix and add comments for all type checking errors
2024-07-05 09:06:56 -03:00
Brandon Hancock (bhancock_ai)
57fc079267 Bugfix/kickoff for each usage metrics (#844)
* WIP. Figuring out disconnect issue.

* Cleaned up logs now that I've isolated the issue to the LLM

* more wip.

* WIP. It looks like usage metrics has always been broken for async

* Update parent crew who is managing for_each loop

* Merge in main to bugfix/kickoff-for-each-usage-metrics

* Clean up code for review

* Add new tests

* Final cleanup. Ready for review.

* Moving copy functionality from Agent to BaseAgent

* Fix renaming issue

* Fix linting errors

* use BaseAgent instead of Agent where applicable
2024-07-03 15:30:53 -03:00
Gui Vieira
5b66e87621 Improve telemetry (#818)
* Improve telemetry

* Minor adjustments

* Try to fix typing error

* Try to fix typing error [2]
2024-06-28 20:05:47 -03:00
Lorenze Jay
10997dd175 Lorenzejay/byoa (#776)
* better spacing

* works with llama index

* works on langchain custom just need delegation to work

* cleanup for custom_agent class

* works with different argument expectations for agent_executor

* cleanup for hierarchial process, better agent_executor args handler and added to the crew agent doc page

* removed code examples for langchain + llama index, added to docs instead

* added key output if return is not a str for and added some tests

* added hinting for CustomAgent class

* removed pass as it was not needed

* closer just need to figuire ou agentTools

* running agents - llamaindex and langchain with base agent

* some cleanup on baseAgent

* minimum for agent to run for base class and ensure it works with hierarchical process

* cleanup for original agent to take on BaseAgent class

* Agent takes on langchainagent and cleanup across

* token handling working for usage_metrics to continue working

* installed llama-index, updated docs and added better name

* fixed some type errors

* base agent holds token_process

* heirarchail process uses proper tools and no longer relies on hasattr for token_processes

* removal of test_custom_agent_executions

* this fixes copying agents

* leveraging an executor class for trigger llamaindex agent

* llama index now has ask_human

* executor mixins added

* added output converter base class

* type listed

* cleanup for output conversions and tokenprocess eliminated redundancy

* properly handling tokens

* simplified token calc handling

* original agent with base agent builder structure setup

* better docs

* no more llama-index dep

* cleaner docs

* test fixes

* poetry reverts and better docs

* base_agent_tools set for third party agents

* updated task and test fix
2024-06-27 14:56:08 -03:00
Lorenze Jay
d3b6640b4a added usage_metrics to full output (#756)
* added extra parameter for kickoff to return token usage count after result

* added output_token_usage to class and in full_output

* logger duplicated

* added more types

* added usage_metrics to full output instead

* added more to the description on full_output

* possible mispacing
2024-06-12 14:18:52 -03:00
Brandon Hancock (bhancock_ai)
946c56494e Feature/kickoff for each sync (#680)
* Sync with deep copy working now

* async working!!

* Clean up code for review

* Fix naming

---------

Co-authored-by: João Moura <joaomdmoura@gmail.com>
2024-06-11 12:51:39 -03:00
Eduardo Chiarotti
24ed8a2549 feat: Add crew train cli (#624)
* fix: fix crewai-tools cli command

* feat: add crewai train CLI command

* feat: add the tests

* fix: fix typing hinting issue on code

* fix: test.yml

* fix: fix test

* fix: removed fix since it didnt changed the test
2024-05-23 18:46:45 -03:00
João Moura
a336381849 adding agent to task output 2024-05-16 05:12:32 -03:00
Saif Mahmud
23f10418d7 Fixes #603 (#604) 2024-05-13 02:34:52 -03:00
Eduardo Chiarotti
1ec4da6947 feat: add mypy as type checker, update code and add comment to reference (#591)
* fix: fix test actually running

* fix: fix test to not send request to openai

* fix: fix linting to remove cli files

* fix: exclude only files that breaks black

* fix: Fix all Ruff checkings on the code and Fix Test with repeated name

* fix: Change linter name on yml file

* feat: update pre-commit

* feat: remove need for isort on the code

* feat: add mypy as type checker, update code and add comment to reference

* feat: remove black linter

* feat: remove poetry to run the command

* feat: change logic to test mypy

* feat: update tests yml to try to fix the tests gh action

* feat: try to add just mypy to run on gh action

* feat: fix yml file

* feat: add comment to avoid issue on gh action

* feat: decouple pytest from the necessity of poetry install

* feat: change tests.yml to test different approach

* feat: change to poetry run

* fix: parameter field on yml file

* fix: update parameters to be on the pyproject

* fix: update pyproject to remove import untyped errors
2024-05-10 16:37:52 -03:00
Eduardo Chiarotti
aeba64feaf Feat: Add Ruff to improve linting/formatting (#588)
* fix: fix test actually running

* fix: fix test to not send request to openai

* fix: fix linting to remove cli files

* fix: exclude only files that breaks black

* fix: Fix all Ruff checkings on the code and Fix Test with repeated name

* fix: Change linter name on yml file

* feat: update pre-commit

* feat: remove need for isort on the code

* feat: remove black linter

* feat: update tests yml to try to fix the tests gh action
2024-05-10 11:53:53 -03:00
Eduardo Chiarotti
7eb4fcdaf4 fix: Add validation fix output_file issue when have '/' (#585)
* fix: Add validation fix output_file issue when have /

* fix: run black to format code

* fix: run black to format code
2024-05-09 08:11:00 -03:00
ftoppi
0a35868367 Update task.py: try to find json in task output using regex (#491)
* Update task.py: try to find json in task output using regex

Sometimes the model replies with a valid and additional text, let's try to extract and validate it first. It's cheaper than calling LLM for that.

* Update task.py

---------

Co-authored-by: João Moura <joaomdmoura@gmail.com>
2024-05-02 03:26:34 -03:00
Victor Carvalho Tavernari
dae0aedc99 Add conditional check for output file directory creation (#523)
This commit adds a conditional check to ensure that the output file directory exists before attempting to create it. This ensures that the code does not
fail in cases where the directory does not exist and needs to be created. The condition is added in the `_save_file` method of the `Task` class, ensuring
that the correct behavior is maintained for saving results to a file.
2024-05-02 03:13:51 -03:00
shivam singh
326f524e7c doc: Add documentation to Task model. (#363) 2024-04-05 19:49:36 -03:00
ftoppi
8ed4c66117 tasks.py: don't call Converter when model response is valid (#406)
* tasks.py: don't call Converter when model response is valid

Try to convert the task output to the expected Pydantic model before sending it to Converter, maybe the model got it right.
2024-04-04 10:11:46 -03:00
João Moura
5b59e450f7 Adding long term, short term, entity and contextual memory 2024-04-01 10:45:17 -03:00
GabeKoga
bcf701b287 feature: human input per task (#395)
* feature: human input per task

* Update executor.py

* Update executor.py

* Update executor.py

* Update executor.py

* Update executor.py

* feat: change human input for unit testing
added documentation and unit test

* Create test_agent_human_input.yaml

add yaml for test

---------

Co-authored-by: João Moura <joaomdmoura@gmail.com>
2024-04-01 10:04:56 -03:00
Gui Vieira
128ce91951 Fix input interpolation bug (#369) 2024-03-22 03:08:54 -03:00
João Moura
99be4f5a61 Overridding classes __repr__ 2024-03-05 10:12:49 -03:00
João Moura
ba28ab1680 adding support for agents and tasks to be defined of configs 2024-03-05 01:26:07 -03:00