Commit Graph

519 Commits

Author SHA1 Message Date
Greyson LaLonde
e29ca9ec28 feat: replace embedchain with native crewai adapter (#451)
- Remove embedchain adapter; add crewai rag adapter and update all search tools  
- Add loaders: pdf, youtube (video & channel), github, docs site, mysql, postgresql  
- Add configurable similarity threshold, limit params, and embedding_model support  
- Improve chromadb compatibility (sanitize metadata, convert columns, fix chunking)  
- Fix xml encoding, Python 3.10 issues, and youtube url spoofing  
- Update crewai dependency and instructions; refresh uv.lock  
- Update tests for new rag adapter and search params
2025-09-18 19:02:22 -04:00
Lucas Gomide
8d9cee45f2 feat: attempt to make embedchain optional (#450)
* fix: attempt to make embedchain optional

* fix: drop pydantic_settings dependency

* fix: ensure the package is importable without any extra dependency

After making embedchain option many packages were unstalled which caused errors in some tools due to failing import directives
2025-09-12 13:35:17 -04:00
Lucas Gomide
f9925887aa Add CrewAIPlatformTools (#449)
* chore: add deprecation warning in CrewaiEnterpriseTools

* feat: add CrewAI Platform Tool

* feat: drop support to oldest env-var token
2025-09-12 12:04:26 -04:00
Vini Brasil
6f2301c945 Clarify tool support for both local and remote URLs (#447)
This commit updates tool prompts to explicitly highlight that some tools
can accept both local file paths and remote URLs.

The improved prompts ensure LLMs understand they may pass remote
resources.
2025-09-10 09:38:14 -03:00
Tony Kipkemboi
cb8a1da730 feat(parallel): add ParallelSearchTool (Search API v1beta) (#445)
* docs: add BUILDING_TOOLS.md

* feat(parallel): add ParallelSearchTool (Search API v1beta), tests, README; register exports; regenerate tool.specs.json

* test(parallel): replace URL substring assertion with hostname allowlist (CodeQL)
2025-09-08 10:53:06 -04:00
Lucas Gomide
47b64d3507 fix: use explicit DeprecationWarning for legacy token detection (#440) 2025-09-02 16:28:22 -04:00
Lucas Gomide
33241ef363 refactor: fetch & execute enterprise tool actions from platform (#437)
* refactor: fetch enterprise tool actions from platform

* chore: logging legacy token detected
2025-09-02 15:41:00 -04:00
Vini Brasil
93b841fc86 Create tool for generating automations in Studio (#438)
* Create tool for generating automations in Studio

This commit creates a tool to use CrewAI Enterprise API to generate
crews using CrewAI Studio.

* Replace CREWAI_BASE_URL with CREWAI_PLUS_URL

* Add missing /crewai_plus in URL
2025-09-02 11:46:18 -03:00
Jinash Rouniyar
cb84d2ddfa feat: Add modular contextual AI tools with async functionality (#431)
* Add contextual AI tools with async support

* Fix package version issues and update README

* Rename contextual tools to contextualai and update contents

* Update tools init for contextualai tools

* feat: Resolved no module found error for nest_asyncio

* Updated nest_asyncio import

---------

Co-authored-by: QJ <qj@QJs-MacBook-Pro.local>
Co-authored-by: Qile-Jiang <qile.jiang@contextual.ai>
2025-08-28 08:56:53 -04:00
Mike Plachta
1f581fa9ac fix: add OCRTool to crewai_tools package exports (#433) 2025-08-27 14:11:57 -07:00
Mike Plachta
6562587cba feat: add InvokeCrewAIAutomationTool for external crew API integration (#430)
* feat: add InvokeCrewAIAutomationTool for external crew API integration

* feat: add InvokeCrewAIAutomationTool class for executing CrewAI tasks programmatically
2025-08-27 10:42:19 -07:00
Erika Shorten
992cd726c4 Weaviate hybrid search (#428)
* Update tool to use hybrid search

* Set default alpha to 0.75
2025-08-27 10:33:16 -04:00
Mike Plachta
403bb7e208 feat: add descriptive header to scraped website content output (#426)
* feat: add descriptive header to scraped website content output

* fix: correct typo in scraped website content header text
2025-08-26 08:10:24 -07:00
Lucas Gomide
dc039cfac8 Adds RAG feature (#406)
* feat: initialize rag

* refactor: using cosine distance metric for chromadb

* feat: use RecursiveCharacterTextSplitter as chunker strategy

* feat: support chucker and loader per data_type

* feat: adding JSON loader

* feat: adding CSVLoader

* feat: adding loader for DOCX files

* feat: add loader for MDX files

* feat: add loader for XML files

* feat: add loader for parser Webpage

* feat: support to load files from an entire directory

* feat: support to auto-load the loaders for additional DataType

* feat: add chuckers for some specific data type

- Each chunker uses separators specific to its content type

* feat: prevent document duplication and centralize content management

- Implement document deduplication logic in RAG
  * Check for existing documents by source reference
  * Compare doc IDs to detect content changes
  * Automatically replace outdated content while preventing duplicates

- Centralize common functionality for better maintainability
  * Create SourceContent class to handle URLs, files, and text uniformly
  * Extract shared utilities (compute_sha256) to misc.py
  * Standardize doc ID generation across all loaders

- Improve RAG system architecture
  * All loaders now inherit consistent patterns from centralized BaseLoader
  * Better separation of concerns with dedicated content management classes
  * Standardized LoaderResult structure across all loader implementations

* chore: split text loaders file

* test: adding missing tests about RAG loaders

* refactor: QOL

* fix: add missing uv syntax on DOCXLoader
2025-08-19 18:30:35 -04:00
Joao Moura
1ce016df8b Revert "working around OAI new update for now"
This reverts commit 23a16eb446.
2025-08-19 00:09:25 -07:00
Joao Moura
23a16eb446 working around OAI new update for now 2025-08-18 22:40:30 -07:00
Gabe Milani
16d613488b fix: use json_schema_extra instead of deprecated Field extra args (#417) 2025-08-14 17:52:56 -04:00
nicoferdi96
99e174e575 Stagehand tool improvements (#415)
* Stagehand tool improvements

This commit significantly improves the StagehandTool reliability and usability when working with CrewAI agents by addressing several critical
  issues:

  ## Key Improvements

  ### 1. Atomic Action Support
  - Added _extract_steps() method to break complex instructions into individual steps
  - Added _simplify_instruction() method for intelligent error recovery
  - Sequential execution of micro-actions with proper DOM settling between steps
  - Prevents token limit issues on complex pages by encouraging scoped actions

  ### 2. Enhanced Schema Design
  - Made instruction field optional to handle navigation-only commands
  - Added smart defaults for missing instructions based on command_type
  - Improved field descriptions to guide agents toward atomic actions with location context
  - Prevents "instruction Field required" validation errors

  ### 3. Intelligent API Key Management
  - Added _get_model_api_key() method with automatic detection based on model type
  - Support for OpenAI (GPT), Anthropic (Claude), and Google (Gemini) API keys
  - Removes need for manual model API key configuration

  ### 4. Robust Error Recovery
  - Step-by-step execution with individual error handling per atomic action
  - Automatic retry with simplified instructions when complex actions fail
  - Comprehensive error logging and reporting for debugging
  - Graceful degradation instead of complete failure

  ### 5. Token Management & Performance
  - Tool descriptions encourage atomic, scoped actions (e.g., "click search box in header")
  - Prevents "prompt too long" errors on complex pages like Wikipedia
  - Location-aware instruction patterns for better DOM targeting
  - Reduced observe-act cycles through better instruction decomposition

  ### 6. Enhanced Testing Support
  - Comprehensive async mock objects for testing mode
  - Proper async/sync compatibility for different execution contexts
  - Enhanced resource cleanup and session management

* Update stagehand_tool.py

removeing FixedStagehandTool in favour of StagehandTool

* removed comment

* Cleanup

Revoved unused class
Improved tool description
2025-08-13 08:57:11 -04:00
meirk-brd
41ce4981ac feat: Add Bright Data tools (#314)
* Initial commit of BrightData tools

* Renamed the BrightData test file names

* Refactored and improved the overall BrightData tools

* Add BrightData tools

* Add tools to init

* Added config class

* Fix test failures and add missing __init__.py files

- Remove problematic brightdata_dataset_tool_test.py that referenced non-existent classes
- Fix brightdata_serp_tool_test.py to expect string responses instead of dict
- Fix brightdata_webunlocker_tool_test.py to expect string responses instead of dict
- Add missing tests/tools/__init__.py for proper test imports

---------

Co-authored-by: Ranjan Dailata <ranjancse@gmail.com>
Co-authored-by: Tony Kipkemboi <iamtonykipkemboi@gmail.com>
2025-08-07 10:29:51 -04:00
Volodymyr Tkachuk
d00c9764fc feat: Adding SingleStoreSearchTool (#349)
* initial commit

* Add actual SinglesStore Search Tool implementation

* add the implementation

* update readme

* add tool's description

* add tests

* fix tests

* review comments

* remove schema from public exports

* fix test failure

* revert tools.specs.json

* added dependencies and env vars descriptions
2025-08-07 09:51:37 -04:00
Lucas Gomide
4daf18256d feat: allow custom client_timeout for MCPAdapter (#409) 2025-08-05 09:53:57 -04:00
Lucas Gomide
9220cfba28 fix: use proper ArxivPaperTool import (#408) 2025-08-05 09:46:11 -04:00
Lucas Gomide
707c8583f4 chore: make tavily tools exportable (#400) 2025-07-29 14:56:20 -04:00
Harikrishnan K
30df46445b Adding Arxiv Paper tool (#310)
* arxiv_paper_tool.py

* Updating as per the review

* Update __init__.py

* Update __init__.py

* Update arxiv_paper_tool.py

* added test cases

* Create README.md

* Create Examples.md

* Update Examples.md

* Updated logger

* Updated with package_dependencies,env_vars
2025-07-29 12:53:30 -04:00
Vini Brasil
b8bd3000c6 Make RagTool processs-safe (#399)
This commit adds a file lock to `RagTool`. As it uses Embedchain and
Chroma internally, this tool was not process-safe.
2025-07-28 17:09:53 -03:00
Steven Silvester
e6ec6cc332 Fix MongoDBVectorSearchTool serialization and schema (#389)
* Fix MongoDBVectorSearchTool serialization

* fix base class
2025-07-23 10:28:07 -07:00
Mike Plachta
104485d18b feat: add SerperScrapeWebsiteTool for extracting clean content from URLs (#392)
* feat: add SerperScrapeWebsiteTool for extracting clean content from URLs

* feat: add required SERPER_API_KEY env var validation to SerperScrapeWebsiteTool
2025-07-23 10:22:47 -07:00
Filip Michalsky
c3e87fc31f Fm/update stagehand (#387)
* Update tool specifications for

* Update stagehand dependency from stagehand-py to stagehand v0.4.1

* uv add

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-07-22 07:05:29 -07:00
Piyush Jain
2c38d1d448 Bedrock AgentCore browser and code interpreter toolkits (#385)
* Added browser and code tools

* Added dependencies, moved imports inside class

* Added instructions in README

* Updated imports

* Updated imports

* Updated dependencies

* Fix 'get_current_page' utilities for Browser tools

* Support browser session cleanup from synchronous code

* Update browser tool examples for new changes

* Manually override _run->_arun and set nested loop when in crew event loop

* Browser async example

* update examples with uv

* Fix toolkit fields for code interpreter

* Update code interpreter examples

* update uv.lock

* Move nest_asyncio import

---------

Co-authored-by: Michael Chin <mchin188@yahoo.com>
2025-07-18 14:33:39 -07:00
Thiago Moretto
9f6002a9dd Declaring and make some tool configurable using env vars (#376) 2025-07-14 09:48:36 -04:00
Lorenze Jay
78f5144bde Enhance EnterpriseActionTool with improved schema processing and erro… (#371)
* Enhance EnterpriseActionTool with improved schema processing and error handling

- Added methods for sanitizing names and processing schema types, including support for nested models and nullable types.
- Improved error handling during schema creation and processing, with warnings for failures.
- Updated parameter handling in the `_run` method to clean up `kwargs` before sending requests.
- Introduced a detailed description generation for nested schema structures to enhance tool documentation.

* Add tests for EnterpriseActionTool schema conversion and validation

- Introduced a new test class for validating complex nested schemas in EnterpriseActionTool.
- Added tests for schema conversion, optional fields, enum validation, and required nested fields.
- Implemented execution tests to ensure the tool can handle complex validated input correctly.
- Verified model naming conventions and added tests for simpler schemas with basic enum validation.
- Enhanced overall test coverage for the EnterpriseActionTool functionality.

* Update chromadb dependency version in pyproject.toml and uv.lock

- Changed chromadb version from >=0.4.22 to ==0.5.23 in both pyproject.toml and uv.lock to ensure compatibility and stability.

* Update test workflow configuration

- Changed EMBEDCHAIN_DB_URI to point to a temporary test database location.
- Added CHROMA_PERSIST_PATH for specifying the path to the Chroma test database.
- Cleaned up the test run command in the workflow file.

* reverted
2025-07-11 10:18:54 -07:00
Vini Brasil
eb09f2718f Use environment variable to initialize scrapegraph_py.Client (#362)
This commit fixes a bug where `SCRAPEGRAPH_API_KEY` were never used to
initialize `scrapegraph_py.Client`.
2025-07-09 15:03:16 -03:00
Steven Silvester
e0de166592 Add MongoDB Vector Search Tool (#319)
* INTPYTHON-580 Design and Implement MongoDBVectorSearchTool

* add implementation

* wip

* wip

* finish tests

* add todo

* refactor to wrap langchain-mongodb

* cleanup

* address review

* Fix usage of EnvVar class

* inline code

* lint

* lint

* fix usage of SearchIndexModel

* Refactor: Update EnvVar import path and remove unused tests.utils module

- Changed import of EnvVar from tests.utils to crewai.tools in multiple files.
- Updated README.md for MongoDB vector search tool with additional context.
- Modified subprocess command in vector_search.py for package installation.
- Cleaned up test_generate_tool_specs.py to improve mock patching syntax.
- Deleted unused tests/utils.py file.

* update the crewai dep and the lockfile

* chore: update package versions and dependencies in uv.lock

- Removed `auth0-python` package.
- Updated `crewai` version to 0.140.0 and adjusted its dependencies.
- Changed `json-repair` version to 0.25.2.
- Updated `litellm` version to 1.72.6.
- Modified dependency markers for several packages to improve compatibility with Python versions.

* refactor: improve MongoDB vector search tool with enhanced error handling and new dimensions field

- Added logging for error handling in the _run method and during client cleanup.
- Introduced a new 'dimensions' field in the MongoDBVectorSearchConfig for embedding vector size.
- Refactored the _run method to return JSON formatted results and handle exceptions gracefully.
- Cleaned up import statements and improved code readability.

* address review

* update tests

* debug

* fix test

* fix test

* fix test

* support azure openai

---------

Co-authored-by: lorenzejay <lorenzejaytech@gmail.com>
2025-07-09 08:44:23 -07:00
gautham
c45e92bd17 Add Couchbase as a tool (#264)
* - Added CouchbaseFTSVectorStore as a CrewAI tool.
- Wrote a README to setup the tool.
- Wrote test cases.
- Added Couchbase as an optional dependency in the project.

* Fixed naming in some places. Added docstrings. Added instructions on how to create a vector search index.

* Fixed pyproject.toml

* error handling and response format

- Removed unnecessary ImportError for missing 'couchbase' package.
- Changed response format from a concatenated string to a JSON array for search results.
- Updated error handling to return error messages instead of raising exceptions in certain cases.
- Adjusted tests to reflect changes in response format and error handling.

* Update dependencies in pyproject.toml and uv.lock

- Changed pydantic version from 2.6.1 to 2.10.6 in both pyproject.toml and uv.lock.
- Updated crewai-tools version from 0.42.2 to 0.42.3 in uv.lock.
- Adjusted pydantic-core version from 2.33.1 to 2.27.2 in uv.lock, reflecting the new pydantic version.

* Removed restrictive pydantic version and updated uv.lock

* synced lockfile

* regenerated lockfile

* updated lockfile

* regenerated lockfile

* Update tool specifications for

* Fix test cases

---------

Co-authored-by: AayushTyagi1 <tyagiaayush5@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-07-08 10:54:54 -04:00
Lorenze Jay
b4786d86b0 refactor: enhance schema handling in EnterpriseActionTool (#355)
* refactor: enhance schema handling in EnterpriseActionTool

- Extracted schema property and required field extraction into separate methods for better readability and maintainability.
- Introduced methods to analyze field types and create Pydantic field definitions based on nullability and requirement status.
- Updated the _run method to handle required nullable fields, ensuring they are set to None if not provided in kwargs.

* refactor: streamline nullable field handling in EnterpriseActionTool

- Removed commented-out code related to handling required nullable fields for clarity.
- Simplified the logic in the _run method to focus on processing parameters without unnecessary comments.
2025-07-02 12:54:09 -07:00
Emmanuel Ferdman
d53e96fcd7 fix: update Pydantic schema access (#337)
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2025-07-02 09:25:26 -07:00
Thiago Moretto
26652e5e24 Mapping required env vars of more tools (#353) 2025-07-01 14:08:30 -04:00
Lorenze Jay
180cc38330 refactor: update Firecrawl tools to improve configuration and error handling (#351)
- Added TYPE_CHECKING imports for FirecrawlApp to enhance type safety.
- Updated configuration keys in FirecrawlCrawlWebsiteTool and FirecrawlScrapeWebsiteTool to camelCase for consistency.
- Introduced error handling in the _run methods of both tools to ensure FirecrawlApp is properly initialized before usage.
- Adjusted parameters passed to crawl_url and scrape_url methods to use 'params' instead of unpacking the config dictionary directly.
2025-06-27 11:27:48 -07:00
Lorenze Jay
8723e66807 feat: add support for parsing actions list from environment variables (#346)
* feat: add support for parsing actions list from environment variables

This commit introduces a new function, _parse_actions_list, to handle the parsing of a string representation of a list of tool names from environment variables. The CrewaiEnterpriseTools now utilizes this function to filter tools based on the parsed actions list, enhancing flexibility in tool selection. Additionally, a new test case is added to verify the correct usage of the environment actions list.

* test: simplify environment actions list test setup

This commit refactors the test for CrewaiEnterpriseTools to streamline the setup of environment variables. The environment token and actions list are now set in a single patch.dict call, improving readability and reducing redundancy in the test code.
2025-06-25 11:14:41 -07:00
Lorenze Jay
e4cb8bf797 Lorenze/better env vars setup enterprise tools (#343)
* refactor: remove token validation from EnterpriseActionKitToolAdapter and CrewaiEnterpriseTools

This commit simplifies the initialization of the EnterpriseActionKitToolAdapter and CrewaiEnterpriseTools by removing the explicit validation for the enterprise action token. The token can now be set to None without raising an error, allowing for more flexible usage.

* added loggers for monitoring

* fixed typo

* fix: enhance token handling in EnterpriseActionKitToolAdapter and CrewaiEnterpriseTools

This commit improves the handling of the enterprise action token by allowing it to be fetched from environment variables if not provided. It adds checks to ensure the token is set before making API requests, enhancing robustness and flexibility.

* removed redundancy

* test: add new test for environment token fallback in CrewaiEnterpriseTools

This update introduces a new test case to verify that the environment token is used when no token is provided during the initialization of CrewaiEnterpriseTools. Additionally, minor formatting adjustments were made to existing assertions for consistency.

* test: update environment token test to clear environment variables

This change modifies the test for CrewaiEnterpriseTools to ensure that the environment variables are cleared before setting the test token. This ensures a clean test environment and prevents potential interference from other tests.

* drop redundancy
2025-06-25 10:17:26 -07:00
Lucas Gomide
03917411b4 Support to filter available MCP Tools (#345)
* feat: support to complex filter on ToolCollection

* refactor: use proper tool collection methot to filter tool in CrewAiEnterpriseTools

* feat: allow to filter available MCP tools
2025-06-25 12:32:22 -04:00
Lucas Gomide
e8825d071a feat: support api_key fallback to EXA_API_KEY env-var (#341) 2025-06-24 14:47:39 -04:00
Rostyslav Borovyk
78a062a907 Add Oxylabs Web Scraping tools (#312)
* Add Oxylabs tools

* Review updates

* Add package_dependencies attribute
2025-06-24 09:56:47 -04:00
Lucas Gomide
c13b08de2e fix: add support for case-insensitive Enterprise filter (#340) 2025-06-20 11:09:45 -04:00
Lucas Gomide
31b3dd2b94 feat: Add ToolCollection class for named tool access (#339)
This change allows accessing tools by name (tools["tool_name"]) in addition to
index (tools[0]), making it more intuitive and convenient to work with multiple
tools without needing to remember their position in the list
2025-06-20 09:27:10 -04:00
Lucas Gomide
9e92b84bcc feat: mapping explicitly tool environment variables (#338) 2025-06-20 08:06:11 -04:00
Lucas Gomide
2cca45b45a refactor: renaming init_params and run_params to reflect their schema. (#332) (#333)
We’re currently using the JSON Schema standard for these fields
2025-06-17 07:52:51 -04:00
Lucas Gomide
fac32d9503 Support to collect extra package dependencies of Tools (#330)
* feat: add explictly package_dependencies in the Tools

* feat: collect package_dependencies from Tool to add in tool.specs.json

* feat: add default value in run_params Tool' specs

* fix: support get boolean values

This commit also refactor test to make easier define newest attributes into a Tool
2025-06-16 10:09:19 -04:00
Lorenze Jay
5a99f07765 refactor: remove token validation from EnterpriseActionKitToolAdapter… (#331)
* refactor: remove token validation from EnterpriseActionKitToolAdapter and CrewaiEnterpriseTools

This commit simplifies the initialization of the EnterpriseActionKitToolAdapter and CrewaiEnterpriseTools by removing the explicit validation for the enterprise action token. The token can now be set to None without raising an error, allowing for more flexible usage.

* added loggers for monitoring

* fixed typo
2025-06-14 12:21:18 -07:00
Lucas Gomide
dc2d4af8ea fix: ensure the entire file will be read when the start_line is None (#325) 2025-06-11 09:02:56 -04:00