From 24f1a19310bd1044e6ac528ca554c2eb439c7fd1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Kopeck=C3=BD?= Date: Sun, 16 Mar 2025 17:29:57 +0100 Subject: [PATCH 01/29] feat: add docs for ApifyActorsTool (#2254) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * add docs for ApifyActorsTool * improve readme, add link to template * format * improve tool docs * improve readme * Update apifyactorstool.mdx (#1) * Update apifyactorstool.mdx * Update apifyactorstool.mdx * dans suggestions * custom apify icon * update descripton * Update apifyactorstool.mdx --------- Co-authored-by: Jan Čurn Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- docs/concepts/tools.mdx | 1 + docs/mint.json | 3 +- docs/tools/apifyactorstool.mdx | 99 ++++++++++++++++++++++++++++++++++ 3 files changed, 102 insertions(+), 1 deletion(-) create mode 100644 docs/tools/apifyactorstool.mdx diff --git a/docs/concepts/tools.mdx b/docs/concepts/tools.mdx index fa823d0b9..6910735ab 100644 --- a/docs/concepts/tools.mdx +++ b/docs/concepts/tools.mdx @@ -106,6 +106,7 @@ Here is a list of the available tools and their descriptions: | Tool | Description | | :------------------------------- | :--------------------------------------------------------------------------------------------- | +| **ApifyActorsTool** | A tool that integrates Apify Actors with your workflows for web scraping and automation tasks. | | **BrowserbaseLoadTool** | A tool for interacting with and extracting data from web browsers. | | **CodeDocsSearchTool** | A RAG tool optimized for searching through code documentation and related technical documents. | | **CodeInterpreterTool** | A tool for interpreting python code. | diff --git a/docs/mint.json b/docs/mint.json index 8e2e270f7..87cb26760 100644 --- a/docs/mint.json +++ b/docs/mint.json @@ -154,6 +154,7 @@ "group": "Tools", "pages": [ "tools/aimindtool", + "tools/apifyactorstool", "tools/bravesearchtool", "tools/browserbaseloadtool", "tools/codedocssearchtool", @@ -220,4 +221,4 @@ "linkedin": "https://www.linkedin.com/company/crewai-inc", "youtube": "https://youtube.com/@crewAIInc" } -} \ No newline at end of file +} diff --git a/docs/tools/apifyactorstool.mdx b/docs/tools/apifyactorstool.mdx new file mode 100644 index 000000000..0f0835b9f --- /dev/null +++ b/docs/tools/apifyactorstool.mdx @@ -0,0 +1,99 @@ +--- +title: Apify Actors +description: "`ApifyActorsTool` lets you call Apify Actors to provide your CrewAI workflows with web scraping, crawling, data extraction, and web automation capabilities." +# hack to use custom Apify icon +icon: "); -webkit-mask-image: url('https://upload.wikimedia.org/wikipedia/commons/a/ae/Apify.svg');/*" +--- + +# `ApifyActorsTool` + +Integrate [Apify Actors](https://apify.com/actors) into your CrewAI workflows. + +## Description + +The `ApifyActorsTool` connects [Apify Actors](https://apify.com/actors), cloud-based programs for web scraping and automation, to your CrewAI workflows. +Use any of the 4,000+ Actors on [Apify Store](https://apify.com/store) for use cases such as extracting data from social media, search engines, online maps, e-commerce sites, travel portals, or general websites. + +For details, see the [Apify CrewAI integration](https://docs.apify.com/platform/integrations/crewai) in Apify documentation. + +## Steps to get started + + + + Install `crewai[tools]` and `langchain-apify` using pip: `pip install 'crewai[tools]' langchain-apify`. + + + Sign up to [Apify Console](https://console.apify.com/) and get your [Apify API token](https://console.apify.com/settings/integrations).. + + + Set your Apify API token as the `APIFY_API_TOKEN` environment variable to enable the tool's functionality. + + + +## Usage example + +Use the `ApifyActorsTool` manually to run the [RAG Web Browser Actor](https://apify.com/apify/rag-web-browser) to perform a web search: + +```python +from crewai_tools import ApifyActorsTool + +# Initialize the tool with an Apify Actor +tool = ApifyActorsTool(actor_name="apify/rag-web-browser") + +# Run the tool with input parameters +results = tool.run(run_input={"query": "What is CrewAI?", "maxResults": 5}) + +# Process the results +for result in results: + print(f"URL: {result['metadata']['url']}") + print(f"Content: {result.get('markdown', 'N/A')[:100]}...") +``` + +### Expected output + +Here is the output from running the code above: + +```text +URL: https://www.example.com/crewai-intro +Content: CrewAI is a framework for building AI-powered workflows... +URL: https://docs.crewai.com/ +Content: Official documentation for CrewAI... +``` + +The `ApifyActorsTool` automatically fetches the Actor definition and input schema from Apify using the provided `actor_name` and then constructs the tool description and argument schema. This means you need to specify only a valid `actor_name`, and the tool handles the rest when used with agents—no need to specify the `run_input`. Here's how it works: + +```python +from crewai import Agent +from crewai_tools import ApifyActorsTool + +rag_browser = ApifyActorsTool(actor_name="apify/rag-web-browser") + +agent = Agent( + role="Research Analyst", + goal="Find and summarize information about specific topics", + backstory="You are an experienced researcher with attention to detail", + tools=[rag_browser], +) +``` + +You can run other Actors from [Apify Store](https://apify.com/store) simply by changing the `actor_name` and, when using it manually, adjusting the `run_input` based on the Actor input schema. + +For an example of usage with agents, see the [CrewAI Actor template](https://apify.com/templates/python-crewai). + +## Configuration + +The `ApifyActorsTool` requires these inputs to work: + +- **`actor_name`** + The ID of the Apify Actor to run, e.g., `"apify/rag-web-browser"`. Browse all Actors on [Apify Store](https://apify.com/store). +- **`run_input`** + A dictionary of input parameters for the Actor when running the tool manually. + - For example, for the `apify/rag-web-browser` Actor: `{"query": "search term", "maxResults": 5}` + - See the Actor's [input schema](https://apify.com/apify/rag-web-browser/input-schema) for the list of input parameters. + +## Resources + +- **[Apify](https://apify.com/)**: Explore the Apify platform. +- **[How to build an AI agent on Apify](https://blog.apify.com/how-to-build-an-ai-agent/)** - A complete step-by-step guide to creating, publishing, and monetizing AI agents on the Apify platform. +- **[RAG Web Browser Actor](https://apify.com/apify/rag-web-browser)**: A popular Actor for web search for LLMs. +- **[CrewAI Integration Guide](https://docs.apify.com/platform/integrations/crewai)**: Follow the official guide for integrating Apify and CrewAI. From e723e5ca3fb7e4cb890c4befda47746aedbd7408 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jo=C3=A3o=20Moura?= Date: Mon, 17 Mar 2025 09:13:21 -0700 Subject: [PATCH 02/29] preparign new version --- pyproject.toml | 2 +- src/crewai/__init__.py | 2 +- src/crewai/cli/templates/crew/pyproject.toml | 2 +- src/crewai/cli/templates/flow/pyproject.toml | 2 +- src/crewai/cli/templates/tool/pyproject.toml | 2 +- uv.lock | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index ba6bdcccc..2e319e8d0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "crewai" -version = "0.105.0" +version = "0.108.0" description = "Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks." readme = "README.md" requires-python = ">=3.10,<3.13" diff --git a/src/crewai/__init__.py b/src/crewai/__init__.py index a0c38915c..4a992ff88 100644 --- a/src/crewai/__init__.py +++ b/src/crewai/__init__.py @@ -14,7 +14,7 @@ warnings.filterwarnings( category=UserWarning, module="pydantic.main", ) -__version__ = "0.105.0" +__version__ = "0.108.0" __all__ = [ "Agent", "Crew", diff --git a/src/crewai/cli/templates/crew/pyproject.toml b/src/crewai/cli/templates/crew/pyproject.toml index 6108d4c59..54a6e82f9 100644 --- a/src/crewai/cli/templates/crew/pyproject.toml +++ b/src/crewai/cli/templates/crew/pyproject.toml @@ -5,7 +5,7 @@ description = "{{name}} using crewAI" authors = [{ name = "Your Name", email = "you@example.com" }] requires-python = ">=3.10,<3.13" dependencies = [ - "crewai[tools]>=0.105.0,<1.0.0" + "crewai[tools]>=0.108.0,<1.0.0" ] [project.scripts] diff --git a/src/crewai/cli/templates/flow/pyproject.toml b/src/crewai/cli/templates/flow/pyproject.toml index 2991ba265..0a3d0de03 100644 --- a/src/crewai/cli/templates/flow/pyproject.toml +++ b/src/crewai/cli/templates/flow/pyproject.toml @@ -5,7 +5,7 @@ description = "{{name}} using crewAI" authors = [{ name = "Your Name", email = "you@example.com" }] requires-python = ">=3.10,<3.13" dependencies = [ - "crewai[tools]>=0.105.0,<1.0.0", + "crewai[tools]>=0.108.0,<1.0.0", ] [project.scripts] diff --git a/src/crewai/cli/templates/tool/pyproject.toml b/src/crewai/cli/templates/tool/pyproject.toml index 8733f50d1..e96ef65df 100644 --- a/src/crewai/cli/templates/tool/pyproject.toml +++ b/src/crewai/cli/templates/tool/pyproject.toml @@ -5,7 +5,7 @@ description = "Power up your crews with {{folder_name}}" readme = "README.md" requires-python = ">=3.10,<3.13" dependencies = [ - "crewai[tools]>=0.105.0" + "crewai[tools]>=0.108.0" ] [tool.crewai] diff --git a/uv.lock b/uv.lock index 7a0140f1d..8dfc754c2 100644 --- a/uv.lock +++ b/uv.lock @@ -619,7 +619,7 @@ wheels = [ [[package]] name = "crewai" -version = "0.105.0" +version = "0.108.0" source = { editable = "." } dependencies = [ { name = "appdirs" }, From 33cebea15b51302448dede7edc2589d3cccfd8d9 Mon Sep 17 00:00:00 2001 From: "Brandon Hancock (bhancock_ai)" <109994880+bhancockio@users.noreply.github.com> Date: Mon, 17 Mar 2025 16:31:23 -0400 Subject: [PATCH 03/29] spelling and tab fix (#2394) --- docs/concepts/{event-listner.mdx => event-listener.mdx} | 0 docs/mint.json | 1 + 2 files changed, 1 insertion(+) rename docs/concepts/{event-listner.mdx => event-listener.mdx} (100%) diff --git a/docs/concepts/event-listner.mdx b/docs/concepts/event-listener.mdx similarity index 100% rename from docs/concepts/event-listner.mdx rename to docs/concepts/event-listener.mdx diff --git a/docs/mint.json b/docs/mint.json index 87cb26760..f39557110 100644 --- a/docs/mint.json +++ b/docs/mint.json @@ -115,6 +115,7 @@ "concepts/testing", "concepts/cli", "concepts/tools", + "concepts/event-listener", "concepts/langchain-tools", "concepts/llamaindex-tools" ] From fe0813e831bf930146b7ad12356eaecfcf600b49 Mon Sep 17 00:00:00 2001 From: Vini Brasil Date: Tue, 18 Mar 2025 13:52:23 -0300 Subject: [PATCH 04/29] Improve `MethodExecutionFailedEvent.error` typing (#2401) --- src/crewai/utilities/events/flow_events.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/crewai/utilities/events/flow_events.py b/src/crewai/utilities/events/flow_events.py index 435d64214..8800b301b 100644 --- a/src/crewai/utilities/events/flow_events.py +++ b/src/crewai/utilities/events/flow_events.py @@ -1,6 +1,6 @@ from typing import Any, Dict, Optional, Union -from pydantic import BaseModel +from pydantic import BaseModel, ConfigDict from .base_events import CrewEvent @@ -52,9 +52,11 @@ class MethodExecutionFailedEvent(FlowEvent): flow_name: str method_name: str - error: Any + error: Exception type: str = "method_execution_failed" + model_config = ConfigDict(arbitrary_types_allowed=True) + class FlowFinishedEvent(FlowEvent): """Event emitted when a flow completes execution""" From 520933b4c51f2ae3c15d2a8c30415b3e82ebb3d0 Mon Sep 17 00:00:00 2001 From: elda27 Date: Thu, 20 Mar 2025 22:28:31 +0900 Subject: [PATCH 05/29] Fix: More comfortable validation #2177 (#2178) * Fix: More confortable validation * Fix: union type support --------- Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- src/crewai/task.py | 22 ++++++++++++-- tests/task_test.py | 71 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 90 insertions(+), 3 deletions(-) diff --git a/src/crewai/task.py b/src/crewai/task.py index be400e99a..0c063e4f9 100644 --- a/src/crewai/task.py +++ b/src/crewai/task.py @@ -19,6 +19,8 @@ from typing import ( Tuple, Type, Union, + get_args, + get_origin, ) from pydantic import ( @@ -178,15 +180,29 @@ class Task(BaseModel): """ if v is not None: sig = inspect.signature(v) - if len(sig.parameters) != 1: + positional_args = [ + param + for param in sig.parameters.values() + if param.default is inspect.Parameter.empty + ] + if len(positional_args) != 1: raise ValueError("Guardrail function must accept exactly one parameter") # Check return annotation if present, but don't require it return_annotation = sig.return_annotation if return_annotation != inspect.Signature.empty: + + return_annotation_args = get_args(return_annotation) if not ( - return_annotation == Tuple[bool, Any] - or str(return_annotation) == "Tuple[bool, Any]" + get_origin(return_annotation) is tuple + and len(return_annotation_args) == 2 + and return_annotation_args[0] is bool + and ( + return_annotation_args[1] is Any + or return_annotation_args[1] is str + or return_annotation_args[1] is TaskOutput + or return_annotation_args[1] == Union[str, TaskOutput] + ) ): raise ValueError( "If return type is annotated, it must be Tuple[bool, Any]" diff --git a/tests/task_test.py b/tests/task_test.py index 3cd11cfc7..ac25a14f8 100644 --- a/tests/task_test.py +++ b/tests/task_test.py @@ -3,6 +3,8 @@ import hashlib import json import os +from functools import partial +from typing import Tuple, Union from unittest.mock import MagicMock, patch import pytest @@ -215,6 +217,75 @@ def test_multiple_output_type_error(): ) +def test_guardrail_type_error(): + desc = "Give me a list of 5 interesting ideas to explore for na article, what makes them unique and interesting." + expected_output = "Bullet point list of 5 interesting ideas." + # Lambda function + Task( + description=desc, + expected_output=expected_output, + guardrail=lambda x: (True, x), + ) + + # Function + def guardrail_fn(x: TaskOutput) -> tuple[bool, TaskOutput]: + return (True, x) + + Task( + description=desc, + expected_output=expected_output, + guardrail=guardrail_fn, + ) + + class Object: + def guardrail_fn(self, x: TaskOutput) -> tuple[bool, TaskOutput]: + return (True, x) + + @classmethod + def guardrail_class_fn(cls, x: TaskOutput) -> tuple[bool, str]: + return (True, x) + + @staticmethod + def guardrail_static_fn(x: TaskOutput) -> tuple[bool, Union[str, TaskOutput]]: + return (True, x) + + obj = Object() + # Method + Task( + description=desc, + expected_output=expected_output, + guardrail=obj.guardrail_fn, + ) + # Class method + Task( + description=desc, + expected_output=expected_output, + guardrail=Object.guardrail_class_fn, + ) + # Static method + Task( + description=desc, + expected_output=expected_output, + guardrail=Object.guardrail_static_fn, + ) + + def error_fn(x: TaskOutput, y: bool) -> Tuple[bool, TaskOutput]: + return (y, x) + + Task( + description=desc, + expected_output=expected_output, + guardrail=partial(error_fn, y=True), + ) + + with pytest.raises(ValidationError): + Task( + description=desc, + expected_output=expected_output, + guardrail=error_fn, + ) + + @pytest.mark.vcr(filter_headers=["authorization"]) def test_output_pydantic_sequential(): class ScoreOutput(BaseModel): From 90b793779699d0a7dac4fcde43663d41618d43c4 Mon Sep 17 00:00:00 2001 From: Fernando Galves <157684778+cardofe@users.noreply.github.com> Date: Thu, 20 Mar 2025 14:42:23 +0100 Subject: [PATCH 06/29] Update documentation (#2199) * Update llms.mdx Update Amazon Bedrock section with more information about the foundation models available. * Update llms.mdx fix the description of Amazon Bedrock section * Update llms.mdx Remove the incorrect tag * Update llms.mdx Add Claude 3.7 Sonnet to the Amazon Bedrock list --------- Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- docs/concepts/llms.mdx | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/docs/concepts/llms.mdx b/docs/concepts/llms.mdx index 8d815246f..10ee03683 100644 --- a/docs/concepts/llms.mdx +++ b/docs/concepts/llms.mdx @@ -250,6 +250,40 @@ In this section, you'll find detailed examples that help you select, configure, model="bedrock/anthropic.claude-3-sonnet-20240229-v1:0" ) ``` + + Before using Amazon Bedrock, make sure you have boto3 installed in your environment + + [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html) is a managed service that provides access to multiple foundation models from top AI companies through a unified API, enabling secure and responsible AI application development. + + | Model | Context Window | Best For | + |-------------------------|----------------------|-------------------------------------------------------------------| + | Amazon Nova Pro | Up to 300k tokens | High-performance, model balancing accuracy, speed, and cost-effectiveness across diverse tasks. | + | Amazon Nova Micro | Up to 128k tokens | High-performance, cost-effective text-only model optimized for lowest latency responses. | + | Amazon Nova Lite | Up to 300k tokens | High-performance, affordable multimodal processing for images, video, and text with real-time capabilities. | + | Claude 3.7 Sonnet | Up to 128k tokens | High-performance, best for complex reasoning, coding & AI agents | + | Claude 3.5 Sonnet v2 | Up to 200k tokens | State-of-the-art model specialized in software engineering, agentic capabilities, and computer interaction at optimized cost. | + | Claude 3.5 Sonnet | Up to 200k tokens | High-performance model delivering superior intelligence and reasoning across diverse tasks with optimal speed-cost balance. | + | Claude 3.5 Haiku | Up to 200k tokens | Fast, compact multimodal model optimized for quick responses and seamless human-like interactions | + | Claude 3 Sonnet | Up to 200k tokens | Multimodal model balancing intelligence and speed for high-volume deployments. | + | Claude 3 Haiku | Up to 200k tokens | Compact, high-speed multimodal model optimized for quick responses and natural conversational interactions | + | Claude 3 Opus | Up to 200k tokens | Most advanced multimodal model excelling at complex tasks with human-like reasoning and superior contextual understanding. | + | Claude 2.1 | Up to 200k tokens | Enhanced version with expanded context window, improved reliability, and reduced hallucinations for long-form and RAG applications | + | Claude | Up to 100k tokens | Versatile model excelling in sophisticated dialogue, creative content, and precise instruction following. | + | Claude Instant | Up to 100k tokens | Fast, cost-effective model for everyday tasks like dialogue, analysis, summarization, and document Q&A | + | Llama 3.1 405B Instruct | Up to 128k tokens | Advanced LLM for synthetic data generation, distillation, and inference for chatbots, coding, and domain-specific tasks. | + | Llama 3.1 70B Instruct | Up to 128k tokens | Powers complex conversations with superior contextual understanding, reasoning and text generation. | + | Llama 3.1 8B Instruct | Up to 128k tokens | Advanced state-of-the-art model with language understanding, superior reasoning, and text generation. | + | Llama 3 70B Instruct | Up to 8k tokens | Powers complex conversations with superior contextual understanding, reasoning and text generation. | + | Llama 3 8B Instruct | Up to 8k tokens | Advanced state-of-the-art LLM with language understanding, superior reasoning, and text generation. | + | Titan Text G1 - Lite | Up to 4k tokens | Lightweight, cost-effective model optimized for English tasks and fine-tuning with focus on summarization and content generation. | + | Titan Text G1 - Express | Up to 8k tokens | Versatile model for general language tasks, chat, and RAG applications with support for English and 100+ languages. | + | Cohere Command | Up to 4k tokens | Model specialized in following user commands and delivering practical enterprise solutions. | + | Jurassic-2 Mid | Up to 8,191 tokens | Cost-effective model balancing quality and affordability for diverse language tasks like Q&A, summarization, and content generation. | + | Jurassic-2 Ultra | Up to 8,191 tokens | Model for advanced text generation and comprehension, excelling in complex tasks like analysis and content creation. | + | Jamba-Instruct | Up to 256k tokens | Model with extended context window optimized for cost-effective text generation, summarization, and Q&A. | + | Mistral 7B Instruct | Up to 32k tokens | This LLM follows instructions, completes requests, and generates creative text. | + | Mistral 8x7B Instruct | Up to 32k tokens | An MOE LLM that follows instructions, completes requests, and generates creative text. | + From 92980544362dbdc55eff19a6b9250a39b8b55054 Mon Sep 17 00:00:00 2001 From: Seyed Mostafa Meshkati Date: Thu, 20 Mar 2025 17:18:11 +0330 Subject: [PATCH 07/29] docs: add base_url env for anthropic llm example (#2204) Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- docs/concepts/llms.mdx | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/concepts/llms.mdx b/docs/concepts/llms.mdx index 10ee03683..f1d586bee 100644 --- a/docs/concepts/llms.mdx +++ b/docs/concepts/llms.mdx @@ -158,7 +158,11 @@ In this section, you'll find detailed examples that help you select, configure, ```toml Code + # Required ANTHROPIC_API_KEY=sk-ant-... + + # Optional + ANTHROPIC_API_BASE= ``` Example usage in your CrewAI project: From bbe896d48c3d749da02fe5a575d55ac0b375d1d9 Mon Sep 17 00:00:00 2001 From: Vini Brasil Date: Thu, 20 Mar 2025 10:59:17 -0300 Subject: [PATCH 08/29] Support wildcard handling in `emit()` (#2424) * Support wildcard handling in `emit()` Change `emit()` to call handlers registered for parent classes using `isinstance()`. Ensures that base event handlers receive derived events. * Fix failing test * Remove unused variable --- .../utilities/events/crewai_event_bus.py | 13 +++---- .../utilities/events/test_crewai_event_bus.py | 34 +++++++++++++++++++ 2 files changed, 39 insertions(+), 8 deletions(-) create mode 100644 tests/utilities/events/test_crewai_event_bus.py diff --git a/src/crewai/utilities/events/crewai_event_bus.py b/src/crewai/utilities/events/crewai_event_bus.py index c0cf50908..5df5ee689 100644 --- a/src/crewai/utilities/events/crewai_event_bus.py +++ b/src/crewai/utilities/events/crewai_event_bus.py @@ -67,15 +67,12 @@ class CrewAIEventsBus: source: The object emitting the event event: The event instance to emit """ - event_type = type(event) - if event_type in self._handlers: - for handler in self._handlers[event_type]: - handler(source, event) - self._signal.send(source, event=event) + for event_type, handlers in self._handlers.items(): + if isinstance(event, event_type): + for handler in handlers: + handler(source, event) - def clear_handlers(self) -> None: - """Clear all registered event handlers - useful for testing""" - self._handlers.clear() + self._signal.send(source, event=event) def register_handler( self, event_type: Type[EventTypes], handler: Callable[[Any, EventTypes], None] diff --git a/tests/utilities/events/test_crewai_event_bus.py b/tests/utilities/events/test_crewai_event_bus.py new file mode 100644 index 000000000..0dd8c8b34 --- /dev/null +++ b/tests/utilities/events/test_crewai_event_bus.py @@ -0,0 +1,34 @@ +from unittest.mock import Mock + +from crewai.utilities.events.base_events import CrewEvent +from crewai.utilities.events.crewai_event_bus import crewai_event_bus + + +class TestEvent(CrewEvent): + pass + + +def test_specific_event_handler(): + mock_handler = Mock() + + @crewai_event_bus.on(TestEvent) + def handler(source, event): + mock_handler(source, event) + + event = TestEvent(type="test_event") + crewai_event_bus.emit("source_object", event) + + mock_handler.assert_called_once_with("source_object", event) + + +def test_wildcard_event_handler(): + mock_handler = Mock() + + @crewai_event_bus.on(CrewEvent) + def handler(source, event): + mock_handler(source, event) + + event = TestEvent(type="test_event") + crewai_event_bus.emit("source_object", event) + + mock_handler.assert_called_once_with("source_object", event) From 6e94edb777aac75f4c677119009dd69a6d109aea Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jo=C3=A3o=20Moura?= Date: Thu, 20 Mar 2025 08:19:58 -0700 Subject: [PATCH 09/29] TYPO --- docs/guides/advanced/customizing-prompts.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/guides/advanced/customizing-prompts.mdx b/docs/guides/advanced/customizing-prompts.mdx index 2622cdcca..4458184fc 100644 --- a/docs/guides/advanced/customizing-prompts.mdx +++ b/docs/guides/advanced/customizing-prompts.mdx @@ -1,4 +1,5 @@ ----title: Customizing Prompts +--- +title: Customizing Prompts description: Dive deeper into low-level prompt customization for CrewAI, enabling super custom and complex use cases for different models and languages. icon: message-pen --- From f8f9df6d1d5e167a89f5d984d460a869e924591f Mon Sep 17 00:00:00 2001 From: Amine Saihi <80285747+amine759@users.noreply.github.com> Date: Thu, 20 Mar 2025 16:06:21 +0000 Subject: [PATCH 10/29] update doc SpaceNewsKnowledgeSource code snippet (#2275) Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- docs/concepts/knowledge.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/concepts/knowledge.mdx b/docs/concepts/knowledge.mdx index b5827551a..13a875409 100644 --- a/docs/concepts/knowledge.mdx +++ b/docs/concepts/knowledge.mdx @@ -460,12 +460,12 @@ class SpaceNewsKnowledgeSource(BaseKnowledgeSource): data = response.json() articles = data.get('results', []) - formatted_data = self._format_articles(articles) + formatted_data = self.validate_content(articles) return {self.api_endpoint: formatted_data} except Exception as e: raise ValueError(f"Failed to fetch space news: {str(e)}") - def _format_articles(self, articles: list) -> str: + def validate_content(self, articles: list) -> str: """Format articles into readable text.""" formatted = "Space News Articles:\n\n" for article in articles: From 9fc84fc1ac59f916b615192d743c470a7e383bac Mon Sep 17 00:00:00 2001 From: "devin-ai-integration[bot]" <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Thu, 20 Mar 2025 12:17:26 -0400 Subject: [PATCH 11/29] Fix incorrect import statement in memory examples documentation (fixes #2395) (#2396) Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Joe Moura Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- docs/concepts/memory.mdx | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/concepts/memory.mdx b/docs/concepts/memory.mdx index 298e8814c..bb4e885cd 100644 --- a/docs/concepts/memory.mdx +++ b/docs/concepts/memory.mdx @@ -60,7 +60,8 @@ my_crew = Crew( ```python Code from crewai import Crew, Process from crewai.memory import LongTermMemory, ShortTermMemory, EntityMemory -from crewai.memory.storage import LTMSQLiteStorage, RAGStorage +from crewai.memory.storage.rag_storage import RAGStorage +from crewai.memory.storage.ltm_sqlite_storage import LTMSQLiteStorage from typing import List, Optional # Assemble your crew with memory capabilities @@ -119,7 +120,7 @@ Example using environment variables: import os from crewai import Crew from crewai.memory import LongTermMemory -from crewai.memory.storage import LTMSQLiteStorage +from crewai.memory.storage.ltm_sqlite_storage import LTMSQLiteStorage # Configure storage path using environment variable storage_path = os.getenv("CREWAI_STORAGE_DIR", "./storage") @@ -148,7 +149,7 @@ crew = Crew(memory=True) # Uses default storage locations ```python from crewai import Crew from crewai.memory import LongTermMemory -from crewai.memory.storage import LTMSQLiteStorage +from crewai.memory.storage.ltm_sqlite_storage import LTMSQLiteStorage # Configure custom storage paths crew = Crew( From 66b19311a7fc55b1f71af09630a8e08eb1e066d8 Mon Sep 17 00:00:00 2001 From: Sir Qasim Date: Thu, 20 Mar 2025 21:48:02 +0500 Subject: [PATCH 12/29] Fix crewai run Command Issue for Flow Projects and Cloud Deployment (#2291) This PR addresses an issue with the crewai run command following the creation of a flow project. Previously, the update command interfered with execution, causing it not to work as expected. With these changes, the command now runs according to the instructions in the readme.md, and it also improves deployment support when using CrewAI Cloud. Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- src/crewai/cli/templates/flow/pyproject.toml | 1 + 1 file changed, 1 insertion(+) diff --git a/src/crewai/cli/templates/flow/pyproject.toml b/src/crewai/cli/templates/flow/pyproject.toml index 0a3d0de03..93e7c1de7 100644 --- a/src/crewai/cli/templates/flow/pyproject.toml +++ b/src/crewai/cli/templates/flow/pyproject.toml @@ -10,6 +10,7 @@ dependencies = [ [project.scripts] kickoff = "{{folder_name}}.main:kickoff" +run_crew = "{{folder_name}}.main:kickoff" plot = "{{folder_name}}.main:plot" [build-system] From 794574957ea01075e011062f81e1b309774954eb Mon Sep 17 00:00:00 2001 From: Sir Qasim Date: Thu, 20 Mar 2025 21:54:17 +0500 Subject: [PATCH 13/29] Add note to create ./knowldge folder for source file management (#2297) This update includes a note in the documentation instructing users to create a ./knowldge folder. All source files (such as .txt, .pdf, .xlsx, .json) should be placed in this folder for centralized management. This change aims to streamline file organization and improve accessibility across projects. Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- docs/concepts/knowledge.mdx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/concepts/knowledge.mdx b/docs/concepts/knowledge.mdx index 13a875409..ae74ee50a 100644 --- a/docs/concepts/knowledge.mdx +++ b/docs/concepts/knowledge.mdx @@ -150,6 +150,8 @@ result = crew.kickoff( Here are examples of how to use different types of knowledge sources: +Note: Please ensure that you create the ./knowldge folder. All source files (e.g., .txt, .pdf, .xlsx, .json) should be placed in this folder for centralized management. + ### Text File Knowledge Source ```python from crewai.knowledge.source.text_file_knowledge_source import TextFileKnowledgeSource From 2155acb3a36058c12b3a909fb9aef2be3dbf9877 Mon Sep 17 00:00:00 2001 From: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com> Date: Thu, 20 Mar 2025 10:11:37 -0700 Subject: [PATCH 14/29] docs: Update JSONSearchTool and RagTool configuration parameter from 'embedder' to 'embedding_model' (#2311) Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- docs/tools/jsonsearchtool.mdx | 10 ++++++---- docs/tools/ragtool.mdx | 8 ++++---- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/tools/jsonsearchtool.mdx b/docs/tools/jsonsearchtool.mdx index 38267ff73..d7f8b853e 100644 --- a/docs/tools/jsonsearchtool.mdx +++ b/docs/tools/jsonsearchtool.mdx @@ -7,8 +7,10 @@ icon: file-code # `JSONSearchTool` - The JSONSearchTool is currently in an experimental phase. This means the tool is under active development, and users might encounter unexpected behavior or changes. - We highly encourage feedback on any issues or suggestions for improvements. + The JSONSearchTool is currently in an experimental phase. This means the tool + is under active development, and users might encounter unexpected behavior or + changes. We highly encourage feedback on any issues or suggestions for + improvements. ## Description @@ -60,7 +62,7 @@ tool = JSONSearchTool( # stream=true, }, }, - "embedder": { + "embedding_model": { "provider": "google", # or openai, ollama, ... "config": { "model": "models/embedding-001", @@ -70,4 +72,4 @@ tool = JSONSearchTool( }, } ) -``` \ No newline at end of file +``` diff --git a/docs/tools/ragtool.mdx b/docs/tools/ragtool.mdx index 841a2a278..b03059152 100644 --- a/docs/tools/ragtool.mdx +++ b/docs/tools/ragtool.mdx @@ -8,8 +8,8 @@ icon: vector-square ## Description -The `RagTool` is designed to answer questions by leveraging the power of Retrieval-Augmented Generation (RAG) through EmbedChain. -It provides a dynamic knowledge base that can be queried to retrieve relevant information from various data sources. +The `RagTool` is designed to answer questions by leveraging the power of Retrieval-Augmented Generation (RAG) through EmbedChain. +It provides a dynamic knowledge base that can be queried to retrieve relevant information from various data sources. This tool is particularly useful for applications that require access to a vast array of information and need to provide contextually relevant answers. ## Example @@ -138,7 +138,7 @@ config = { "model": "gpt-4", } }, - "embedder": { + "embedding_model": { "provider": "openai", "config": { "model": "text-embedding-ada-002" @@ -151,4 +151,4 @@ rag_tool = RagTool(config=config, summarize=True) ## Conclusion -The `RagTool` provides a powerful way to create and query knowledge bases from various data sources. By leveraging Retrieval-Augmented Generation, it enables agents to access and retrieve relevant information efficiently, enhancing their ability to provide accurate and contextually appropriate responses. \ No newline at end of file +The `RagTool` provides a powerful way to create and query knowledge bases from various data sources. By leveraging Retrieval-Augmented Generation, it enables agents to access and retrieve relevant information efficiently, enhancing their ability to provide accurate and contextually appropriate responses. From df266bda011c14beb900a7745a9e71b487cfba59 Mon Sep 17 00:00:00 2001 From: Tony Kipkemboi Date: Thu, 20 Mar 2025 11:44:21 -0700 Subject: [PATCH 15/29] Update documentation: Add changelog, fix formatting issues, replace mint.json with docs.json (#2400) --- docs/changelog.mdx | 187 ++++++++++++++++++++++++++++++++++ docs/concepts/llms.mdx | 4 +- docs/docs.json | 223 ++++++++++++++++++++++++++++++++++++++++ docs/mint.json | 225 ----------------------------------------- 4 files changed, 412 insertions(+), 227 deletions(-) create mode 100644 docs/changelog.mdx create mode 100644 docs/docs.json delete mode 100644 docs/mint.json diff --git a/docs/changelog.mdx b/docs/changelog.mdx new file mode 100644 index 000000000..3f17934bc --- /dev/null +++ b/docs/changelog.mdx @@ -0,0 +1,187 @@ +--- +title: Changelog +description: View the latest updates and changes to CrewAI +icon: timeline +--- + + + **Features** + - Converted tabs to spaces in `crew.py` template + - Enhanced LLM Streaming Response Handling and Event System + - Included `model_name` + - Enhanced Event Listener with rich visualization and improved logging + - Added fingerprints + + **Bug Fixes** + - Fixed Mistral issues + - Fixed a bug in documentation + - Fixed type check error in fingerprint property + + **Documentation Updates** + - Improved tool documentation + - Updated installation guide for the `uv` tool package + - Added instructions for upgrading crewAI with the `uv` tool + - Added documentation for `ApifyActorsTool` + + + + **Core Improvements & Fixes** + - Fixed issues with missing template variables and user memory configuration + - Improved async flow support and addressed agent response formatting + - Enhanced memory reset functionality and fixed CLI memory commands + - Fixed type issues, tool calling properties, and telemetry decoupling + + **New Features & Enhancements** + - Added Flow state export and improved state utilities + - Enhanced agent knowledge setup with optional crew embedder + - Introduced event emitter for better observability and LLM call tracking + - Added support for Python 3.10 and ChatOllama from langchain_ollama + - Integrated context window size support for the o3-mini model + - Added support for multiple router calls + + **Documentation & Guides** + - Improved documentation layout and hierarchical structure + - Added QdrantVectorSearchTool guide and clarified event listener usage + - Fixed typos in prompts and updated Amazon Bedrock model listings + + + + **Core Improvements & Fixes** + - Enhanced LLM Support: Improved structured LLM output, parameter handling, and formatting for Anthropic models + - Crew & Agent Stability: Fixed issues with cloning agents/crews using knowledge sources, multiple task outputs in conditional tasks, and ignored Crew task callbacks + - Memory & Storage Fixes: Fixed short-term memory handling with Bedrock, ensured correct embedder initialization, and added a reset memories function in the crew class + - Training & Execution Reliability: Fixed broken training and interpolation issues with dict and list input types + + **New Features & Enhancements** + - Advanced Knowledge Management: Improved naming conventions and enhanced embedding configuration with custom embedder support + - Expanded Logging & Observability: Added JSON format support for logging and integrated MLflow tracing documentation + - Data Handling Improvements: Updated excel_knowledge_source.py to process multi-tab files + - General Performance & Codebase Clean-Up: Streamlined enterprise code alignment and resolved linting issues + - Adding new tool: `QdrantVectorSearchTool` + + **Documentation & Guides** + - Updated AI & Memory Docs: Improved Bedrock, Google AI, and long-term memory documentation + - Task & Workflow Clarity: Added "Human Input" row to Task Attributes, Langfuse guide, and FileWriterTool documentation + - Fixed Various Typos & Formatting Issues + + + + **Features** + - Add Composio docs + - Add SageMaker as a LLM provider + + **Fixes** + - Overall LLM connection issues + - Using safe accessors on training + - Add version check to crew_chat.py + + **Documentation** + - New docs for crewai chat + - Improve formatting and clarity in CLI and Composio Tool docs + + + + **Features** + - Conversation crew v1 + - Add unique ID to flow states + - Add @persist decorator with FlowPersistence interface + + **Integrations** + - Add SambaNova integration + - Add NVIDIA NIM provider in cli + - Introducing VoyageAI + + **Fixes** + - Fix API Key Behavior and Entity Handling in Mem0 Integration + - Fixed core invoke loop logic and relevant tests + - Make tool inputs actual objects and not strings + - Add important missing parts to creating tools + - Drop litellm version to prevent windows issue + - Before kickoff if inputs are none + - Fixed typos, nested pydantic model issue, and docling issues + + + + **New Features** + - Adding Multimodal Abilities to Crew + - Programatic Guardrails + - HITL multiple rounds + - Gemini 2.0 Support + - CrewAI Flows Improvements + - Add Workflow Permissions + - Add support for langfuse with litellm + - Portkey Integration with CrewAI + - Add interpolate_only method and improve error handling + - Docling Support + - Weviate Support + + **Fixes** + - output_file not respecting system path + - disk I/O error when resetting short-term memory + - CrewJSONEncoder now accepts enums + - Python max version + - Interpolation for output_file in Task + - Handle coworker role name case/whitespace properly + - Add tiktoken as explicit dependency and document Rust requirement + - Include agent knowledge in planning process + - Change storage initialization to None for KnowledgeStorage + - Fix optional storage checks + - include event emitter in flows + - Docstring, Error Handling, and Type Hints Improvements + - Suppressed userWarnings from litellm pydantic issues + + + + **Changes** + - Remove all references to pipeline and pipeline router + - Add Nvidia NIM as provider in Custom LLM + - Add knowledge demo + improve knowledge docs + - Add HITL multiple rounds of followup + - New docs about yaml crew with decorators + - Simplify template crew + + + + **Features** + - Added knowledge to agent level + - Feat/remove langchain + - Improve typed task outputs + - Log in to Tool Repository on crewai login + + **Fixes** + - Fixes issues with result as answer not properly exiting LLM loop + - Fix missing key name when running with ollama provider + - Fix spelling issue found + + **Documentation** + - Update readme for running mypy + - Add knowledge to mint.json + - Update Github actions + - Update Agents docs to include two approaches for creating an agent + - Improvements to LLM Configuration and Usage + + + + **New Features** + - New before_kickoff and after_kickoff crew callbacks + - Support to pre-seed agents with Knowledge + - Add support for retrieving user preferences and memories using Mem0 + + **Fixes** + - Fix Async Execution + - Upgrade chroma and adjust embedder function generator + - Update CLI Watson supported models + docs + - Reduce level for Bandit + - Fixing all tests + + **Documentation** + - Update Docs + + + + **Fixes** + - Fixing Tokens callback replacement bug + - Fixing Step callback issue + - Add cached prompt tokens info on usage metrics + - Fix crew_train_success test + \ No newline at end of file diff --git a/docs/concepts/llms.mdx b/docs/concepts/llms.mdx index f1d586bee..2712de77a 100644 --- a/docs/concepts/llms.mdx +++ b/docs/concepts/llms.mdx @@ -746,5 +746,5 @@ Learn how to get the most out of your LLM configuration: Use larger context models for extensive tasks - - ``` + + diff --git a/docs/docs.json b/docs/docs.json new file mode 100644 index 000000000..d2c7e55b0 --- /dev/null +++ b/docs/docs.json @@ -0,0 +1,223 @@ +{ + "$schema": "https://mintlify.com/docs.json", + "theme": "palm", + "name": "CrewAI", + "colors": { + "primary": "#EB6658", + "light": "#F3A78B", + "dark": "#C94C3C" + }, + "favicon": "favicon.svg", + "navigation": { + "tabs": [ + { + "tab": "Get Started", + "groups": [ + { + "group": "Get Started", + "pages": [ + "introduction", + "installation", + "quickstart", + "changelog" + ] + }, + { + "group": "Guides", + "pages": [ + { + "group": "Concepts", + "pages": [ + "guides/concepts/evaluating-use-cases" + ] + }, + { + "group": "Agents", + "pages": [ + "guides/agents/crafting-effective-agents" + ] + }, + { + "group": "Crews", + "pages": [ + "guides/crews/first-crew" + ] + }, + { + "group": "Flows", + "pages": [ + "guides/flows/first-flow", + "guides/flows/mastering-flow-state" + ] + }, + { + "group": "Advanced", + "pages": [ + "guides/advanced/customizing-prompts", + "guides/advanced/fingerprinting" + ] + } + ] + }, + { + "group": "Core Concepts", + "pages": [ + "concepts/agents", + "concepts/tasks", + "concepts/crews", + "concepts/flows", + "concepts/knowledge", + "concepts/llms", + "concepts/processes", + "concepts/collaboration", + "concepts/training", + "concepts/memory", + "concepts/planning", + "concepts/testing", + "concepts/cli", + "concepts/tools", + "concepts/event-listener", + "concepts/langchain-tools", + "concepts/llamaindex-tools" + ] + }, + { + "group": "How to Guides", + "pages": [ + "how-to/create-custom-tools", + "how-to/sequential-process", + "how-to/hierarchical-process", + "how-to/custom-manager-agent", + "how-to/llm-connections", + "how-to/customizing-agents", + "how-to/multimodal-agents", + "how-to/coding-agents", + "how-to/force-tool-output-as-result", + "how-to/human-input-on-execution", + "how-to/kickoff-async", + "how-to/kickoff-for-each", + "how-to/replay-tasks-from-latest-crew-kickoff", + "how-to/conditional-tasks", + "how-to/agentops-observability", + "how-to/langtrace-observability", + "how-to/mlflow-observability", + "how-to/openlit-observability", + "how-to/portkey-observability", + "how-to/langfuse-observability" + ] + }, + { + "group": "Tools", + "pages": [ + "tools/aimindtool", + "tools/apifyactorstool", + "tools/bravesearchtool", + "tools/browserbaseloadtool", + "tools/codedocssearchtool", + "tools/codeinterpretertool", + "tools/composiotool", + "tools/csvsearchtool", + "tools/dalletool", + "tools/directorysearchtool", + "tools/directoryreadtool", + "tools/docxsearchtool", + "tools/exasearchtool", + "tools/filereadtool", + "tools/filewritetool", + "tools/firecrawlcrawlwebsitetool", + "tools/firecrawlscrapewebsitetool", + "tools/firecrawlsearchtool", + "tools/githubsearchtool", + "tools/hyperbrowserloadtool", + "tools/linkupsearchtool", + "tools/llamaindextool", + "tools/serperdevtool", + "tools/s3readertool", + "tools/s3writertool", + "tools/scrapegraphscrapetool", + "tools/scrapeelementfromwebsitetool", + "tools/jsonsearchtool", + "tools/mdxsearchtool", + "tools/mysqltool", + "tools/multiontool", + "tools/nl2sqltool", + "tools/patronustools", + "tools/pdfsearchtool", + "tools/pgsearchtool", + "tools/qdrantvectorsearchtool", + "tools/ragtool", + "tools/scrapewebsitetool", + "tools/scrapflyscrapetool", + "tools/seleniumscrapingtool", + "tools/snowflakesearchtool", + "tools/spidertool", + "tools/txtsearchtool", + "tools/visiontool", + "tools/weaviatevectorsearchtool", + "tools/websitesearchtool", + "tools/xmlsearchtool", + "tools/youtubechannelsearchtool", + "tools/youtubevideosearchtool" + ] + }, + { + "group": "Telemetry", + "pages": [ + "telemetry" + ] + } + ] + }, + { + "tab": "Examples", + "groups": [ + { + "group": "Examples", + "pages": [ + "examples/example" + ] + } + ] + } + ], + "global": { + "anchors": [ + { + "anchor": "Community", + "href": "https://community.crewai.com", + "icon": "discourse" + } + ] + } + }, + "logo": { + "light": "crew_only_logo.png", + "dark": "crew_only_logo.png" + }, + "appearance": { + "default": "dark", + "strict": false + }, + "navbar": { + "primary": { + "type": "github", + "href": "https://github.com/crewAIInc/crewAI" + } + }, + "search": { + "prompt": "Search CrewAI docs" + }, + "seo": { + "indexing": "navigable" + }, + "footer": { + "socials": { + "website": "https://crewai.com", + "x": "https://x.com/crewAIInc", + "github": "https://github.com/crewAIInc/crewAI", + "linkedin": "https://www.linkedin.com/company/crewai-inc", + "youtube": "https://youtube.com/@crewAIInc", + "reddit": "https://www.reddit.com/r/crewAIInc/" + } + } +} \ No newline at end of file diff --git a/docs/mint.json b/docs/mint.json deleted file mode 100644 index f39557110..000000000 --- a/docs/mint.json +++ /dev/null @@ -1,225 +0,0 @@ -{ - "name": "CrewAI", - "theme": "venus", - "logo": { - "dark": "crew_only_logo.png", - "light": "crew_only_logo.png" - }, - "favicon": "favicon.svg", - "colors": { - "primary": "#EB6658", - "light": "#F3A78B", - "dark": "#C94C3C", - "anchors": { - "from": "#737373", - "to": "#EB6658" - } - }, - "seo": { - "indexHiddenPages": false - }, - "modeToggle": { - "default": "dark", - "isHidden": false - }, - "feedback": { - "suggestEdit": true, - "raiseIssue": true, - "thumbsRating": true - }, - "topbarCtaButton": { - "type": "github", - "url": "https://github.com/crewAIInc/crewAI" - }, - "primaryTab": { - "name": "Get Started" - }, - "tabs": [ - { - "name": "Examples", - "url": "examples" - } - ], - "anchors": [ - { - "name": "Community", - "icon": "discourse", - "url": "https://community.crewai.com" - }, - { - "name": "Changelog", - "icon": "timeline", - "url": "https://github.com/crewAIInc/crewAI/releases" - } - ], - "navigation": [ - { - "group": "Get Started", - "pages": [ - "introduction", - "installation", - "quickstart" - ] - }, - { - "group": "Guides", - "pages": [ - { - "group": "Concepts", - "pages": [ - "guides/concepts/evaluating-use-cases" - ] - }, - { - "group": "Agents", - "pages": [ - "guides/agents/crafting-effective-agents" - ] - }, - { - "group": "Crews", - "pages": [ - "guides/crews/first-crew" - ] - }, - { - "group": "Flows", - "pages": [ - "guides/flows/first-flow", - "guides/flows/mastering-flow-state" - ] - }, - { - "group": "Advanced", - "pages": [ - "guides/advanced/customizing-prompts", - "guides/advanced/fingerprinting" - ] - } - ] - }, - { - "group": "Core Concepts", - "pages": [ - "concepts/agents", - "concepts/tasks", - "concepts/crews", - "concepts/flows", - "concepts/knowledge", - "concepts/llms", - "concepts/processes", - "concepts/collaboration", - "concepts/training", - "concepts/memory", - "concepts/planning", - "concepts/testing", - "concepts/cli", - "concepts/tools", - "concepts/event-listener", - "concepts/langchain-tools", - "concepts/llamaindex-tools" - ] - }, - { - "group": "How to Guides", - "pages": [ - "how-to/create-custom-tools", - "how-to/sequential-process", - "how-to/hierarchical-process", - "how-to/custom-manager-agent", - "how-to/llm-connections", - "how-to/customizing-agents", - "how-to/multimodal-agents", - "how-to/coding-agents", - "how-to/force-tool-output-as-result", - "how-to/human-input-on-execution", - "how-to/kickoff-async", - "how-to/kickoff-for-each", - "how-to/replay-tasks-from-latest-crew-kickoff", - "how-to/conditional-tasks", - "how-to/agentops-observability", - "how-to/langtrace-observability", - "how-to/mlflow-observability", - "how-to/openlit-observability", - "how-to/portkey-observability", - "how-to/langfuse-observability" - ] - }, - { - "group": "Examples", - "pages": [ - "examples/example" - ] - }, - { - "group": "Tools", - "pages": [ - "tools/aimindtool", - "tools/apifyactorstool", - "tools/bravesearchtool", - "tools/browserbaseloadtool", - "tools/codedocssearchtool", - "tools/codeinterpretertool", - "tools/composiotool", - "tools/csvsearchtool", - "tools/dalletool", - "tools/directorysearchtool", - "tools/directoryreadtool", - "tools/docxsearchtool", - "tools/exasearchtool", - "tools/filereadtool", - "tools/filewritetool", - "tools/firecrawlcrawlwebsitetool", - "tools/firecrawlscrapewebsitetool", - "tools/firecrawlsearchtool", - "tools/githubsearchtool", - "tools/hyperbrowserloadtool", - "tools/linkupsearchtool", - "tools/llamaindextool", - "tools/serperdevtool", - "tools/s3readertool", - "tools/s3writertool", - "tools/scrapegraphscrapetool", - "tools/scrapeelementfromwebsitetool", - "tools/jsonsearchtool", - "tools/mdxsearchtool", - "tools/mysqltool", - "tools/multiontool", - "tools/nl2sqltool", - "tools/patronustools", - "tools/pdfsearchtool", - "tools/pgsearchtool", - "tools/qdrantvectorsearchtool", - "tools/ragtool", - "tools/scrapewebsitetool", - "tools/scrapflyscrapetool", - "tools/seleniumscrapingtool", - "tools/snowflakesearchtool", - "tools/spidertool", - "tools/txtsearchtool", - "tools/visiontool", - "tools/weaviatevectorsearchtool", - "tools/websitesearchtool", - "tools/xmlsearchtool", - "tools/youtubechannelsearchtool", - "tools/youtubevideosearchtool" - ] - }, - { - "group": "Telemetry", - "pages": [ - "telemetry" - ] - } - ], - "search": { - "prompt": "Search CrewAI docs" - }, - "footerSocials": { - "website": "https://crewai.com", - "x": "https://x.com/crewAIInc", - "github": "https://github.com/crewAIInc/crewAI", - "linkedin": "https://www.linkedin.com/company/crewai-inc", - "youtube": "https://youtube.com/@crewAIInc" - } -} From b2c8779f4c28da7497647e7d7b7bb97db2b9c96b Mon Sep 17 00:00:00 2001 From: Tony Kipkemboi Date: Thu, 20 Mar 2025 12:39:37 -0700 Subject: [PATCH 16/29] Add documentation for Local NVIDIA NIM with WSL2 --- docs/concepts/llms.mdx | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/docs/concepts/llms.mdx b/docs/concepts/llms.mdx index 2712de77a..e17098f6a 100644 --- a/docs/concepts/llms.mdx +++ b/docs/concepts/llms.mdx @@ -270,7 +270,7 @@ In this section, you'll find detailed examples that help you select, configure, | Claude 3.5 Haiku | Up to 200k tokens | Fast, compact multimodal model optimized for quick responses and seamless human-like interactions | | Claude 3 Sonnet | Up to 200k tokens | Multimodal model balancing intelligence and speed for high-volume deployments. | | Claude 3 Haiku | Up to 200k tokens | Compact, high-speed multimodal model optimized for quick responses and natural conversational interactions | - | Claude 3 Opus | Up to 200k tokens | Most advanced multimodal model excelling at complex tasks with human-like reasoning and superior contextual understanding. | + | Claude 3 Opus | Up to 200k tokens | Most advanced multimodal model exceling at complex tasks with human-like reasoning and superior contextual understanding. | | Claude 2.1 | Up to 200k tokens | Enhanced version with expanded context window, improved reliability, and reduced hallucinations for long-form and RAG applications | | Claude | Up to 100k tokens | Versatile model excelling in sophisticated dialogue, creative content, and precise instruction following. | | Claude Instant | Up to 100k tokens | Fast, cost-effective model for everyday tasks like dialogue, analysis, summarization, and document Q&A | @@ -406,6 +406,46 @@ In this section, you'll find detailed examples that help you select, configure, | baichuan-inc/baichuan2-13b-chat | 4,096 tokens | Support Chinese and English chat, coding, math, instruction following, solving quizzes | + + + NVIDIA NIM enables you to run powerful LLMs locally on your Windows machine using WSL2 (Windows Subsystem for Linux). + This approach allows you to leverage your NVIDIA GPU for private, secure, and cost-effective AI inference without relying on cloud services. + Perfect for development, testing, or production scenarios where data privacy or offline capabilities are required. + + Here is a step-by-step guide to setting up a local NVIDIA NIM model: + + 1. Follow installation instructions from [NVIDIA Website](https://docs.nvidia.com/nim/wsl2/latest/getting-started.html) + + 2. Install the local model. For Llama 3.1-8b follow [instructions](https://build.nvidia.com/meta/llama-3_1-8b-instruct/deploy) + + 3. Configure your crewai local models: + + ```python Code + from crewai.llm import LLM + + local_nvidia_nim_llm = LLM( + model="openai/meta/llama-3.1-8b-instruct", # it's an openai-api compatible model + base_url="http://localhost:8000/v1", + api_key="", # api_key is required, but you can use any text + ) + + # Then you can use it in your crew: + + @CrewBase + class MyCrew(): + # ... + + @agent + def researcher(self) -> Agent: + return Agent( + config=self.agents_config['researcher'], + llm=local_nvidia_nim_llm + ) + + # ... + ``` + + Set the following environment variables in your `.env` file: From b766af75f2b573a0a07cef54741849b91d9c5f4a Mon Sep 17 00:00:00 2001 From: Arthur Chien Date: Fri, 21 Mar 2025 03:44:44 +0800 Subject: [PATCH 17/29] fix the _extract_thought (#2398) * fix the _extract_thought the regex string should be same with prompt in en.json:129 ...\nThought: I now know the final answer\nFinal Answer: the... * fix Action match --------- Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- src/crewai/agents/parser.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/crewai/agents/parser.py b/src/crewai/agents/parser.py index 1bda4df5c..05c5bc003 100644 --- a/src/crewai/agents/parser.py +++ b/src/crewai/agents/parser.py @@ -124,9 +124,9 @@ class CrewAgentParser: ) def _extract_thought(self, text: str) -> str: - thought_index = text.find("\n\nAction") + thought_index = text.find("\nAction") if thought_index == -1: - thought_index = text.find("\n\nFinal Answer") + thought_index = text.find("\nFinal Answer") if thought_index == -1: return "" thought = text[:thought_index].strip() From 3aa48dcd588dcfa8681e7309213fbdffaa4456fe Mon Sep 17 00:00:00 2001 From: Gustavo Satheler Date: Fri, 21 Mar 2025 13:32:54 -0300 Subject: [PATCH 18/29] fix: move agent tools for a variable instead of use format (#2319) Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- src/crewai/utilities/planning_handler.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/crewai/utilities/planning_handler.py b/src/crewai/utilities/planning_handler.py index 6ce74f236..1bd14a0c8 100644 --- a/src/crewai/utilities/planning_handler.py +++ b/src/crewai/utilities/planning_handler.py @@ -96,6 +96,10 @@ class CrewPlanner: tasks_summary = [] for idx, task in enumerate(self.tasks): knowledge_list = self._get_agent_knowledge(task) + agent_tools = ( + f"[{', '.join(str(tool) for tool in task.agent.tools)}]" if task.agent and task.agent.tools else '"agent has no tools"', + f',\n "agent_knowledge": "[\\"{knowledge_list[0]}\\"]"' if knowledge_list and str(knowledge_list) != "None" else "" + ) task_summary = f""" Task Number {idx + 1} - {task.description} "task_description": {task.description} @@ -103,10 +107,7 @@ class CrewPlanner: "agent": {task.agent.role if task.agent else "None"} "agent_goal": {task.agent.goal if task.agent else "None"} "task_tools": {task.tools} - "agent_tools": %s%s""" % ( - f"[{', '.join(str(tool) for tool in task.agent.tools)}]" if task.agent and task.agent.tools else '"agent has no tools"', - f',\n "agent_knowledge": "[\\"{knowledge_list[0]}\\"]"' if knowledge_list and str(knowledge_list) != "None" else "" - ) + "agent_tools": {"".join(agent_tools)}""" tasks_summary.append(task_summary) return " ".join(tasks_summary) From 32da76a2ca644d8472d0778c1b522b83928b9bd1 Mon Sep 17 00:00:00 2001 From: Jorge Gonzalez Date: Fri, 21 Mar 2025 13:17:43 -0400 Subject: [PATCH 19/29] Use task in the note about how methods names need to match task names (#2355) The note is about the task but mentions the agent incorrectly. Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- docs/quickstart.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/quickstart.mdx b/docs/quickstart.mdx index df57f756f..1edccee0e 100644 --- a/docs/quickstart.mdx +++ b/docs/quickstart.mdx @@ -300,7 +300,7 @@ email_summarizer: ``` - Note how we use the same name for the agent in the `tasks.yaml` (`email_summarizer_task`) file as the method name in the `crew.py` (`email_summarizer_task`) file. + Note how we use the same name for the task in the `tasks.yaml` (`email_summarizer_task`) file as the method name in the `crew.py` (`email_summarizer_task`) file. ```yaml tasks.yaml From 80f1a88b6356371ca622b74f80adf0cf62108661 Mon Sep 17 00:00:00 2001 From: Patcher Date: Fri, 21 Mar 2025 22:56:50 +0530 Subject: [PATCH 20/29] Upgrade OTel SDK version to 1.30.0 (#2375) Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- pyproject.toml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index 2e319e8d0..6e895be32 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -17,9 +17,9 @@ dependencies = [ "pdfplumber>=0.11.4", "regex>=2024.9.11", # Telemetry and Monitoring - "opentelemetry-api>=1.22.0", - "opentelemetry-sdk>=1.22.0", - "opentelemetry-exporter-otlp-proto-http>=1.22.0", + "opentelemetry-api>=1.30.0", + "opentelemetry-sdk>=1.30.0", + "opentelemetry-exporter-otlp-proto-http>=1.30.0", # Data Handling "chromadb>=0.5.23", "openpyxl>=3.1.5", From 6b1cf78e410776ca2e3ac0a9a399c20cb9ed7753 Mon Sep 17 00:00:00 2001 From: Julio Peixoto <96303574+JulioPeixoto@users.noreply.github.com> Date: Fri, 21 Mar 2025 14:34:16 -0300 Subject: [PATCH 21/29] docs: add detailed docstrings to Telemetry class methods (#2377) Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- src/crewai/telemetry/telemetry.py | 80 +++++++++++++++++++++++++++++-- 1 file changed, 76 insertions(+), 4 deletions(-) diff --git a/src/crewai/telemetry/telemetry.py b/src/crewai/telemetry/telemetry.py index 984a4938d..559ca8d4f 100644 --- a/src/crewai/telemetry/telemetry.py +++ b/src/crewai/telemetry/telemetry.py @@ -281,8 +281,16 @@ class Telemetry: return self._safe_telemetry_operation(operation) def task_ended(self, span: Span, task: Task, crew: Crew): - """Records task execution in a crew.""" + """Records the completion of a task execution in a crew. + Args: + span (Span): The OpenTelemetry span tracking the task execution + task (Task): The task that was completed + crew (Crew): The crew context in which the task was executed + + Note: + If share_crew is enabled, this will also record the task output + """ def operation(): if crew.share_crew: self._add_attribute( @@ -297,8 +305,13 @@ class Telemetry: self._safe_telemetry_operation(operation) def tool_repeated_usage(self, llm: Any, tool_name: str, attempts: int): - """Records the repeated usage 'error' of a tool by an agent.""" + """Records when a tool is used repeatedly, which might indicate an issue. + Args: + llm (Any): The language model being used + tool_name (str): Name of the tool being repeatedly used + attempts (int): Number of attempts made with this tool + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Tool Repeated Usage") @@ -317,8 +330,13 @@ class Telemetry: self._safe_telemetry_operation(operation) def tool_usage(self, llm: Any, tool_name: str, attempts: int): - """Records the usage of a tool by an agent.""" + """Records the usage of a tool by an agent. + Args: + llm (Any): The language model being used + tool_name (str): Name of the tool being used + attempts (int): Number of attempts made with this tool + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Tool Usage") @@ -337,8 +355,11 @@ class Telemetry: self._safe_telemetry_operation(operation) def tool_usage_error(self, llm: Any): - """Records the usage of a tool by an agent.""" + """Records when a tool usage results in an error. + Args: + llm (Any): The language model being used when the error occurred + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Tool Usage Error") @@ -357,6 +378,14 @@ class Telemetry: def individual_test_result_span( self, crew: Crew, quality: float, exec_time: int, model_name: str ): + """Records individual test results for a crew execution. + + Args: + crew (Crew): The crew being tested + quality (float): Quality score of the execution + exec_time (int): Execution time in seconds + model_name (str): Name of the model used + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Crew Individual Test Result") @@ -383,6 +412,14 @@ class Telemetry: inputs: dict[str, Any] | None, model_name: str, ): + """Records the execution of a test suite for a crew. + + Args: + crew (Crew): The crew being tested + iterations (int): Number of test iterations + inputs (dict[str, Any] | None): Input parameters for the test + model_name (str): Name of the model used in testing + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Crew Test Execution") @@ -408,6 +445,7 @@ class Telemetry: self._safe_telemetry_operation(operation) def deploy_signup_error_span(self): + """Records when an error occurs during the deployment signup process.""" def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Deploy Signup Error") @@ -417,6 +455,11 @@ class Telemetry: self._safe_telemetry_operation(operation) def start_deployment_span(self, uuid: Optional[str] = None): + """Records the start of a deployment process. + + Args: + uuid (Optional[str]): Unique identifier for the deployment + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Start Deployment") @@ -428,6 +471,7 @@ class Telemetry: self._safe_telemetry_operation(operation) def create_crew_deployment_span(self): + """Records the creation of a new crew deployment.""" def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Create Crew Deployment") @@ -437,6 +481,12 @@ class Telemetry: self._safe_telemetry_operation(operation) def get_crew_logs_span(self, uuid: Optional[str], log_type: str = "deployment"): + """Records the retrieval of crew logs. + + Args: + uuid (Optional[str]): Unique identifier for the crew + log_type (str, optional): Type of logs being retrieved. Defaults to "deployment". + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Get Crew Logs") @@ -449,6 +499,11 @@ class Telemetry: self._safe_telemetry_operation(operation) def remove_crew_span(self, uuid: Optional[str] = None): + """Records the removal of a crew. + + Args: + uuid (Optional[str]): Unique identifier for the crew being removed + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Remove Crew") @@ -574,6 +629,11 @@ class Telemetry: self._safe_telemetry_operation(operation) def flow_creation_span(self, flow_name: str): + """Records the creation of a new flow. + + Args: + flow_name (str): Name of the flow being created + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Flow Creation") @@ -584,6 +644,12 @@ class Telemetry: self._safe_telemetry_operation(operation) def flow_plotting_span(self, flow_name: str, node_names: list[str]): + """Records flow visualization/plotting activity. + + Args: + flow_name (str): Name of the flow being plotted + node_names (list[str]): List of node names in the flow + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Flow Plotting") @@ -595,6 +661,12 @@ class Telemetry: self._safe_telemetry_operation(operation) def flow_execution_span(self, flow_name: str, node_names: list[str]): + """Records the execution of a flow. + + Args: + flow_name (str): Name of the flow being executed + node_names (list[str]): List of nodes being executed in the flow + """ def operation(): tracer = trace.get_tracer("crewai.telemetry") span = tracer.start_span("Flow Execution") From 4d7aacb5f243aed4426914982506518126772f01 Mon Sep 17 00:00:00 2001 From: Saurabh Misra Date: Fri, 21 Mar 2025 10:43:48 -0700 Subject: [PATCH 22/29] =?UTF-8?q?=E2=9A=A1=EF=B8=8F=20Speed=20up=20method?= =?UTF-8?q?=20`Repository.is=5Fgit=5Frepo`=20by=2072,270%=20(#2381)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Here is the optimized version of the `Repository` class. Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com> Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- src/crewai/cli/git.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/crewai/cli/git.py b/src/crewai/cli/git.py index 94c3648b0..58836e733 100644 --- a/src/crewai/cli/git.py +++ b/src/crewai/cli/git.py @@ -1,4 +1,5 @@ import subprocess +from functools import lru_cache class Repository: @@ -35,6 +36,7 @@ class Repository: encoding="utf-8", ).strip() + @lru_cache(maxsize=None) def is_git_repo(self) -> bool: """Check if the current directory is a git repository.""" try: From d3a09c3180e86fc1d73492660627f0a39e6d4b2c Mon Sep 17 00:00:00 2001 From: Saurabh Misra Date: Fri, 21 Mar 2025 10:51:14 -0700 Subject: [PATCH 23/29] =?UTF-8?q?=E2=9A=A1=EF=B8=8F=20Speed=20up=20method?= =?UTF-8?q?=20`CrewAgentParser.=5Fclean=5Faction`=20by=20427,565%=20(#2382?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Here is the optimized version of the program. Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com> Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- src/crewai/agents/parser.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/crewai/agents/parser.py b/src/crewai/agents/parser.py index 05c5bc003..88a869c16 100644 --- a/src/crewai/agents/parser.py +++ b/src/crewai/agents/parser.py @@ -136,7 +136,7 @@ class CrewAgentParser: def _clean_action(self, text: str) -> str: """Clean action string by removing non-essential formatting characters.""" - return re.sub(r"^\s*\*+\s*|\s*\*+\s*$", "", text).strip() + return text.strip().strip("*").strip() def _safe_repair_json(self, tool_input: str) -> str: UNABLE_TO_REPAIR_JSON_RESULTS = ['""', "{}"] From 53067f8b9290986f22e2ef94eb92e5df1c24fb66 Mon Sep 17 00:00:00 2001 From: Parth Patel <64201651+parthbs@users.noreply.github.com> Date: Fri, 21 Mar 2025 23:27:24 +0530 Subject: [PATCH 24/29] add Mem0 OSS support (#2429) Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- src/crewai/memory/storage/mem0_storage.py | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/src/crewai/memory/storage/mem0_storage.py b/src/crewai/memory/storage/mem0_storage.py index be889afff..0319c6a8a 100644 --- a/src/crewai/memory/storage/mem0_storage.py +++ b/src/crewai/memory/storage/mem0_storage.py @@ -1,7 +1,7 @@ import os from typing import Any, Dict, List -from mem0 import MemoryClient +from mem0 import Memory, MemoryClient from crewai.memory.storage.interface import Storage @@ -32,13 +32,16 @@ class Mem0Storage(Storage): mem0_org_id = config.get("org_id") mem0_project_id = config.get("project_id") - # Initialize MemoryClient with available parameters - if mem0_org_id and mem0_project_id: - self.memory = MemoryClient( - api_key=mem0_api_key, org_id=mem0_org_id, project_id=mem0_project_id - ) + # Initialize MemoryClient or Memory based on the presence of the mem0_api_key + if mem0_api_key: + if mem0_org_id and mem0_project_id: + self.memory = MemoryClient( + api_key=mem0_api_key, org_id=mem0_org_id, project_id=mem0_project_id + ) + else: + self.memory = MemoryClient(api_key=mem0_api_key) else: - self.memory = MemoryClient(api_key=mem0_api_key) + self.memory = Memory() # Fallback to Memory if no Mem0 API key is provided def _sanitize_role(self, role: str) -> str: """ From 4daa88fa59f22e77c5ba83b2c457c9ab0de9c9b0 Mon Sep 17 00:00:00 2001 From: Stefano Baccianella <4247706+mangiucugna@users.noreply.github.com> Date: Fri, 21 Mar 2025 19:25:19 +0100 Subject: [PATCH 25/29] As explained in https://github.com/mangiucugna/json_repair?tab=readme-ov-file#performance-considerations we can skip a wasteful json.loads() here and save quite some time (#2397) Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com> --- src/crewai/tools/tool_usage.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/crewai/tools/tool_usage.py b/src/crewai/tools/tool_usage.py index 25e4b126a..9c924027d 100644 --- a/src/crewai/tools/tool_usage.py +++ b/src/crewai/tools/tool_usage.py @@ -455,7 +455,7 @@ class ToolUsage: # Attempt 4: Repair JSON try: - repaired_input = repair_json(tool_input) + repaired_input = repair_json(tool_input, skip_json_loads=True) self._printer.print( content=f"Repaired JSON: {repaired_input}", color="blue" ) From 0a116202f04a8438404c2c3bcfaa4fde812f6917 Mon Sep 17 00:00:00 2001 From: Fernando Galves <157684778+cardofe@users.noreply.github.com> Date: Fri, 21 Mar 2025 19:48:25 +0100 Subject: [PATCH 26/29] Update the context window size for Amazon Bedrock FM- llm.py (#2304) Update the context window size for Amazon Bedrock Foundation Models. Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com> --- src/crewai/llm.py | 54 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/src/crewai/llm.py b/src/crewai/llm.py index fb8367dfe..68ddbacc7 100644 --- a/src/crewai/llm.py +++ b/src/crewai/llm.py @@ -114,6 +114,60 @@ LLM_CONTEXT_WINDOW_SIZES = { "Llama-3.2-11B-Vision-Instruct": 16384, "Meta-Llama-3.2-3B-Instruct": 4096, "Meta-Llama-3.2-1B-Instruct": 16384, + # bedrock + "us.amazon.nova-pro-v1:0": 300000, + "us.amazon.nova-micro-v1:0": 128000, + "us.amazon.nova-lite-v1:0": 300000, + "us.anthropic.claude-3-5-sonnet-20240620-v1:0": 200000, + "us.anthropic.claude-3-5-haiku-20241022-v1:0": 200000, + "us.anthropic.claude-3-5-sonnet-20241022-v2:0": 200000, + "us.anthropic.claude-3-7-sonnet-20250219-v1:0": 200000, + "us.anthropic.claude-3-sonnet-20240229-v1:0": 200000, + "us.anthropic.claude-3-opus-20240229-v1:0": 200000, + "us.anthropic.claude-3-haiku-20240307-v1:0": 200000, + "us.meta.llama3-2-11b-instruct-v1:0": 128000, + "us.meta.llama3-2-3b-instruct-v1:0": 131000, + "us.meta.llama3-2-90b-instruct-v1:0": 128000, + "us.meta.llama3-2-1b-instruct-v1:0": 131000, + "us.meta.llama3-1-8b-instruct-v1:0": 128000, + "us.meta.llama3-1-70b-instruct-v1:0": 128000, + "us.meta.llama3-3-70b-instruct-v1:0": 128000, + "us.meta.llama3-1-405b-instruct-v1:0": 128000, + "eu.anthropic.claude-3-5-sonnet-20240620-v1:0": 200000, + "eu.anthropic.claude-3-sonnet-20240229-v1:0": 200000, + "eu.anthropic.claude-3-haiku-20240307-v1:0": 200000, + "eu.meta.llama3-2-3b-instruct-v1:0": 131000, + "eu.meta.llama3-2-1b-instruct-v1:0": 131000, + "apac.anthropic.claude-3-5-sonnet-20240620-v1:0": 200000, + "apac.anthropic.claude-3-5-sonnet-20241022-v2:0": 200000, + "apac.anthropic.claude-3-sonnet-20240229-v1:0": 200000, + "apac.anthropic.claude-3-haiku-20240307-v1:0": 200000, + "amazon.nova-pro-v1:0": 300000, + "amazon.nova-micro-v1:0": 128000, + "amazon.nova-lite-v1:0": 300000, + "anthropic.claude-3-5-sonnet-20240620-v1:0": 200000, + "anthropic.claude-3-5-haiku-20241022-v1:0": 200000, + "anthropic.claude-3-5-sonnet-20241022-v2:0": 200000, + "anthropic.claude-3-7-sonnet-20250219-v1:0": 200000, + "anthropic.claude-3-sonnet-20240229-v1:0": 200000, + "anthropic.claude-3-opus-20240229-v1:0": 200000, + "anthropic.claude-3-haiku-20240307-v1:0": 200000, + "anthropic.claude-v2:1": 200000, + "anthropic.claude-v2": 100000, + "anthropic.claude-instant-v1": 100000, + "meta.llama3-1-405b-instruct-v1:0": 128000, + "meta.llama3-1-70b-instruct-v1:0": 128000, + "meta.llama3-1-8b-instruct-v1:0": 128000, + "meta.llama3-70b-instruct-v1:0": 8000, + "meta.llama3-8b-instruct-v1:0": 8000, + "amazon.titan-text-lite-v1": 4000, + "amazon.titan-text-express-v1": 8000, + "cohere.command-text-v14": 4000, + "ai21.j2-mid-v1": 8191, + "ai21.j2-ultra-v1": 8191, + "ai21.jamba-instruct-v1:0": 256000, + "mistral.mistral-7b-instruct-v0:2": 32000, + "mistral.mixtral-8x7b-instruct-v0:1": 32000, # mistral "mistral-tiny": 32768, "mistral-small-latest": 32768, From bb3829a9ed4886da7a284bc89fc8bd7eda168ce2 Mon Sep 17 00:00:00 2001 From: Matisse <54796148+AMatisse@users.noreply.github.com> Date: Fri, 21 Mar 2025 15:12:26 -0400 Subject: [PATCH 27/29] docs: Update model reference in LLM configuration (#2267) Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com> --- docs/concepts/llms.mdx | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/docs/concepts/llms.mdx b/docs/concepts/llms.mdx index e17098f6a..2aada5fac 100644 --- a/docs/concepts/llms.mdx +++ b/docs/concepts/llms.mdx @@ -59,7 +59,7 @@ There are three ways to configure LLMs in CrewAI. Choose the method that best fi goal: Conduct comprehensive research and analysis backstory: A dedicated research professional with years of experience verbose: true - llm: openai/gpt-4o-mini # your model here + llm: openai/gpt-4o-mini # your model here # (see provider configuration examples below for more) ``` @@ -111,7 +111,7 @@ There are three ways to configure LLMs in CrewAI. Choose the method that best fi ## Provider Configuration Examples -CrewAI supports a multitude of LLM providers, each offering unique features, authentication methods, and model capabilities. +CrewAI supports a multitude of LLM providers, each offering unique features, authentication methods, and model capabilities. In this section, you'll find detailed examples that help you select, configure, and optimize the LLM that best fits your project's needs. @@ -121,7 +121,7 @@ In this section, you'll find detailed examples that help you select, configure, ```toml Code # Required OPENAI_API_KEY=sk-... - + # Optional OPENAI_API_BASE= OPENAI_ORGANIZATION= @@ -226,7 +226,7 @@ In this section, you'll find detailed examples that help you select, configure, AZURE_API_KEY= AZURE_API_BASE= AZURE_API_VERSION= - + # Optional AZURE_AD_TOKEN= AZURE_API_TYPE= @@ -289,7 +289,7 @@ In this section, you'll find detailed examples that help you select, configure, | Mistral 8x7B Instruct | Up to 32k tokens | An MOE LLM that follows instructions, completes requests, and generates creative text. | - + ```toml Code AWS_ACCESS_KEY_ID= @@ -474,7 +474,7 @@ In this section, you'll find detailed examples that help you select, configure, WATSONX_URL= WATSONX_APIKEY= WATSONX_PROJECT_ID= - + # Optional WATSONX_TOKEN= WATSONX_DEPLOYMENT_SPACE_ID= @@ -491,7 +491,7 @@ In this section, you'll find detailed examples that help you select, configure, 1. Install Ollama: [ollama.ai](https://ollama.ai/) - 2. Run a model: `ollama run llama2` + 2. Run a model: `ollama run llama3` 3. Configure: ```python Code @@ -600,7 +600,7 @@ In this section, you'll find detailed examples that help you select, configure, ```toml Code OPENROUTER_API_KEY= ``` - + Example usage in your CrewAI project: ```python Code llm = LLM( @@ -723,7 +723,7 @@ Learn how to get the most out of your LLM configuration: - Small tasks (up to 4K tokens): Standard models - Medium tasks (between 4K-32K): Enhanced models - Large tasks (over 32K): Large context models - + ```python # Configure model with appropriate settings llm = LLM( @@ -760,11 +760,11 @@ Learn how to get the most out of your LLM configuration: Most authentication issues can be resolved by checking API key format and environment variable names. - + ```bash # OpenAI OPENAI_API_KEY=sk-... - + # Anthropic ANTHROPIC_API_KEY=sk-ant-... ``` @@ -773,11 +773,11 @@ Learn how to get the most out of your LLM configuration: Always include the provider prefix in model names - + ```python # Correct llm = LLM(model="openai/gpt-4") - + # Incorrect llm = LLM(model="gpt-4") ``` @@ -786,5 +786,10 @@ Learn how to get the most out of your LLM configuration: Use larger context models for extensive tasks + + ```python + # Large context model + llm = LLM(model="openai/gpt-4o") # 128K tokens + ``` From ed1f009c6492f52fc8c2856fe99ca02dbd8ba355 Mon Sep 17 00:00:00 2001 From: "Brandon Hancock (bhancock_ai)" <109994880+bhancockio@users.noreply.github.com> Date: Fri, 21 Mar 2025 21:59:55 -0400 Subject: [PATCH 28/29] Feat/improve yaml extraction (#2428) * Support wildcard handling in `emit()` Change `emit()` to call handlers registered for parent classes using `isinstance()`. Ensures that base event handlers receive derived events. * Fix failing test * Remove unused variable * update interpolation to work with example response types in yaml docs * make tests * fix circular deps * Fixing interpolation imports * Improve test --------- Co-authored-by: Vinicius Brasil Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com> --- src/crewai/agents/agent_builder/base_agent.py | 13 +- src/crewai/task.py | 76 +------ src/crewai/utilities/formatter.py | 12 +- src/crewai/utilities/string_utils.py | 82 ++++++++ tests/task_test.py | 65 +++--- tests/utilities/test_string_utils.py | 187 ++++++++++++++++++ 6 files changed, 323 insertions(+), 112 deletions(-) create mode 100644 src/crewai/utilities/string_utils.py create mode 100644 tests/utilities/test_string_utils.py diff --git a/src/crewai/agents/agent_builder/base_agent.py b/src/crewai/agents/agent_builder/base_agent.py index 47515d087..4413e0a97 100644 --- a/src/crewai/agents/agent_builder/base_agent.py +++ b/src/crewai/agents/agent_builder/base_agent.py @@ -25,6 +25,7 @@ from crewai.tools.base_tool import BaseTool, Tool from crewai.utilities import I18N, Logger, RPMController from crewai.utilities.config import process_config from crewai.utilities.converter import Converter +from crewai.utilities.string_utils import interpolate_only T = TypeVar("T", bound="BaseAgent") @@ -333,9 +334,15 @@ class BaseAgent(ABC, BaseModel): self._original_backstory = self.backstory if inputs: - self.role = self._original_role.format(**inputs) - self.goal = self._original_goal.format(**inputs) - self.backstory = self._original_backstory.format(**inputs) + self.role = interpolate_only( + input_string=self._original_role, inputs=inputs + ) + self.goal = interpolate_only( + input_string=self._original_goal, inputs=inputs + ) + self.backstory = interpolate_only( + input_string=self._original_backstory, inputs=inputs + ) def set_cache_handler(self, cache_handler: CacheHandler) -> None: """Set the cache handler for the agent. diff --git a/src/crewai/task.py b/src/crewai/task.py index 0c063e4f9..10358147c 100644 --- a/src/crewai/task.py +++ b/src/crewai/task.py @@ -2,6 +2,7 @@ import datetime import inspect import json import logging +import re import threading import uuid from concurrent.futures import Future @@ -49,6 +50,7 @@ from crewai.utilities.events import ( from crewai.utilities.events.crewai_event_bus import crewai_event_bus from crewai.utilities.i18n import I18N from crewai.utilities.printer import Printer +from crewai.utilities.string_utils import interpolate_only class Task(BaseModel): @@ -507,7 +509,9 @@ class Task(BaseModel): return try: - self.description = self._original_description.format(**inputs) + self.description = interpolate_only( + input_string=self._original_description, inputs=inputs + ) except KeyError as e: raise ValueError( f"Missing required template variable '{e.args[0]}' in description" @@ -516,7 +520,7 @@ class Task(BaseModel): raise ValueError(f"Error interpolating description: {str(e)}") from e try: - self.expected_output = self.interpolate_only( + self.expected_output = interpolate_only( input_string=self._original_expected_output, inputs=inputs ) except (KeyError, ValueError) as e: @@ -524,7 +528,7 @@ class Task(BaseModel): if self.output_file is not None: try: - self.output_file = self.interpolate_only( + self.output_file = interpolate_only( input_string=self._original_output_file, inputs=inputs ) except (KeyError, ValueError) as e: @@ -555,72 +559,6 @@ class Task(BaseModel): f"\n\n{conversation_instruction}\n\n{conversation_history}" ) - def interpolate_only( - self, - input_string: Optional[str], - inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]], - ) -> str: - """Interpolate placeholders (e.g., {key}) in a string while leaving JSON untouched. - - Args: - input_string: The string containing template variables to interpolate. - Can be None or empty, in which case an empty string is returned. - inputs: Dictionary mapping template variables to their values. - Supported value types are strings, integers, floats, and dicts/lists - containing only these types and other nested dicts/lists. - - Returns: - The interpolated string with all template variables replaced with their values. - Empty string if input_string is None or empty. - - Raises: - ValueError: If a value contains unsupported types - """ - - # Validation function for recursive type checking - def validate_type(value: Any) -> None: - if value is None: - return - if isinstance(value, (str, int, float, bool)): - return - if isinstance(value, (dict, list)): - for item in value.values() if isinstance(value, dict) else value: - validate_type(item) - return - raise ValueError( - f"Unsupported type {type(value).__name__} in inputs. " - "Only str, int, float, bool, dict, and list are allowed." - ) - - # Validate all input values - for key, value in inputs.items(): - try: - validate_type(value) - except ValueError as e: - raise ValueError(f"Invalid value for key '{key}': {str(e)}") from e - - if input_string is None or not input_string: - return "" - if "{" not in input_string and "}" not in input_string: - return input_string - if not inputs: - raise ValueError( - "Inputs dictionary cannot be empty when interpolating variables" - ) - try: - escaped_string = input_string.replace("{", "{{").replace("}", "}}") - - for key in inputs.keys(): - escaped_string = escaped_string.replace(f"{{{{{key}}}}}", f"{{{key}}}") - - return escaped_string.format(**inputs) - except KeyError as e: - raise KeyError( - f"Template variable '{e.args[0]}' not found in inputs dictionary" - ) from e - except ValueError as e: - raise ValueError(f"Error during string interpolation: {str(e)}") from e - def increment_tools_errors(self) -> None: """Increment the tools errors counter.""" self.tools_errors += 1 diff --git a/src/crewai/utilities/formatter.py b/src/crewai/utilities/formatter.py index 34da6cc43..19b2a74f9 100644 --- a/src/crewai/utilities/formatter.py +++ b/src/crewai/utilities/formatter.py @@ -1,10 +1,12 @@ -from typing import List +import re +from typing import TYPE_CHECKING, List -from crewai.task import Task -from crewai.tasks.task_output import TaskOutput +if TYPE_CHECKING: + from crewai.task import Task + from crewai.tasks.task_output import TaskOutput -def aggregate_raw_outputs_from_task_outputs(task_outputs: List[TaskOutput]) -> str: +def aggregate_raw_outputs_from_task_outputs(task_outputs: List["TaskOutput"]) -> str: """Generate string context from the task outputs.""" dividers = "\n\n----------\n\n" @@ -13,7 +15,7 @@ def aggregate_raw_outputs_from_task_outputs(task_outputs: List[TaskOutput]) -> s return context -def aggregate_raw_outputs_from_tasks(tasks: List[Task]) -> str: +def aggregate_raw_outputs_from_tasks(tasks: List["Task"]) -> str: """Generate string context from the tasks.""" task_outputs = [task.output for task in tasks if task.output is not None] diff --git a/src/crewai/utilities/string_utils.py b/src/crewai/utilities/string_utils.py new file mode 100644 index 000000000..9a1857781 --- /dev/null +++ b/src/crewai/utilities/string_utils.py @@ -0,0 +1,82 @@ +import re +from typing import Any, Dict, List, Optional, Union + + +def interpolate_only( + input_string: Optional[str], + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]], +) -> str: + """Interpolate placeholders (e.g., {key}) in a string while leaving JSON untouched. + Only interpolates placeholders that follow the pattern {variable_name} where + variable_name starts with a letter/underscore and contains only letters, numbers, and underscores. + + Args: + input_string: The string containing template variables to interpolate. + Can be None or empty, in which case an empty string is returned. + inputs: Dictionary mapping template variables to their values. + Supported value types are strings, integers, floats, and dicts/lists + containing only these types and other nested dicts/lists. + + Returns: + The interpolated string with all template variables replaced with their values. + Empty string if input_string is None or empty. + + Raises: + ValueError: If a value contains unsupported types or a template variable is missing + """ + + # Validation function for recursive type checking + def validate_type(value: Any) -> None: + if value is None: + return + if isinstance(value, (str, int, float, bool)): + return + if isinstance(value, (dict, list)): + for item in value.values() if isinstance(value, dict) else value: + validate_type(item) + return + raise ValueError( + f"Unsupported type {type(value).__name__} in inputs. " + "Only str, int, float, bool, dict, and list are allowed." + ) + + # Validate all input values + for key, value in inputs.items(): + try: + validate_type(value) + except ValueError as e: + raise ValueError(f"Invalid value for key '{key}': {str(e)}") from e + + if input_string is None or not input_string: + return "" + if "{" not in input_string and "}" not in input_string: + return input_string + if not inputs: + raise ValueError( + "Inputs dictionary cannot be empty when interpolating variables" + ) + + # The regex pattern to find valid variable placeholders + # Matches {variable_name} where variable_name starts with a letter/underscore + # and contains only letters, numbers, and underscores + pattern = r"\{([A-Za-z_][A-Za-z0-9_]*)\}" + + # Find all matching variables in the input string + variables = re.findall(pattern, input_string) + result = input_string + + # Check if all variables exist in inputs + missing_vars = [var for var in variables if var not in inputs] + if missing_vars: + raise KeyError( + f"Template variable '{missing_vars[0]}' not found in inputs dictionary" + ) + + # Replace each variable with its value + for var in variables: + if var in inputs: + placeholder = "{" + var + "}" + value = str(inputs[var]) + result = result.replace(placeholder, value) + + return result diff --git a/tests/task_test.py b/tests/task_test.py index ac25a14f8..67ce99910 100644 --- a/tests/task_test.py +++ b/tests/task_test.py @@ -15,6 +15,7 @@ from crewai import Agent, Crew, Process, Task from crewai.tasks.conditional_task import ConditionalTask from crewai.tasks.task_output import TaskOutput from crewai.utilities.converter import Converter +from crewai.utilities.string_utils import interpolate_only def test_task_tool_reflect_agent_tools(): @@ -822,7 +823,7 @@ def test_interpolate_only(): # Test JSON structure preservation json_string = '{"info": "Look at {placeholder}", "nested": {"val": "{nestedVal}"}}' - result = task.interpolate_only( + result = interpolate_only( input_string=json_string, inputs={"placeholder": "the data", "nestedVal": "something else"}, ) @@ -833,20 +834,18 @@ def test_interpolate_only(): # Test normal string interpolation normal_string = "Hello {name}, welcome to {place}!" - result = task.interpolate_only( + result = interpolate_only( input_string=normal_string, inputs={"name": "John", "place": "CrewAI"} ) assert result == "Hello John, welcome to CrewAI!" # Test empty string - result = task.interpolate_only(input_string="", inputs={"unused": "value"}) + result = interpolate_only(input_string="", inputs={"unused": "value"}) assert result == "" # Test string with no placeholders no_placeholders = "Hello, this is a test" - result = task.interpolate_only( - input_string=no_placeholders, inputs={"unused": "value"} - ) + result = interpolate_only(input_string=no_placeholders, inputs={"unused": "value"}) assert result == no_placeholders @@ -858,7 +857,7 @@ def test_interpolate_only_with_dict_inside_expected_output(): ) json_string = '{"questions": {"main_question": "What is the user\'s name?", "secondary_question": "What is the user\'s age?"}}' - result = task.interpolate_only( + result = interpolate_only( input_string=json_string, inputs={ "questions": { @@ -872,18 +871,16 @@ def test_interpolate_only_with_dict_inside_expected_output(): assert result == json_string normal_string = "Hello {name}, welcome to {place}!" - result = task.interpolate_only( + result = interpolate_only( input_string=normal_string, inputs={"name": "John", "place": "CrewAI"} ) assert result == "Hello John, welcome to CrewAI!" - result = task.interpolate_only(input_string="", inputs={"unused": "value"}) + result = interpolate_only(input_string="", inputs={"unused": "value"}) assert result == "" no_placeholders = "Hello, this is a test" - result = task.interpolate_only( - input_string=no_placeholders, inputs={"unused": "value"} - ) + result = interpolate_only(input_string=no_placeholders, inputs={"unused": "value"}) assert result == no_placeholders @@ -1085,12 +1082,12 @@ def test_interpolate_with_list_of_strings(): # Test simple list of strings input_str = "Available items: {items}" inputs = {"items": ["apple", "banana", "cherry"]} - result = task.interpolate_only(input_str, inputs) + result = interpolate_only(input_str, inputs) assert result == f"Available items: {inputs['items']}" # Test empty list empty_list_input = {"items": []} - result = task.interpolate_only(input_str, empty_list_input) + result = interpolate_only(input_str, empty_list_input) assert result == "Available items: []" @@ -1106,7 +1103,7 @@ def test_interpolate_with_list_of_dicts(): {"name": "Bob", "age": 25, "skills": ["Java", "Cloud"]}, ] } - result = task.interpolate_only("{people}", input_data) + result = interpolate_only("{people}", input_data) parsed_result = eval(result) assert isinstance(parsed_result, list) @@ -1138,7 +1135,7 @@ def test_interpolate_with_nested_structures(): ], } } - result = task.interpolate_only("{company}", input_data) + result = interpolate_only("{company}", input_data) parsed = eval(result) assert parsed["name"] == "TechCorp" @@ -1161,7 +1158,7 @@ def test_interpolate_with_special_characters(): "empty": "", } } - result = task.interpolate_only("{special_data}", input_data) + result = interpolate_only("{special_data}", input_data) parsed = eval(result) assert parsed["quotes"] == """This has "double" and 'single' quotes""" @@ -1188,7 +1185,7 @@ def test_interpolate_mixed_types(): }, } } - result = task.interpolate_only("{data}", input_data) + result = interpolate_only("{data}", input_data) parsed = eval(result) assert parsed["name"] == "Test Dataset" @@ -1216,7 +1213,7 @@ def test_interpolate_complex_combination(): }, ] } - result = task.interpolate_only("{report}", input_data) + result = interpolate_only("{report}", input_data) parsed = eval(result) assert len(parsed) == 2 @@ -1233,7 +1230,7 @@ def test_interpolate_invalid_type_validation(): # Test with invalid top-level type with pytest.raises(ValueError) as excinfo: - task.interpolate_only("{data}", {"data": set()}) # type: ignore we are purposely testing this failure + interpolate_only("{data}", {"data": set()}) # type: ignore we are purposely testing this failure assert "Unsupported type set" in str(excinfo.value) @@ -1246,7 +1243,7 @@ def test_interpolate_invalid_type_validation(): } } with pytest.raises(ValueError) as excinfo: - task.interpolate_only("{data}", {"data": invalid_nested}) + interpolate_only("{data}", {"data": invalid_nested}) assert "Unsupported type set" in str(excinfo.value) @@ -1265,24 +1262,22 @@ def test_interpolate_custom_object_validation(): # Test with custom object at top level with pytest.raises(ValueError) as excinfo: - task.interpolate_only("{obj}", {"obj": CustomObject(5)}) # type: ignore we are purposely testing this failure + interpolate_only("{obj}", {"obj": CustomObject(5)}) # type: ignore we are purposely testing this failure assert "Unsupported type CustomObject" in str(excinfo.value) # Test with nested custom object in dictionary with pytest.raises(ValueError) as excinfo: - task.interpolate_only( - "{data}", {"data": {"valid": 1, "invalid": CustomObject(5)}} - ) + interpolate_only("{data}", {"data": {"valid": 1, "invalid": CustomObject(5)}}) assert "Unsupported type CustomObject" in str(excinfo.value) # Test with nested custom object in list with pytest.raises(ValueError) as excinfo: - task.interpolate_only("{data}", {"data": [1, "valid", CustomObject(5)]}) + interpolate_only("{data}", {"data": [1, "valid", CustomObject(5)]}) assert "Unsupported type CustomObject" in str(excinfo.value) # Test with deeply nested custom object with pytest.raises(ValueError) as excinfo: - task.interpolate_only( + interpolate_only( "{data}", {"data": {"level1": {"level2": [{"level3": CustomObject(5)}]}}} ) assert "Unsupported type CustomObject" in str(excinfo.value) @@ -1306,7 +1301,7 @@ def test_interpolate_valid_complex_types(): } # Should not raise any errors - result = task.interpolate_only("{data}", {"data": valid_data}) + result = interpolate_only("{data}", {"data": valid_data}) parsed = eval(result) assert parsed["name"] == "Valid Dataset" assert parsed["stats"]["nested"]["deeper"]["b"] == 2.5 @@ -1319,16 +1314,16 @@ def test_interpolate_edge_cases(): ) # Test empty dict and list - assert task.interpolate_only("{}", {"data": {}}) == "{}" - assert task.interpolate_only("[]", {"data": []}) == "[]" + assert interpolate_only("{}", {"data": {}}) == "{}" + assert interpolate_only("[]", {"data": []}) == "[]" # Test numeric types - assert task.interpolate_only("{num}", {"num": 42}) == "42" - assert task.interpolate_only("{num}", {"num": 3.14}) == "3.14" + assert interpolate_only("{num}", {"num": 42}) == "42" + assert interpolate_only("{num}", {"num": 3.14}) == "3.14" # Test boolean values (valid JSON types) - assert task.interpolate_only("{flag}", {"flag": True}) == "True" - assert task.interpolate_only("{flag}", {"flag": False}) == "False" + assert interpolate_only("{flag}", {"flag": True}) == "True" + assert interpolate_only("{flag}", {"flag": False}) == "False" def test_interpolate_valid_types(): @@ -1346,7 +1341,7 @@ def test_interpolate_valid_types(): "nested": {"flag": True, "empty": None}, } - result = task.interpolate_only("{data}", {"data": valid_data}) + result = interpolate_only("{data}", {"data": valid_data}) parsed = eval(result) assert parsed["active"] is True diff --git a/tests/utilities/test_string_utils.py b/tests/utilities/test_string_utils.py new file mode 100644 index 000000000..441aae8c0 --- /dev/null +++ b/tests/utilities/test_string_utils.py @@ -0,0 +1,187 @@ +from typing import Any, Dict, List, Union + +import pytest + +from crewai.utilities.string_utils import interpolate_only + + +class TestInterpolateOnly: + """Tests for the interpolate_only function in string_utils.py.""" + + def test_basic_variable_interpolation(self): + """Test basic variable interpolation works correctly.""" + template = "Hello, {name}! Welcome to {company}." + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]] = { + "name": "Alice", + "company": "CrewAI", + } + + result = interpolate_only(template, inputs) + + assert result == "Hello, Alice! Welcome to CrewAI." + + def test_multiple_occurrences_of_same_variable(self): + """Test that multiple occurrences of the same variable are replaced.""" + template = "{name} is using {name}'s account." + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]] = { + "name": "Bob" + } + + result = interpolate_only(template, inputs) + + assert result == "Bob is using Bob's account." + + def test_json_structure_preservation(self): + """Test that JSON structures are preserved and not interpolated incorrectly.""" + template = """ + Instructions for {agent}: + + Please return the following object: + + {"name": "person's name", "age": 25, "skills": ["coding", "testing"]} + """ + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]] = { + "agent": "DevAgent" + } + + result = interpolate_only(template, inputs) + + assert "Instructions for DevAgent:" in result + assert ( + '{"name": "person\'s name", "age": 25, "skills": ["coding", "testing"]}' + in result + ) + + def test_complex_nested_json(self): + """Test with complex JSON structures containing curly braces.""" + template = """ + {agent} needs to process: + { + "config": { + "nested": { + "value": 42 + }, + "arrays": [1, 2, {"inner": "value"}] + } + } + """ + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]] = { + "agent": "DataProcessor" + } + + result = interpolate_only(template, inputs) + + assert "DataProcessor needs to process:" in result + assert '"nested": {' in result + assert '"value": 42' in result + assert '[1, 2, {"inner": "value"}]' in result + + def test_missing_variable(self): + """Test that an error is raised when a required variable is missing.""" + template = "Hello, {name}!" + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]] = { + "not_name": "Alice" + } + + with pytest.raises(KeyError) as excinfo: + interpolate_only(template, inputs) + + assert "template variable" in str(excinfo.value).lower() + assert "name" in str(excinfo.value) + + def test_invalid_input_types(self): + """Test that an error is raised with invalid input types.""" + template = "Hello, {name}!" + # Using Any for this test since we're intentionally testing an invalid type + inputs: Dict[str, Any] = {"name": object()} # Object is not a valid input type + + with pytest.raises(ValueError) as excinfo: + interpolate_only(template, inputs) + + assert "unsupported type" in str(excinfo.value).lower() + + def test_empty_input_string(self): + """Test handling of empty or None input string.""" + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]] = { + "name": "Alice" + } + + assert interpolate_only("", inputs) == "" + assert interpolate_only(None, inputs) == "" + + def test_no_variables_in_template(self): + """Test a template with no variables to replace.""" + template = "This is a static string with no variables." + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]] = { + "name": "Alice" + } + + result = interpolate_only(template, inputs) + + assert result == template + + def test_variable_name_starting_with_underscore(self): + """Test variables starting with underscore are replaced correctly.""" + template = "Variable: {_special_var}" + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]] = { + "_special_var": "Special Value" + } + + result = interpolate_only(template, inputs) + + assert result == "Variable: Special Value" + + def test_preserves_non_matching_braces(self): + """Test that non-matching braces patterns are preserved.""" + template = ( + "This {123} and {!var} should not be replaced but {valid_var} should." + ) + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]] = { + "valid_var": "works" + } + + result = interpolate_only(template, inputs) + + assert ( + result == "This {123} and {!var} should not be replaced but works should." + ) + + def test_complex_mixed_scenario(self): + """Test a complex scenario with both valid variables and JSON structures.""" + template = """ + {agent_name} is working on task {task_id}. + + Instructions: + 1. Process the data + 2. Return results as: + + { + "taskId": "{task_id}", + "results": { + "processed_by": "agent_name", + "status": "complete", + "values": [1, 2, 3] + } + } + """ + inputs: Dict[str, Union[str, int, float, Dict[str, Any], List[Any]]] = { + "agent_name": "AnalyticsAgent", + "task_id": "T-12345", + } + + result = interpolate_only(template, inputs) + + assert "AnalyticsAgent is working on task T-12345" in result + assert '"taskId": "T-12345"' in result + assert '"processed_by": "agent_name"' in result # This shouldn't be replaced + assert '"values": [1, 2, 3]' in result + + def test_empty_inputs_dictionary(self): + """Test that an error is raised with empty inputs dictionary.""" + template = "Hello, {name}!" + inputs: Dict[str, Any] = {} + + with pytest.raises(ValueError) as excinfo: + interpolate_only(template, inputs) + + assert "inputs dictionary cannot be empty" in str(excinfo.value).lower() From bdc92deade4198c0de2de1bc59bb6c986eaac266 Mon Sep 17 00:00:00 2001 From: Tony Kipkemboi Date: Mon, 24 Mar 2025 12:06:50 -0400 Subject: [PATCH 29/29] docs: update changelog dates (#2437) * docs: update changelog dates * docs: add aws bedrock tools docs * docs: fix incorrect respect_context_window parameter in Crew example --- docs/changelog.mdx | 20 +-- docs/docs.json | 2 + docs/how-to/kickoff-for-each.mdx | 3 +- docs/tools/bedrockinvokeagenttool.mdx | 187 ++++++++++++++++++++++++++ docs/tools/bedrockkbretriever.mdx | 165 +++++++++++++++++++++++ 5 files changed, 365 insertions(+), 12 deletions(-) create mode 100644 docs/tools/bedrockinvokeagenttool.mdx create mode 100644 docs/tools/bedrockkbretriever.mdx diff --git a/docs/changelog.mdx b/docs/changelog.mdx index 3f17934bc..e88d1f632 100644 --- a/docs/changelog.mdx +++ b/docs/changelog.mdx @@ -4,7 +4,7 @@ description: View the latest updates and changes to CrewAI icon: timeline --- - + **Features** - Converted tabs to spaces in `crew.py` template - Enhanced LLM Streaming Response Handling and Event System @@ -24,7 +24,7 @@ icon: timeline - Added documentation for `ApifyActorsTool` - + **Core Improvements & Fixes** - Fixed issues with missing template variables and user memory configuration - Improved async flow support and addressed agent response formatting @@ -45,7 +45,7 @@ icon: timeline - Fixed typos in prompts and updated Amazon Bedrock model listings - + **Core Improvements & Fixes** - Enhanced LLM Support: Improved structured LLM output, parameter handling, and formatting for Anthropic models - Crew & Agent Stability: Fixed issues with cloning agents/crews using knowledge sources, multiple task outputs in conditional tasks, and ignored Crew task callbacks @@ -65,7 +65,7 @@ icon: timeline - Fixed Various Typos & Formatting Issues - + **Features** - Add Composio docs - Add SageMaker as a LLM provider @@ -80,7 +80,7 @@ icon: timeline - Improve formatting and clarity in CLI and Composio Tool docs - + **Features** - Conversation crew v1 - Add unique ID to flow states @@ -101,7 +101,7 @@ icon: timeline - Fixed typos, nested pydantic model issue, and docling issues - + **New Features** - Adding Multimodal Abilities to Crew - Programatic Guardrails @@ -131,7 +131,7 @@ icon: timeline - Suppressed userWarnings from litellm pydantic issues - + **Changes** - Remove all references to pipeline and pipeline router - Add Nvidia NIM as provider in Custom LLM @@ -141,7 +141,7 @@ icon: timeline - Simplify template crew - + **Features** - Added knowledge to agent level - Feat/remove langchain @@ -161,7 +161,7 @@ icon: timeline - Improvements to LLM Configuration and Usage - + **New Features** - New before_kickoff and after_kickoff crew callbacks - Support to pre-seed agents with Knowledge @@ -178,7 +178,7 @@ icon: timeline - Update Docs - + **Fixes** - Fixing Tokens callback replacement bug - Fixing Step callback issue diff --git a/docs/docs.json b/docs/docs.json index d2c7e55b0..7991e5953 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -111,6 +111,8 @@ "pages": [ "tools/aimindtool", "tools/apifyactorstool", + "tools/bedrockinvokeagenttool", + "tools/bedrockkbretriever", "tools/bravesearchtool", "tools/browserbaseloadtool", "tools/codedocssearchtool", diff --git a/docs/how-to/kickoff-for-each.mdx b/docs/how-to/kickoff-for-each.mdx index 4a83f7a5f..68e7ecd15 100644 --- a/docs/how-to/kickoff-for-each.mdx +++ b/docs/how-to/kickoff-for-each.mdx @@ -39,8 +39,7 @@ analysis_crew = Crew( agents=[coding_agent], tasks=[data_analysis_task], verbose=True, - memory=False, - respect_context_window=True # enable by default + memory=False ) datasets = [ diff --git a/docs/tools/bedrockinvokeagenttool.mdx b/docs/tools/bedrockinvokeagenttool.mdx new file mode 100644 index 000000000..c9b2c6aac --- /dev/null +++ b/docs/tools/bedrockinvokeagenttool.mdx @@ -0,0 +1,187 @@ +--- +title: Bedrock Invoke Agent Tool +description: Enables CrewAI agents to invoke Amazon Bedrock Agents and leverage their capabilities within your workflows +icon: aws +--- + +# `BedrockInvokeAgentTool` + +The `BedrockInvokeAgentTool` enables CrewAI agents to invoke Amazon Bedrock Agents and leverage their capabilities within your workflows. + +## Installation + +```bash +uv pip install 'crewai[tools]' +``` + +## Requirements + +- AWS credentials configured (either through environment variables or AWS CLI) +- `boto3` and `python-dotenv` packages +- Access to Amazon Bedrock Agents + +## Usage + +Here's how to use the tool with a CrewAI agent: + +```python {2, 4-8} +from crewai import Agent, Task, Crew +from crewai_tools.aws.bedrock.agents.invoke_agent_tool import BedrockInvokeAgentTool + +# Initialize the tool +agent_tool = BedrockInvokeAgentTool( + agent_id="your-agent-id", + agent_alias_id="your-agent-alias-id" +) + +# Create a CrewAI agent that uses the tool +aws_expert = Agent( + role='AWS Service Expert', + goal='Help users understand AWS services and quotas', + backstory='I am an expert in AWS services and can provide detailed information about them.', + tools=[agent_tool], + verbose=True +) + +# Create a task for the agent +quota_task = Task( + description="Find out the current service quotas for EC2 in us-west-2 and explain any recent changes.", + agent=aws_expert +) + +# Create a crew with the agent +crew = Crew( + agents=[aws_expert], + tasks=[quota_task], + verbose=2 +) + +# Run the crew +result = crew.kickoff() +print(result) +``` + +## Tool Arguments + +| Argument | Type | Required | Default | Description | +|:---------|:-----|:---------|:--------|:------------| +| **agent_id** | `str` | Yes | None | The unique identifier of the Bedrock agent | +| **agent_alias_id** | `str` | Yes | None | The unique identifier of the agent alias | +| **session_id** | `str` | No | timestamp | The unique identifier of the session | +| **enable_trace** | `bool` | No | False | Whether to enable trace for debugging | +| **end_session** | `bool` | No | False | Whether to end the session after invocation | +| **description** | `str` | No | None | Custom description for the tool | + +## Environment Variables + +```bash +BEDROCK_AGENT_ID=your-agent-id # Alternative to passing agent_id +BEDROCK_AGENT_ALIAS_ID=your-agent-alias-id # Alternative to passing agent_alias_id +AWS_REGION=your-aws-region # Defaults to us-west-2 +AWS_ACCESS_KEY_ID=your-access-key # Required for AWS authentication +AWS_SECRET_ACCESS_KEY=your-secret-key # Required for AWS authentication +``` + +## Advanced Usage + +### Multi-Agent Workflow with Session Management + +```python {2, 4-22} +from crewai import Agent, Task, Crew, Process +from crewai_tools.aws.bedrock.agents.invoke_agent_tool import BedrockInvokeAgentTool + +# Initialize tools with session management +initial_tool = BedrockInvokeAgentTool( + agent_id="your-agent-id", + agent_alias_id="your-agent-alias-id", + session_id="custom-session-id" +) + +followup_tool = BedrockInvokeAgentTool( + agent_id="your-agent-id", + agent_alias_id="your-agent-alias-id", + session_id="custom-session-id" +) + +final_tool = BedrockInvokeAgentTool( + agent_id="your-agent-id", + agent_alias_id="your-agent-alias-id", + session_id="custom-session-id", + end_session=True +) + +# Create agents for different stages +researcher = Agent( + role='AWS Service Researcher', + goal='Gather information about AWS services', + backstory='I am specialized in finding detailed AWS service information.', + tools=[initial_tool] +) + +analyst = Agent( + role='Service Compatibility Analyst', + goal='Analyze service compatibility and requirements', + backstory='I analyze AWS services for compatibility and integration possibilities.', + tools=[followup_tool] +) + +summarizer = Agent( + role='Technical Documentation Writer', + goal='Create clear technical summaries', + backstory='I specialize in creating clear, concise technical documentation.', + tools=[final_tool] +) + +# Create tasks +research_task = Task( + description="Find all available AWS services in us-west-2 region.", + agent=researcher +) + +analysis_task = Task( + description="Analyze which services support IPv6 and their implementation requirements.", + agent=analyst +) + +summary_task = Task( + description="Create a summary of IPv6-compatible services and their key features.", + agent=summarizer +) + +# Create a crew with the agents and tasks +crew = Crew( + agents=[researcher, analyst, summarizer], + tasks=[research_task, analysis_task, summary_task], + process=Process.sequential, + verbose=2 +) + +# Run the crew +result = crew.kickoff() +``` + +## Use Cases + +### Hybrid Multi-Agent Collaborations +- Create workflows where CrewAI agents collaborate with managed Bedrock agents running as services in AWS +- Enable scenarios where sensitive data processing happens within your AWS environment while other agents operate externally +- Bridge on-premises CrewAI agents with cloud-based Bedrock agents for distributed intelligence workflows + +### Data Sovereignty and Compliance +- Keep data-sensitive agentic workflows within your AWS environment while allowing external CrewAI agents to orchestrate tasks +- Maintain compliance with data residency requirements by processing sensitive information only within your AWS account +- Enable secure multi-agent collaborations where some agents cannot access your organization's private data + +### Seamless AWS Service Integration +- Access any AWS service through Amazon Bedrock Actions without writing complex integration code +- Enable CrewAI agents to interact with AWS services through natural language requests +- Leverage pre-built Bedrock agent capabilities to interact with AWS services like Bedrock Knowledge Bases, Lambda, and more + +### Scalable Hybrid Agent Architectures +- Offload computationally intensive tasks to managed Bedrock agents while lightweight tasks run in CrewAI +- Scale agent processing by distributing workloads between local CrewAI agents and cloud-based Bedrock agents + +### Cross-Organizational Agent Collaboration +- Enable secure collaboration between your organization's CrewAI agents and partner organizations' Bedrock agents +- Create workflows where external expertise from Bedrock agents can be incorporated without exposing sensitive data +- Build agent ecosystems that span organizational boundaries while maintaining security and data control \ No newline at end of file diff --git a/docs/tools/bedrockkbretriever.mdx b/docs/tools/bedrockkbretriever.mdx new file mode 100644 index 000000000..3868f636a --- /dev/null +++ b/docs/tools/bedrockkbretriever.mdx @@ -0,0 +1,165 @@ +--- +title: 'Bedrock Knowledge Base Retriever' +description: 'Retrieve information from Amazon Bedrock Knowledge Bases using natural language queries' +icon: aws +--- + +# `BedrockKBRetrieverTool` + +The `BedrockKBRetrieverTool` enables CrewAI agents to retrieve information from Amazon Bedrock Knowledge Bases using natural language queries. + +## Installation + +```bash +uv pip install 'crewai[tools]' +``` + +## Requirements + +- AWS credentials configured (either through environment variables or AWS CLI) +- `boto3` and `python-dotenv` packages +- Access to Amazon Bedrock Knowledge Base + +## Usage + +Here's how to use the tool with a CrewAI agent: + +```python {2, 4-17} +from crewai import Agent, Task, Crew +from crewai_tools.aws.bedrock.knowledge_base.retriever_tool import BedrockKBRetrieverTool + +# Initialize the tool +kb_tool = BedrockKBRetrieverTool( + knowledge_base_id="your-kb-id", + number_of_results=5 +) + +# Create a CrewAI agent that uses the tool +researcher = Agent( + role='Knowledge Base Researcher', + goal='Find information about company policies', + backstory='I am a researcher specialized in retrieving and analyzing company documentation.', + tools=[kb_tool], + verbose=True +) + +# Create a task for the agent +research_task = Task( + description="Find our company's remote work policy and summarize the key points.", + agent=researcher +) + +# Create a crew with the agent +crew = Crew( + agents=[researcher], + tasks=[research_task], + verbose=2 +) + +# Run the crew +result = crew.kickoff() +print(result) +``` + +## Tool Arguments + +| Argument | Type | Required | Default | Description | +|:---------|:-----|:---------|:---------|:-------------| +| **knowledge_base_id** | `str` | Yes | None | The unique identifier of the knowledge base (0-10 alphanumeric characters) | +| **number_of_results** | `int` | No | 5 | Maximum number of results to return | +| **retrieval_configuration** | `dict` | No | None | Custom configurations for the knowledge base query | +| **guardrail_configuration** | `dict` | No | None | Content filtering settings | +| **next_token** | `str` | No | None | Token for pagination | + +## Environment Variables + +```bash +BEDROCK_KB_ID=your-knowledge-base-id # Alternative to passing knowledge_base_id +AWS_REGION=your-aws-region # Defaults to us-east-1 +AWS_ACCESS_KEY_ID=your-access-key # Required for AWS authentication +AWS_SECRET_ACCESS_KEY=your-secret-key # Required for AWS authentication +``` + +## Response Format + +The tool returns results in JSON format: + +```json +{ + "results": [ + { + "content": "Retrieved text content", + "content_type": "text", + "source_type": "S3", + "source_uri": "s3://bucket/document.pdf", + "score": 0.95, + "metadata": { + "additional": "metadata" + } + } + ], + "nextToken": "pagination-token", + "guardrailAction": "NONE" +} +``` + +## Advanced Usage + +### Custom Retrieval Configuration + +```python +kb_tool = BedrockKBRetrieverTool( + knowledge_base_id="your-kb-id", + retrieval_configuration={ + "vectorSearchConfiguration": { + "numberOfResults": 10, + "overrideSearchType": "HYBRID" + } + } +) + +policy_expert = Agent( + role='Policy Expert', + goal='Analyze company policies in detail', + backstory='I am an expert in corporate policy analysis with deep knowledge of regulatory requirements.', + tools=[kb_tool] +) +``` + +## Supported Data Sources + +- Amazon S3 +- Confluence +- Salesforce +- SharePoint +- Web pages +- Custom document locations +- Amazon Kendra +- SQL databases + +## Use Cases + +### Enterprise Knowledge Integration +- Enable CrewAI agents to access your organization's proprietary knowledge without exposing sensitive data +- Allow agents to make decisions based on your company's specific policies, procedures, and documentation +- Create agents that can answer questions based on your internal documentation while maintaining data security + +### Specialized Domain Knowledge +- Connect CrewAI agents to domain-specific knowledge bases (legal, medical, technical) without retraining models +- Leverage existing knowledge repositories that are already maintained in your AWS environment +- Combine CrewAI's reasoning with domain-specific information from your knowledge bases + +### Data-Driven Decision Making +- Ground CrewAI agent responses in your actual company data rather than general knowledge +- Ensure agents provide recommendations based on your specific business context and documentation +- Reduce hallucinations by retrieving factual information from your knowledge bases + +### Scalable Information Access +- Access terabytes of organizational knowledge without embedding it all into your models +- Dynamically query only the relevant information needed for specific tasks +- Leverage AWS's scalable infrastructure to handle large knowledge bases efficiently + +### Compliance and Governance +- Ensure CrewAI agents provide responses that align with your company's approved documentation +- Create auditable trails of information sources used by your agents +- Maintain control over what information sources your agents can access