Merge branch 'bugfix/restrict-python-version-compatibility' of https://github.com/joaomdmoura/crewAI into bugfix/restrict-python-version-compatibility

This commit is contained in:
Brandon Hancock
2024-12-09 13:54:47 -05:00
10 changed files with 92 additions and 26 deletions

View File

@@ -8,8 +8,8 @@ icon: book
## What is Knowledge?
Knowledge in CrewAI is a powerful system that allows AI agents to access and utilize external information sources during their tasks.
Think of it as giving your agents a reference library they can consult while working.
Knowledge in CrewAI is a powerful system that allows AI agents to access and utilize external information sources during their tasks.
Think of it as giving your agents a reference library they can consult while working.
<Info>
Key benefits of using Knowledge:
@@ -47,7 +47,7 @@ from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSourc
# Create a knowledge source
content = "Users name is John. He is 30 years old and lives in San Francisco."
string_source = StringKnowledgeSource(
content=content,
content=content,
)
# Create an LLM with a temperature of 0 to ensure deterministic outputs
@@ -122,7 +122,6 @@ crewai reset-memories --knowledge
This is useful when you've updated your knowledge sources and want to ensure that the agents are using the most recent information.
## Custom Knowledge Sources
CrewAI allows you to create custom knowledge sources for any type of data by extending the `BaseKnowledgeSource` class. Let's create a practical example that fetches and processes space news articles.
@@ -141,10 +140,10 @@ from pydantic import BaseModel, Field
class SpaceNewsKnowledgeSource(BaseKnowledgeSource):
"""Knowledge source that fetches data from Space News API."""
api_endpoint: str = Field(description="API endpoint URL")
limit: int = Field(default=10, description="Number of articles to fetch")
def load_content(self) -> Dict[Any, str]:
"""Fetch and format space news articles."""
try:
@@ -152,15 +151,15 @@ class SpaceNewsKnowledgeSource(BaseKnowledgeSource):
f"{self.api_endpoint}?limit={self.limit}"
)
response.raise_for_status()
data = response.json()
articles = data.get('results', [])
formatted_data = self._format_articles(articles)
return {self.api_endpoint: formatted_data}
except Exception as e:
raise ValueError(f"Failed to fetch space news: {str(e)}")
def _format_articles(self, articles: list) -> str:
"""Format articles into readable text."""
formatted = "Space News Articles:\n\n"
@@ -180,7 +179,7 @@ class SpaceNewsKnowledgeSource(BaseKnowledgeSource):
for _, text in content.items():
chunks = self._chunk_text(text)
self.chunks.extend(chunks)
self._save_documents()
# Create knowledge source
@@ -193,7 +192,7 @@ recent_news = SpaceNewsKnowledgeSource(
space_analyst = Agent(
role="Space News Analyst",
goal="Answer questions about space news accurately and comprehensively",
backstory="""You are a space industry analyst with expertise in space exploration,
backstory="""You are a space industry analyst with expertise in space exploration,
satellite technology, and space industry trends. You excel at answering questions
about space news and providing detailed, accurate information.""",
knowledge_sources=[recent_news],
@@ -220,13 +219,14 @@ result = crew.kickoff(
inputs={"user_question": "What are the latest developments in space exploration?"}
)
```
```output Output
# Agent: Space News Analyst
## Task: Answer this question about space news: What are the latest developments in space exploration?
# Agent: Space News Analyst
## Final Answer:
## Final Answer:
The latest developments in space exploration, based on recent space news articles, include the following:
1. SpaceX has received the final regulatory approvals to proceed with the second integrated Starship/Super Heavy launch, scheduled for as soon as the morning of Nov. 17, 2023. This is a significant step in SpaceX's ambitious plans for space exploration and colonization. [Source: SpaceNews](https://spacenews.com/starship-cleared-for-nov-17-launch/)
@@ -242,11 +242,13 @@ The latest developments in space exploration, based on recent space news article
6. The National Natural Science Foundation of China has outlined a five-year project for researchers to study the assembly of ultra-large spacecraft. This could lead to significant advancements in spacecraft technology and space exploration capabilities. [Source: SpaceNews](https://spacenews.com/china-researching-challenges-of-kilometer-scale-ultra-large-spacecraft/)
7. The Center for AEroSpace Autonomy Research (CAESAR) at Stanford University is focusing on spacecraft autonomy. The center held a kickoff event on May 22, 2024, to highlight the industry, academia, and government collaboration it seeks to foster. This could lead to significant advancements in autonomous spacecraft technology. [Source: SpaceNews](https://spacenews.com/stanford-center-focuses-on-spacecraft-autonomy/)
```
```
</CodeGroup>
#### Key Components Explained
1. **Custom Knowledge Source (`SpaceNewsKnowledgeSource`)**:
- Extends `BaseKnowledgeSource` for integration with CrewAI
- Configurable API endpoint and article limit
- Implements three key methods:
@@ -255,10 +257,12 @@ The latest developments in space exploration, based on recent space news article
- `add()`: Processes and stores the content
2. **Agent Configuration**:
- Specialized role as a Space News Analyst
- Uses the knowledge source to access space news
3. **Task Setup**:
- Takes a user question as input through `{user_question}`
- Designed to provide detailed answers based on the knowledge source
@@ -267,6 +271,7 @@ The latest developments in space exploration, based on recent space news article
- Handles input/output through the kickoff method
This example demonstrates how to:
- Create a custom knowledge source that fetches real-time data
- Process and format external data for AI consumption
- Use the knowledge source to answer specific user questions
@@ -274,13 +279,15 @@ This example demonstrates how to:
#### About the Spaceflight News API
The example uses the [Spaceflight News API](https://api.spaceflightnewsapi.net/v4/documentation), which:
The example uses the [Spaceflight News API](https://api.spaceflightnewsapi.net/v4/docs/), which:
- Provides free access to space-related news articles
- Requires no authentication
- Returns structured data about space news
- Supports pagination and filtering
You can customize the API query by modifying the endpoint URL:
```python
# Fetch more articles
recent_news = SpaceNewsKnowledgeSource(
@@ -303,9 +310,9 @@ recent_news = SpaceNewsKnowledgeSource(
- Consider content overlap for context preservation
- Organize related information into separate knowledge sources
</Accordion>
<Accordion title="Performance Tips">
- Adjust chunk sizes based on content complexity
- Adjust chunk sizes based on content complexity
- Configure appropriate embedding models
- Consider using local embedding providers for faster processing
</Accordion>

View File

@@ -172,6 +172,48 @@ def my_tool(question: str) -> str:
return "Result from your custom tool"
```
### Structured Tools
The `StructuredTool` class wraps functions as tools, providing flexibility and validation while reducing boilerplate. It supports custom schemas and dynamic logic for seamless integration of complex functionalities.
#### Example:
Using `StructuredTool.from_function`, you can wrap a function that interacts with an external API or system, providing a structured interface. This enables robust validation and consistent execution, making it easier to integrate complex functionalities into your applications as demonstrated in the following example:
```python
from crewai.tools.structured_tool import CrewStructuredTool
from pydantic import BaseModel
# Define the schema for the tool's input using Pydantic
class APICallInput(BaseModel):
endpoint: str
parameters: dict
# Wrapper function to execute the API call
def tool_wrapper(*args, **kwargs):
# Here, you would typically call the API using the parameters
# For demonstration, we'll return a placeholder string
return f"Call the API at {kwargs['endpoint']} with parameters {kwargs['parameters']}"
# Create and return the structured tool
def create_structured_tool():
return CrewStructuredTool.from_function(
name='Wrapper API',
description="A tool to wrap API calls with structured input.",
args_schema=APICallInput,
func=tool_wrapper,
)
# Example usage
structured_tool = create_structured_tool()
# Execute the tool with structured input
result = structured_tool._run(**{
"endpoint": "https://example.com/api",
"parameters": {"key1": "value1", "key2": "value2"}
})
print(result) # Output: Call the API at https://example.com/api with parameters {'key1': 'value1', 'key2': 'value2'}
```
### Custom Caching Mechanism
<Tip>

View File

@@ -143,6 +143,9 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
tool_result = self._execute_tool_and_check_finality(
formatted_answer
)
if self.step_callback:
self.step_callback(tool_result)
formatted_answer.text += f"\nObservation: {tool_result.result}"
formatted_answer.result = tool_result.result
if tool_result.result_as_answer:
@@ -299,7 +302,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
self._i18n.slice("summarizer_system_message"), role="system"
),
self._format_msg(
self._i18n.slice("sumamrize_instruction").format(group=group),
self._i18n.slice("summarize_instruction").format(group=group),
),
],
callbacks=self.callbacks,

View File

@@ -3,6 +3,7 @@ import hashlib
import io
import logging
import os
import shutil
from typing import Any, Dict, List, Optional, Union, cast
import chromadb
@@ -15,6 +16,7 @@ from crewai.knowledge.storage.base_knowledge_storage import BaseKnowledgeStorage
from crewai.utilities import EmbeddingConfigurator
from crewai.utilities.logger import Logger
from crewai.utilities.paths import db_storage_path
from crewai.utilities.constants import KNOWLEDGE_DIRECTORY
@contextlib.contextmanager
@@ -105,15 +107,17 @@ class KnowledgeStorage(BaseKnowledgeStorage):
raise Exception("Failed to create or get collection")
def reset(self):
if self.app:
self.app.reset()
else:
base_path = os.path.join(db_storage_path(), "knowledge")
base_path = os.path.join(db_storage_path(), KNOWLEDGE_DIRECTORY)
if not self.app:
self.app = chromadb.PersistentClient(
path=base_path,
settings=Settings(allow_reset=True),
)
self.app.reset()
self.app.reset()
shutil.rmtree(base_path)
self.app = None
self.collection = None
def save(
self,

View File

@@ -150,9 +150,11 @@ class RAGStorage(BaseRAGStorage):
def reset(self) -> None:
try:
shutil.rmtree(f"{db_storage_path()}/{self.type}")
if self.app:
self.app.reset()
shutil.rmtree(f"{db_storage_path()}/{self.type}")
self.app = None
self.collection = None
except Exception as e:
if "attempt to write a readonly database" in str(e):
# Ignore this specific error

View File

@@ -37,7 +37,7 @@ class UserMemory(Memory):
limit: int = 3,
score_threshold: float = 0.35,
):
results = super().search(
results = self.storage.search(
query=query,
limit=limit,
score_threshold=score_threshold,

View File

@@ -66,6 +66,8 @@ def cache_handler(func):
def crew(func) -> Callable[..., Crew]:
@wraps(func)
def wrapper(self, *args, **kwargs) -> Crew:
instantiated_tasks = []
instantiated_agents = []

View File

@@ -213,4 +213,8 @@ def CrewBase(cls: T) -> T:
callback_functions[callback]() for callback in callbacks
]
# Include base class (qual)name in the wrapper class (qual)name.
WrappedClass.__name__ = CrewBase.__name__ + "(" + cls.__name__ + ")"
WrappedClass.__qualname__ = CrewBase.__qualname__ + "(" + cls.__name__ + ")"
return cast(T, WrappedClass)

View File

@@ -1,11 +1,13 @@
from functools import wraps
def memoize(func):
cache = {}
@wraps(func)
def memoized_func(*args, **kwargs):
key = (args, tuple(kwargs.items()))
if key not in cache:
cache[key] = func(*args, **kwargs)
return cache[key]
memoized_func.__dict__.update(func.__dict__)
return memoized_func

View File

@@ -19,7 +19,7 @@
"human_feedback": "You got human feedback on your work, re-evaluate it and give a new Final Answer when ready.\n {human_feedback}",
"getting_input": "This is the agent's final answer: {final_answer}\n\n",
"summarizer_system_message": "You are a helpful assistant that summarizes text.",
"sumamrize_instruction": "Summarize the following text, make sure to include all the important information: {group}",
"summarize_instruction": "Summarize the following text, make sure to include all the important information: {group}",
"summary": "This is a summary of our conversation so far:\n{merged_summary}",
"manager_request": "Your best answer to your coworker asking you this, accounting for the context shared.",
"formatted_task_instructions": "Ensure your final answer contains only the content in the following format: {output_format}\n\nEnsure the final output does not include any code block markers like ```json or ```python.",