From dfc4255f2f3461d1ecf6239b7f586857a87d0be9 Mon Sep 17 00:00:00 2001 From: Tony Kipkemboi Date: Tue, 27 May 2025 13:08:40 -0400 Subject: [PATCH] docs: Add transparency features for prompts and memory systems (#2902) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs: Fix major memory system documentation issues - Remove misleading deprecation warnings, fix confusing comments, clearly separate three memory approaches, provide accurate examples that match implementation * fix: Correct broken image paths in README - Update crewai_logo.png and asset.png paths to point to docs/images/ directory instead of docs/ directly * docs: Add system prompt transparency and customization guide - Add 'Understanding Default System Instructions' section to address black-box concerns - Document what CrewAI automatically injects into prompts - Provide code examples to inspect complete system prompts - Show 3 methods to override default instructions - Include observability integration examples with Langfuse - Add best practices for production prompt management * docs: Fix implementation accuracy issues in memory documentation - Fix Ollama embedding URL parameter and remove unsupported Cohere input_type parameter * docs: Reference observability docs instead of showing specific tool examples * docs: Reorganize knowledge documentation for better developer experience - Move quickstart examples right after overview for immediate hands-on experience - Create logical learning progression: basics → configuration → advanced → troubleshooting - Add comprehensive agent vs crew knowledge guide with working examples - Consolidate debugging and troubleshooting in dedicated section - Organize best practices by topic in accordion format - Improve content flow from simple concepts to advanced features - Ensure all examples are grounded in actual codebase implementation * docs: enhance custom LLM documentation with comprehensive examples and accurate imports * docs: reorganize observability tools into dedicated section with comprehensive overview and improved navigation * docs: rename how-to section to learn and add comprehensive overview page * docs: finalize documentation reorganization and update navigation labels * docs: enhance README with comprehensive badges, navigation links, and getting started video --- README.md | 86 +- docs/concepts/knowledge.mdx | 1161 ++++++++++------- docs/concepts/memory.mdx | 625 ++++++++- docs/docs.json | 64 +- docs/guides/advanced/customizing-prompts.mdx | 177 ++- docs/how-to/custom-llm.mdx | 646 --------- .../before-and-after-kickoff-hooks.mdx | 0 .../bring-your-own-agent.mdx | 0 docs/{how-to => learn}/coding-agents.mdx | 0 docs/{how-to => learn}/conditional-tasks.mdx | 0 .../{how-to => learn}/create-custom-tools.mdx | 0 docs/learn/custom-llm.mdx | 350 +++++ .../custom-manager-agent.mdx | 0 docs/{how-to => learn}/customizing-agents.mdx | 0 .../dalle-image-generation.mdx | 0 .../force-tool-output-as-result.mdx | 0 .../hierarchical-process.mdx | 0 docs/{how-to => learn}/human-in-the-loop.mdx | 0 .../human-input-on-execution.mdx | 0 docs/{how-to => learn}/kickoff-async.mdx | 0 docs/{how-to => learn}/kickoff-for-each.mdx | 0 docs/{how-to => learn}/llm-connections.mdx | 0 docs/{how-to => learn}/multimodal-agents.mdx | 0 docs/learn/overview.mdx | 158 +++ .../replay-tasks-from-latest-crew-kickoff.mdx | 0 docs/{how-to => learn}/sequential-process.mdx | 0 docs/{how-to => learn}/using-annotations.mdx | 0 .../agentops.mdx} | 0 .../arize-phoenix.mdx} | 0 .../langfuse.mdx} | 0 .../langtrace.mdx} | 0 .../mlflow.mdx} | 0 .../openlit.mdx} | 0 .../opik.mdx} | 0 docs/observability/overview.mdx | 118 ++ .../patronus-evaluation.mdx} | 24 +- .../portkey.mdx} | 0 .../weave.mdx} | 0 docs/tools/ai-ml/overview.mdx | 4 +- 39 files changed, 2241 insertions(+), 1172 deletions(-) delete mode 100644 docs/how-to/custom-llm.mdx rename docs/{how-to => learn}/before-and-after-kickoff-hooks.mdx (100%) rename docs/{how-to => learn}/bring-your-own-agent.mdx (100%) rename docs/{how-to => learn}/coding-agents.mdx (100%) rename docs/{how-to => learn}/conditional-tasks.mdx (100%) rename docs/{how-to => learn}/create-custom-tools.mdx (100%) create mode 100644 docs/learn/custom-llm.mdx rename docs/{how-to => learn}/custom-manager-agent.mdx (100%) rename docs/{how-to => learn}/customizing-agents.mdx (100%) rename docs/{how-to => learn}/dalle-image-generation.mdx (100%) rename docs/{how-to => learn}/force-tool-output-as-result.mdx (100%) rename docs/{how-to => learn}/hierarchical-process.mdx (100%) rename docs/{how-to => learn}/human-in-the-loop.mdx (100%) rename docs/{how-to => learn}/human-input-on-execution.mdx (100%) rename docs/{how-to => learn}/kickoff-async.mdx (100%) rename docs/{how-to => learn}/kickoff-for-each.mdx (100%) rename docs/{how-to => learn}/llm-connections.mdx (100%) rename docs/{how-to => learn}/multimodal-agents.mdx (100%) create mode 100644 docs/learn/overview.mdx rename docs/{how-to => learn}/replay-tasks-from-latest-crew-kickoff.mdx (100%) rename docs/{how-to => learn}/sequential-process.mdx (100%) rename docs/{how-to => learn}/using-annotations.mdx (100%) rename docs/{how-to/agentops-observability.mdx => observability/agentops.mdx} (100%) rename docs/{how-to/arize-phoenix-observability.mdx => observability/arize-phoenix.mdx} (100%) rename docs/{how-to/langfuse-observability.mdx => observability/langfuse.mdx} (100%) rename docs/{how-to/langtrace-observability.mdx => observability/langtrace.mdx} (100%) rename docs/{how-to/mlflow-observability.mdx => observability/mlflow.mdx} (100%) rename docs/{how-to/openlit-observability.mdx => observability/openlit.mdx} (100%) rename docs/{how-to/opik-observability.mdx => observability/opik.mdx} (100%) create mode 100644 docs/observability/overview.mdx rename docs/{tools/ai-ml/patronustools.mdx => observability/patronus-evaluation.mdx} (85%) rename docs/{how-to/portkey-observability.mdx => observability/portkey.mdx} (100%) rename docs/{how-to/weave-integration.mdx => observability/weave.mdx} (100%) diff --git a/README.md b/README.md index 279c35017..ad8f8634a 100644 --- a/README.md +++ b/README.md @@ -1,27 +1,70 @@ -
+

+ + Open source Multi-AI Agent orchestration framework + +

+

+ + crewAIInc%2FcrewAI | Trendshift + +

-![Logo of CrewAI](./docs/images/crewai_logo.png) +

+ Homepage + · + Docs + · + Start Cloud Trial + · + Blog + · + Forum +

-
+

+ + GitHub Repo stars + + + GitHub forks + + + GitHub issues + + + GitHub pull requests + + + License: MIT + +

+ +

+ + PyPI version + + + PyPI downloads + + + Twitter Follow + +

### Fast and Flexible Multi-Agent Automation Framework -CrewAI is a lean, lightning-fast Python framework built entirely from -scratch—completely **independent of LangChain or other agent frameworks**. -It empowers developers with both high-level simplicity and precise low-level -control, ideal for creating autonomous AI agents tailored to any scenario. +> CrewAI is a lean, lightning-fast Python framework built entirely from scratch—completely **independent of LangChain or other agent frameworks**. +> It empowers developers with both high-level simplicity and precise low-level control, ideal for creating autonomous AI agents tailored to any scenario. - **CrewAI Crews**: Optimize for autonomy and collaborative intelligence. - **CrewAI Flows**: Enable granular, event-driven control, single LLM calls for precise task orchestration and supports Crews natively -With over 100,000 developers certified through our community courses at -[learn.crewai.com](https://learn.crewai.com), CrewAI is rapidly becoming the +With over 100,000 developers certified through our community courses at [learn.crewai.com](https://learn.crewai.com), CrewAI is rapidly becoming the standard for enterprise-ready AI automation. # CrewAI Enterprise Suite -CrewAI Enterprise Suite is a comprehensive bundle tailored for organizations -that require secure, scalable, and easy-to-manage agent-driven automation. +CrewAI Enterprise Suite is a comprehensive bundle tailored for organizations that require secure, scalable, and easy-to-manage agent-driven automation. You can try one part of the suite the [Crew Control Plane for free](https://app.crewai.com) @@ -35,21 +78,9 @@ You can try one part of the suite the [Crew Control Plane for free](https://app. - **24/7 Support**: Dedicated enterprise support to ensure uninterrupted operation and quick resolution of issues. - **On-premise and Cloud Deployment Options**: Deploy CrewAI Enterprise on-premise or in the cloud, depending on your security and compliance requirements. -CrewAI Enterprise is designed for enterprises seeking a powerful, -reliable solution to transform complex business processes into efficient, +CrewAI Enterprise is designed for enterprises seeking a powerful, reliable solution to transform complex business processes into efficient, intelligent automations. -

- -[Homepage](https://www.crewai.com/) | [Documentation](https://docs.crewai.com/) | [Chat with Docs](https://chatg.pt/DWjSBZn) | [Discourse](https://community.crewai.com) - -

- -[![GitHub Repo stars](https://img.shields.io/github/stars/joaomdmoura/crewAI)](https://github.com/crewAIInc/crewAI) -[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT) - - - ## Table of contents - [Why CrewAI?](#why-crewai) @@ -88,7 +119,12 @@ CrewAI empowers developers and enterprises to confidently build intelligent auto ## Getting Started -### Learning Resources +Setup and run your first CrewAI agents by following this tutorial. + +[![CrewAI Getting Started Tutorial](https://img.youtube.com/vi/-kSOTtYzgEw/hqdefault.jpg)](https://www.youtube.com/watch?v=-kSOTtYzgEw "CrewAI Getting Started Tutorial") + +### + Learning Resources Learn CrewAI through our comprehensive courses: diff --git a/docs/concepts/knowledge.mdx b/docs/concepts/knowledge.mdx index 774847abc..022d090d9 100644 --- a/docs/concepts/knowledge.mdx +++ b/docs/concepts/knowledge.mdx @@ -17,6 +17,103 @@ Think of it as giving your agents a reference library they can consult while wor - Ground responses in factual information +## Quickstart Examples + + +For file-based Knowledge Sources, make sure to place your files in a `knowledge` directory at the root of your project. +Also, use relative paths from the `knowledge` directory when creating the source. + + +### Basic String Knowledge Example + +```python Code +from crewai import Agent, Task, Crew, Process, LLM +from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource + +# Create a knowledge source +content = "Users name is John. He is 30 years old and lives in San Francisco." +string_source = StringKnowledgeSource(content=content) + +# Create an LLM with a temperature of 0 to ensure deterministic outputs +llm = LLM(model="gpt-4o-mini", temperature=0) + +# Create an agent with the knowledge store +agent = Agent( + role="About User", + goal="You know everything about the user.", + backstory="You are a master at understanding people and their preferences.", + verbose=True, + allow_delegation=False, + llm=llm, +) + +task = Task( + description="Answer the following questions about the user: {question}", + expected_output="An answer to the question.", + agent=agent, +) + +crew = Crew( + agents=[agent], + tasks=[task], + verbose=True, + process=Process.sequential, + knowledge_sources=[string_source], # Enable knowledge by adding the sources here +) + +result = crew.kickoff(inputs={"question": "What city does John live in and how old is he?"}) +``` + +### Web Content Knowledge Example + + + You need to install `docling` for the following example to work: `uv add docling` + + +```python Code +from crewai import LLM, Agent, Crew, Process, Task +from crewai.knowledge.source.crew_docling_source import CrewDoclingSource + +# Create a knowledge source from web content +content_source = CrewDoclingSource( + file_paths=[ + "https://lilianweng.github.io/posts/2024-11-28-reward-hacking", + "https://lilianweng.github.io/posts/2024-07-07-hallucination", + ], +) + +# Create an LLM with a temperature of 0 to ensure deterministic outputs +llm = LLM(model="gpt-4o-mini", temperature=0) + +# Create an agent with the knowledge store +agent = Agent( + role="About papers", + goal="You know everything about the papers.", + backstory="You are a master at understanding papers and their content.", + verbose=True, + allow_delegation=False, + llm=llm, +) + +task = Task( + description="Answer the following questions about the papers: {question}", + expected_output="An answer to the question.", + agent=agent, +) + +crew = Crew( + agents=[agent], + tasks=[task], + verbose=True, + process=Process.sequential, + knowledge_sources=[content_source], +) + +result = crew.kickoff( + inputs={"question": "What is the reward hacking paper about? Be sure to provide sources."} +) +``` + ## Supported Knowledge Sources CrewAI supports various types of knowledge sources out of the box: @@ -34,17 +131,235 @@ CrewAI supports various types of knowledge sources out of the box: -## Supported Knowledge Parameters +### Text File Knowledge Source +```python +from crewai.knowledge.source.text_file_knowledge_source import TextFileKnowledgeSource - - List of knowledge sources that provide content to be stored and queried. Can include PDF, CSV, Excel, JSON, text files, or string content. - - - Name of the collection where the knowledge will be stored. Used to identify different sets of knowledge. Defaults to \"knowledge\" if not provided. - - -Custom storage configuration for managing how the knowledge is stored and retrieved. If not provided, a default storage will be created. - +text_source = TextFileKnowledgeSource( + file_paths=["document.txt", "another.txt"] +) +``` + +### PDF Knowledge Source +```python +from crewai.knowledge.source.pdf_knowledge_source import PDFKnowledgeSource + +pdf_source = PDFKnowledgeSource( + file_paths=["document.pdf", "another.pdf"] +) +``` + +### CSV Knowledge Source +```python +from crewai.knowledge.source.csv_knowledge_source import CSVKnowledgeSource + +csv_source = CSVKnowledgeSource( + file_paths=["data.csv"] +) +``` + +### Excel Knowledge Source +```python +from crewai.knowledge.source.excel_knowledge_source import ExcelKnowledgeSource + +excel_source = ExcelKnowledgeSource( + file_paths=["spreadsheet.xlsx"] +) +``` + +### JSON Knowledge Source +```python +from crewai.knowledge.source.json_knowledge_source import JSONKnowledgeSource + +json_source = JSONKnowledgeSource( + file_paths=["data.json"] +) +``` + + + Please ensure that you create the ./knowledge folder. All source files (e.g., .txt, .pdf, .xlsx, .json) should be placed in this folder for centralized management. + + +## Agent vs Crew Knowledge: Complete Guide + + +**Understanding Knowledge Levels**: CrewAI supports knowledge at both agent and crew levels. This section clarifies exactly how each works, when they're initialized, and addresses common misconceptions about dependencies. + + +### How Knowledge Initialization Actually Works + +Here's exactly what happens when you use knowledge: + +#### Agent-Level Knowledge (Independent) +```python +from crewai import Agent, Task, Crew +from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource + +# Agent with its own knowledge - NO crew knowledge needed +specialist_knowledge = StringKnowledgeSource( + content="Specialized technical information for this agent only" +) + +specialist_agent = Agent( + role="Technical Specialist", + goal="Provide technical expertise", + backstory="Expert in specialized technical domains", + knowledge_sources=[specialist_knowledge] # Agent-specific knowledge +) + +task = Task( + description="Answer technical questions", + agent=specialist_agent, + expected_output="Technical answer" +) + +# No crew-level knowledge required +crew = Crew( + agents=[specialist_agent], + tasks=[task] +) + +result = crew.kickoff() # Agent knowledge works independently +``` + +#### What Happens During `crew.kickoff()` + +When you call `crew.kickoff()`, here's the exact sequence: + +```python +# During kickoff +for agent in self.agents: + agent.crew = self # Agent gets reference to crew + agent.set_knowledge(crew_embedder=self.embedder) # Agent knowledge initialized + agent.create_agent_executor() +``` + +#### Storage Independence + +Each knowledge level uses independent storage collections: + +```python +# Agent knowledge storage +agent_collection_name = agent.role # e.g., "Technical Specialist" + +# Crew knowledge storage +crew_collection_name = "crew" + +# Both stored in same ChromaDB instance but different collections +# Path: ~/.local/share/CrewAI/{project}/knowledge/ +# ├── crew/ # Crew knowledge collection +# ├── Technical Specialist/ # Agent knowledge collection +# └── Another Agent Role/ # Another agent's collection +``` + +### Complete Working Examples + +#### Example 1: Agent-Only Knowledge +```python +from crewai import Agent, Task, Crew +from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource + +# Agent-specific knowledge +agent_knowledge = StringKnowledgeSource( + content="Agent-specific information that only this agent needs" +) + +agent = Agent( + role="Specialist", + goal="Use specialized knowledge", + backstory="Expert with specific knowledge", + knowledge_sources=[agent_knowledge], + embedder={ # Agent can have its own embedder + "provider": "openai", + "config": {"model": "text-embedding-3-small"} + } +) + +task = Task( + description="Answer using your specialized knowledge", + agent=agent, + expected_output="Answer based on agent knowledge" +) + +# No crew knowledge needed +crew = Crew(agents=[agent], tasks=[task]) +result = crew.kickoff() # Works perfectly +``` + +#### Example 2: Both Agent and Crew Knowledge +```python +# Crew-wide knowledge (shared by all agents) +crew_knowledge = StringKnowledgeSource( + content="Company policies and general information for all agents" +) + +# Agent-specific knowledge +specialist_knowledge = StringKnowledgeSource( + content="Technical specifications only the specialist needs" +) + +specialist = Agent( + role="Technical Specialist", + goal="Provide technical expertise", + backstory="Technical expert", + knowledge_sources=[specialist_knowledge] # Agent-specific +) + +generalist = Agent( + role="General Assistant", + goal="Provide general assistance", + backstory="General helper" + # No agent-specific knowledge +) + +crew = Crew( + agents=[specialist, generalist], + tasks=[...], + knowledge_sources=[crew_knowledge] # Crew-wide knowledge +) + +# Result: +# - specialist gets: crew_knowledge + specialist_knowledge +# - generalist gets: crew_knowledge only +``` + +#### Example 3: Multiple Agents with Different Knowledge +```python +# Different knowledge for different agents +sales_knowledge = StringKnowledgeSource(content="Sales procedures and pricing") +tech_knowledge = StringKnowledgeSource(content="Technical documentation") +support_knowledge = StringKnowledgeSource(content="Support procedures") + +sales_agent = Agent( + role="Sales Representative", + knowledge_sources=[sales_knowledge], + embedder={"provider": "openai", "config": {"model": "text-embedding-3-small"}} +) + +tech_agent = Agent( + role="Technical Expert", + knowledge_sources=[tech_knowledge], + embedder={"provider": "ollama", "config": {"model": "mxbai-embed-large"}} +) + +support_agent = Agent( + role="Support Specialist", + knowledge_sources=[support_knowledge] + # Will use crew embedder as fallback +) + +crew = Crew( + agents=[sales_agent, tech_agent, support_agent], + tasks=[...], + embedder={ # Fallback embedder for agents without their own + "provider": "google", + "config": {"model": "text-embedding-004"} + } +) + +# Each agent gets only their specific knowledge +# Each can use different embedding providers +``` Unlike retrieval from a vector database using a tool, agents preloaded with knowledge will not need a retrieval persona or task. @@ -55,110 +370,6 @@ Crew level knowledge sources will be used by **all agents** in the crew. Agent level knowledge sources will be used by the **specific agent** that is preloaded with the knowledge. -## Quickstart Example - - -For file-Based Knowledge Sources, make sure to place your files in a `knowledge` directory at the root of your project. -Also, use relative paths from the `knowledge` directory when creating the source. - - -Here's an example using string-based knowledge: - -```python Code -from crewai import Agent, Task, Crew, Process, LLM -from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource - -# Create a knowledge source -content = "Users name is John. He is 30 years old and lives in San Francisco." -string_source = StringKnowledgeSource( - content=content, -) - -# Create an LLM with a temperature of 0 to ensure deterministic outputs -llm = LLM(model="gpt-4o-mini", temperature=0) - -# Create an agent with the knowledge store -agent = Agent( - role="About User", - goal="You know everything about the user.", - backstory="""You are a master at understanding people and their preferences.""", - verbose=True, - allow_delegation=False, - llm=llm, -) -task = Task( - description="Answer the following questions about the user: {question}", - expected_output="An answer to the question.", - agent=agent, -) - -crew = Crew( - agents=[agent], - tasks=[task], - verbose=True, - process=Process.sequential, - knowledge_sources=[string_source], # Enable knowledge by adding the sources here. You can also add more sources to the sources list. -) - -result = crew.kickoff(inputs={"question": "What city does John live in and how old is he?"}) -``` - - -Here's another example with the `CrewDoclingSource`. The CrewDoclingSource is actually quite versatile and can handle multiple file formats including MD, PDF, DOCX, HTML, and more. - - - You need to install `docling` for the following example to work: `uv add docling` - - - - -```python Code -from crewai import LLM, Agent, Crew, Process, Task -from crewai.knowledge.source.crew_docling_source import CrewDoclingSource - -# Create a knowledge source -content_source = CrewDoclingSource( - file_paths=[ - "https://lilianweng.github.io/posts/2024-11-28-reward-hacking", - "https://lilianweng.github.io/posts/2024-07-07-hallucination", - ], -) - -# Create an LLM with a temperature of 0 to ensure deterministic outputs -llm = LLM(model="gpt-4o-mini", temperature=0) - -# Create an agent with the knowledge store -agent = Agent( - role="About papers", - goal="You know everything about the papers.", - backstory="""You are a master at understanding papers and their content.""", - verbose=True, - allow_delegation=False, - llm=llm, -) -task = Task( - description="Answer the following questions about the papers: {question}", - expected_output="An answer to the question.", - agent=agent, -) - -crew = Crew( - agents=[agent], - tasks=[task], - verbose=True, - process=Process.sequential, - knowledge_sources=[ - content_source - ], # Enable knowledge by adding the sources here. You can also add more sources to the sources list. -) - -result = crew.kickoff( - inputs={ - "question": "What is the reward hacking paper about? Be sure to provide sources." - } -) -``` - ## Knowledge Configuration You can configure the knowledge configuration for the crew or agent. @@ -179,239 +390,231 @@ agent = Agent( `score_threshold`: is the minimum score for a document to be considered relevant. Default is 0.35. -## More Examples +## Supported Knowledge Parameters -Here are examples of how to use different types of knowledge sources: + + List of knowledge sources that provide content to be stored and queried. Can include PDF, CSV, Excel, JSON, text files, or string content. + + + Name of the collection where the knowledge will be stored. Used to identify different sets of knowledge. Defaults to \"knowledge\" if not provided. + + +Custom storage configuration for managing how the knowledge is stored and retrieved. If not provided, a default storage will be created. + -Note: Please ensure that you create the ./knowldge folder. All source files (e.g., .txt, .pdf, .xlsx, .json) should be placed in this folder for centralized management. +## Knowledge Storage Transparency -### Text File Knowledge Source -```python -from crewai.knowledge.source.text_file_knowledge_source import TextFileKnowledgeSource + +**Understanding Knowledge Storage**: CrewAI automatically stores knowledge sources in platform-specific directories using ChromaDB for vector storage. Understanding these locations and defaults helps with production deployments, debugging, and storage management. + -# Create a text file knowledge source -text_source = TextFileKnowledgeSource( - file_paths=["document.txt", "another.txt"] -) +### Where CrewAI Stores Knowledge Files -# Create crew with text file source on agents or crew level -agent = Agent( - ... - knowledge_sources=[text_source] -) +By default, CrewAI uses the same storage system as memory, storing knowledge in platform-specific directories: -crew = Crew( - ... - knowledge_sources=[text_source] -) +#### Default Storage Locations by Platform + +**macOS:** +``` +~/Library/Application Support/CrewAI/{project_name}/ +└── knowledge/ # Knowledge ChromaDB files + ├── chroma.sqlite3 # ChromaDB metadata + ├── {collection_id}/ # Vector embeddings + └── knowledge_{collection}/ # Named collections ``` -### PDF Knowledge Source -```python -from crewai.knowledge.source.pdf_knowledge_source import PDFKnowledgeSource - -# Create a PDF knowledge source -pdf_source = PDFKnowledgeSource( - file_paths=["document.pdf", "another.pdf"] -) - -# Create crew with PDF knowledge source on agents or crew level -agent = Agent( - ... - knowledge_sources=[pdf_source] -) - -crew = Crew( - ... - knowledge_sources=[pdf_source] -) +**Linux:** +``` +~/.local/share/CrewAI/{project_name}/ +└── knowledge/ + ├── chroma.sqlite3 + ├── {collection_id}/ + └── knowledge_{collection}/ ``` -### CSV Knowledge Source -```python -from crewai.knowledge.source.csv_knowledge_source import CSVKnowledgeSource - -# Create a CSV knowledge source -csv_source = CSVKnowledgeSource( - file_paths=["data.csv"] -) - -# Create crew with CSV knowledge source or on agent level -agent = Agent( - ... - knowledge_sources=[csv_source] -) - -crew = Crew( - ... - knowledge_sources=[csv_source] -) +**Windows:** +``` +C:\Users\{username}\AppData\Local\CrewAI\{project_name}\ +└── knowledge\ + ├── chroma.sqlite3 + ├── {collection_id}\ + └── knowledge_{collection}\ ``` -### Excel Knowledge Source -```python -from crewai.knowledge.source.excel_knowledge_source import ExcelKnowledgeSource +### Finding Your Knowledge Storage Location -# Create an Excel knowledge source -excel_source = ExcelKnowledgeSource( - file_paths=["spreadsheet.xlsx"] -) - -# Create crew with Excel knowledge source on agents or crew level -agent = Agent( - ... - knowledge_sources=[excel_source] -) - -crew = Crew( - ... - knowledge_sources=[excel_source] -) -``` - -### JSON Knowledge Source -```python -from crewai.knowledge.source.json_knowledge_source import JSONKnowledgeSource - -# Create a JSON knowledge source -json_source = JSONKnowledgeSource( - file_paths=["data.json"] -) - -# Create crew with JSON knowledge source on agents or crew level -agent = Agent( - ... - knowledge_sources=[json_source] -) - -crew = Crew( - ... - knowledge_sources=[json_source] -) -``` - -## Knowledge Configuration - -### Chunking Configuration - -Knowledge sources automatically chunk content for better processing. -You can configure chunking behavior in your knowledge sources: +To see exactly where CrewAI is storing your knowledge files: ```python -from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource - -source = StringKnowledgeSource( - content="Your content here", - chunk_size=4000, # Maximum size of each chunk (default: 4000) - chunk_overlap=200 # Overlap between chunks (default: 200) -) -``` - -The chunking configuration helps in: -- Breaking down large documents into manageable pieces -- Maintaining context through chunk overlap -- Optimizing retrieval accuracy - -### Embeddings Configuration - -You can also configure the embedder for the knowledge store. -This is useful if you want to use a different embedder for the knowledge store than the one used for the agents. -The `embedder` parameter supports various embedding model providers that include: -- `openai`: OpenAI's embedding models -- `google`: Google's text embedding models -- `azure`: Azure OpenAI embeddings -- `ollama`: Local embeddings with Ollama -- `vertexai`: Google Cloud VertexAI embeddings -- `cohere`: Cohere's embedding models -- `voyageai`: VoyageAI's embedding models -- `bedrock`: AWS Bedrock embeddings -- `huggingface`: Hugging Face models -- `watson`: IBM Watson embeddings - -Here's an example of how to configure the embedder for the knowledge store using Google's `text-embedding-004` model: - -```python Example -from crewai import Agent, Task, Crew, Process, LLM -from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource +from crewai.utilities.paths import db_storage_path import os -# Get the GEMINI API key -GEMINI_API_KEY = os.environ.get("GEMINI_API_KEY") +# Get the knowledge storage path +knowledge_path = os.path.join(db_storage_path(), "knowledge") +print(f"Knowledge storage location: {knowledge_path}") -# Create a knowledge source -content = "Users name is John. He is 30 years old and lives in San Francisco." -string_source = StringKnowledgeSource( - content=content, +# List knowledge collections and files +if os.path.exists(knowledge_path): + print("\nKnowledge storage contents:") + for item in os.listdir(knowledge_path): + item_path = os.path.join(knowledge_path, item) + if os.path.isdir(item_path): + print(f"📁 Collection: {item}/") + # Show collection contents + try: + for subitem in os.listdir(item_path): + print(f" └── {subitem}") + except PermissionError: + print(f" └── (permission denied)") + else: + print(f"📄 {item}") +else: + print("No knowledge storage found yet.") +``` + +### Controlling Knowledge Storage Locations + +#### Option 1: Environment Variable (Recommended) +```python +import os +from crewai import Crew + +# Set custom storage location for all CrewAI data +os.environ["CREWAI_STORAGE_DIR"] = "./my_project_storage" + +# All knowledge will now be stored in ./my_project_storage/knowledge/ +crew = Crew( + agents=[...], + tasks=[...], + knowledge_sources=[...] ) +``` -# Create an LLM with a temperature of 0 to ensure deterministic outputs -gemini_llm = LLM( - model="gemini/gemini-1.5-pro-002", - api_key=GEMINI_API_KEY, - temperature=0, -) +#### Option 2: Custom Knowledge Storage +```python +from crewai.knowledge.storage.knowledge_storage import KnowledgeStorage +from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource -# Create an agent with the knowledge store -agent = Agent( - role="About User", - goal="You know everything about the user.", - backstory="""You are a master at understanding people and their preferences.""", - verbose=True, - allow_delegation=False, - llm=gemini_llm, +# Create custom storage with specific embedder +custom_storage = KnowledgeStorage( embedder={ - "provider": "google", - "config": { - "model": "models/text-embedding-004", - "api_key": GEMINI_API_KEY, - } - } + "provider": "ollama", + "config": {"model": "mxbai-embed-large"} + }, + collection_name="my_custom_knowledge" ) -task = Task( - description="Answer the following questions about the user: {question}", - expected_output="An answer to the question.", - agent=agent, +# Use with knowledge sources +knowledge_source = StringKnowledgeSource( + content="Your knowledge content here" ) +knowledge_source.storage = custom_storage +``` + +#### Option 3: Project-Specific Knowledge Storage +```python +import os +from pathlib import Path + +# Store knowledge in project directory +project_root = Path(__file__).parent +knowledge_dir = project_root / "knowledge_storage" + +os.environ["CREWAI_STORAGE_DIR"] = str(knowledge_dir) + +# Now all knowledge will be stored in your project directory +``` + +### Default Embedding Provider Behavior + + +**Default Embedding Provider**: CrewAI defaults to OpenAI embeddings (`text-embedding-3-small`) for knowledge storage, even when using different LLM providers. You can easily customize this to match your setup. + + +#### Understanding Default Behavior +```python +from crewai import Agent, Crew, LLM +from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource + +# When using Claude as your LLM... +agent = Agent( + role="Researcher", + goal="Research topics", + backstory="Expert researcher", + llm=LLM(provider="anthropic", model="claude-3-sonnet") # Using Claude +) + +# CrewAI will still use OpenAI embeddings by default for knowledge +# This ensures consistency but may not match your LLM provider preference +knowledge_source = StringKnowledgeSource(content="Research data...") crew = Crew( agents=[agent], - tasks=[task], - verbose=True, - process=Process.sequential, - knowledge_sources=[string_source], + tasks=[...], + knowledge_sources=[knowledge_source] + # Default: Uses OpenAI embeddings even with Claude LLM +) +``` + +#### Customizing Knowledge Embedding Providers +```python +# Option 1: Use Voyage AI (recommended by Anthropic for Claude users) +crew = Crew( + agents=[agent], + tasks=[...], + knowledge_sources=[knowledge_source], embedder={ - "provider": "google", + "provider": "voyageai", # Recommended for Claude users "config": { - "model": "models/text-embedding-004", - "api_key": GEMINI_API_KEY, + "api_key": "your-voyage-api-key", + "model": "voyage-3" # or "voyage-3-large" for best quality } } ) -result = crew.kickoff(inputs={"question": "What city does John live in and how old is he?"}) -``` -```text Output -# Agent: About User -## Task: Answer the following questions about the user: What city does John live in and how old is he? +# Option 2: Use local embeddings (no external API calls) +crew = Crew( + agents=[agent], + tasks=[...], + knowledge_sources=[knowledge_source], + embedder={ + "provider": "ollama", + "config": { + "model": "mxbai-embed-large", + "url": "http://localhost:11434/api/embeddings" + } + } +) -# Agent: About User -## Final Answer: -John is 30 years old and lives in San Francisco. +# Option 3: Agent-level embedding customization +agent = Agent( + role="Researcher", + goal="Research topics", + backstory="Expert researcher", + knowledge_sources=[knowledge_source], + embedder={ + "provider": "google", + "config": { + "model": "models/text-embedding-004", + "api_key": "your-google-key" + } + } +) ``` - -## Query Rewriting +## Advanced Features + +### Query Rewriting CrewAI implements an intelligent query rewriting mechanism to optimize knowledge retrieval. When an agent needs to search through knowledge sources, the raw task prompt is automatically transformed into a more effective search query. -### How Query Rewriting Works +#### How Query Rewriting Works 1. When an agent executes a task with knowledge sources available, the `_get_knowledge_search_query` method is triggered 2. The agent's LLM is used to transform the original task prompt into an optimized search query 3. This optimized query is then used to retrieve relevant information from knowledge sources -### Benefits of Query Rewriting +#### Benefits of Query Rewriting @@ -422,24 +625,27 @@ CrewAI implements an intelligent query rewriting mechanism to optimize knowledge -### Implementation Details +#### Example -Query rewriting happens transparently using a system prompt that instructs the LLM to: +```python +# Original task prompt +task_prompt = "Answer the following questions about the user's favorite movies: What movie did John watch last week? Format your answer in JSON." -- Focus on key words of the intended task -- Make the query more specific and context-aware -- Remove irrelevant content like output format instructions -- Generate only the rewritten query without preamble or postamble +# Behind the scenes, this might be rewritten as: +rewritten_query = "What movies did John watch last week?" +``` + +The rewritten query is more focused on the core information need and removes irrelevant instructions about output formatting. This mechanism is fully automatic and requires no configuration from users. The agent's LLM is used to perform the query rewriting, so using a more capable LLM can improve the quality of rewritten queries. -## Knowledge Events +### Knowledge Events CrewAI emits events during the knowledge retrieval process that you can listen for using the event system. These events allow you to monitor, debug, and analyze how knowledge is being retrieved and used by your agents. -### Available Knowledge Events +#### Available Knowledge Events - **KnowledgeRetrievalStartedEvent**: Emitted when an agent starts retrieving knowledge from sources - **KnowledgeRetrievalCompletedEvent**: Emitted when knowledge retrieval is completed, including the query used and the retrieved content @@ -448,7 +654,7 @@ CrewAI emits events during the knowledge retrieval process that you can listen f - **KnowledgeQueryFailedEvent**: Emitted when a query to knowledge sources fails - **KnowledgeSearchQueryFailedEvent**: Emitted when a search query fails -### Example: Monitoring Knowledge Retrieval +#### Example: Monitoring Knowledge Retrieval ```python from crewai.utilities.events import ( @@ -475,88 +681,7 @@ knowledge_monitor = KnowledgeMonitorListener() For more information on using events, see the [Event Listeners](https://docs.crewai.com/concepts/event-listener) documentation. -### Example - -```python -# Original task prompt -task_prompt = "Answer the following questions about the user's favorite movies: What movie did John watch last week? Format your answer in JSON." - -# Behind the scenes, this might be rewritten as: -rewritten_query = "What movies did John watch last week?" -``` - -The rewritten query is more focused on the core information need and removes irrelevant instructions about output formatting. - -## Clearing Knowledge - -If you need to clear the knowledge stored in CrewAI, you can use the `crewai reset-memories` command with the `--knowledge` option. - -```bash Command -crewai reset-memories --knowledge -``` - -This is useful when you've updated your knowledge sources and want to ensure that the agents are using the most recent information. - -## Agent-Specific Knowledge - -While knowledge can be provided at the crew level using `crew.knowledge_sources`, individual agents can also have their own knowledge sources using the `knowledge_sources` parameter: - -```python Code -from crewai import Agent, Task, Crew -from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource - -# Create agent-specific knowledge about a product -product_specs = StringKnowledgeSource( - content="""The XPS 13 laptop features: - - 13.4-inch 4K display - - Intel Core i7 processor - - 16GB RAM - - 512GB SSD storage - - 12-hour battery life""", - metadata={"category": "product_specs"} -) - -# Create a support agent with product knowledge -support_agent = Agent( - role="Technical Support Specialist", - goal="Provide accurate product information and support.", - backstory="You are an expert on our laptop products and specifications.", - knowledge_sources=[product_specs] # Agent-specific knowledge -) - -# Create a task that requires product knowledge -support_task = Task( - description="Answer this customer question: {question}", - agent=support_agent -) - -# Create and run the crew -crew = Crew( - agents=[support_agent], - tasks=[support_task] -) - -# Get answer about the laptop's specifications -result = crew.kickoff( - inputs={"question": "What is the storage capacity of the XPS 13?"} -) - -# Resetting the agent specific knowledge via crew object -crew.reset_memories(command_type = 'agent_knowledge') - -# Resetting the agent specific knowledge via CLI -crewai reset-memories --agent-knowledge -crewai reset-memories -akn -``` - - - Benefits of agent-specific knowledge: - - Give agents specialized information for their roles - - Maintain separation of concerns between agents - - Combine with crew-level knowledge for layered information access - - -## Custom Knowledge Sources +### Custom Knowledge Sources CrewAI allows you to create custom knowledge sources for any type of data by extending the `BaseKnowledgeSource` class. Let's create a practical example that fetches and processes space news articles. @@ -679,63 +804,193 @@ The latest developments in space exploration, based on recent space news article ``` -#### Key Components Explained -1. **Custom Knowledge Source (`SpaceNewsKnowledgeSource`)**: +## Debugging and Troubleshooting - - Extends `BaseKnowledgeSource` for integration with CrewAI - - Configurable API endpoint and article limit - - Implements three key methods: - - `load_content()`: Fetches articles from the API - - `_format_articles()`: Structures the articles into readable text - - `add()`: Processes and stores the content - -2. **Agent Configuration**: - - - Specialized role as a Space News Analyst - - Uses the knowledge source to access space news - -3. **Task Setup**: - - - Takes a user question as input through `{user_question}` - - Designed to provide detailed answers based on the knowledge source - -4. **Crew Orchestration**: - - Manages the workflow between agent and task - - Handles input/output through the kickoff method - -This example demonstrates how to: - -- Create a custom knowledge source that fetches real-time data -- Process and format external data for AI consumption -- Use the knowledge source to answer specific user questions -- Integrate everything seamlessly with CrewAI's agent system - -#### About the Spaceflight News API - -The example uses the [Spaceflight News API](https://api.spaceflightnewsapi.net/v4/docs/), which: - -- Provides free access to space-related news articles -- Requires no authentication -- Returns structured data about space news -- Supports pagination and filtering - -You can customize the API query by modifying the endpoint URL: +### Debugging Knowledge Issues +#### Check Agent Knowledge Initialization ```python -# Fetch more articles -recent_news = SpaceNewsKnowledgeSource( - api_endpoint="https://api.spaceflightnewsapi.net/v4/articles", - limit=20, # Increase the number of articles +from crewai import Agent, Crew, Task +from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource + +knowledge_source = StringKnowledgeSource(content="Test knowledge") + +agent = Agent( + role="Test Agent", + goal="Test knowledge", + backstory="Testing", + knowledge_sources=[knowledge_source] ) -# Add search parameters -recent_news = SpaceNewsKnowledgeSource( - api_endpoint="https://api.spaceflightnewsapi.net/v4/articles?search=NASA", # Search for NASA news - limit=10, +crew = Crew(agents=[agent], tasks=[Task(...)]) + +# Before kickoff - knowledge not initialized +print(f"Before kickoff - Agent knowledge: {getattr(agent, 'knowledge', None)}") + +crew.kickoff() + +# After kickoff - knowledge initialized +print(f"After kickoff - Agent knowledge: {agent.knowledge}") +print(f"Agent knowledge collection: {agent.knowledge.storage.collection_name}") +print(f"Number of sources: {len(agent.knowledge.sources)}") +``` + +#### Verify Knowledge Storage Locations +```python +import os +from crewai.utilities.paths import db_storage_path + +# Check storage structure +storage_path = db_storage_path() +knowledge_path = os.path.join(storage_path, "knowledge") + +if os.path.exists(knowledge_path): + print("Knowledge collections found:") + for collection in os.listdir(knowledge_path): + collection_path = os.path.join(knowledge_path, collection) + if os.path.isdir(collection_path): + print(f" - {collection}/") + # Show collection contents + for item in os.listdir(collection_path): + print(f" └── {item}") +``` + +#### Test Knowledge Retrieval +```python +# Test agent knowledge retrieval +if hasattr(agent, 'knowledge') and agent.knowledge: + test_query = ["test query"] + results = agent.knowledge.query(test_query) + print(f"Agent knowledge results: {len(results)} documents found") + + # Test crew knowledge retrieval (if exists) + if hasattr(crew, 'knowledge') and crew.knowledge: + crew_results = crew.query_knowledge(test_query) + print(f"Crew knowledge results: {len(crew_results)} documents found") +``` + +#### Inspect Knowledge Collections +```python +import chromadb +from crewai.utilities.paths import db_storage_path +import os + +# Connect to CrewAI's knowledge ChromaDB +knowledge_path = os.path.join(db_storage_path(), "knowledge") + +if os.path.exists(knowledge_path): + client = chromadb.PersistentClient(path=knowledge_path) + collections = client.list_collections() + + print("Knowledge Collections:") + for collection in collections: + print(f" - {collection.name}: {collection.count()} documents") + + # Sample a few documents to verify content + if collection.count() > 0: + sample = collection.peek(limit=2) + print(f" Sample content: {sample['documents'][0][:100]}...") +else: + print("No knowledge storage found") +``` + +#### Check Knowledge Processing +```python +from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource + +# Create a test knowledge source +test_source = StringKnowledgeSource( + content="Test knowledge content for debugging", + chunk_size=100, # Small chunks for testing + chunk_overlap=20 +) + +# Check chunking behavior +print(f"Original content length: {len(test_source.content)}") +print(f"Chunk size: {test_source.chunk_size}") +print(f"Chunk overlap: {test_source.chunk_overlap}") + +# Process and inspect chunks +test_source.add() +print(f"Number of chunks created: {len(test_source.chunks)}") +for i, chunk in enumerate(test_source.chunks[:3]): # Show first 3 chunks + print(f"Chunk {i+1}: {chunk[:50]}...") +``` + +### Common Knowledge Storage Issues + +**"File not found" errors:** +```python +# Ensure files are in the correct location +from crewai.utilities.constants import KNOWLEDGE_DIRECTORY +import os + +knowledge_dir = KNOWLEDGE_DIRECTORY # Usually "knowledge" +file_path = os.path.join(knowledge_dir, "your_file.pdf") + +if not os.path.exists(file_path): + print(f"File not found: {file_path}") + print(f"Current working directory: {os.getcwd()}") + print(f"Expected knowledge directory: {os.path.abspath(knowledge_dir)}") +``` + +**"Embedding dimension mismatch" errors:** +```python +# This happens when switching embedding providers +# Reset knowledge storage to clear old embeddings +crew.reset_memories(command_type='knowledge') + +# Or use consistent embedding providers +crew = Crew( + agents=[...], + tasks=[...], + knowledge_sources=[...], + embedder={"provider": "openai", "config": {"model": "text-embedding-3-small"}} ) ``` +**"ChromaDB permission denied" errors:** +```bash +# Fix storage permissions +chmod -R 755 ~/.local/share/CrewAI/ +``` + +**Knowledge not persisting between runs:** +```python +# Verify storage location consistency +import os +from crewai.utilities.paths import db_storage_path + +print("CREWAI_STORAGE_DIR:", os.getenv("CREWAI_STORAGE_DIR")) +print("Computed storage path:", db_storage_path()) +print("Knowledge path:", os.path.join(db_storage_path(), "knowledge")) +``` + +### Knowledge Reset Commands + +```python +# Reset only agent-specific knowledge +crew.reset_memories(command_type='agent_knowledge') + +# Reset both crew and agent knowledge +crew.reset_memories(command_type='knowledge') + +# CLI commands +# crewai reset-memories --agent-knowledge # Agent knowledge only +# crewai reset-memories --knowledge # All knowledge +``` + +### Clearing Knowledge + +If you need to clear the knowledge stored in CrewAI, you can use the `crewai reset-memories` command with the `--knowledge` option. + +```bash Command +crewai reset-memories --knowledge +``` + +This is useful when you've updated your knowledge sources and want to ensure that the agents are using the most recent information. + ## Best Practices @@ -756,5 +1011,27 @@ recent_news = SpaceNewsKnowledgeSource( - If the knowledge sources are large, this leads to inefficiency and increased latency, as the same data is embedded each time. - To resolve this, directly initialize the knowledge parameter instead of the knowledge_sources parameter. - Link to the issue to get complete idea [Github Issue](https://github.com/crewAIInc/crewAI/issues/2755) - + + + + - Use agent-level knowledge for role-specific information + - Use crew-level knowledge for shared information all agents need + - Set embedders at agent level if you need different embedding strategies + - Use consistent collection naming by keeping agent roles descriptive + - Test knowledge initialization by checking agent.knowledge after kickoff + - Monitor storage locations to understand where knowledge is stored + - Reset knowledge appropriately using the correct command types + + + + - Set `CREWAI_STORAGE_DIR` to a known location in production + - Choose explicit embedding providers to match your LLM setup and avoid API key conflicts + - Monitor knowledge storage size as it grows with document additions + - Organize knowledge sources by domain or purpose using collection names + - Include knowledge directories in your backup and deployment strategies + - Set appropriate file permissions for knowledge files and storage directories + - Use environment variables for API keys and sensitive configuration + + + diff --git a/docs/concepts/memory.mdx b/docs/concepts/memory.mdx index ec9e76cae..da228f4e9 100644 --- a/docs/concepts/memory.mdx +++ b/docs/concepts/memory.mdx @@ -46,22 +46,96 @@ crew = Crew( - **Storage Location**: Platform-specific location via `appdirs` package - **Custom Storage Directory**: Set `CREWAI_STORAGE_DIR` environment variable -### Custom Embedder Configuration +## Storage Location Transparency + + +**Understanding Storage Locations**: CrewAI uses platform-specific directories to store memory and knowledge files following OS conventions. Understanding these locations helps with production deployments, backups, and debugging. + + +### Where CrewAI Stores Files + +By default, CrewAI uses the `appdirs` library to determine storage locations following platform conventions. Here's exactly where your files are stored: + +#### Default Storage Locations by Platform + +**macOS:** +``` +~/Library/Application Support/CrewAI/{project_name}/ +├── knowledge/ # Knowledge base ChromaDB files +├── short_term_memory/ # Short-term memory ChromaDB files +├── long_term_memory/ # Long-term memory ChromaDB files +├── entities/ # Entity memory ChromaDB files +└── long_term_memory_storage.db # SQLite database +``` + +**Linux:** +``` +~/.local/share/CrewAI/{project_name}/ +├── knowledge/ +├── short_term_memory/ +├── long_term_memory/ +├── entities/ +└── long_term_memory_storage.db +``` + +**Windows:** +``` +C:\Users\{username}\AppData\Local\CrewAI\{project_name}\ +├── knowledge\ +├── short_term_memory\ +├── long_term_memory\ +├── entities\ +└── long_term_memory_storage.db +``` + +### Finding Your Storage Location + +To see exactly where CrewAI is storing files on your system: + ```python +from crewai.utilities.paths import db_storage_path +import os + +# Get the base storage path +storage_path = db_storage_path() +print(f"CrewAI storage location: {storage_path}") + +# List all CrewAI storage directories +if os.path.exists(storage_path): + print("\nStored files and directories:") + for item in os.listdir(storage_path): + item_path = os.path.join(storage_path, item) + if os.path.isdir(item_path): + print(f"📁 {item}/") + # Show ChromaDB collections + if os.path.exists(item_path): + for subitem in os.listdir(item_path): + print(f" └── {subitem}") + else: + print(f"📄 {item}") +else: + print("No CrewAI storage directory found yet.") +``` + +### Controlling Storage Locations + +#### Option 1: Environment Variable (Recommended) +```python +import os +from crewai import Crew + +# Set custom storage location +os.environ["CREWAI_STORAGE_DIR"] = "./my_project_storage" + +# All memory and knowledge will now be stored in ./my_project_storage/ crew = Crew( agents=[...], tasks=[...], - memory=True, - embedder={ - "provider": "openai", - "config": { - "model": "text-embedding-3-small" - } - } + memory=True ) ``` -### Custom Storage Paths +#### Option 2: Custom Storage Paths ```python import os from crewai import Crew @@ -69,16 +143,547 @@ from crewai.memory import LongTermMemory from crewai.memory.storage.ltm_sqlite_storage import LTMSQLiteStorage # Configure custom storage location +custom_storage_path = "./storage" +os.makedirs(custom_storage_path, exist_ok=True) + crew = Crew( memory=True, long_term_memory=LongTermMemory( storage=LTMSQLiteStorage( - db_path=os.getenv("CREWAI_STORAGE_DIR", "./storage") + "/memory.db" + db_path=f"{custom_storage_path}/memory.db" ) ) ) ``` +#### Option 3: Project-Specific Storage +```python +import os +from pathlib import Path + +# Store in project directory +project_root = Path(__file__).parent +storage_dir = project_root / "crewai_storage" + +os.environ["CREWAI_STORAGE_DIR"] = str(storage_dir) + +# Now all storage will be in your project directory +``` + +### Embedding Provider Defaults + + +**Default Embedding Provider**: CrewAI defaults to OpenAI embeddings for consistency and reliability. You can easily customize this to match your LLM provider or use local embeddings. + + +#### Understanding Default Behavior +```python +# When using Claude as your LLM... +from crewai import Agent, LLM + +agent = Agent( + role="Analyst", + goal="Analyze data", + backstory="Expert analyst", + llm=LLM(provider="anthropic", model="claude-3-sonnet") # Using Claude +) + +# CrewAI will use OpenAI embeddings by default for consistency +# You can easily customize this to match your preferred provider +``` + +#### Customizing Embedding Providers +```python +from crewai import Crew + +# Option 1: Match your LLM provider +crew = Crew( + agents=[agent], + tasks=[task], + memory=True, + embedder={ + "provider": "anthropic", # Match your LLM provider + "config": { + "api_key": "your-anthropic-key", + "model": "text-embedding-3-small" + } + } +) + +# Option 2: Use local embeddings (no external API calls) +crew = Crew( + agents=[agent], + tasks=[task], + memory=True, + embedder={ + "provider": "ollama", + "config": {"model": "mxbai-embed-large"} + } +) +``` + +### Debugging Storage Issues + +#### Check Storage Permissions +```python +import os +from crewai.utilities.paths import db_storage_path + +storage_path = db_storage_path() +print(f"Storage path: {storage_path}") +print(f"Path exists: {os.path.exists(storage_path)}") +print(f"Is writable: {os.access(storage_path, os.W_OK) if os.path.exists(storage_path) else 'Path does not exist'}") + +# Create with proper permissions +if not os.path.exists(storage_path): + os.makedirs(storage_path, mode=0o755, exist_ok=True) + print(f"Created storage directory: {storage_path}") +``` + +#### Inspect ChromaDB Collections +```python +import chromadb +from crewai.utilities.paths import db_storage_path + +# Connect to CrewAI's ChromaDB +storage_path = db_storage_path() +chroma_path = os.path.join(storage_path, "knowledge") + +if os.path.exists(chroma_path): + client = chromadb.PersistentClient(path=chroma_path) + collections = client.list_collections() + + print("ChromaDB Collections:") + for collection in collections: + print(f" - {collection.name}: {collection.count()} documents") +else: + print("No ChromaDB storage found") +``` + +#### Reset Storage (Debugging) +```python +from crewai import Crew + +# Reset all memory storage +crew = Crew(agents=[...], tasks=[...], memory=True) + +# Reset specific memory types +crew.reset_memories(command_type='short') # Short-term memory +crew.reset_memories(command_type='long') # Long-term memory +crew.reset_memories(command_type='entity') # Entity memory +crew.reset_memories(command_type='knowledge') # Knowledge storage +``` + +### Production Best Practices + +1. **Set `CREWAI_STORAGE_DIR`** to a known location in production for better control +2. **Choose explicit embedding providers** to match your LLM setup +3. **Monitor storage directory size** for large-scale deployments +4. **Include storage directories** in your backup strategy +5. **Set appropriate file permissions** (0o755 for directories, 0o644 for files) +6. **Use project-relative paths** for containerized deployments + +### Common Storage Issues + +**"ChromaDB permission denied" errors:** +```bash +# Fix permissions +chmod -R 755 ~/.local/share/CrewAI/ +``` + +**"Database is locked" errors:** +```python +# Ensure only one CrewAI instance accesses storage +import fcntl +import os + +storage_path = db_storage_path() +lock_file = os.path.join(storage_path, ".crewai.lock") + +with open(lock_file, 'w') as f: + fcntl.flock(f.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + # Your CrewAI code here +``` + +**Storage not persisting between runs:** +```python +# Verify storage location is consistent +import os +print("CREWAI_STORAGE_DIR:", os.getenv("CREWAI_STORAGE_DIR")) +print("Current working directory:", os.getcwd()) +print("Computed storage path:", db_storage_path()) +``` + +## Custom Embedder Configuration + +CrewAI supports multiple embedding providers to give you flexibility in choosing the best option for your use case. Here's a comprehensive guide to configuring different embedding providers for your memory system. + +### Why Choose Different Embedding Providers? + +- **Cost Optimization**: Local embeddings (Ollama) are free after initial setup +- **Privacy**: Keep your data local with Ollama or use your preferred cloud provider +- **Performance**: Some models work better for specific domains or languages +- **Consistency**: Match your embedding provider with your LLM provider +- **Compliance**: Meet specific regulatory or organizational requirements + +### OpenAI Embeddings (Default) + +OpenAI provides reliable, high-quality embeddings that work well for most use cases. + +```python +from crewai import Crew + +# Basic OpenAI configuration (uses environment OPENAI_API_KEY) +crew = Crew( + agents=[...], + tasks=[...], + memory=True, + embedder={ + "provider": "openai", + "config": { + "model": "text-embedding-3-small" # or "text-embedding-3-large" + } + } +) + +# Advanced OpenAI configuration +crew = Crew( + memory=True, + embedder={ + "provider": "openai", + "config": { + "api_key": "your-openai-api-key", # Optional: override env var + "model": "text-embedding-3-large", + "dimensions": 1536, # Optional: reduce dimensions for smaller storage + "organization_id": "your-org-id" # Optional: for organization accounts + } + } +) +``` + +### Azure OpenAI Embeddings + +For enterprise users with Azure OpenAI deployments. + +```python +crew = Crew( + memory=True, + embedder={ + "provider": "openai", # Use openai provider for Azure + "config": { + "api_key": "your-azure-api-key", + "api_base": "https://your-resource.openai.azure.com/", + "api_type": "azure", + "api_version": "2023-05-15", + "model": "text-embedding-3-small", + "deployment_id": "your-deployment-name" # Azure deployment name + } + } +) +``` + +### Google AI Embeddings + +Use Google's text embedding models for integration with Google Cloud services. + +```python +crew = Crew( + memory=True, + embedder={ + "provider": "google", + "config": { + "api_key": "your-google-api-key", + "model": "text-embedding-004" # or "text-embedding-preview-0409" + } + } +) +``` + +### Vertex AI Embeddings + +For Google Cloud users with Vertex AI access. + +```python +crew = Crew( + memory=True, + embedder={ + "provider": "vertexai", + "config": { + "project_id": "your-gcp-project-id", + "region": "us-central1", # or your preferred region + "api_key": "your-service-account-key", + "model_name": "textembedding-gecko" + } + } +) +``` + +### Ollama Embeddings (Local) + +Run embeddings locally for privacy and cost savings. + +```python +# First, install and run Ollama locally, then pull an embedding model: +# ollama pull mxbai-embed-large + +crew = Crew( + memory=True, + embedder={ + "provider": "ollama", + "config": { + "model": "mxbai-embed-large", # or "nomic-embed-text" + "url": "http://localhost:11434/api/embeddings" # Default Ollama URL + } + } +) + +# For custom Ollama installations +crew = Crew( + memory=True, + embedder={ + "provider": "ollama", + "config": { + "model": "mxbai-embed-large", + "url": "http://your-ollama-server:11434/api/embeddings" + } + } +) +``` + +### Cohere Embeddings + +Use Cohere's embedding models for multilingual support. + +```python +crew = Crew( + memory=True, + embedder={ + "provider": "cohere", + "config": { + "api_key": "your-cohere-api-key", + "model": "embed-english-v3.0" # or "embed-multilingual-v3.0" + } + } +) +``` + +### VoyageAI Embeddings + +High-performance embeddings optimized for retrieval tasks. + +```python +crew = Crew( + memory=True, + embedder={ + "provider": "voyageai", + "config": { + "api_key": "your-voyage-api-key", + "model": "voyage-large-2", # or "voyage-code-2" for code + "input_type": "document" # or "query" + } + } +) +``` + +### AWS Bedrock Embeddings + +For AWS users with Bedrock access. + +```python +crew = Crew( + memory=True, + embedder={ + "provider": "bedrock", + "config": { + "aws_access_key_id": "your-access-key", + "aws_secret_access_key": "your-secret-key", + "region_name": "us-east-1", + "model": "amazon.titan-embed-text-v1" + } + } +) +``` + +### Hugging Face Embeddings + +Use open-source models from Hugging Face. + +```python +crew = Crew( + memory=True, + embedder={ + "provider": "huggingface", + "config": { + "api_key": "your-hf-token", # Optional for public models + "model": "sentence-transformers/all-MiniLM-L6-v2", + "api_url": "https://api-inference.huggingface.co" # or your custom endpoint + } + } +) +``` + +### IBM Watson Embeddings + +For IBM Cloud users. + +```python +crew = Crew( + memory=True, + embedder={ + "provider": "watson", + "config": { + "api_key": "your-watson-api-key", + "url": "your-watson-instance-url", + "model": "ibm/slate-125m-english-rtrvr" + } + } +) +``` + +### Choosing the Right Embedding Provider + +| Provider | Best For | Pros | Cons | +|:---------|:----------|:------|:------| +| **OpenAI** | General use, reliability | High quality, well-tested | Cost, requires API key | +| **Ollama** | Privacy, cost savings | Free, local, private | Requires local setup | +| **Google AI** | Google ecosystem | Good performance | Requires Google account | +| **Azure OpenAI** | Enterprise, compliance | Enterprise features | Complex setup | +| **Cohere** | Multilingual content | Great language support | Specialized use case | +| **VoyageAI** | Retrieval tasks | Optimized for search | Newer provider | + +### Environment Variable Configuration + +For security, store API keys in environment variables: + +```python +import os + +# Set environment variables +os.environ["OPENAI_API_KEY"] = "your-openai-key" +os.environ["GOOGLE_API_KEY"] = "your-google-key" +os.environ["COHERE_API_KEY"] = "your-cohere-key" + +# Use without exposing keys in code +crew = Crew( + memory=True, + embedder={ + "provider": "openai", + "config": { + "model": "text-embedding-3-small" + # API key automatically loaded from environment + } + } +) +``` + +### Testing Different Embedding Providers + +Compare embedding providers for your specific use case: + +```python +from crewai import Crew +from crewai.utilities.paths import db_storage_path + +# Test different providers with the same data +providers_to_test = [ + { + "name": "OpenAI", + "config": { + "provider": "openai", + "config": {"model": "text-embedding-3-small"} + } + }, + { + "name": "Ollama", + "config": { + "provider": "ollama", + "config": {"model": "mxbai-embed-large"} + } + } +] + +for provider in providers_to_test: + print(f"\nTesting {provider['name']} embeddings...") + + # Create crew with specific embedder + crew = Crew( + agents=[...], + tasks=[...], + memory=True, + embedder=provider['config'] + ) + + # Run your test and measure performance + result = crew.kickoff() + print(f"{provider['name']} completed successfully") +``` + +### Troubleshooting Embedding Issues + +**Model not found errors:** +```python +# Verify model availability +from crewai.utilities.embedding_configurator import EmbeddingConfigurator + +configurator = EmbeddingConfigurator() +try: + embedder = configurator.configure_embedder({ + "provider": "ollama", + "config": {"model": "mxbai-embed-large"} + }) + print("Embedder configured successfully") +except Exception as e: + print(f"Configuration error: {e}") +``` + +**API key issues:** +```python +import os + +# Check if API keys are set +required_keys = ["OPENAI_API_KEY", "GOOGLE_API_KEY", "COHERE_API_KEY"] +for key in required_keys: + if os.getenv(key): + print(f"✅ {key} is set") + else: + print(f"❌ {key} is not set") +``` + +**Performance comparison:** +```python +import time + +def test_embedding_performance(embedder_config, test_text="This is a test document"): + start_time = time.time() + + crew = Crew( + agents=[...], + tasks=[...], + memory=True, + embedder=embedder_config + ) + + # Simulate memory operation + crew.kickoff() + + end_time = time.time() + return end_time - start_time + +# Compare performance +openai_time = test_embedding_performance({ + "provider": "openai", + "config": {"model": "text-embedding-3-small"} +}) + +ollama_time = test_embedding_performance({ + "provider": "ollama", + "config": {"model": "mxbai-embed-large"} +}) + +print(f"OpenAI: {openai_time:.2f}s") +print(f"Ollama: {ollama_time:.2f}s") +``` + ## 2. User Memory with Mem0 (Legacy) diff --git a/docs/docs.json b/docs/docs.json index 7947b16cb..0d29bc7bb 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -164,8 +164,7 @@ "tools/ai-ml/llamaindextool", "tools/ai-ml/langchaintool", "tools/ai-ml/ragtool", - "tools/ai-ml/codeinterpretertool", - "tools/ai-ml/patronustools" + "tools/ai-ml/codeinterpretertool" ] }, { @@ -190,40 +189,43 @@ ] }, { - "group": "Agent Monitoring & Observability", + "group": "Observability", "pages": [ - "how-to/agentops-observability", - "how-to/arize-phoenix-observability", - "how-to/langfuse-observability", - "how-to/langtrace-observability", - "how-to/mlflow-observability", - "how-to/openlit-observability", - "how-to/opik-observability", - "how-to/portkey-observability", - "how-to/weave-integration" + "observability/overview", + "observability/agentops", + "observability/arize-phoenix", + "observability/langfuse", + "observability/langtrace", + "observability/mlflow", + "observability/openlit", + "observability/opik", + "observability/patronus-evaluation", + "observability/portkey", + "observability/weave" ] }, { "group": "Learn", "pages": [ - "how-to/conditional-tasks", - "how-to/coding-agents", - "how-to/create-custom-tools", - "how-to/custom-llm", - "how-to/custom-manager-agent", - "how-to/customizing-agents", - "how-to/dalle-image-generation", - "how-to/force-tool-output-as-result", - "how-to/hierarchical-process", - "how-to/human-in-the-loop", - "how-to/human-input-on-execution", - "how-to/kickoff-async", - "how-to/kickoff-for-each", - "how-to/llm-connections", - "how-to/multimodal-agents", - "how-to/replay-tasks-from-latest-crew-kickoff", - "how-to/sequential-process", - "how-to/using-annotations" + "learn/overview", + "learn/conditional-tasks", + "learn/coding-agents", + "learn/create-custom-tools", + "learn/custom-llm", + "learn/custom-manager-agent", + "learn/customizing-agents", + "learn/dalle-image-generation", + "learn/force-tool-output-as-result", + "learn/hierarchical-process", + "learn/human-in-the-loop", + "learn/human-input-on-execution", + "learn/kickoff-async", + "learn/kickoff-for-each", + "learn/llm-connections", + "learn/multimodal-agents", + "learn/replay-tasks-from-latest-crew-kickoff", + "learn/sequential-process", + "learn/using-annotations" ] }, { @@ -352,7 +354,7 @@ "navbar": { "links": [ { - "label": "Start Free Trial", + "label": "Start Cloud Trial", "href": "https://app.crewai.com" } ], diff --git a/docs/guides/advanced/customizing-prompts.mdx b/docs/guides/advanced/customizing-prompts.mdx index fb7c3c02a..72850bc99 100644 --- a/docs/guides/advanced/customizing-prompts.mdx +++ b/docs/guides/advanced/customizing-prompts.mdx @@ -6,7 +6,7 @@ icon: message-pen ## Why Customize Prompts? -Although CrewAI's default prompts work well for many scenarios, low-level customization opens the door to significantly more flexible and powerful agent behavior. Here’s why you might want to take advantage of this deeper control: +Although CrewAI's default prompts work well for many scenarios, low-level customization opens the door to significantly more flexible and powerful agent behavior. Here's why you might want to take advantage of this deeper control: 1. **Optimize for specific LLMs** – Different models (such as GPT-4, Claude, or Llama) thrive with prompt formats tailored to their unique architectures. 2. **Change the language** – Build agents that operate exclusively in languages beyond English, handling nuances with precision. @@ -20,13 +20,174 @@ This guide explores how to tap into CrewAI's prompts at a lower level, giving yo Under the hood, CrewAI employs a modular prompt system that you can customize extensively: -- **Agent templates** – Govern each agent’s approach to their assigned role. +- **Agent templates** – Govern each agent's approach to their assigned role. - **Prompt slices** – Control specialized behaviors such as tasks, tool usage, and output structure. - **Error handling** – Direct how agents respond to failures, exceptions, or timeouts. - **Tool-specific prompts** – Define detailed instructions for how tools are invoked or utilized. Check out the [original prompt templates in CrewAI's repository](https://github.com/crewAIInc/crewAI/blob/main/src/crewai/translations/en.json) to see how these elements are organized. From there, you can override or adapt them as needed to unlock advanced behaviors. +## Understanding Default System Instructions + + +**Production Transparency Issue**: CrewAI automatically injects default instructions into your prompts that you might not be aware of. This section explains what's happening under the hood and how to gain full control. + + +When you define an agent with `role`, `goal`, and `backstory`, CrewAI automatically adds additional system instructions that control formatting and behavior. Understanding these default injections is crucial for production systems where you need full prompt transparency. + +### What CrewAI Automatically Injects + +Based on your agent configuration, CrewAI adds different default instructions: + +#### For Agents Without Tools +```text +"I MUST use these formats, my job depends on it!" +``` + +#### For Agents With Tools +```text +"IMPORTANT: Use the following format in your response: + +Thought: you should always think about what to do +Action: the action to take, only one name of [tool_names] +Action Input: the input to the action, just a simple JSON object... +``` + +#### For Structured Outputs (JSON/Pydantic) +```text +"Ensure your final answer contains only the content in the following format: {output_format} +Ensure the final output does not include any code block markers like ```json or ```python." +``` + +### Viewing the Complete System Prompt + +To see exactly what prompt is being sent to your LLM, you can inspect the generated prompt: + +```python +from crewai import Agent, Crew, Task +from crewai.utilities.prompts import Prompts + +# Create your agent +agent = Agent( + role="Data Analyst", + goal="Analyze data and provide insights", + backstory="You are an expert data analyst with 10 years of experience.", + verbose=True +) + +# Create a sample task +task = Task( + description="Analyze the sales data and identify trends", + expected_output="A detailed analysis with key insights and trends", + agent=agent +) + +# Create the prompt generator +prompt_generator = Prompts( + agent=agent, + has_tools=len(agent.tools) > 0, + use_system_prompt=agent.use_system_prompt +) + +# Generate and inspect the actual prompt +generated_prompt = prompt_generator.task_execution() + +# Print the complete system prompt that will be sent to the LLM +if "system" in generated_prompt: + print("=== SYSTEM PROMPT ===") + print(generated_prompt["system"]) + print("\n=== USER PROMPT ===") + print(generated_prompt["user"]) +else: + print("=== COMPLETE PROMPT ===") + print(generated_prompt["prompt"]) + +# You can also see how the task description gets formatted +print("\n=== TASK CONTEXT ===") +print(f"Task Description: {task.description}") +print(f"Expected Output: {task.expected_output}") +``` + +### Overriding Default Instructions + +You have several options to gain full control over the prompts: + +#### Option 1: Custom Templates (Recommended) +```python +from crewai import Agent + +# Define your own system template without default instructions +custom_system_template = """You are {role}. {backstory} +Your goal is: {goal} + +Respond naturally and conversationally. Focus on providing helpful, accurate information.""" + +custom_prompt_template = """Task: {input} + +Please complete this task thoughtfully.""" + +agent = Agent( + role="Research Assistant", + goal="Help users find accurate information", + backstory="You are a helpful research assistant.", + system_template=custom_system_template, + prompt_template=custom_prompt_template, + use_system_prompt=True # Use separate system/user messages +) +``` + +#### Option 2: Custom Prompt File +Create a `custom_prompts.json` file to override specific prompt slices: + +```json +{ + "slices": { + "no_tools": "\nProvide your best answer in a natural, conversational way.", + "tools": "\nYou have access to these tools: {tools}\n\nUse them when helpful, but respond naturally.", + "formatted_task_instructions": "Format your response as: {output_format}" + } +} +``` + +Then use it in your crew: + +```python +crew = Crew( + agents=[agent], + tasks=[task], + prompt_file="custom_prompts.json", + verbose=True +) +``` + +#### Option 3: Disable System Prompts for o1 Models +```python +agent = Agent( + role="Analyst", + goal="Analyze data", + backstory="Expert analyst", + use_system_prompt=False # Disables system prompt separation +) +``` + +### Debugging with Observability Tools + +For production transparency, integrate with observability platforms to monitor all prompts and LLM interactions. This allows you to see exactly what prompts (including default instructions) are being sent to your LLMs. + +See our [Observability documentation](/how-to/observability) for detailed integration guides with various platforms including Langfuse, MLflow, Weights & Biases, and custom logging solutions. + +### Best Practices for Production + +1. **Always inspect generated prompts** before deploying to production +2. **Use custom templates** when you need full control over prompt content +3. **Integrate observability tools** for ongoing prompt monitoring (see [Observability docs](/how-to/observability)) +4. **Test with different LLMs** as default instructions may work differently across models +5. **Document your prompt customizations** for team transparency + + +The default instructions exist to ensure consistent agent behavior, but they can interfere with domain-specific requirements. Use the customization options above to maintain full control over your agent's behavior in production systems. + + ## Best Practices for Managing Prompt Files When engaging in low-level prompt customization, follow these guidelines to keep things organized and maintainable: @@ -44,7 +205,7 @@ One straightforward approach is to create a JSON file for the prompts you want t 1. Craft a JSON file with your updated prompt slices. 2. Reference that file via the `prompt_file` parameter in your Crew. -CrewAI then merges your customizations with the defaults, so you don’t have to redefine every prompt. Here’s how: +CrewAI then merges your customizations with the defaults, so you don't have to redefine every prompt. Here's how: ### Example: Basic Prompt Customization @@ -93,14 +254,14 @@ With these few edits, you gain low-level control over how your agents communicat ## Optimizing for Specific Models -Different models thrive on differently structured prompts. Making deeper adjustments can significantly boost performance by aligning your prompts with a model’s nuances. +Different models thrive on differently structured prompts. Making deeper adjustments can significantly boost performance by aligning your prompts with a model's nuances. ### Example: Llama 3.3 Prompting Template -For instance, when dealing with Meta’s Llama 3.3, deeper-level customization may reflect the recommended structure described at: +For instance, when dealing with Meta's Llama 3.3, deeper-level customization may reflect the recommended structure described at: https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_1/#prompt-template -Here’s an example to highlight how you might fine-tune an Agent to leverage Llama 3.3 in code: +Here's an example to highlight how you might fine-tune an Agent to leverage Llama 3.3 in code: ```python from crewai import Agent, Crew, Task, Process @@ -148,8 +309,8 @@ Through this deeper configuration, you can exercise comprehensive, low-level con ## Conclusion -Low-level prompt customization in CrewAI opens the door to super custom, complex use cases. By establishing well-organized prompt files (or direct inline templates), you can accommodate various models, languages, and specialized domains. This level of flexibility ensures you can craft precisely the AI behavior you need, all while knowing CrewAI still provides reliable defaults when you don’t override them. +Low-level prompt customization in CrewAI opens the door to super custom, complex use cases. By establishing well-organized prompt files (or direct inline templates), you can accommodate various models, languages, and specialized domains. This level of flexibility ensures you can craft precisely the AI behavior you need, all while knowing CrewAI still provides reliable defaults when you don't override them. -You now have the foundation for advanced prompt customizations in CrewAI. Whether you’re adapting for model-specific structures or domain-specific constraints, this low-level approach lets you shape agent interactions in highly specialized ways. +You now have the foundation for advanced prompt customizations in CrewAI. Whether you're adapting for model-specific structures or domain-specific constraints, this low-level approach lets you shape agent interactions in highly specialized ways. \ No newline at end of file diff --git a/docs/how-to/custom-llm.mdx b/docs/how-to/custom-llm.mdx deleted file mode 100644 index 448e60b37..000000000 --- a/docs/how-to/custom-llm.mdx +++ /dev/null @@ -1,646 +0,0 @@ ---- -title: Custom LLM Implementation -description: Learn how to create custom LLM implementations in CrewAI. -icon: code ---- - -## Custom LLM Implementations - -CrewAI now supports custom LLM implementations through the `BaseLLM` abstract base class. This allows you to create your own LLM implementations that don't rely on litellm's authentication mechanism. - -To create a custom LLM implementation, you need to: - -1. Inherit from the `BaseLLM` abstract base class -2. Implement the required methods: - - `call()`: The main method to call the LLM with messages - - `supports_function_calling()`: Whether the LLM supports function calling - - `supports_stop_words()`: Whether the LLM supports stop words - - `get_context_window_size()`: The context window size of the LLM - -## Example: Basic Custom LLM - -```python -from crewai import BaseLLM -from typing import Any, Dict, List, Optional, Union - -class CustomLLM(BaseLLM): - def __init__(self, api_key: str, endpoint: str): - super().__init__() # Initialize the base class to set default attributes - if not api_key or not isinstance(api_key, str): - raise ValueError("Invalid API key: must be a non-empty string") - if not endpoint or not isinstance(endpoint, str): - raise ValueError("Invalid endpoint URL: must be a non-empty string") - self.api_key = api_key - self.endpoint = endpoint - self.stop = [] # You can customize stop words if needed - - def call( - self, - messages: Union[str, List[Dict[str, str]]], - tools: Optional[List[dict]] = None, - callbacks: Optional[List[Any]] = None, - available_functions: Optional[Dict[str, Any]] = None, - ) -> Union[str, Any]: - """Call the LLM with the given messages. - - Args: - messages: Input messages for the LLM. - tools: Optional list of tool schemas for function calling. - callbacks: Optional list of callback functions. - available_functions: Optional dict mapping function names to callables. - - Returns: - Either a text response from the LLM or the result of a tool function call. - - Raises: - TimeoutError: If the LLM request times out. - RuntimeError: If the LLM request fails for other reasons. - ValueError: If the response format is invalid. - """ - # Implement your own logic to call the LLM - # For example, using requests: - import requests - - try: - headers = { - "Authorization": f"Bearer {self.api_key}", - "Content-Type": "application/json" - } - - # Convert string message to proper format if needed - if isinstance(messages, str): - messages = [{"role": "user", "content": messages}] - - data = { - "messages": messages, - "tools": tools - } - - response = requests.post( - self.endpoint, - headers=headers, - json=data, - timeout=30 # Set a reasonable timeout - ) - response.raise_for_status() # Raise an exception for HTTP errors - return response.json()["choices"][0]["message"]["content"] - except requests.Timeout: - raise TimeoutError("LLM request timed out") - except requests.RequestException as e: - raise RuntimeError(f"LLM request failed: {str(e)}") - except (KeyError, IndexError, ValueError) as e: - raise ValueError(f"Invalid response format: {str(e)}") - - def supports_function_calling(self) -> bool: - """Check if the LLM supports function calling. - - Returns: - True if the LLM supports function calling, False otherwise. - """ - # Return True if your LLM supports function calling - return True - - def supports_stop_words(self) -> bool: - """Check if the LLM supports stop words. - - Returns: - True if the LLM supports stop words, False otherwise. - """ - # Return True if your LLM supports stop words - return True - - def get_context_window_size(self) -> int: - """Get the context window size of the LLM. - - Returns: - The context window size as an integer. - """ - # Return the context window size of your LLM - return 8192 -``` - -## Error Handling Best Practices - -When implementing custom LLMs, it's important to handle errors properly to ensure robustness and reliability. Here are some best practices: - -### 1. Implement Try-Except Blocks for API Calls - -Always wrap API calls in try-except blocks to handle different types of errors: - -```python -def call( - self, - messages: Union[str, List[Dict[str, str]]], - tools: Optional[List[dict]] = None, - callbacks: Optional[List[Any]] = None, - available_functions: Optional[Dict[str, Any]] = None, -) -> Union[str, Any]: - try: - # API call implementation - response = requests.post( - self.endpoint, - headers=self.headers, - json=self.prepare_payload(messages), - timeout=30 # Set a reasonable timeout - ) - response.raise_for_status() # Raise an exception for HTTP errors - return response.json()["choices"][0]["message"]["content"] - except requests.Timeout: - raise TimeoutError("LLM request timed out") - except requests.RequestException as e: - raise RuntimeError(f"LLM request failed: {str(e)}") - except (KeyError, IndexError, ValueError) as e: - raise ValueError(f"Invalid response format: {str(e)}") -``` - -### 2. Implement Retry Logic for Transient Failures - -For transient failures like network issues or rate limiting, implement retry logic with exponential backoff: - -```python -def call( - self, - messages: Union[str, List[Dict[str, str]]], - tools: Optional[List[dict]] = None, - callbacks: Optional[List[Any]] = None, - available_functions: Optional[Dict[str, Any]] = None, -) -> Union[str, Any]: - import time - - max_retries = 3 - retry_delay = 1 # seconds - - for attempt in range(max_retries): - try: - response = requests.post( - self.endpoint, - headers=self.headers, - json=self.prepare_payload(messages), - timeout=30 - ) - response.raise_for_status() - return response.json()["choices"][0]["message"]["content"] - except (requests.Timeout, requests.ConnectionError) as e: - if attempt < max_retries - 1: - time.sleep(retry_delay * (2 ** attempt)) # Exponential backoff - continue - raise TimeoutError(f"LLM request failed after {max_retries} attempts: {str(e)}") - except requests.RequestException as e: - raise RuntimeError(f"LLM request failed: {str(e)}") -``` - -### 3. Validate Input Parameters - -Always validate input parameters to prevent runtime errors: - -```python -def __init__(self, api_key: str, endpoint: str): - super().__init__() - if not api_key or not isinstance(api_key, str): - raise ValueError("Invalid API key: must be a non-empty string") - if not endpoint or not isinstance(endpoint, str): - raise ValueError("Invalid endpoint URL: must be a non-empty string") - self.api_key = api_key - self.endpoint = endpoint -``` - -### 4. Handle Authentication Errors Gracefully - -Provide clear error messages for authentication failures: - -```python -def call( - self, - messages: Union[str, List[Dict[str, str]]], - tools: Optional[List[dict]] = None, - callbacks: Optional[List[Any]] = None, - available_functions: Optional[Dict[str, Any]] = None, -) -> Union[str, Any]: - try: - response = requests.post(self.endpoint, headers=self.headers, json=data) - if response.status_code == 401: - raise ValueError("Authentication failed: Invalid API key or token") - elif response.status_code == 403: - raise ValueError("Authorization failed: Insufficient permissions") - response.raise_for_status() - # Process response - except Exception as e: - # Handle error - raise -``` - -## Example: JWT-based Authentication - -For services that use JWT-based authentication instead of API keys, you can implement a custom LLM like this: - -```python -from crewai import BaseLLM, Agent, Task -from typing import Any, Dict, List, Optional, Union - -class JWTAuthLLM(BaseLLM): - def __init__(self, jwt_token: str, endpoint: str): - super().__init__() # Initialize the base class to set default attributes - if not jwt_token or not isinstance(jwt_token, str): - raise ValueError("Invalid JWT token: must be a non-empty string") - if not endpoint or not isinstance(endpoint, str): - raise ValueError("Invalid endpoint URL: must be a non-empty string") - self.jwt_token = jwt_token - self.endpoint = endpoint - self.stop = [] # You can customize stop words if needed - - def call( - self, - messages: Union[str, List[Dict[str, str]]], - tools: Optional[List[dict]] = None, - callbacks: Optional[List[Any]] = None, - available_functions: Optional[Dict[str, Any]] = None, - ) -> Union[str, Any]: - """Call the LLM with JWT authentication. - - Args: - messages: Input messages for the LLM. - tools: Optional list of tool schemas for function calling. - callbacks: Optional list of callback functions. - available_functions: Optional dict mapping function names to callables. - - Returns: - Either a text response from the LLM or the result of a tool function call. - - Raises: - TimeoutError: If the LLM request times out. - RuntimeError: If the LLM request fails for other reasons. - ValueError: If the response format is invalid. - """ - # Implement your own logic to call the LLM with JWT authentication - import requests - - try: - headers = { - "Authorization": f"Bearer {self.jwt_token}", - "Content-Type": "application/json" - } - - # Convert string message to proper format if needed - if isinstance(messages, str): - messages = [{"role": "user", "content": messages}] - - data = { - "messages": messages, - "tools": tools - } - - response = requests.post( - self.endpoint, - headers=headers, - json=data, - timeout=30 # Set a reasonable timeout - ) - - if response.status_code == 401: - raise ValueError("Authentication failed: Invalid JWT token") - elif response.status_code == 403: - raise ValueError("Authorization failed: Insufficient permissions") - - response.raise_for_status() # Raise an exception for HTTP errors - return response.json()["choices"][0]["message"]["content"] - except requests.Timeout: - raise TimeoutError("LLM request timed out") - except requests.RequestException as e: - raise RuntimeError(f"LLM request failed: {str(e)}") - except (KeyError, IndexError, ValueError) as e: - raise ValueError(f"Invalid response format: {str(e)}") - - def supports_function_calling(self) -> bool: - """Check if the LLM supports function calling. - - Returns: - True if the LLM supports function calling, False otherwise. - """ - return True - - def supports_stop_words(self) -> bool: - """Check if the LLM supports stop words. - - Returns: - True if the LLM supports stop words, False otherwise. - """ - return True - - def get_context_window_size(self) -> int: - """Get the context window size of the LLM. - - Returns: - The context window size as an integer. - """ - return 8192 -``` - -## Troubleshooting - -Here are some common issues you might encounter when implementing custom LLMs and how to resolve them: - -### 1. Authentication Failures - -**Symptoms**: 401 Unauthorized or 403 Forbidden errors - -**Solutions**: -- Verify that your API key or JWT token is valid and not expired -- Check that you're using the correct authentication header format -- Ensure that your token has the necessary permissions - -### 2. Timeout Issues - -**Symptoms**: Requests taking too long or timing out - -**Solutions**: -- Implement timeout handling as shown in the examples -- Use retry logic with exponential backoff -- Consider using a more reliable network connection - -### 3. Response Parsing Errors - -**Symptoms**: KeyError, IndexError, or ValueError when processing responses - -**Solutions**: -- Validate the response format before accessing nested fields -- Implement proper error handling for malformed responses -- Check the API documentation for the expected response format - -### 4. Rate Limiting - -**Symptoms**: 429 Too Many Requests errors - -**Solutions**: -- Implement rate limiting in your custom LLM -- Add exponential backoff for retries -- Consider using a token bucket algorithm for more precise rate control - -## Advanced Features - -### Logging - -Adding logging to your custom LLM can help with debugging and monitoring: - -```python -import logging -from typing import Any, Dict, List, Optional, Union - -class LoggingLLM(BaseLLM): - def __init__(self, api_key: str, endpoint: str): - super().__init__() - self.api_key = api_key - self.endpoint = endpoint - self.logger = logging.getLogger("crewai.llm.custom") - - def call( - self, - messages: Union[str, List[Dict[str, str]]], - tools: Optional[List[dict]] = None, - callbacks: Optional[List[Any]] = None, - available_functions: Optional[Dict[str, Any]] = None, - ) -> Union[str, Any]: - self.logger.info(f"Calling LLM with {len(messages) if isinstance(messages, list) else 1} messages") - try: - # API call implementation - response = self._make_api_call(messages, tools) - self.logger.debug(f"LLM response received: {response[:100]}...") - return response - except Exception as e: - self.logger.error(f"LLM call failed: {str(e)}") - raise -``` - -### Rate Limiting - -Implementing rate limiting can help avoid overwhelming the LLM API: - -```python -import time -from typing import Any, Dict, List, Optional, Union - -class RateLimitedLLM(BaseLLM): - def __init__( - self, - api_key: str, - endpoint: str, - requests_per_minute: int = 60 - ): - super().__init__() - self.api_key = api_key - self.endpoint = endpoint - self.requests_per_minute = requests_per_minute - self.request_times: List[float] = [] - - def call( - self, - messages: Union[str, List[Dict[str, str]]], - tools: Optional[List[dict]] = None, - callbacks: Optional[List[Any]] = None, - available_functions: Optional[Dict[str, Any]] = None, - ) -> Union[str, Any]: - self._enforce_rate_limit() - # Record this request time - self.request_times.append(time.time()) - # Make the actual API call - return self._make_api_call(messages, tools) - - def _enforce_rate_limit(self) -> None: - """Enforce the rate limit by waiting if necessary.""" - now = time.time() - # Remove request times older than 1 minute - self.request_times = [t for t in self.request_times if now - t < 60] - - if len(self.request_times) >= self.requests_per_minute: - # Calculate how long to wait - oldest_request = min(self.request_times) - wait_time = 60 - (now - oldest_request) - if wait_time > 0: - time.sleep(wait_time) -``` - -### Metrics Collection - -Collecting metrics can help you monitor your LLM usage: - -```python -import time -from typing import Any, Dict, List, Optional, Union - -class MetricsCollectingLLM(BaseLLM): - def __init__(self, api_key: str, endpoint: str): - super().__init__() - self.api_key = api_key - self.endpoint = endpoint - self.metrics: Dict[str, Any] = { - "total_calls": 0, - "total_tokens": 0, - "errors": 0, - "latency": [] - } - - def call( - self, - messages: Union[str, List[Dict[str, str]]], - tools: Optional[List[dict]] = None, - callbacks: Optional[List[Any]] = None, - available_functions: Optional[Dict[str, Any]] = None, - ) -> Union[str, Any]: - start_time = time.time() - self.metrics["total_calls"] += 1 - - try: - response = self._make_api_call(messages, tools) - # Estimate tokens (simplified) - if isinstance(messages, str): - token_estimate = len(messages) // 4 - else: - token_estimate = sum(len(m.get("content", "")) // 4 for m in messages) - self.metrics["total_tokens"] += token_estimate - return response - except Exception as e: - self.metrics["errors"] += 1 - raise - finally: - latency = time.time() - start_time - self.metrics["latency"].append(latency) - - def get_metrics(self) -> Dict[str, Any]: - """Return the collected metrics.""" - avg_latency = sum(self.metrics["latency"]) / len(self.metrics["latency"]) if self.metrics["latency"] else 0 - return { - **self.metrics, - "avg_latency": avg_latency - } -``` - -## Advanced Usage: Function Calling - -If your LLM supports function calling, you can implement the function calling logic in your custom LLM: - -```python -import json -from typing import Any, Dict, List, Optional, Union - -def call( - self, - messages: Union[str, List[Dict[str, str]]], - tools: Optional[List[dict]] = None, - callbacks: Optional[List[Any]] = None, - available_functions: Optional[Dict[str, Any]] = None, -) -> Union[str, Any]: - import requests - - try: - headers = { - "Authorization": f"Bearer {self.jwt_token}", - "Content-Type": "application/json" - } - - # Convert string message to proper format if needed - if isinstance(messages, str): - messages = [{"role": "user", "content": messages}] - - data = { - "messages": messages, - "tools": tools - } - - response = requests.post( - self.endpoint, - headers=headers, - json=data, - timeout=30 - ) - response.raise_for_status() - response_data = response.json() - - # Check if the LLM wants to call a function - if response_data["choices"][0]["message"].get("tool_calls"): - tool_calls = response_data["choices"][0]["message"]["tool_calls"] - - # Process each tool call - for tool_call in tool_calls: - function_name = tool_call["function"]["name"] - function_args = json.loads(tool_call["function"]["arguments"]) - - if available_functions and function_name in available_functions: - function_to_call = available_functions[function_name] - function_response = function_to_call(**function_args) - - # Add the function response to the messages - messages.append({ - "role": "tool", - "tool_call_id": tool_call["id"], - "name": function_name, - "content": str(function_response) - }) - - # Call the LLM again with the updated messages - return self.call(messages, tools, callbacks, available_functions) - - # Return the text response if no function call - return response_data["choices"][0]["message"]["content"] - except requests.Timeout: - raise TimeoutError("LLM request timed out") - except requests.RequestException as e: - raise RuntimeError(f"LLM request failed: {str(e)}") - except (KeyError, IndexError, ValueError) as e: - raise ValueError(f"Invalid response format: {str(e)}") -``` - -## Using Your Custom LLM with CrewAI - -Once you've implemented your custom LLM, you can use it with CrewAI agents and crews: - -```python -from crewai import Agent, Task, Crew -from typing import Dict, Any - -# Create your custom LLM instance -jwt_llm = JWTAuthLLM( - jwt_token="your.jwt.token", - endpoint="https://your-llm-endpoint.com/v1/chat/completions" -) - -# Use it with an agent -agent = Agent( - role="Research Assistant", - goal="Find information on a topic", - backstory="You are a research assistant tasked with finding information.", - llm=jwt_llm, -) - -# Create a task for the agent -task = Task( - description="Research the benefits of exercise", - agent=agent, - expected_output="A summary of the benefits of exercise", -) - -# Execute the task -result = agent.execute_task(task) -print(result) - -# Or use it with a crew -crew = Crew( - agents=[agent], - tasks=[task], - manager_llm=jwt_llm, # Use your custom LLM for the manager -) - -# Run the crew -result = crew.kickoff() -print(result) -``` - -## Implementing Your Own Authentication Mechanism - -The `BaseLLM` class allows you to implement any authentication mechanism you need, not just JWT or API keys. You can use: - -- OAuth tokens -- Client certificates -- Custom headers -- Session-based authentication -- Any other authentication method required by your LLM provider - -Simply implement the appropriate authentication logic in your custom LLM class. diff --git a/docs/how-to/before-and-after-kickoff-hooks.mdx b/docs/learn/before-and-after-kickoff-hooks.mdx similarity index 100% rename from docs/how-to/before-and-after-kickoff-hooks.mdx rename to docs/learn/before-and-after-kickoff-hooks.mdx diff --git a/docs/how-to/bring-your-own-agent.mdx b/docs/learn/bring-your-own-agent.mdx similarity index 100% rename from docs/how-to/bring-your-own-agent.mdx rename to docs/learn/bring-your-own-agent.mdx diff --git a/docs/how-to/coding-agents.mdx b/docs/learn/coding-agents.mdx similarity index 100% rename from docs/how-to/coding-agents.mdx rename to docs/learn/coding-agents.mdx diff --git a/docs/how-to/conditional-tasks.mdx b/docs/learn/conditional-tasks.mdx similarity index 100% rename from docs/how-to/conditional-tasks.mdx rename to docs/learn/conditional-tasks.mdx diff --git a/docs/how-to/create-custom-tools.mdx b/docs/learn/create-custom-tools.mdx similarity index 100% rename from docs/how-to/create-custom-tools.mdx rename to docs/learn/create-custom-tools.mdx diff --git a/docs/learn/custom-llm.mdx b/docs/learn/custom-llm.mdx new file mode 100644 index 000000000..469f0cc00 --- /dev/null +++ b/docs/learn/custom-llm.mdx @@ -0,0 +1,350 @@ +--- +title: Custom LLM Implementation +description: Learn how to create custom LLM implementations in CrewAI. +icon: code +--- + +## Overview + +CrewAI supports custom LLM implementations through the `BaseLLM` abstract base class. This allows you to integrate any LLM provider that doesn't have built-in support in LiteLLM, or implement custom authentication mechanisms. + +## Quick Start + +Here's a minimal custom LLM implementation: + +```python +from crewai import BaseLLM +from typing import Any, Dict, List, Optional, Union +import requests + +class CustomLLM(BaseLLM): + def __init__(self, model: str, api_key: str, endpoint: str, temperature: Optional[float] = None): + # IMPORTANT: Call super().__init__() with required parameters + super().__init__(model=model, temperature=temperature) + + self.api_key = api_key + self.endpoint = endpoint + + def call( + self, + messages: Union[str, List[Dict[str, str]]], + tools: Optional[List[dict]] = None, + callbacks: Optional[List[Any]] = None, + available_functions: Optional[Dict[str, Any]] = None, + ) -> Union[str, Any]: + """Call the LLM with the given messages.""" + # Convert string to message format if needed + if isinstance(messages, str): + messages = [{"role": "user", "content": messages}] + + # Prepare request + payload = { + "model": self.model, + "messages": messages, + "temperature": self.temperature, + } + + # Add tools if provided and supported + if tools and self.supports_function_calling(): + payload["tools"] = tools + + # Make API call + response = requests.post( + self.endpoint, + headers={ + "Authorization": f"Bearer {self.api_key}", + "Content-Type": "application/json" + }, + json=payload, + timeout=30 + ) + response.raise_for_status() + + result = response.json() + return result["choices"][0]["message"]["content"] + + def supports_function_calling(self) -> bool: + """Override if your LLM supports function calling.""" + return True # Change to False if your LLM doesn't support tools + + def get_context_window_size(self) -> int: + """Return the context window size of your LLM.""" + return 8192 # Adjust based on your model's actual context window +``` + +## Using Your Custom LLM + +```python +from crewai import Agent, Task, Crew + +# Assuming you have the CustomLLM class defined above +# Create your custom LLM +custom_llm = CustomLLM( + model="my-custom-model", + api_key="your-api-key", + endpoint="https://api.example.com/v1/chat/completions", + temperature=0.7 +) + +# Use with an agent +agent = Agent( + role="Research Assistant", + goal="Find and analyze information", + backstory="You are a research assistant.", + llm=custom_llm +) + +# Create and execute tasks +task = Task( + description="Research the latest developments in AI", + expected_output="A comprehensive summary", + agent=agent +) + +crew = Crew(agents=[agent], tasks=[task]) +result = crew.kickoff() +``` + +## Required Methods + +### Constructor: `__init__()` + +**Critical**: You must call `super().__init__(model, temperature)` with the required parameters: + +```python +def __init__(self, model: str, api_key: str, temperature: Optional[float] = None): + # REQUIRED: Call parent constructor with model and temperature + super().__init__(model=model, temperature=temperature) + + # Your custom initialization + self.api_key = api_key +``` + +### Abstract Method: `call()` + +The `call()` method is the heart of your LLM implementation. It must: + +- Accept messages (string or list of dicts with 'role' and 'content') +- Return a string response +- Handle tools and function calling if supported +- Raise appropriate exceptions for errors + +### Optional Methods + +```python +def supports_function_calling(self) -> bool: + """Return True if your LLM supports function calling.""" + return True # Default is True + +def supports_stop_words(self) -> bool: + """Return True if your LLM supports stop sequences.""" + return True # Default is True + +def get_context_window_size(self) -> int: + """Return the context window size.""" + return 4096 # Default is 4096 +``` + +## Common Patterns + +### Error Handling + +```python +import requests + +def call(self, messages, tools=None, callbacks=None, available_functions=None): + try: + response = requests.post( + self.endpoint, + headers={"Authorization": f"Bearer {self.api_key}"}, + json=payload, + timeout=30 + ) + response.raise_for_status() + return response.json()["choices"][0]["message"]["content"] + + except requests.Timeout: + raise TimeoutError("LLM request timed out") + except requests.RequestException as e: + raise RuntimeError(f"LLM request failed: {str(e)}") + except (KeyError, IndexError) as e: + raise ValueError(f"Invalid response format: {str(e)}") +``` + +### Custom Authentication + +```python +from crewai import BaseLLM +from typing import Optional + +class CustomAuthLLM(BaseLLM): + def __init__(self, model: str, auth_token: str, endpoint: str, temperature: Optional[float] = None): + super().__init__(model=model, temperature=temperature) + self.auth_token = auth_token + self.endpoint = endpoint + + def call(self, messages, tools=None, callbacks=None, available_functions=None): + headers = { + "Authorization": f"Custom {self.auth_token}", # Custom auth format + "Content-Type": "application/json" + } + # Rest of implementation... +``` + +### Stop Words Support + +CrewAI automatically adds `"\nObservation:"` as a stop word to control agent behavior. If your LLM supports stop words: + +```python +def call(self, messages, tools=None, callbacks=None, available_functions=None): + payload = { + "model": self.model, + "messages": messages, + "stop": self.stop # Include stop words in API call + } + # Make API call... + +def supports_stop_words(self) -> bool: + return True # Your LLM supports stop sequences +``` + +If your LLM doesn't support stop words natively: + +```python +def call(self, messages, tools=None, callbacks=None, available_functions=None): + response = self._make_api_call(messages, tools) + content = response["choices"][0]["message"]["content"] + + # Manually truncate at stop words + if self.stop: + for stop_word in self.stop: + if stop_word in content: + content = content.split(stop_word)[0] + break + + return content + +def supports_stop_words(self) -> bool: + return False # Tell CrewAI we handle stop words manually +``` + +## Function Calling + +If your LLM supports function calling, implement the complete flow: + +```python +import json + +def call(self, messages, tools=None, callbacks=None, available_functions=None): + # Convert string to message format + if isinstance(messages, str): + messages = [{"role": "user", "content": messages}] + + # Make API call + response = self._make_api_call(messages, tools) + message = response["choices"][0]["message"] + + # Check for function calls + if "tool_calls" in message and available_functions: + return self._handle_function_calls( + message["tool_calls"], messages, tools, available_functions + ) + + return message["content"] + +def _handle_function_calls(self, tool_calls, messages, tools, available_functions): + """Handle function calling with proper message flow.""" + for tool_call in tool_calls: + function_name = tool_call["function"]["name"] + + if function_name in available_functions: + # Parse and execute function + function_args = json.loads(tool_call["function"]["arguments"]) + function_result = available_functions[function_name](**function_args) + + # Add function call and result to message history + messages.append({ + "role": "assistant", + "content": None, + "tool_calls": [tool_call] + }) + messages.append({ + "role": "tool", + "tool_call_id": tool_call["id"], + "name": function_name, + "content": str(function_result) + }) + + # Call LLM again with updated context + return self.call(messages, tools, None, available_functions) + + return "Function call failed" +``` + +## Troubleshooting + +### Common Issues + +**Constructor Errors** +```python +# ❌ Wrong - missing required parameters +def __init__(self, api_key: str): + super().__init__() + +# ✅ Correct +def __init__(self, model: str, api_key: str, temperature: Optional[float] = None): + super().__init__(model=model, temperature=temperature) +``` + +**Function Calling Not Working** +- Ensure `supports_function_calling()` returns `True` +- Check that you handle `tool_calls` in the response +- Verify `available_functions` parameter is used correctly + +**Authentication Failures** +- Verify API key format and permissions +- Check authentication header format +- Ensure endpoint URLs are correct + +**Response Parsing Errors** +- Validate response structure before accessing nested fields +- Handle cases where content might be None +- Add proper error handling for malformed responses + +## Testing Your Custom LLM + +```python +from crewai import Agent, Task, Crew + +def test_custom_llm(): + llm = CustomLLM( + model="test-model", + api_key="test-key", + endpoint="https://api.test.com" + ) + + # Test basic call + result = llm.call("Hello, world!") + assert isinstance(result, str) + assert len(result) > 0 + + # Test with CrewAI agent + agent = Agent( + role="Test Agent", + goal="Test custom LLM", + backstory="A test agent.", + llm=llm + ) + + task = Task( + description="Say hello", + expected_output="A greeting", + agent=agent + ) + + crew = Crew(agents=[agent], tasks=[task]) + result = crew.kickoff() + assert "hello" in result.raw.lower() +``` + +This guide covers the essentials of implementing custom LLMs in CrewAI. diff --git a/docs/how-to/custom-manager-agent.mdx b/docs/learn/custom-manager-agent.mdx similarity index 100% rename from docs/how-to/custom-manager-agent.mdx rename to docs/learn/custom-manager-agent.mdx diff --git a/docs/how-to/customizing-agents.mdx b/docs/learn/customizing-agents.mdx similarity index 100% rename from docs/how-to/customizing-agents.mdx rename to docs/learn/customizing-agents.mdx diff --git a/docs/how-to/dalle-image-generation.mdx b/docs/learn/dalle-image-generation.mdx similarity index 100% rename from docs/how-to/dalle-image-generation.mdx rename to docs/learn/dalle-image-generation.mdx diff --git a/docs/how-to/force-tool-output-as-result.mdx b/docs/learn/force-tool-output-as-result.mdx similarity index 100% rename from docs/how-to/force-tool-output-as-result.mdx rename to docs/learn/force-tool-output-as-result.mdx diff --git a/docs/how-to/hierarchical-process.mdx b/docs/learn/hierarchical-process.mdx similarity index 100% rename from docs/how-to/hierarchical-process.mdx rename to docs/learn/hierarchical-process.mdx diff --git a/docs/how-to/human-in-the-loop.mdx b/docs/learn/human-in-the-loop.mdx similarity index 100% rename from docs/how-to/human-in-the-loop.mdx rename to docs/learn/human-in-the-loop.mdx diff --git a/docs/how-to/human-input-on-execution.mdx b/docs/learn/human-input-on-execution.mdx similarity index 100% rename from docs/how-to/human-input-on-execution.mdx rename to docs/learn/human-input-on-execution.mdx diff --git a/docs/how-to/kickoff-async.mdx b/docs/learn/kickoff-async.mdx similarity index 100% rename from docs/how-to/kickoff-async.mdx rename to docs/learn/kickoff-async.mdx diff --git a/docs/how-to/kickoff-for-each.mdx b/docs/learn/kickoff-for-each.mdx similarity index 100% rename from docs/how-to/kickoff-for-each.mdx rename to docs/learn/kickoff-for-each.mdx diff --git a/docs/how-to/llm-connections.mdx b/docs/learn/llm-connections.mdx similarity index 100% rename from docs/how-to/llm-connections.mdx rename to docs/learn/llm-connections.mdx diff --git a/docs/how-to/multimodal-agents.mdx b/docs/learn/multimodal-agents.mdx similarity index 100% rename from docs/how-to/multimodal-agents.mdx rename to docs/learn/multimodal-agents.mdx diff --git a/docs/learn/overview.mdx b/docs/learn/overview.mdx new file mode 100644 index 000000000..f2c942690 --- /dev/null +++ b/docs/learn/overview.mdx @@ -0,0 +1,158 @@ +--- +title: "Overview" +description: "Learn how to build, customize, and optimize your CrewAI applications with comprehensive guides and tutorials" +icon: "face-smile" +--- + +## Learn CrewAI + +This section provides comprehensive guides and tutorials to help you master CrewAI, from basic concepts to advanced techniques. Whether you're just getting started or looking to optimize your existing implementations, these resources will guide you through every aspect of building powerful AI agent workflows. + +## Getting Started Guides + +### Core Concepts + + + Learn how to execute tasks in a sequential order for structured workflows. + + + + Implement hierarchical task execution with manager agents overseeing workflows. + + + + Create dynamic workflows with conditional task execution based on outcomes. + + + + Execute crews asynchronously for improved performance and concurrency. + + + +### Agent Development + + + Learn how to customize agent behavior, roles, and capabilities. + + + + Build agents that can write, execute, and debug code automatically. + + + + Create agents that can process text, images, and other media types. + + + + Implement custom manager agents for complex hierarchical workflows. + + + +## Advanced Features + +### Workflow Control + + + Integrate human oversight and intervention into agent workflows. + + + + Allow human input during task execution for dynamic decision making. + + + + Replay and resume tasks from previous crew executions. + + + + Execute crews multiple times with different inputs efficiently. + + + +### Customization & Integration + + + Integrate custom language models and providers with CrewAI. + + + + Configure and manage connections to various LLM providers. + + + + Build custom tools to extend agent capabilities. + + + + Use Python annotations for cleaner, more maintainable code. + + + +## Specialized Applications + +### Content & Media + + + Generate images using DALL-E integration with your agents. + + + + Integrate existing agents and models into CrewAI workflows. + + + +### Tool Management + + + Configure tools to return their output directly as task results. + + + +## Learning Path Recommendations + +### For Beginners +1. Start with **Sequential Process** to understand basic workflow execution +2. Learn **Customizing Agents** to create effective agent configurations +3. Explore **Create Custom Tools** to extend functionality +4. Try **Human in the Loop** for interactive workflows + +### For Intermediate Users +1. Master **Hierarchical Process** for complex multi-agent systems +2. Implement **Conditional Tasks** for dynamic workflows +3. Use **Async Kickoff** for performance optimization +4. Integrate **Custom LLM** for specialized models + +### For Advanced Users +1. Build **Multimodal Agents** for complex media processing +2. Create **Custom Manager Agents** for sophisticated orchestration +3. Implement **Bring Your Own Agent** for hybrid systems +4. Use **Replay Tasks** for robust error recovery + +## Best Practices + +### Development +- **Start Simple**: Begin with basic sequential workflows before adding complexity +- **Test Incrementally**: Test each component before integrating into larger systems +- **Use Annotations**: Leverage Python annotations for cleaner, more maintainable code +- **Custom Tools**: Build reusable tools that can be shared across different agents + +### Production +- **Error Handling**: Implement robust error handling and recovery mechanisms +- **Performance**: Use async execution and optimize LLM calls for better performance +- **Monitoring**: Integrate observability tools to track agent performance +- **Human Oversight**: Include human checkpoints for critical decisions + +### Optimization +- **Resource Management**: Monitor and optimize token usage and API costs +- **Workflow Design**: Design workflows that minimize unnecessary LLM calls +- **Tool Efficiency**: Create efficient tools that provide maximum value with minimal overhead +- **Iterative Improvement**: Use feedback and metrics to continuously improve agent performance + +## Getting Help + +- **Documentation**: Each guide includes detailed examples and explanations +- **Community**: Join the [CrewAI Forum](https://community.crewai.com) for discussions and support +- **Examples**: Check the Examples section for complete working implementations +- **Support**: Contact [support@crewai.com](mailto:support@crewai.com) for technical assistance + +Start with the guides that match your current needs and gradually explore more advanced topics as you become comfortable with the fundamentals. \ No newline at end of file diff --git a/docs/how-to/replay-tasks-from-latest-crew-kickoff.mdx b/docs/learn/replay-tasks-from-latest-crew-kickoff.mdx similarity index 100% rename from docs/how-to/replay-tasks-from-latest-crew-kickoff.mdx rename to docs/learn/replay-tasks-from-latest-crew-kickoff.mdx diff --git a/docs/how-to/sequential-process.mdx b/docs/learn/sequential-process.mdx similarity index 100% rename from docs/how-to/sequential-process.mdx rename to docs/learn/sequential-process.mdx diff --git a/docs/how-to/using-annotations.mdx b/docs/learn/using-annotations.mdx similarity index 100% rename from docs/how-to/using-annotations.mdx rename to docs/learn/using-annotations.mdx diff --git a/docs/how-to/agentops-observability.mdx b/docs/observability/agentops.mdx similarity index 100% rename from docs/how-to/agentops-observability.mdx rename to docs/observability/agentops.mdx diff --git a/docs/how-to/arize-phoenix-observability.mdx b/docs/observability/arize-phoenix.mdx similarity index 100% rename from docs/how-to/arize-phoenix-observability.mdx rename to docs/observability/arize-phoenix.mdx diff --git a/docs/how-to/langfuse-observability.mdx b/docs/observability/langfuse.mdx similarity index 100% rename from docs/how-to/langfuse-observability.mdx rename to docs/observability/langfuse.mdx diff --git a/docs/how-to/langtrace-observability.mdx b/docs/observability/langtrace.mdx similarity index 100% rename from docs/how-to/langtrace-observability.mdx rename to docs/observability/langtrace.mdx diff --git a/docs/how-to/mlflow-observability.mdx b/docs/observability/mlflow.mdx similarity index 100% rename from docs/how-to/mlflow-observability.mdx rename to docs/observability/mlflow.mdx diff --git a/docs/how-to/openlit-observability.mdx b/docs/observability/openlit.mdx similarity index 100% rename from docs/how-to/openlit-observability.mdx rename to docs/observability/openlit.mdx diff --git a/docs/how-to/opik-observability.mdx b/docs/observability/opik.mdx similarity index 100% rename from docs/how-to/opik-observability.mdx rename to docs/observability/opik.mdx diff --git a/docs/observability/overview.mdx b/docs/observability/overview.mdx new file mode 100644 index 000000000..72d4ec276 --- /dev/null +++ b/docs/observability/overview.mdx @@ -0,0 +1,118 @@ +--- +title: "Overview" +description: "Monitor, evaluate, and optimize your CrewAI agents with comprehensive observability tools" +icon: "face-smile" +--- + +## Observability for CrewAI + +Observability is crucial for understanding how your CrewAI agents perform, identifying bottlenecks, and ensuring reliable operation in production environments. This section covers various tools and platforms that provide monitoring, evaluation, and optimization capabilities for your agent workflows. + +## Why Observability Matters + +- **Performance Monitoring**: Track agent execution times, token usage, and resource consumption +- **Quality Assurance**: Evaluate output quality and consistency across different scenarios +- **Debugging**: Identify and resolve issues in agent behavior and task execution +- **Cost Management**: Monitor LLM API usage and associated costs +- **Continuous Improvement**: Gather insights to optimize agent performance over time + +## Available Observability Tools + +### Monitoring & Tracing Platforms + + + + Session replays, metrics, and monitoring for agent development and production. + + + + OpenTelemetry-native monitoring with cost tracking and performance analytics. + + + + Machine learning lifecycle management with tracing and evaluation capabilities. + + + + LLM engineering platform with detailed tracing and analytics. + + + + Open-source observability for LLMs and agent frameworks. + + + + AI observability platform for monitoring and troubleshooting. + + + + AI gateway with comprehensive monitoring and reliability features. + + + + Debug, evaluate, and monitor LLM applications with comprehensive tracing. + + + + Weights & Biases platform for tracking and evaluating AI applications. + + + +### Evaluation & Quality Assurance + + + + Comprehensive evaluation platform for LLM outputs and agent behaviors. + + + +## Key Observability Metrics + +### Performance Metrics +- **Execution Time**: How long agents take to complete tasks +- **Token Usage**: Input/output tokens consumed by LLM calls +- **API Latency**: Response times from external services +- **Success Rate**: Percentage of successfully completed tasks + +### Quality Metrics +- **Output Accuracy**: Correctness of agent responses +- **Consistency**: Reliability across similar inputs +- **Relevance**: How well outputs match expected results +- **Safety**: Compliance with content policies and guidelines + +### Cost Metrics +- **API Costs**: Expenses from LLM provider usage +- **Resource Utilization**: Compute and memory consumption +- **Cost per Task**: Economic efficiency of agent operations +- **Budget Tracking**: Monitoring against spending limits + +## Getting Started + +1. **Choose Your Tools**: Select observability platforms that match your needs +2. **Instrument Your Code**: Add monitoring to your CrewAI applications +3. **Set Up Dashboards**: Configure visualizations for key metrics +4. **Define Alerts**: Create notifications for important events +5. **Establish Baselines**: Measure initial performance for comparison +6. **Iterate and Improve**: Use insights to optimize your agents + +## Best Practices + +### Development Phase +- Use detailed tracing to understand agent behavior +- Implement evaluation metrics early in development +- Monitor resource usage during testing +- Set up automated quality checks + +### Production Phase +- Implement comprehensive monitoring and alerting +- Track performance trends over time +- Monitor for anomalies and degradation +- Maintain cost visibility and control + +### Continuous Improvement +- Regular performance reviews and optimization +- A/B testing of different agent configurations +- Feedback loops for quality improvement +- Documentation of lessons learned + +Choose the observability tools that best fit your use case, infrastructure, and monitoring requirements to ensure your CrewAI agents perform reliably and efficiently. \ No newline at end of file diff --git a/docs/tools/ai-ml/patronustools.mdx b/docs/observability/patronus-evaluation.mdx similarity index 85% rename from docs/tools/ai-ml/patronustools.mdx rename to docs/observability/patronus-evaluation.mdx index 0a5a1d25f..391719d30 100644 --- a/docs/tools/ai-ml/patronustools.mdx +++ b/docs/observability/patronus-evaluation.mdx @@ -1,16 +1,26 @@ --- -title: Patronus Evaluation Tools -description: The Patronus evaluation tools enable CrewAI agents to evaluate and score model inputs and outputs using the Patronus AI platform. -icon: check +title: Patronus AI Evaluation +description: Monitor and evaluate CrewAI agent performance using Patronus AI's comprehensive evaluation platform for LLM outputs and agent behaviors. +icon: shield-check --- -# `Patronus Evaluation Tools` +# Patronus AI Evaluation -## Description +## Overview -The [Patronus evaluation tools](https://patronus.ai) are designed to enable CrewAI agents to evaluate and score model inputs and outputs using the Patronus AI platform. These tools provide different levels of control over the evaluation process, from allowing agents to select the most appropriate evaluator and criteria to using predefined criteria or custom local evaluators. +[Patronus AI](https://patronus.ai) provides comprehensive evaluation and monitoring capabilities for CrewAI agents, enabling you to assess model outputs, agent behaviors, and overall system performance. This integration allows you to implement continuous evaluation workflows that help maintain quality and reliability in production environments. -There are three main Patronus evaluation tools: +## Key Features + +- **Automated Evaluation**: Real-time assessment of agent outputs and behaviors +- **Custom Criteria**: Define specific evaluation criteria tailored to your use cases +- **Performance Monitoring**: Track agent performance metrics over time +- **Quality Assurance**: Ensure consistent output quality across different scenarios +- **Safety & Compliance**: Monitor for potential issues and policy violations + +## Evaluation Tools + +Patronus provides three main evaluation tools for different use cases: 1. **PatronusEvalTool**: Allows agents to select the most appropriate evaluator and criteria for the evaluation task. 2. **PatronusPredefinedCriteriaEvalTool**: Uses predefined evaluator and criteria specified by the user. diff --git a/docs/how-to/portkey-observability.mdx b/docs/observability/portkey.mdx similarity index 100% rename from docs/how-to/portkey-observability.mdx rename to docs/observability/portkey.mdx diff --git a/docs/how-to/weave-integration.mdx b/docs/observability/weave.mdx similarity index 100% rename from docs/how-to/weave-integration.mdx rename to docs/observability/weave.mdx diff --git a/docs/tools/ai-ml/overview.mdx b/docs/tools/ai-ml/overview.mdx index 3fcaf0ef4..dc92df561 100644 --- a/docs/tools/ai-ml/overview.mdx +++ b/docs/tools/ai-ml/overview.mdx @@ -37,9 +37,7 @@ These tools integrate with AI and machine learning services to enhance your agen Execute Python code and perform data analysis. - - AI safety and content moderation capabilities. - + ## **Common Use Cases**