Merge branch 'main' into lorenze/integration-docs-2

- Introduced detailed documentation for integrations including Asana, Box, ClickUp, GitHub, Gmail, Google Calendar, Google Sheets, HubSpot, Jira, Linear, Notion, Salesforce, Shopify, Slack, Stripe, and Zendesk.
- Updated main docs.json to include a new "Integration Docs" section, organizing the documentation for easy access.
- Each integration includes setup instructions, available actions, and example tasks to streamline user onboarding and usage.

.github/workflows/mkdocs.yml

@@ -0,0 +1,45 @@
+name: Deploy MkDocs
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  contents: write
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.10'
+
+      - name: Calculate requirements hash
+        id: req-hash
+        run: echo "::set-output name=hash::$(sha256sum requirements-doc.txt | awk '{print $1}')"
+
+      - name: Setup cache
+        uses: actions/cache@v4
+        with:
+          key: mkdocs-material-${{ steps.req-hash.outputs.hash }}
+          path: .cache
+          restore-keys: |
+            mkdocs-material-
+
+      - name: Install Requirements
+        run: |
+          sudo apt-get update &&
+          sudo apt-get install pngquant &&
+          pip install mkdocs-material mkdocs-material-extensions pillow cairosvg
+
+        env:
+          GH_TOKEN: ${{ secrets.GH_TOKEN }}
+
+      - name: Build and deploy MkDocs
+        run: mkdocs gh-deploy --force

docs/concepts/cli.mdx

@@ -285,32 +285,25 @@ Watch this video tutorial for a step-by-step demonstration of deploying your cre
 
 ### 11. API Keys
 
-When running ```crewai create crew``` command, the CLI will show you a list of available LLM providers to choose from, followed by model selection for your chosen provider.
+When running ```crewai create crew``` command, the CLI will first show you the top 5 most common LLM providers and ask you to select one.
 
-Once you've selected an LLM provider and model, you will be prompted for API keys.
+Once you've selected an LLM provider, you will be prompted for API keys.
 
-#### Available LLM Providers
+#### Initial API key providers
 
-The CLI will show you the following LLM providers to choose from:
+The CLI will initially prompt for API keys for the following services:
 
 * OpenAI
+* Groq
 * Anthropic
 * Google Gemini
-* NVIDIA NIM
-* Groq
-* Hugging Face
-* Ollama
-* Watson
-* AWS Bedrock
-* Azure
-* Cerebras
 * SambaNova
 
-When you select a provider, the CLI will then show you available models for that provider and prompt you to enter your API key.
+When you select a provider, the CLI will prompt you to enter your API key.
 
 #### Other Options
 
-If you select "other", you will be able to select from a list of LiteLLM supported providers.
+If you select option 6, you will be able to select from a list of LiteLLM supported providers.
 
 When you select a provider, the CLI will prompt you to enter the Key name and the API key.
 

docs/docs.json

@@ -134,7 +134,7 @@
                   "tools/web-scraping/stagehandtool",
                   "tools/web-scraping/firecrawlcrawlwebsitetool",
                   "tools/web-scraping/firecrawlscrapewebsitetool",
-                  "tools/web-scraping/oxylabsscraperstool"
+                  "tools/web-scraping/firecrawlsearchtool"
                 ]
               },
               {

docs/enterprise/features/integrations.mdx

@@ -124,7 +124,7 @@ from crewai_tools import CrewaiEnterpriseTools
 enterprise_tools = CrewaiEnterpriseTools(
     actions_list=["gmail_find_email"] # only gmail_find_email tool will be available
 )
-gmail_tool = enterprise_tools["gmail_find_email"]
+gmail_tool = enterprise_tools[0]
 
 gmail_agent = Agent(
     role="Gmail Manager",

docs/mcp/overview.mdx

@@ -85,22 +85,6 @@ with MCPServerAdapter(server_params) as mcp_tools:
 ```
 This general pattern shows how to integrate tools. For specific examples tailored to each transport, refer to the detailed guides below.
 
-## Filtering Tools
-
-```python
-with MCPServerAdapter(server_params) as mcp_tools:
-    print(f"Available tools: {[tool.name for tool in mcp_tools]}")
-
-    my_agent = Agent(
-        role="MCP Tool User",
-        goal="Utilize tools from an MCP server.",
-        backstory="I can connect to MCP servers and use their tools.",
-        tools=mcp_tools["tool_name"], # Pass the loaded tools to your agent
-        reasoning=True,
-        verbose=True
-    )
-    # ... rest of your crew setup ...
-```
 ## Explore MCP Integrations
 
 <CardGroup cols={2}>

docs/observability/langfuse.mdx

@@ -30,29 +30,18 @@ Set your Langfuse API keys and configure OpenTelemetry export settings to send t
 
 ```python
 import os
- 
-# Get keys for your project from the project settings page: https://cloud.langfuse.com
-os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..." 
-os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."
-os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com" # 🇪🇺 EU region
-# os.environ["LANGFUSE_HOST"] = "https://us.cloud.langfuse.com" # 🇺🇸 US region
- 
- 
-# Your OpenAI key
-os.environ["OPENAI_API_KEY"] = "sk-proj-..."
-```
-With the environment variables set, we can now initialize the Langfuse client. get_client() initializes the Langfuse client using the credentials provided in the environment variables.
+import base64
 
-```python
-from langfuse import get_client
- 
-langfuse = get_client()
- 
-# Verify connection
-if langfuse.auth_check():
-    print("Langfuse client is authenticated and ready!")
-else:
-    print("Authentication failed. Please check your credentials and host.")
+LANGFUSE_PUBLIC_KEY="pk-lf-..."
+LANGFUSE_SECRET_KEY="sk-lf-..."
+LANGFUSE_AUTH=base64.b64encode(f"{LANGFUSE_PUBLIC_KEY}:{LANGFUSE_SECRET_KEY}".encode()).decode()
+
+os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://cloud.langfuse.com/api/public/otel" # EU data region
+# os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://us.cloud.langfuse.com/api/public/otel" # US data region
+os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = f"Authorization=Basic {LANGFUSE_AUTH}"
+
+# your openai key
+os.environ["OPENAI_API_KEY"] = "sk-..."
 ```
 
 ### Step 3: Initialize OpenLit

docs/tools/web-scraping/overview.mdx

@@ -56,10 +56,6 @@ These tools enable your agents to interact with the web, extract data from websi
   <Card title="Stagehand Tool" icon="hand" href="/tools/web-scraping/stagehandtool">
     Intelligent browser automation with natural language commands.
   </Card>
-
-  <Card title="Oxylabs Scraper Tool" icon="globe" href="/tools/web-scraping/oxylabsscraperstool">
-    Access web data at scale with Oxylabs.
-  </Card>
 </CardGroup>
 
 ## **Common Use Cases**
@@ -104,4 +100,4 @@ agent = Agent(
 - **JavaScript-Heavy Sites**: Use `SeleniumScrapingTool` for dynamic content
 - **Scale & Performance**: Use `FirecrawlScrapeWebsiteTool` for high-volume scraping
 - **Cloud Infrastructure**: Use `BrowserBaseLoadTool` for scalable browser automation
-- **Complex Workflows**: Use `StagehandTool` for intelligent browser interactions
+- **Complex Workflows**: Use `StagehandTool` for intelligent browser interactions 

docs/tools/web-scraping/oxylabsscraperstool.mdx

@@ -1,236 +0,0 @@
----
-title: Oxylabs Scrapers
-description: >
-  Oxylabs Scrapers allow to easily access the information from the respective sources. Please see the list of available sources below:
-    - `Amazon Product`
-    - `Amazon Search`
-    - `Google Seach`
-    - `Universal`
-icon: globe
----
-
-## Installation
-
-Get the credentials by creating an Oxylabs Account [here](https://oxylabs.io).
-```shell
-pip install 'crewai[tools]' oxylabs
-```
-Check [Oxylabs Documentation](https://developers.oxylabs.io/scraping-solutions/web-scraper-api/targets) to get more information about API parameters.
-
-# `OxylabsAmazonProductScraperTool`
-
-### Example
-
-```python
-from crewai_tools import OxylabsAmazonProductScraperTool
-
-# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
-tool = OxylabsAmazonProductScraperTool()
-
-result = tool.run(query="AAAAABBBBCC")
-
-print(result)
-```
-
-### Parameters
-
-- `query` - 10-symbol ASIN code.
-- `domain` - domain localization for Amazon.
-- `geo_location` - the _Deliver to_ location.
-- `user_agent_type` - device type and browser.
-- `render` - enables JavaScript rendering when set to `html`.
-- `callback_url` - URL to your callback endpoint.
-- `context` - Additional advanced settings and controls for specialized requirements.
-- `parse` - returns parsed data when set to true.
-- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
-
-### Advanced example
-
-```python
-from crewai_tools import OxylabsAmazonProductScraperTool
-
-# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
-tool = OxylabsAmazonProductScraperTool(
-    config={
-        "domain": "com",
-        "parse": True,
-        "context": [
-            {
-                "key": "autoselect_variant",
-                "value": True
-            }
-        ]
-    }
-)
-
-result = tool.run(query="AAAAABBBBCC")
-
-print(result)
-```
-
-# `OxylabsAmazonSearchScraperTool`
-
-### Example
-
-```python
-from crewai_tools import OxylabsAmazonSearchScraperTool
-
-# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
-tool = OxylabsAmazonSearchScraperTool()
-
-result = tool.run(query="headsets")
-
-print(result)
-```
-
-### Parameters
-
-- `query` - Amazon search term.
-- `domain` - Domain localization for Bestbuy.
-- `start_page` - starting page number.
-- `pages` - number of pages to retrieve.
-- `geo_location` - the _Deliver to_ location.
-- `user_agent_type` - device type and browser.
-- `render` - enables JavaScript rendering when set to `html`.
-- `callback_url` - URL to your callback endpoint.
-- `context` - Additional advanced settings and controls for specialized requirements.
-- `parse` - returns parsed data when set to true.
-- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
-
-### Advanced example
-
-```python
-from crewai_tools import OxylabsAmazonSearchScraperTool
-
-# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
-tool = OxylabsAmazonSearchScraperTool(
-    config={
-        "domain": 'nl',
-        "start_page": 2,
-        "pages": 2,
-        "parse": True,
-        "context": [
-            {'key': 'category_id', 'value': 16391693031}
-        ],
-    }
-)
-
-result = tool.run(query='nirvana tshirt')
-
-print(result)
-```
-
-# `OxylabsGoogleSearchScraperTool`
-
-### Example
-
-```python
-from crewai_tools import OxylabsGoogleSearchScraperTool
-
-# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
-tool = OxylabsGoogleSearchScraperTool()
-
-result = tool.run(query="iPhone 16")
-
-print(result)
-```
-
-### Parameters
-
-- `query` - search keyword.
-- `domain` - domain localization for Google.
-- `start_page` - starting page number.
-- `pages` - number of pages to retrieve.
-- `limit` - number of results to retrieve in each page.
-- `locale` - `Accept-Language` header value which changes your Google search page web interface language.
-- `geo_location` - the geographical location that the result should be adapted for. Using this parameter correctly is extremely important to get the right data.
-- `user_agent_type` - device type and browser.
-- `render` - enables JavaScript rendering when set to `html`.
-- `callback_url` - URL to your callback endpoint.
-- `context` - Additional advanced settings and controls for specialized requirements.
-- `parse` - returns parsed data when set to true.
-- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
-
-### Advanced example
-
-```python
-from crewai_tools import OxylabsGoogleSearchScraperTool
-
-# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
-tool = OxylabsGoogleSearchScraperTool(
-    config={
-        "parse": True,
-        "geo_location": "Paris, France",
-        "user_agent_type": "tablet",
-    }
-)
-
-result = tool.run(query="iPhone 16")
-
-print(result)
-```
-
-# `OxylabsUniversalScraperTool`
-
-### Example
-
-```python
-from crewai_tools import OxylabsUniversalScraperTool
-
-# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
-tool = OxylabsUniversalScraperTool()
-
-result = tool.run(url="https://ip.oxylabs.io")
-
-print(result)
-```
-
-### Parameters
-
-- `url` - website url to scrape.
-- `user_agent_type` - device type and browser.
-- `geo_location` - sets the proxy's geolocation to retrieve data.
-- `render` - enables JavaScript rendering when set to `html`.
-- `callback_url` - URL to your callback endpoint.
-- `context` - Additional advanced settings and controls for specialized requirements.
-- `parse` - returns parsed data when set to `true`, as long as a dedicated parser exists for the submitted URL's page type.
-- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
-
-
-### Advanced example
-
-```python
-from crewai_tools import OxylabsUniversalScraperTool
-
-# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
-tool = OxylabsUniversalScraperTool(
-    config={
-        "render": "html",
-        "user_agent_type": "mobile",
-        "context": [
-            {"key": "force_headers", "value": True},
-            {"key": "force_cookies", "value": True},
-            {
-                "key": "headers",
-                "value": {
-                    "Custom-Header-Name": "custom header content",
-                },
-            },
-            {
-                "key": "cookies",
-                "value": [
-                    {"key": "NID", "value": "1234567890"},
-                    {"key": "1P JAR", "value": "0987654321"},
-                ],
-            },
-            {"key": "http_method", "value": "get"},
-            {"key": "follow_redirects", "value": True},
-            {"key": "successful_status_codes", "value": [808, 909]},
-        ],
-    }
-)
-
-result = tool.run(url="https://ip.oxylabs.io")
-
-print(result)
-```

mkdocs.yml

@@ -0,0 +1,216 @@
+site_name: crewAI
+site_author: crewAI, Inc
+site_description: Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
+repo_name: crewAI
+repo_url: https://github.com/crewAIInc/crewAI
+site_url: https://docs.crewai.com
+edit_uri: edit/main/docs/
+copyright: Copyright © 2024 crewAI, Inc
+
+markdown_extensions:
+  - abbr
+  - admonition
+  - pymdownx.details
+  - attr_list
+  - def_list
+  - footnotes
+  - md_in_html
+  - toc:
+      permalink: true
+  - pymdownx.arithmatex:
+      generic: true
+  - pymdownx.betterem:
+      smart_enable: all
+  - pymdownx.caret
+  - pymdownx.emoji:
+      emoji_generator: !!python/name:material.extensions.emoji.to_svg
+      emoji_index: !!python/name:material.extensions.emoji.twemoji
+  - pymdownx.highlight:
+      anchor_linenums: true
+      line_spans: __span
+      pygments_lang_class: true
+  - pymdownx.inlinehilite
+  - pymdownx.keys
+  - pymdownx.magiclink:
+      normalize_issue_symbols: true
+      repo_url_shorthand: true
+      user: joaomdmoura
+      repo: crewAI
+  - pymdownx.mark
+  - pymdownx.smartsymbols
+  - pymdownx.snippets:
+      auto_append:
+        - includes/mkdocs.md
+  - pymdownx.superfences:
+      custom_fences:
+        - name: mermaid
+          class: mermaid
+          format: !!python/name:pymdownx.superfences.fence_code_format
+  - pymdownx.tabbed:
+      alternate_style: true
+      combine_header_slug: true
+      slugify: !!python/object/apply:pymdownx.slugs.slugify
+        kwds:
+          case: lower
+  - pymdownx.tasklist:
+      custom_checkbox: true
+  - pymdownx.tilde
+theme:
+  name: material
+  language: en
+  icon:
+    repo: fontawesome/brands/github
+    edit: material/pencil
+    view: material/eye
+    admonition:
+      note: octicons/light-bulb-16
+      abstract: octicons/checklist-16
+      info: octicons/info-16
+      tip: octicons/squirrel-16
+      success: octicons/check-16
+      question: octicons/question-16
+      warning: octicons/alert-16
+      failure: octicons/x-circle-16
+      danger: octicons/zap-16
+      bug: octicons/bug-16
+      example: octicons/beaker-16
+      quote: octicons/quote-16
+
+  palette:
+    - scheme: default
+      primary: deep orange
+      accent: deep orange
+      toggle:
+        icon: material/brightness-7
+        name: Switch to dark mode
+    - scheme: slate
+      primary: deep orange
+      accent: deep orange
+      toggle:
+        icon: material/brightness-4
+        name: Switch to light mode
+  features:
+    - announce.dismiss
+    - content.action.edit
+    - content.action.view
+    - content.code.annotate
+    - content.code.copy
+    - content.code.select
+    - content.tabs.link
+    - content.tooltips
+    - header.autohide
+    - navigation.footer
+    - navigation.indexes
+    # - navigation.prune
+    # - navigation.sections
+    # - navigation.tabs
+    - search.suggest
+    - navigation.instant
+    - navigation.instant.progress
+    - navigation.instant.prefetch
+    - navigation.tracking
+    # - navigation.expand
+    - navigation.path
+    - navigation.top
+    - toc.follow
+    - toc.integrate
+    - search.highlight
+    - search.share
+
+nav:
+  - Home: '/'
+  - Getting Started:
+    - Installing CrewAI: 'getting-started/Installing-CrewAI.md'
+    - Starting a new CrewAI project: 'getting-started/Start-a-New-CrewAI-Project-Template-Method.md'
+  - Core Concepts:
+    - Agents: 'core-concepts/Agents.md'
+    - Tasks: 'core-concepts/Tasks.md'
+    - Tools: 'core-concepts/Tools.md'
+    - Processes: 'core-concepts/Processes.md'
+    - Crews: 'core-concepts/Crews.md'
+    - Collaboration: 'core-concepts/Collaboration.md'
+    - Training: 'core-concepts/Training-Crew.md'
+    - Memory: 'core-concepts/Memory.md'
+    - Planning: 'core-concepts/Planning.md'
+    - Testing: 'core-concepts/Testing.md'
+    - Using LangChain Tools: 'core-concepts/Using-LangChain-Tools.md'
+    - Using LlamaIndex Tools: 'core-concepts/Using-LlamaIndex-Tools.md'
+  - How to Guides:
+    - Create Custom Tools: 'how-to/Create-Custom-Tools.md'
+    - Using Sequential Process: 'how-to/Sequential.md'
+    - Using Hierarchical Process: 'how-to/Hierarchical.md'
+    - Create your own Manager Agent: 'how-to/Your-Own-Manager-Agent.md'
+    - Connecting to any LLM: 'how-to/LLM-Connections.md'
+    - Customizing Agents: 'how-to/Customizing-Agents.md'
+    - Coding Agents: 'how-to/Coding-Agents.md'
+    - Forcing Tool Output as Result: 'how-to/Force-Tool-Ouput-as-Result.md'
+    - Human Input on Execution: 'how-to/Human-Input-on-Execution.md'
+    - Kickoff a Crew Asynchronously: 'how-to/Kickoff-async.md'
+    - Kickoff a Crew for a List: 'how-to/Kickoff-for-each.md'
+    - Replay from a specific task from a kickoff: 'how-to/Replay-tasks-from-latest-Crew-Kickoff.md'
+    - Conditional Tasks: 'how-to/Conditional-Tasks.md'
+    - Agent Monitoring with AgentOps: 'how-to/AgentOps-Observability.md'
+    - Agent Monitoring with LangTrace: 'how-to/Langtrace-Observability.md'
+    - Agent Monitoring with OpenLIT: 'how-to/openlit-Observability.md'
+    - Agent Monitoring with MLflow: 'how-to/mlflow-Observability.md'
+  - Tools Docs:
+    - Browserbase Web Loader: 'tools/BrowserbaseLoadTool.md'
+    - Code Docs RAG Search: 'tools/CodeDocsSearchTool.md'
+    - Code Interpreter: 'tools/CodeInterpreterTool.md'
+    - Composio Tools: 'tools/ComposioTool.md'
+    - CSV RAG Search: 'tools/CSVSearchTool.md'
+    - DALL-E Tool: 'tools/DALL-ETool.md'
+    - Directory RAG Search: 'tools/DirectorySearchTool.md'
+    - Directory Read: 'tools/DirectoryReadTool.md'
+    - Docx Rag Search: 'tools/DOCXSearchTool.md'
+    - EXA Search Web Loader: 'tools/EXASearchTool.md'
+    - File Read: 'tools/FileReadTool.md'
+    - File Write: 'tools/FileWriteTool.md'
+    - Firecrawl Crawl Website Tool: 'tools/FirecrawlCrawlWebsiteTool.md'
+    - Firecrawl Scrape Website Tool: 'tools/FirecrawlScrapeWebsiteTool.md'
+    - Firecrawl Search Tool: 'tools/FirecrgstawlSearchTool.md'
+    - Github RAG Search: 'tools/GitHubSearchTool.md'
+    - Google Serper Search: 'tools/SerperDevTool.md'
+    - JSON RAG Search: 'tools/JSONSearchTool.md'
+    - MDX RAG Search: 'tools/MDXSearchTool.md'
+    - MySQL Tool: 'tools/MySQLTool.md'
+    - NL2SQL Tool: 'tools/NL2SQLTool.md'
+    - PDF RAG Search: 'tools/PDFSearchTool.md'
+    - PG RAG Search: 'tools/PGSearchTool.md'
+    - Scrape Website: 'tools/ScrapeWebsiteTool.md'
+    - Selenium Scraper: 'tools/SeleniumScrapingTool.md'
+    - Spider Scraper: 'tools/SpiderTool.md'
+    - TXT RAG Search: 'tools/TXTSearchTool.md'
+    - Vision Tool: 'tools/VisionTool.md'
+    - Website RAG Search: 'tools/WebsiteSearchTool.md'
+    - XML RAG Search: 'tools/XMLSearchTool.md'
+    - Youtube Channel RAG Search: 'tools/YoutubeChannelSearchTool.md'
+    - Youtube Video RAG Search: 'tools/YoutubeVideoSearchTool.md'
+  - Examples:
+    - Trip Planner Crew: https://github.com/joaomdmoura/crewAI-examples/tree/main/trip_planner"
+    - Create Instagram Post: https://github.com/joaomdmoura/crewAI-examples/tree/main/instagram_post"
+    - Stock Analysis: https://github.com/joaomdmoura/crewAI-examples/tree/main/stock_analysis"
+    - Game Generator: https://github.com/joaomdmoura/crewAI-examples/tree/main/game-builder-crew"
+    - Drafting emails with LangGraph: https://github.com/joaomdmoura/crewAI-examples/tree/main/CrewAI-LangGraph"
+    - Landing Page Generator: https://github.com/joaomdmoura/crewAI-examples/tree/main/landing_page_generator"
+    - Prepare for meetings: https://github.com/joaomdmoura/crewAI-examples/tree/main/prep-for-a-meeting"
+  - Telemetry: 'telemetry/Telemetry.md'
+  - Change Log: 'https://github.com/crewAIInc/crewAI/releases'
+
+extra_css:
+  - stylesheets/output.css
+  - stylesheets/extra.css
+
+plugins:
+  - social
+  - search
+
+extra:
+  analytics:
+    provider: google
+    property: G-N3Q505TMQ6
+  social:
+    - icon: fontawesome/brands/x-twitter
+      link: https://x.com/crewAIInc
+    - icon: fontawesome/brands/github
+      link: https://github.com/crewAIInc/crewAI

pyproject.toml

@@ -74,6 +74,11 @@ dev-dependencies = [
     "ruff>=0.8.2",
     "mypy>=1.10.0",
     "pre-commit>=3.6.0",
+    "mkdocs>=1.4.3",
+    "mkdocstrings>=0.22.0",
+    "mkdocstrings-python>=1.1.2",
+    "mkdocs-material>=9.5.7",
+    "mkdocs-material-extensions>=1.3.1",
     "pillow>=10.2.0",
     "cairosvg>=2.7.1",
     "pytest>=8.0.0",

src/crewai/task.py

@@ -39,7 +39,7 @@ from crewai.tasks.output_format import OutputFormat
 from crewai.tasks.task_output import TaskOutput
 from crewai.tools.base_tool import BaseTool
 from crewai.utilities.config import process_config
-from crewai.utilities.constants import NOT_SPECIFIED, _NotSpecified
+from crewai.utilities.constants import NOT_SPECIFIED
 from crewai.utilities.guardrail import process_guardrail, GuardrailResult
 from crewai.utilities.converter import Converter, convert_to_model
 from crewai.utilities.events import (
@@ -95,9 +95,9 @@ class Task(BaseModel):
     agent: Optional[BaseAgent] = Field(
         description="Agent responsible for execution the task.", default=None
     )
-    context: Union[List["Task"], None, _NotSpecified] = Field(
+    context: Optional[List["Task"]] = Field(
         description="Other tasks that will have their output used as context for this task.",
-        default=NOT_SPECIFIED
+        default=NOT_SPECIFIED,
     )
     async_execution: Optional[bool] = Field(
         description="Whether the task should be executed asynchronously or not.",
@@ -158,9 +158,6 @@ class Task(BaseModel):
     end_time: Optional[datetime.datetime] = Field(
         default=None, description="End time of the task execution"
     )
-    model_config = {
-        "arbitrary_types_allowed": True
-    }
 
     @field_validator("guardrail")
     @classmethod

src/crewai/utilities/agent_utils.py

@@ -476,14 +476,7 @@ def load_agent_from_repository(from_repository: str) -> Dict[str, Any]:
                     try:
                         module = importlib.import_module(tool["module"])
                         tool_class = getattr(module, tool["name"])
-
-                        tool_value = tool_class(**tool["init_params"])
-
-                        if isinstance(tool_value, list):
-                            attributes[key].extend(tool_value)
-                        else:
-                            attributes[key].append(tool_value)
-
+                        attributes[key].append(tool_class())
                     except Exception as e:
                         raise AgentRepositoryError(
                             f"Tool {tool['name']} could not be loaded: {e}"

src/crewai/utilities/formatter.py

@@ -1,5 +1,5 @@
-from typing import TYPE_CHECKING, List, Union
-from crewai.utilities.constants import _NotSpecified
+from typing import TYPE_CHECKING, List
+
 
 if TYPE_CHECKING:
     from crewai.task import Task
@@ -15,7 +15,7 @@ def aggregate_raw_outputs_from_task_outputs(task_outputs: List["TaskOutput"]) ->
     return context
 
 
-def aggregate_raw_outputs_from_tasks(tasks: Union[List["Task"],_NotSpecified]) -> str:
+def aggregate_raw_outputs_from_tasks(tasks: List["Task"]) -> str:
     """Generate string context from the tasks."""
 
     task_outputs = (

tests/agent_test.py

@@ -2099,7 +2099,7 @@ def mock_get_auth_token():
 
 @patch("crewai.cli.plus_api.PlusAPI.get_agent")
 def test_agent_from_repository(mock_get_agent, mock_get_auth_token):
-    from crewai_tools import SerperDevTool, XMLSearchTool, CSVSearchTool, EnterpriseActionTool
+    from crewai_tools import SerperDevTool, XMLSearchTool
 
     mock_get_response = MagicMock()
     mock_get_response.status_code = 200
@@ -2108,42 +2108,19 @@ def test_agent_from_repository(mock_get_agent, mock_get_auth_token):
         "goal": "test goal",
         "backstory": "test backstory",
         "tools": [
-            {"module": "crewai_tools", "name": "SerperDevTool", "init_params": {"n_results": 30}},
-            {"module": "crewai_tools", "name": "XMLSearchTool", "init_params": {"summarize": True}},
-            {"module": "crewai_tools", "name": "CSVSearchTool", "init_params": {}},
-
-            # using a tools that returns a list of BaseTools
-            {"module": "crewai_tools", "name": "CrewaiEnterpriseTools", "init_params": {"actions_list": [], "enterprise_token": "test_key"}},
+            {"module": "crewai_tools", "name": "SerperDevTool"},
+            {"module": "crewai_tools", "name": "XMLSearchTool"},
         ],
     }
     mock_get_agent.return_value = mock_get_response
-
-    tool_action = EnterpriseActionTool(
-        name="test_name",
-        description="test_description",
-        enterprise_action_token="test_token",
-        action_name="test_action_name",
-        action_schema={"test": "test"},
-    )
-
-    with patch("crewai_tools.CrewaiEnterpriseTools", return_value=[tool_action]):
-        agent = Agent(from_repository="test_agent")
+    agent = Agent(from_repository="test_agent")
 
     assert agent.role == "test role"
     assert agent.goal == "test goal"
     assert agent.backstory == "test backstory"
-    assert len(agent.tools) == 4
-
+    assert len(agent.tools) == 2
     assert isinstance(agent.tools[0], SerperDevTool)
-    assert agent.tools[0].n_results == 30
     assert isinstance(agent.tools[1], XMLSearchTool)
-    assert agent.tools[1].summarize
-
-    assert isinstance(agent.tools[2], CSVSearchTool)
-    assert not agent.tools[2].summarize
-
-    assert isinstance(agent.tools[3], EnterpriseActionTool)
-    assert agent.tools[3].name == "test_name"
 
 
 @patch("crewai.cli.plus_api.PlusAPI.get_agent")
@@ -2156,7 +2133,7 @@ def test_agent_from_repository_override_attributes(mock_get_agent, mock_get_auth
         "role": "test role",
         "goal": "test goal",
         "backstory": "test backstory",
-        "tools": [{"name": "SerperDevTool", "module": "crewai_tools", "init_params": {}}],
+        "tools": [{"name": "SerperDevTool", "module": "crewai_tools"}],
     }
     mock_get_agent.return_value = mock_get_response
     agent = Agent(from_repository="test_agent", role="Custom Role")

tests/test_cli_documentation_sync.py

@@ -1,43 +0,0 @@
-from pathlib import Path
-
-from crewai.cli.constants import PROVIDERS
-
-
-def test_cli_documentation_matches_providers():
-    """Test that CLI documentation accurately reflects the available providers."""
-    docs_path = Path(__file__).parent.parent / "docs" / "concepts" / "cli.mdx"
-    with open(docs_path, 'r') as f:
-        docs_content = f.read()
-    
-    assert "top 5" not in docs_content.lower(), "Documentation should not mention 'top 5' providers"
-    assert "5 most common" not in docs_content.lower(), "Documentation should not mention '5 most common' providers"
-    
-    assert "list of available LLM providers" in docs_content or "following LLM providers" in docs_content, \
-        "Documentation should mention the availability of multiple LLM providers"
-    
-    assert len(PROVIDERS) > 5, f"Expected more than 5 providers, but found {len(PROVIDERS)}"
-    
-    key_providers = ["OpenAI", "Anthropic", "Gemini"]
-    for provider in key_providers:
-        assert provider in docs_content, f"Key provider {provider} should be mentioned in documentation"
-
-
-def test_providers_list_matches_constants():
-    """Test that the actual PROVIDERS list has the expected providers."""
-    expected_providers = [
-        "openai",
-        "anthropic", 
-        "gemini",
-        "nvidia_nim",
-        "groq",
-        "huggingface",
-        "ollama",
-        "watson",
-        "bedrock",
-        "azure",
-        "cerebras",
-        "sambanova",
-    ]
-    
-    assert PROVIDERS == expected_providers, f"PROVIDERS list has changed. Expected {expected_providers}, got {PROVIDERS}"
-    assert len(PROVIDERS) == 12, f"Expected 12 providers, but found {len(PROVIDERS)}"

uv.lock

@@ -337,6 +337,15 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/a2/ee/3fd29bf416eb4f1c5579cf12bf393ae954099258abd7bde03c4f9716ef6b/autoflake-2.3.1-py3-none-any.whl", hash = "sha256:3ae7495db9084b7b32818b4140e6dc4fc280b712fb414f5b8fe57b0a8e85a840", size = 32483 },
 ]
 
+[[package]]
+name = "babel"
+version = "2.16.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/2a/74/f1bc80f23eeba13393b7222b11d95ca3af2c1e28edca18af487137eefed9/babel-2.16.0.tar.gz", hash = "sha256:d1f3554ca26605fe173f3de0c65f750f5a42f924499bf134de6423582298e316", size = 9348104 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/ed/20/bc79bc575ba2e2a7f70e8a1155618bb1301eaa5132a8271373a6903f73f8/babel-2.16.0-py3-none-any.whl", hash = "sha256:368b5b98b37c06b7daf6696391c3240c938b37767d4584413e8438c5c435fa8b", size = 9587599 },
+]
+
 [[package]]
 name = "backoff"
 version = "2.2.1"
@@ -778,6 +787,11 @@ tools = [
 [package.dev-dependencies]
 dev = [
     { name = "cairosvg" },
+    { name = "mkdocs" },
+    { name = "mkdocs-material" },
+    { name = "mkdocs-material-extensions" },
+    { name = "mkdocstrings" },
+    { name = "mkdocstrings-python" },
     { name = "mypy" },
     { name = "pillow" },
     { name = "pre-commit" },
@@ -832,6 +846,11 @@ requires-dist = [
 [package.metadata.requires-dev]
 dev = [
     { name = "cairosvg", specifier = ">=2.7.1" },
+    { name = "mkdocs", specifier = ">=1.4.3" },
+    { name = "mkdocs-material", specifier = ">=9.5.7" },
+    { name = "mkdocs-material-extensions", specifier = ">=1.3.1" },
+    { name = "mkdocstrings", specifier = ">=0.22.0" },
+    { name = "mkdocstrings-python", specifier = ">=1.1.2" },
     { name = "mypy", specifier = ">=1.10.0" },
     { name = "pillow", specifier = ">=10.2.0" },
     { name = "pre-commit", specifier = ">=3.6.0" },
@@ -1409,6 +1428,18 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/c6/b2/454d6e7f0158951d8a78c2e1eb4f69ae81beb8dca5fee9809c6c99e9d0d0/fsspec-2024.10.0-py3-none-any.whl", hash = "sha256:03b9a6785766a4de40368b88906366755e2819e758b83705c88cd7cb5fe81871", size = 179641 },
 ]
 
+[[package]]
+name = "ghp-import"
+version = "2.1.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "python-dateutil" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/d9/29/d40217cbe2f6b1359e00c6c307bb3fc876ba74068cbab3dde77f03ca0dc4/ghp-import-2.1.0.tar.gz", hash = "sha256:9c535c4c61193c2df8871222567d7fd7e5014d835f97dc7b7439069e2413d343", size = 10943 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f7/ec/67fbef5d497f86283db54c22eec6f6140243aae73265799baaaa19cd17fb/ghp_import-2.1.0-py3-none-any.whl", hash = "sha256:8337dd7b50877f163d4c0289bc1f1c7f127550241988d568c1db512c4324a619", size = 11034 },
+]
+
 [[package]]
 name = "google-auth"
 version = "2.35.0"
@@ -1500,6 +1531,18 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/ac/38/08cc303ddddc4b3d7c628c3039a61a3aae36c241ed01393d00c2fd663473/greenlet-3.1.1-cp313-cp313t-musllinux_1_1_x86_64.whl", hash = "sha256:411f015496fec93c1c8cd4e5238da364e1da7a124bcb293f085bf2860c32c6f6", size = 1142112 },
 ]
 
+[[package]]
+name = "griffe"
+version = "1.5.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "colorama" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/d4/c9/8167810358ca129839156dc002526e7398b5fad4a9d7b6e88b875e802d0d/griffe-1.5.1.tar.gz", hash = "sha256:72964f93e08c553257706d6cd2c42d1c172213feb48b2be386f243380b405d4b", size = 384113 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/ab/00/e693a155da0a2a72fd2df75b8fe338146cae59d590ad6f56800adde90cb5/griffe-1.5.1-py3-none-any.whl", hash = "sha256:ad6a7980f8c424c9102160aafa3bcdf799df0e75f7829d75af9ee5aef656f860", size = 127132 },
+]
+
 [[package]]
 name = "grpcio"
 version = "1.67.0"
@@ -2317,6 +2360,15 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/48/22/bc14c6f02e6dccaafb3eba95764c8f096714260c2aa5f76f654fd16a23dd/Mako-1.3.6-py3-none-any.whl", hash = "sha256:a91198468092a2f1a0de86ca92690fb0cfc43ca90ee17e15d93662b4c04b241a", size = 78557 },
 ]
 
+[[package]]
+name = "markdown"
+version = "3.7"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/54/28/3af612670f82f4c056911fbbbb42760255801b3068c48de792d354ff4472/markdown-3.7.tar.gz", hash = "sha256:2ae2471477cfd02dbbf038d5d9bc226d40def84b4fe2986e49b59b6b472bbed2", size = 357086 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/3f/08/83871f3c50fc983b88547c196d11cf8c3340e37c32d2e9d6152abe2c61f7/Markdown-3.7-py3-none-any.whl", hash = "sha256:7eb6df5690b81a1d7942992c97fad2938e956e79df20cbc6186e9c3a77b1c803", size = 106349 },
+]
+
 [[package]]
 name = "markdown-it-py"
 version = "3.0.0"
@@ -2447,6 +2499,131 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/0c/fa/a2043c5a228077423678d9e7e0fe3aab89ab9a625a4f171a30967de1bbb3/mem0ai-0.1.94-py3-none-any.whl", hash = "sha256:862aaccf4ec41d65a5c935dc49c7b8175c96f6b60b29abeda338d8a335027e2c", size = 143077 },
 ]
 
+[[package]]
+name = "mergedeep"
+version = "1.3.4"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/3a/41/580bb4006e3ed0361b8151a01d324fb03f420815446c7def45d02f74c270/mergedeep-1.3.4.tar.gz", hash = "sha256:0096d52e9dad9939c3d975a774666af186eda617e6ca84df4c94dec30004f2a8", size = 4661 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/2c/19/04f9b178c2d8a15b076c8b5140708fa6ffc5601fb6f1e975537072df5b2a/mergedeep-1.3.4-py3-none-any.whl", hash = "sha256:70775750742b25c0d8f36c55aed03d24c3384d17c951b3175d898bd778ef0307", size = 6354 },
+]
+
+[[package]]
+name = "mkdocs"
+version = "1.6.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "click" },
+    { name = "colorama", marker = "platform_system == 'Windows'" },
+    { name = "ghp-import" },
+    { name = "jinja2" },
+    { name = "markdown" },
+    { name = "markupsafe" },
+    { name = "mergedeep" },
+    { name = "mkdocs-get-deps" },
+    { name = "packaging" },
+    { name = "pathspec" },
+    { name = "pyyaml" },
+    { name = "pyyaml-env-tag" },
+    { name = "watchdog" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/bc/c6/bbd4f061bd16b378247f12953ffcb04786a618ce5e904b8c5a01a0309061/mkdocs-1.6.1.tar.gz", hash = "sha256:7b432f01d928c084353ab39c57282f29f92136665bdd6abf7c1ec8d822ef86f2", size = 3889159 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/22/5b/dbc6a8cddc9cfa9c4971d59fb12bb8d42e161b7e7f8cc89e49137c5b279c/mkdocs-1.6.1-py3-none-any.whl", hash = "sha256:db91759624d1647f3f34aa0c3f327dd2601beae39a366d6e064c03468d35c20e", size = 3864451 },
+]
+
+[[package]]
+name = "mkdocs-autorefs"
+version = "1.2.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "markdown" },
+    { name = "markupsafe" },
+    { name = "mkdocs" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/fb/ae/0f1154c614d6a8b8a36fff084e5b82af3a15f7d2060cf0dcdb1c53297a71/mkdocs_autorefs-1.2.0.tar.gz", hash = "sha256:a86b93abff653521bda71cf3fc5596342b7a23982093915cb74273f67522190f", size = 40262 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/71/26/4d39d52ea2219604053a4d05b98e90d6a335511cc01806436ec4886b1028/mkdocs_autorefs-1.2.0-py3-none-any.whl", hash = "sha256:d588754ae89bd0ced0c70c06f58566a4ee43471eeeee5202427da7de9ef85a2f", size = 16522 },
+]
+
+[[package]]
+name = "mkdocs-get-deps"
+version = "0.2.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "mergedeep" },
+    { name = "platformdirs" },
+    { name = "pyyaml" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/98/f5/ed29cd50067784976f25ed0ed6fcd3c2ce9eb90650aa3b2796ddf7b6870b/mkdocs_get_deps-0.2.0.tar.gz", hash = "sha256:162b3d129c7fad9b19abfdcb9c1458a651628e4b1dea628ac68790fb3061c60c", size = 10239 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/9f/d4/029f984e8d3f3b6b726bd33cafc473b75e9e44c0f7e80a5b29abc466bdea/mkdocs_get_deps-0.2.0-py3-none-any.whl", hash = "sha256:2bf11d0b133e77a0dd036abeeb06dec8775e46efa526dc70667d8863eefc6134", size = 9521 },
+]
+
+[[package]]
+name = "mkdocs-material"
+version = "9.5.42"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "babel" },
+    { name = "colorama" },
+    { name = "jinja2" },
+    { name = "markdown" },
+    { name = "mkdocs" },
+    { name = "mkdocs-material-extensions" },
+    { name = "paginate" },
+    { name = "pygments" },
+    { name = "pymdown-extensions" },
+    { name = "regex" },
+    { name = "requests" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/f9/33/b3343ed975fbbd6798b8d8a7c4f1bf8489cc321fc8fd426eba3d87b0242e/mkdocs_material-9.5.42.tar.gz", hash = "sha256:92779b5e9b5934540c574c11647131d217dc540dce72b05feeda088c8eb1b8f2", size = 3963891 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/55/55/ad3e6a60ac1e8e76025543c49c1f24ecd80fb38e8a57000403bf2f0a4293/mkdocs_material-9.5.42-py3-none-any.whl", hash = "sha256:452a7c5d21284b373f36b981a2cbebfff59263feebeede1bc28652e9c5bbe316", size = 8672619 },
+]
+
+[[package]]
+name = "mkdocs-material-extensions"
+version = "1.3.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/79/9b/9b4c96d6593b2a541e1cb8b34899a6d021d208bb357042823d4d2cabdbe7/mkdocs_material_extensions-1.3.1.tar.gz", hash = "sha256:10c9511cea88f568257f960358a467d12b970e1f7b2c0e5fb2bb48cab1928443", size = 11847 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/5b/54/662a4743aa81d9582ee9339d4ffa3c8fd40a4965e033d77b9da9774d3960/mkdocs_material_extensions-1.3.1-py3-none-any.whl", hash = "sha256:adff8b62700b25cb77b53358dad940f3ef973dd6db797907c49e3c2ef3ab4e31", size = 8728 },
+]
+
+[[package]]
+name = "mkdocstrings"
+version = "0.26.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "click" },
+    { name = "jinja2" },
+    { name = "markdown" },
+    { name = "markupsafe" },
+    { name = "mkdocs" },
+    { name = "mkdocs-autorefs" },
+    { name = "platformdirs" },
+    { name = "pymdown-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/c0/76/0475d10d27f3384df3a6ddfdf4a4fdfef83766f77cd4e327d905dc956c15/mkdocstrings-0.26.2.tar.gz", hash = "sha256:34a8b50f1e6cfd29546c6c09fbe02154adfb0b361bb758834bf56aa284ba876e", size = 92512 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/80/b6/4ee320d7c313da3774eff225875eb278f7e6bb26a9cd8e680b8dbc38fdea/mkdocstrings-0.26.2-py3-none-any.whl", hash = "sha256:1248f3228464f3b8d1a15bd91249ce1701fe3104ac517a5f167a0e01ca850ba5", size = 29716 },
+]
+
+[[package]]
+name = "mkdocstrings-python"
+version = "1.12.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "griffe" },
+    { name = "mkdocs-autorefs" },
+    { name = "mkdocstrings" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/23/ec/cb6debe2db77f1ef42b25b21d93b5021474de3037cd82385e586aee72545/mkdocstrings_python-1.12.2.tar.gz", hash = "sha256:7a1760941c0b52a2cd87b960a9e21112ffe52e7df9d0b9583d04d47ed2e186f3", size = 168207 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/5b/c1/ac524e1026d9580cbc654b5d19f5843c8b364a66d30f956372cd09fd2f92/mkdocstrings_python-1.12.2-py3-none-any.whl", hash = "sha256:7f7d40d6db3cb1f5d19dbcd80e3efe4d0ba32b073272c0c0de9de2e604eda62a", size = 111759 },
+]
+
 [[package]]
 name = "mmh3"
 version = "4.1.0"
@@ -3212,6 +3389,15 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl", hash = "sha256:8c491190033a9af7e1d931d0b5dacc2ef47509b34dd0de67ed209b5203fc88c7", size = 53011 },
 ]
 
+[[package]]
+name = "paginate"
+version = "0.5.7"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ec/46/68dde5b6bc00c1296ec6466ab27dddede6aec9af1b99090e1107091b3b84/paginate-0.5.7.tar.gz", hash = "sha256:22bd083ab41e1a8b4f3690544afb2c60c25e5c9a63a30fa2f483f6c60c8e5945", size = 19252 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/90/96/04b8e52da071d28f5e21a805b19cb9390aa17a47462ac87f5e2696b9566d/paginate-0.5.7-py2.py3-none-any.whl", hash = "sha256:b885e2af73abcf01d9559fd5216b57ef722f8c42affbb63942377668e35c7591", size = 13746 },
+]
+
 [[package]]
 name = "pandas"
 version = "2.2.3"
@@ -3278,6 +3464,15 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/c6/ac/dac4a63f978e4dcb3c6d3a78c4d8e0192a113d288502a1216950c41b1027/parso-0.8.4-py2.py3-none-any.whl", hash = "sha256:a418670a20291dacd2dddc80c377c5c3791378ee1e8d12bffc35420643d43f18", size = 103650 },
 ]
 
+[[package]]
+name = "pathspec"
+version = "0.12.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ca/bc/f35b8446f4531a7cb215605d100cd88b7ac6f44ab3fc94870c120ab3adbf/pathspec-0.12.1.tar.gz", hash = "sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712", size = 51043 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/cc/20/ff623b09d963f88bfde16306a54e12ee5ea43e9b597108672ff3a408aad6/pathspec-0.12.1-py3-none-any.whl", hash = "sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08", size = 31191 },
+]
+
 [[package]]
 name = "pdfminer-six"
 version = "20231228"
@@ -3860,6 +4055,19 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/79/84/0fdf9b18ba31d69877bd39c9cd6052b47f3761e9910c15de788e519f079f/PyJWT-2.9.0-py3-none-any.whl", hash = "sha256:3b02fb0f44517787776cf48f2ae25d8e14f300e6d7545a4315cee571a415e850", size = 22344 },
 ]
 
+[[package]]
+name = "pymdown-extensions"
+version = "10.11.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "markdown" },
+    { name = "pyyaml" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/f4/71/2730a20e9e3752393d78998347f8b1085ef9c417646ea9befbeef221e3c4/pymdown_extensions-10.11.2.tar.gz", hash = "sha256:bc8847ecc9e784a098efd35e20cba772bc5a1b529dfcef9dc1972db9021a1049", size = 830241 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/c2/35/c0edf199257ef0a7d407d29cd51c4e70d1dad4370a5f44deb65a7a5475e2/pymdown_extensions-10.11.2-py3-none-any.whl", hash = "sha256:41cdde0a77290e480cf53892f5c5e50921a7ee3e5cd60ba91bf19837b33badcf", size = 259044 },
+]
+
 [[package]]
 name = "pypdf"
 version = "5.0.1"
@@ -4236,6 +4444,18 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/fa/de/02b54f42487e3d3c6efb3f89428677074ca7bf43aae402517bc7cca949f3/PyYAML-6.0.2-cp313-cp313-win_amd64.whl", hash = "sha256:8388ee1976c416731879ac16da0aff3f63b286ffdd57cdeb95f3f2e085687563", size = 156446 },
 ]
 
+[[package]]
+name = "pyyaml-env-tag"
+version = "0.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "pyyaml" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/fb/8e/da1c6c58f751b70f8ceb1eb25bc25d524e8f14fe16edcce3f4e3ba08629c/pyyaml_env_tag-0.1.tar.gz", hash = "sha256:70092675bda14fdec33b31ba77e7543de9ddc88f2e5b99160396572d11525bdb", size = 5631 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/5a/66/bbb1dd374f5c870f59c5bb1db0e18cbe7fa739415a24cbd95b2d1f5ae0c4/pyyaml_env_tag-0.1-py3-none-any.whl", hash = "sha256:af31106dec8a4d68c60207c1886031cbf839b68aa7abccdb19868200532c2069", size = 3911 },
+]
+
 [[package]]
 name = "qdrant-client"
 version = "1.12.0"
@@ -5446,6 +5666,38 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/c8/15/828ec11907aee2349a9342fa71fba4ba7f3af938162a382dd7da339dea16/virtualenv-20.27.0-py3-none-any.whl", hash = "sha256:44a72c29cceb0ee08f300b314848c86e57bf8d1f13107a5e671fb9274138d655", size = 3110969 },
 ]
 
+[[package]]
+name = "watchdog"
+version = "5.0.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/a2/48/a86139aaeab2db0a2482676f64798d8ac4d2dbb457523f50ab37bf02ce2c/watchdog-5.0.3.tar.gz", hash = "sha256:108f42a7f0345042a854d4d0ad0834b741d421330d5f575b81cb27b883500176", size = 129556 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/05/2b/dd2081aab6fc9e785c2eee7146d3c6de58e607f4e70049d715cd170cbf77/watchdog-5.0.3-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:85527b882f3facda0579bce9d743ff7f10c3e1e0db0a0d0e28170a7d0e5ce2ea", size = 96652 },
+    { url = "https://files.pythonhosted.org/packages/9e/4f/f643c0a720d16ef7316aea06a79b96e229e59df4e0d83bec5e12713c1f29/watchdog-5.0.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:53adf73dcdc0ef04f7735066b4a57a4cd3e49ef135daae41d77395f0b5b692cb", size = 88651 },
+    { url = "https://files.pythonhosted.org/packages/2b/72/acb22067d1f18161914c9b1087c703d63638131a9fde78090da290663407/watchdog-5.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e25adddab85f674acac303cf1f5835951345a56c5f7f582987d266679979c75b", size = 89289 },
+    { url = "https://files.pythonhosted.org/packages/70/34/946f08602f8b8e6af45bc725e4a8013975a34883ab5570bd0d827a4c9829/watchdog-5.0.3-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:f01f4a3565a387080dc49bdd1fefe4ecc77f894991b88ef927edbfa45eb10818", size = 96650 },
+    { url = "https://files.pythonhosted.org/packages/96/2b/b84e35d49e8b0bad77e5d086fc1e2c6c833bbfe74d53144cfe8b26117eff/watchdog-5.0.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:91b522adc25614cdeaf91f7897800b82c13b4b8ac68a42ca959f992f6990c490", size = 88653 },
+    { url = "https://files.pythonhosted.org/packages/d5/3f/41b5d77c10f450b79921c17b7d0b416616048867bfe63acaa072a619a0cb/watchdog-5.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:d52db5beb5e476e6853da2e2d24dbbbed6797b449c8bf7ea118a4ee0d2c9040e", size = 89286 },
+    { url = "https://files.pythonhosted.org/packages/1c/9b/8b206a928c188fdeb7b12e1c795199534cd44bdef223b8470129016009dd/watchdog-5.0.3-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:94d11b07c64f63f49876e0ab8042ae034674c8653bfcdaa8c4b32e71cfff87e8", size = 96739 },
+    { url = "https://files.pythonhosted.org/packages/e1/26/129ca9cd0f8016672f37000010c2fedc0b86816e894ebdc0af9bb04a6439/watchdog-5.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:349c9488e1d85d0a58e8cb14222d2c51cbc801ce11ac3936ab4c3af986536926", size = 88708 },
+    { url = "https://files.pythonhosted.org/packages/8f/b3/5e10ec32f0c429cdb55b1369066d6e83faf9985b3a53a4e37bb5c5e29aa0/watchdog-5.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:53a3f10b62c2d569e260f96e8d966463dec1a50fa4f1b22aec69e3f91025060e", size = 89309 },
+    { url = "https://files.pythonhosted.org/packages/54/c4/49af4ab00bcfb688e9962eace2edda07a2cf89b9699ea536da48e8585cff/watchdog-5.0.3-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:950f531ec6e03696a2414b6308f5c6ff9dab7821a768c9d5788b1314e9a46ca7", size = 96740 },
+    { url = "https://files.pythonhosted.org/packages/96/a4/b24de77cc9ae424c1687c9d4fb15aa560d7d7b28ba559aca72f781d0202b/watchdog-5.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ae6deb336cba5d71476caa029ceb6e88047fc1dc74b62b7c4012639c0b563906", size = 88711 },
+    { url = "https://files.pythonhosted.org/packages/a4/71/3f2e9fe8403386b99d788868955b3a790f7a09721501a7e1eb58f514ffaa/watchdog-5.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:1021223c08ba8d2d38d71ec1704496471ffd7be42cfb26b87cd5059323a389a1", size = 89319 },
+    { url = "https://files.pythonhosted.org/packages/a2/d6/1d1ca81c75d903eca3fdb7061d93845485b58a5ba182d146843b88fc51c2/watchdog-5.0.3-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:90a67d7857adb1d985aca232cc9905dd5bc4803ed85cfcdcfcf707e52049eda7", size = 88172 },
+    { url = "https://files.pythonhosted.org/packages/47/bb/d5e0abcfd6d729029a24766682e062526db8b59e9ae0c94aff509e0fd2b9/watchdog-5.0.3-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:720ef9d3a4f9ca575a780af283c8fd3a0674b307651c1976714745090da5a9e8", size = 88644 },
+    { url = "https://files.pythonhosted.org/packages/60/33/7cb71c9df9a77b6927ee5f48d25e1de5562ce0fa7e0c56dcf2b0472e64a2/watchdog-5.0.3-py3-none-manylinux2014_aarch64.whl", hash = "sha256:dd021efa85970bd4824acacbb922066159d0f9e546389a4743d56919b6758b91", size = 79335 },
+    { url = "https://files.pythonhosted.org/packages/f6/91/320bc1496cf951a3cf93a7ffd18a581f0792c304be963d943e0e608c2919/watchdog-5.0.3-py3-none-manylinux2014_armv7l.whl", hash = "sha256:78864cc8f23dbee55be34cc1494632a7ba30263951b5b2e8fc8286b95845f82c", size = 79334 },
+    { url = "https://files.pythonhosted.org/packages/8b/2c/567c5e042ed667d3544c43d48a65cf853450a2d2a9089d9523a65f195e94/watchdog-5.0.3-py3-none-manylinux2014_i686.whl", hash = "sha256:1e9679245e3ea6498494b3028b90c7b25dbb2abe65c7d07423ecfc2d6218ff7c", size = 79333 },
+    { url = "https://files.pythonhosted.org/packages/c3/f0/64059fe162ef3274662e67bbdea6c45b3cd53e846d5bd1365fcdc3dc1d15/watchdog-5.0.3-py3-none-manylinux2014_ppc64.whl", hash = "sha256:9413384f26b5d050b6978e6fcd0c1e7f0539be7a4f1a885061473c5deaa57221", size = 79334 },
+    { url = "https://files.pythonhosted.org/packages/f6/d9/19b7d02965be2801e2d0f6f4bde23e4ae172620071b65430fa0c2f8441ac/watchdog-5.0.3-py3-none-manylinux2014_ppc64le.whl", hash = "sha256:294b7a598974b8e2c6123d19ef15de9abcd282b0fbbdbc4d23dfa812959a9e05", size = 79333 },
+    { url = "https://files.pythonhosted.org/packages/cb/a1/5393ac6d0b095d3a44946b09258e9b5f22cb2fb67bcfa419dd868478826c/watchdog-5.0.3-py3-none-manylinux2014_s390x.whl", hash = "sha256:26dd201857d702bdf9d78c273cafcab5871dd29343748524695cecffa44a8d97", size = 79332 },
+    { url = "https://files.pythonhosted.org/packages/a0/58/edec25190b6403caf4426dd418234f2358a106634b7d6aa4aec6939b104f/watchdog-5.0.3-py3-none-manylinux2014_x86_64.whl", hash = "sha256:0f9332243355643d567697c3e3fa07330a1d1abf981611654a1f2bf2175612b7", size = 79334 },
+    { url = "https://files.pythonhosted.org/packages/97/69/cfb2d17ba8aabc73be2e2d03c8c319b1f32053a02c4b571852983aa24ff2/watchdog-5.0.3-py3-none-win32.whl", hash = "sha256:c66f80ee5b602a9c7ab66e3c9f36026590a0902db3aea414d59a2f55188c1f49", size = 79320 },
+    { url = "https://files.pythonhosted.org/packages/91/b4/2b5b59358dadfa2c8676322f955b6c22cde4937602f40490e2f7403e548e/watchdog-5.0.3-py3-none-win_amd64.whl", hash = "sha256:f00b4cf737f568be9665563347a910f8bdc76f88c2970121c86243c8cfdf90e9", size = 79325 },
+    { url = "https://files.pythonhosted.org/packages/38/b8/0aa69337651b3005f161f7f494e59188a1d8d94171666900d26d29d10f69/watchdog-5.0.3-py3-none-win_ia64.whl", hash = "sha256:49f4d36cb315c25ea0d946e018c01bb028048023b9e103d3d3943f58e109dd45", size = 79324 },
+]
+
 [[package]]
 name = "watchfiles"
 version = "0.24.0"