Compare commits

..

38 Commits

Author SHA1 Message Date
Greyson LaLonde
e1d7de0dba docs: update changelog and version for v1.10.2rc2
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
2026-03-14 00:49:48 -04:00
Greyson LaLonde
96b07bfc84 feat: bump versions to 1.10.2rc2 2026-03-14 00:34:12 -04:00
Greyson LaLonde
b8d7942675 fix: remove exclusive locks from read-only storage operations
* fix: remove exclusive locks from read-only storage operations to eliminate lock contention

read operations like search, list_scopes, get_scope_info, count across
LanceDB, ChromaDB, and RAG adapters were holding exclusive locks unnecessarily.
under multi-process prefork workers this caused RedisLock contention triggering
a portalocker bug where AlreadyLocked is raised with the exceptions module as its arg.

- remove store_lock from 7 LanceDB read methods since MVCC handles concurrent reads
- remove store_lock from ChromaDB search/asearch which are thread-safe since v0.4
- remove store_lock from RAG core query and LanceDB adapter query
- wrap lock_store BaseLockException with actionable error message
- add exception handling in encoding_flow/recall_flow ThreadPoolExecutor calls
- fix flow.py double-logging of ancestor listener errors

* fix: remove dead conditional in filter_and_chunk fallback

both branches of the if/else and the except all produced the same
candidates = [scope_prefix] result, making the get_scope_info call
and conditional pointless

* fix: separate lock acquisition from caller body in lock_store

the try/except wrapped the yield inside the contextmanager, which meant
any BaseLockException raised by the caller's code inside the with block
would be caught and re-raised with a misleading "Failed to acquire lock"
message. split into acquire-then-yield so only actual acquisition
failures get the actionable error message.
2026-03-14 00:21:14 -04:00
Greyson LaLonde
88fd859c26 docs: update changelog and version for v1.10.2rc1
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
2026-03-13 17:07:31 -04:00
Greyson LaLonde
3413f2e671 feat: bump versions to 1.10.2rc1 2026-03-13 16:53:48 -04:00
Greyson LaLonde
326ec15d54 feat(devtools): add release command and trigger PyPI publish
* feat(devtools): add release command and fix automerge on protected branches

Replace gh pr merge --auto with polling-based merge wait that prints the
PR URL for manual review. Add unified release command that chains bump
and tag into a single end-to-end workflow.

* feat(devtools): trigger PyPI publish workflow after GitHub release

* refactor(devtools): extract shared helpers to eliminate duplication

Extract _poll_pr_until_merged, _update_all_versions,
_generate_release_notes, _update_docs_and_create_pr,
_create_tag_and_release, and _trigger_pypi_publish into reusable
helpers. All three commands (bump, tag, release) now compose from
these shared functions.
2026-03-13 16:41:27 -04:00
Greyson LaLonde
c5a8fef118 fix: add cross-process and thread-safe locking to unprotected I/O (#4827)
* fix: add cross-process and thread-safe locking to unprotected I/O

* style: apply ruff formatting and import sorting

* fix: avoid event loop deadlock in snowflake pool lock

* perf: move embedding calls outside cross-process lock in RAG adapter

* fix: close TOCTOU race in browser session manager

* fix: add error handling to update_user_data

* fix: use async lock acquisition in chromadb async methods

* fix: avoid blocking event loop in async browser session wait

* fix: replace dual-lock with single cross-process lock in LanceDB storage

* fix: remove dead _save_user_data function and stale mock

* fix: re-addd file descriptor limit to prevent crashes
2026-03-13 12:28:11 -07:00
Greyson LaLonde
b7af26ff60 ci: add slack notification on successful pypi publish 2026-03-13 12:05:52 -04:00
Greyson LaLonde
48eb7c6937 fix: propagate contextvars across all thread and executor boundaries
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
2026-03-13 00:32:22 -04:00
danglies007
d8e38f2f0b fix: propagate ContextVars into async task threads
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
threading.Thread() does not inherit the parent's contextvars.Context,
causing ContextVar-based state (OpenTelemetry spans, Langfuse trace IDs,
and any other request-scoped vars) to be silently dropped in async tasks.

Fix by calling contextvars.copy_context() before spawning each thread and
using ctx.run() as the thread target, which runs the function inside the
captured context.

Affected locations:
- task.py: execute_async() — the primary async task execution path
- utilities/streaming.py: create_chunk_generator() — streaming execution path

Fixes: #4822
Related: #4168, #4286

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-12 15:33:58 -04:00
Greyson LaLonde
542afe61a8 docs: update changelog and version for v1.10.2a1
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
2026-03-11 11:44:00 -04:00
Greyson LaLonde
8a5b3bc237 feat: bump versions to 1.10.2a1
* feat: bump versions to 1.10.2a1

* chore: update tool specifications

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-11 11:30:11 -04:00
Greyson LaLonde
534f0707ca fix: resolve LockException under concurrent multi-process execution 2026-03-11 11:15:24 -04:00
Giulio Leone
0046f9a96f fix(bedrock): group parallel tool results in single user message (#4775)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* fix(bedrock): group parallel tool results in single user message

When an AWS Bedrock model makes multiple tool calls in a single
response, the Converse API requires all corresponding tool results
to be sent back in a single user message. Previously, each tool
result was emitted as a separate user message, causing:

  ValidationException: Expected toolResult blocks at messages.2.content

Fix: When processing consecutive tool messages, append the toolResult
block to the preceding user message (if it already contains
toolResult blocks) instead of creating a new message. This groups
all parallel tool results together while keeping tool results from
different assistant turns separate.

Fixes #4749

Signed-off-by: Giulio Leone <6887247+giulio-leone@users.noreply.github.com>

* Update lib/crewai/tests/llms/bedrock/test_bedrock.py

* fix: group bedrock tool results

Co-authored-by: João Moura <joaomdmoura@gmail.com>

---------

Signed-off-by: Giulio Leone <6887247+giulio-leone@users.noreply.github.com>
Co-authored-by: Giulio Leone <6887247+giulio-leone@users.noreply.github.com>
Co-authored-by: João Moura <joaomdmoura@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-03-10 17:28:40 -03:00
Lucas Gomide
e72a80be6e Addressing MCP tools resolutions & eliminates all shared mutable connection (#4792)
* fix: allow hyphenated tool names in MCP references like notion#get-page

The _SLUG_RE regex on BaseAgent rejected MCP tool references containing
hyphens (e.g. "notion#get-page") because the fragment pattern only
matched \w (word chars)

* fix: create fresh MCP client per tool invocation to prevent parallel call races

When the LLM dispatches parallel calls to MCP tools on the same server, the executor runs them concurrently via ThreadPoolExecutor. Previously, all tools from a server shared a single MCPClient instance, and even the same tool called twice would reuse one client. Since each thread creates its own asyncio event loop via asyncio.run(), concurrent connect/disconnect calls on the shared client caused anyio cancel-scope errors ("Attempted to exit cancel scope in a different task than it was entered in").

The fix introduces a client_factory pattern: MCPNativeTool now receives a zero-arg callable that produces a fresh MCPClient + transport on every
_run_async() invocation. This eliminates all shared mutable connection state between concurrent calls, whether to the same tool or different tools from the same server.

* test: ensure we can filter hyphenated MCP tool
2026-03-10 14:00:40 -04:00
Lorenze Jay
7cffcab84a ensure we support tool search - saving tokens and dynamically inject appropriate tools during execution - anthropic (#4779)
* ensure we support tool search

* linted

* dont tool search if there is only one tool
2026-03-10 10:48:13 -07:00
João Moura
f070ce8abd fix: update llm parameter handling in human_feedback function (#4801)
Modified the llm parameter assignment to retrieve the model attribute from llm if it is not a string, ensuring compatibility with different llm types.
2026-03-10 14:27:09 -03:00
Sampson
d9f6e2222f Introduce more Brave Search tools (#4446)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* feat: add dedicated Brave Search tools for web, news, image, video, local POIs, and Brave's newest LLM Context endpoint

* fix: normalize transformed response shape

* revert legacy tool name

* fix: schema change prevented property resolution

* Update tool.specs.json

* fix: add fallback for search_langugage

* simplify exports

* makes rate-limiting logic per-instance

* fix(brave-tools): correct _refine_response return type annotations

The abstract method and subclasses annotated _refine_response as returning
dict[str, Any] but most implementations actually return list[dict[str, Any]].
Updated base to return Any, and each subclass to match its actual return type.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Joao Moura <joaomdmoura@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 01:38:54 -03:00
Lucas Gomide
adef605410 fix: add missing list/dict methods to LockedListProxy and LockedDictProxy
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
2026-03-09 09:38:35 -04:00
Greyson LaLonde
cd42bcf035 refactor(memory): convert memory classes to serializable
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* refactor(memory): convert Memory, MemoryScope, and MemorySlice to BaseModel

* fix(test): update mock memory attribute from _read_only to read_only

* fix: handle re-validation in wrap validators and patch BaseModel class in tests
2026-03-08 23:08:10 -04:00
Greyson LaLonde
bc45a7fbe3 feat: create action for nightly releases
Some checks failed
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
2026-03-06 18:32:52 -05:00
Matt Aitchison
87759cdb14 fix(deps): bump gitpython to >=3.1.41 to resolve CVE path traversal vulnerability (#4740)
Some checks failed
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
GitPython ==3.1.38 is affected by a high-severity path traversal
vulnerability (dependabot alert #1). Bump to >=3.1.41,<4 which
includes the fix.
2026-03-05 12:41:24 -06:00
Tiago Freire
059cb93aeb fix(executor): propagate contextvars context to parallel tool call threads
ThreadPoolExecutor threads do not inherit the calling thread's contextvars
context, causing _event_id_stack and _current_celery_task_id to be empty
in worker threads. This broke OTel span parenting for parallel tool calls
(missing parent_event_id) and lost the Celery task ID in the enterprise
tracking layer ([Task ID: no-task]).

Fix by capturing an independent context copy per submission via
contextvars.copy_context().run in CrewAgentExecutor._handle_native_tool_calls,
so each worker thread starts with the correct inherited context without
sharing mutable state across threads.
2026-03-05 08:20:09 -05:00
Lorenze Jay
cebc52694e docs: update changelog and version for v1.10.1
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
2026-03-04 18:20:02 -05:00
Lorenze Jay
53df41989a feat: bump versions to 1.10.1 (#4706)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
2026-03-04 11:03:17 -08:00
Greyson LaLonde
ea70976a5d fix: adjust executor listener value to avoid recursion (#4705)
* fix: adjust executor listener value to avoid recursion

* fix: clear call count to ensure zero state

* feat: expose max method call kwarg
2026-03-04 10:47:22 -08:00
João Moura
3cc6516ae5 Memory overall improvements (#4688)
* feat: enhance memory recall limits and update documentation

- Increased the memory recall limit in the Agent class from 5 to 15.
- Updated the RecallMemoryTool to allow a recall limit of 20.
- Expanded the documentation for the recall_memory feature to emphasize the importance of multiple queries for comprehensive results.

* feat: increase memory recall limit and enhance memory context documentation

- Increased the memory recall limit in the Agent class from 15 to 20.
- Updated the memory context message to clarify the nature of the memories presented and the importance of using the Search memory tool for comprehensive results.

* refactor: remove inferred_categories from RecallState and update category merging logic

- Removed the inferred_categories field from RecallState to simplify state management.
- Updated the _merged_categories method to only merge caller-supplied categories, enhancing clarity in category handling.

* refactor: simplify category handling in RecallFlow

- Updated the _merged_categories method to return only caller-supplied categories, removing the previous merging logic for inferred categories. This change enhances clarity and maintains consistency in category management.
2026-03-04 09:19:07 -08:00
nicoferdi96
ad82e52d39 fix(gemini): group parallel function_response parts in a single Content object (#4693)
* fix(gemini): group parallel function_response parts in a single Content object

When Gemini makes N parallel tool calls, the API requires all N function_response parts in one Content object. Previously each tool result created a separate Content, causing 400 INVALID_ARGUMENT errors. Merge consecutive function_response parts into the existing Content instead of appending new ones.

* Address change requested

- function_response is a declared field on the types.Part Pydantic model so hasattr can be replaced with p.function_response is not None
2026-03-04 12:04:23 +01:00
Matt Aitchison
9336702ebc fix(deps): bump pypdf, urllib3 override, and dev dependencies for security fixes
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
- pypdf ~6.7.4 → ~6.7.5 (CVE: inefficient ASCIIHexDecode stream decoding)
- Add urllib3>=2.6.3 override (CVE: decompression-bomb bypass on redirects)
- ruff 0.14.7 → 0.15.1, mypy 1.19.0 → 1.19.1, pre-commit 4.5.0 → 4.5.1
- types-regex 2024.11.6 → 2026.1.15, boto3-stubs 1.40.54 → 1.42.40
- Auto-fixed 13 lint issues from new ruff rules

Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>
2026-03-04 01:13:38 -05:00
Greyson LaLonde
030f6d6c43 fix: use anon id for ephemeral traces 2026-03-04 00:45:09 -05:00
Mike Plachta
95d51db29f Langgraph migration guide (#4681)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
2026-03-03 11:53:12 -08:00
Greyson LaLonde
a8f51419f6 fix(gemini): surface thought output from thinking models
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
* fix(gemini): surface thought output from thinking models

* chore(llm): remove unreachable hasattr guards on crewai_event_bus
2026-03-03 11:54:55 -05:00
Greyson LaLonde
e7f17d2284 fix: load MCP and platform tools when agent tools is None
Closes #4568
2026-03-03 10:25:25 -05:00
Greyson LaLonde
5d0811258f fix(a2a): support Jupyter environments with running event loops 2026-03-03 10:05:48 -05:00
Greyson LaLonde
7972192d55 fix(deps): bump tokenizers lower bound to >=0.21 to avoid broken 0.20.3
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
2026-03-02 18:04:28 -05:00
Mike Plachta
b3f8a42321 feat: upgrade gemini genai
Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>
Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
2026-03-02 14:27:56 -05:00
Greyson LaLonde
21224f2bc5 fix: conditionally pass plus header
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Empty strings are considered illegal values for bearer auth in `httpx`.
2026-03-02 09:27:54 -05:00
Giulio Leone
b76022c1e7 fix(telemetry): skip signal handler registration in non-main threads
* fix(telemetry): skip signal handler registration in non-main threads

When CrewAI is initialized from a non-main thread (e.g. Streamlit, Flask,
Django, Jupyter), the telemetry module attempted to register signal handlers
which only work in the main thread. This caused multiple noisy ValueError
tracebacks to be printed to stderr, confusing users even though the errors
were caught and non-fatal.

Check `threading.current_thread() is not threading.main_thread()` before
attempting signal registration, and skip silently with a debug-level log
message instead of printing full tracebacks.

Fixes crewAIInc/crewAI#4289

* fix(test): move Telemetry() inside signal.signal mock context

Refs: #4649

* fix(telemetry): move signal.signal mock inside thread to wrap Telemetry() construction

The patch context now activates inside init_in_thread so the mock
is guaranteed to be active before and during Telemetry.__init__,
addressing the Copilot review feedback.

Refs: #4289

* fix(test): mock logger.debug instead of capsys for deterministic assertion

Replace signal.signal-only mock with combined logger + signal mock.
Assert logger.debug was called with the skip message and signal.signal
was never invoked from the non-main thread.

Refs: #4289
2026-03-02 07:42:55 -05:00
139 changed files with 15811 additions and 8453 deletions

127
.github/workflows/nightly.yml vendored Normal file
View File

@@ -0,0 +1,127 @@
name: Nightly Canary Release
on:
schedule:
- cron: '0 6 * * *' # daily at 6am UTC
workflow_dispatch:
jobs:
check:
name: Check for new commits
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
has_changes: ${{ steps.check.outputs.has_changes }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check for commits in last 24h
id: check
run: |
RECENT=$(git log --since="24 hours ago" --oneline | head -1)
if [ -n "$RECENT" ]; then
echo "has_changes=true" >> "$GITHUB_OUTPUT"
else
echo "has_changes=false" >> "$GITHUB_OUTPUT"
fi
build:
name: Build nightly packages
needs: check
if: needs.check.outputs.has_changes == 'true' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install uv
uses: astral-sh/setup-uv@v4
- name: Stamp nightly versions
run: |
DATE=$(date +%Y%m%d)
for init_file in \
lib/crewai/src/crewai/__init__.py \
lib/crewai-tools/src/crewai_tools/__init__.py \
lib/crewai-files/src/crewai_files/__init__.py; do
CURRENT=$(python -c "
import re
text = open('$init_file').read()
print(re.search(r'__version__\s*=\s*\"(.*?)\"\s*$', text, re.MULTILINE).group(1))
")
NIGHTLY="${CURRENT}.dev${DATE}"
sed -i "s/__version__ = .*/__version__ = \"${NIGHTLY}\"/" "$init_file"
echo "$init_file: $CURRENT -> $NIGHTLY"
done
# Update cross-package dependency pins to nightly versions
sed -i "s/\"crewai-tools==[^\"]*\"/\"crewai-tools==${NIGHTLY}\"/" lib/crewai/pyproject.toml
sed -i "s/\"crewai==[^\"]*\"/\"crewai==${NIGHTLY}\"/" lib/crewai-tools/pyproject.toml
echo "Updated cross-package dependency pins to ${NIGHTLY}"
- name: Build packages
run: |
uv build --all-packages
rm dist/.gitignore
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/
publish:
name: Publish nightly to PyPI
needs: build
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/crewai
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
version: "0.8.4"
python-version: "3.12"
enable-cache: false
- name: Download artifacts
uses: actions/download-artifact@v4
with:
name: dist
path: dist
- name: Publish to PyPI
env:
UV_PUBLISH_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
run: |
failed=0
for package in dist/*; do
if [[ "$package" == *"crewai_devtools"* ]]; then
echo "Skipping private package: $package"
continue
fi
echo "Publishing $package"
if ! uv publish "$package"; then
echo "Failed to publish $package"
failed=1
fi
done
if [ $failed -eq 1 ]; then
echo "Some packages failed to publish"
exit 1
fi

View File

@@ -59,6 +59,8 @@ jobs:
contents: read contents: read
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with:
ref: ${{ inputs.release_tag || github.ref }}
- name: Install uv - name: Install uv
uses: astral-sh/setup-uv@v6 uses: astral-sh/setup-uv@v6
@@ -93,3 +95,72 @@ jobs:
echo "Some packages failed to publish" echo "Some packages failed to publish"
exit 1 exit 1
fi fi
- name: Build Slack payload
if: success()
id: slack
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_TAG: ${{ inputs.release_tag }}
run: |
payload=$(uv run python -c "
import json, re, subprocess, sys
with open('lib/crewai/src/crewai/__init__.py') as f:
m = re.search(r\"__version__\s*=\s*[\\\"']([^\\\"']+)\", f.read())
version = m.group(1) if m else 'unknown'
import os
tag = os.environ.get('RELEASE_TAG') or version
try:
r = subprocess.run(['gh','release','view',tag,'--json','body','-q','.body'],
capture_output=True, text=True, check=True)
body = r.stdout.strip()
except Exception:
body = ''
blocks = [
{'type':'section','text':{'type':'mrkdwn',
'text':f':rocket: \`crewai v{version}\` published to PyPI'}},
{'type':'section','text':{'type':'mrkdwn',
'text':f'<https://pypi.org/project/crewai/{version}/|View on PyPI> · <https://github.com/crewAIInc/crewAI/releases/tag/{tag}|Release notes>'}},
{'type':'divider'},
]
if body:
heading, items = '', []
for line in body.split('\n'):
line = line.strip()
if not line: continue
hm = re.match(r'^#{2,3}\s+(.*)', line)
if hm:
if heading and items:
skip = heading in ('What\\'s Changed','') or 'Contributors' in heading
if not skip:
txt = f'*{heading}*\n' + '\n'.join(f'• {i}' for i in items)
blocks.append({'type':'section','text':{'type':'mrkdwn','text':txt}})
heading, items = hm.group(1), []
elif line.startswith('- ') or line.startswith('* '):
items.append(re.sub(r'\*\*([^*]*)\*\*', r'*\1*', line[2:]))
if heading and items:
skip = heading in ('What\\'s Changed','') or 'Contributors' in heading
if not skip:
txt = f'*{heading}*\n' + '\n'.join(f'• {i}' for i in items)
blocks.append({'type':'section','text':{'type':'mrkdwn','text':txt}})
blocks.append({'type':'divider'})
blocks.append({'type':'section','text':{'type':'mrkdwn',
'text':f'\`\`\`uv add \"crewai[tools]=={version}\"\`\`\`'}})
print(json.dumps({'blocks':blocks}))
")
echo "payload=$payload" >> $GITHUB_OUTPUT
- name: Notify Slack
if: success()
uses: slackapi/slack-github-action@v2.1.0
with:
webhook: ${{ secrets.SLACK_WEBHOOK_URL }}
webhook-type: incoming-webhook
payload: ${{ steps.slack.outputs.payload }}

View File

@@ -12,6 +12,7 @@ from dotenv import load_dotenv
import pytest import pytest
from vcr.request import Request # type: ignore[import-untyped] from vcr.request import Request # type: ignore[import-untyped]
try: try:
import vcr.stubs.httpx_stubs as httpx_stubs # type: ignore[import-untyped] import vcr.stubs.httpx_stubs as httpx_stubs # type: ignore[import-untyped]
except ModuleNotFoundError: except ModuleNotFoundError:

File diff suppressed because it is too large Load Diff

View File

@@ -4,6 +4,114 @@ description: "Product updates, improvements, and bug fixes for CrewAI"
icon: "clock" icon: "clock"
mode: "wide" mode: "wide"
--- ---
<Update label="Mar 14, 2026">
## v1.10.2rc2
[View release on GitHub](https://github.com/crewAIInc/crewAI/releases/tag/1.10.2rc2)
## What's Changed
### Bug Fixes
- Remove exclusive locks from read-only storage operations
### Documentation
- Update changelog and version for v1.10.2rc1
## Contributors
@greysonlalonde
</Update>
<Update label="Mar 13, 2026">
## v1.10.2rc1
[View release on GitHub](https://github.com/crewAIInc/crewAI/releases/tag/1.10.2rc1)
## What's Changed
### Features
- Add release command and trigger PyPI publish
### Bug Fixes
- Fix cross-process and thread-safe locking to unprotected I/O
- Propagate contextvars across all thread and executor boundaries
- Propagate ContextVars into async task threads
### Documentation
- Update changelog and version for v1.10.2a1
## Contributors
@danglies007, @greysonlalonde
</Update>
<Update label="Mar 11, 2026">
## v1.10.2a1
[View release on GitHub](https://github.com/crewAIInc/crewAI/releases/tag/1.10.2a1)
## What's Changed
### Features
- Add support for tool search, saving tokens, and dynamically injecting appropriate tools during execution for Anthropics.
- Introduce more Brave Search tools.
- Create action for nightly releases.
### Bug Fixes
- Fix LockException under concurrent multi-process execution.
- Resolve issues with grouping parallel tool results in a single user message.
- Address MCP tools resolutions and eliminate all shared mutable connections.
- Update LLM parameter handling in the human_feedback function.
- Add missing list/dict methods to LockedListProxy and LockedDictProxy.
- Propagate contextvars context to parallel tool call threads.
- Bump gitpython dependency to >=3.1.41 to resolve CVE path traversal vulnerability.
### Refactoring
- Refactor memory classes to be serializable.
### Documentation
- Update changelog and version for v1.10.1.
## Contributors
@akaKuruma, @github-actions[bot], @giulio-leone, @greysonlalonde, @joaomdmoura, @jonathansampson, @lorenzejay, @lucasgomide, @mattatcha
</Update>
<Update label="Mar 04, 2026">
## v1.10.1
[View release on GitHub](https://github.com/crewAIInc/crewAI/releases/tag/1.10.1)
## What's Changed
### Features
- Upgrade Gemini GenAI
### Bug Fixes
- Adjust executor listener value to avoid recursion
- Group parallel function response parts in a single Content object in Gemini
- Surface thought output from thinking models in Gemini
- Load MCP and platform tools when agent tools are None
- Support Jupyter environments with running event loops in A2A
- Use anonymous ID for ephemeral traces
- Conditionally pass plus header
- Skip signal handler registration in non-main threads for telemetry
- Inject tool errors as observations and resolve name collisions
- Upgrade pypdf from 4.x to 6.7.4 to resolve Dependabot alerts
- Resolve critical and high Dependabot security alerts
### Documentation
- Sync Composio tool documentation across locales
## Contributors
@giulio-leone, @greysonlalonde, @haxzie, @joaomdmoura, @lorenzejay, @mattatcha, @mplachta, @nicoferdi96
</Update>
<Update label="Feb 27, 2026"> <Update label="Feb 27, 2026">
## v1.10.1a1 ## v1.10.1a1

View File

@@ -0,0 +1,518 @@
---
title: "Moving from LangGraph to CrewAI: A Practical Guide for Engineers"
description: If you already have built with LangGraph, learn how to quickly port your projects to CrewAI
icon: switch
mode: "wide"
---
You've built agents with LangGraph. You've wrestled with `StateGraph`, wired up conditional edges, and debugged state dictionaries at 2 AM. It works — but somewhere along the way, you started wondering if there's a better path to production.
There is. **CrewAI Flows** gives you the same power — event-driven orchestration, conditional routing, shared state — with dramatically less boilerplate and a mental model that maps cleanly to how you actually think about multi-step AI workflows.
This article walks through the core concepts side by side, shows real code comparisons, and demonstrates why CrewAI Flows is the framework you'll want to reach for next.
---
## The Mental Model Shift
LangGraph asks you to think in **graphs**: nodes, edges, and state dictionaries. Every workflow is a directed graph where you explicitly wire transitions between computation steps. It's powerful, but the abstraction carries overhead — especially when your workflow is fundamentally sequential with a few decision points.
CrewAI Flows asks you to think in **events**: methods that start things, methods that listen for results, and methods that route execution. The topology of your workflow emerges from decorator annotations rather than explicit graph construction. This isn't just syntactic sugar — it changes how you design, read, and maintain your pipelines.
Here's the core mapping:
| LangGraph Concept | CrewAI Flows Equivalent |
| --- | --- |
| `StateGraph` class | `Flow` class |
| `add_node()` | Methods decorated with `@start`, `@listen` |
| `add_edge()` / `add_conditional_edges()` | `@listen()` / `@router()` decorators |
| `TypedDict` state | Pydantic `BaseModel` state |
| `START` / `END` constants | `@start()` decorator / natural method return |
| `graph.compile()` | `flow.kickoff()` |
| Checkpointer / persistence | Built-in memory (LanceDB-backed) |
Let's see what this looks like in practice.
---
## Demo 1: A Simple Sequential Pipeline
Imagine you're building a pipeline that takes a topic, researches it, writes a summary, and formats the output. Here's how each framework handles it.
### LangGraph Approach
```python
from typing import TypedDict
from langgraph.graph import StateGraph, START, END
class ResearchState(TypedDict):
topic: str
raw_research: str
summary: str
formatted_output: str
def research_topic(state: ResearchState) -> dict:
# Call an LLM or search API
result = llm.invoke(f"Research the topic: {state['topic']}")
return {"raw_research": result}
def write_summary(state: ResearchState) -> dict:
result = llm.invoke(
f"Summarize this research:\n{state['raw_research']}"
)
return {"summary": result}
def format_output(state: ResearchState) -> dict:
result = llm.invoke(
f"Format this summary as a polished article section:\n{state['summary']}"
)
return {"formatted_output": result}
# Build the graph
graph = StateGraph(ResearchState)
graph.add_node("research", research_topic)
graph.add_node("summarize", write_summary)
graph.add_node("format", format_output)
graph.add_edge(START, "research")
graph.add_edge("research", "summarize")
graph.add_edge("summarize", "format")
graph.add_edge("format", END)
# Compile and run
app = graph.compile()
result = app.invoke({"topic": "quantum computing advances in 2026"})
print(result["formatted_output"])
```
You define functions, register them as nodes, and manually wire every transition. For a simple sequence like this, there's a lot of ceremony.
### CrewAI Flows Approach
```python
from crewai import LLM, Agent, Crew, Process, Task
from crewai.flow.flow import Flow, listen, start
from pydantic import BaseModel
llm = LLM(model="openai/gpt-5.2")
class ResearchState(BaseModel):
topic: str = ""
raw_research: str = ""
summary: str = ""
formatted_output: str = ""
class ResearchFlow(Flow[ResearchState]):
@start()
def research_topic(self):
# Option 1: Direct LLM call
result = llm.call(f"Research the topic: {self.state.topic}")
self.state.raw_research = result
return result
@listen(research_topic)
def write_summary(self, research_output):
# Option 2: A single agent
summarizer = Agent(
role="Research Summarizer",
goal="Produce concise, accurate summaries of research content",
backstory="You are an expert at distilling complex research into clear, "
"digestible summaries.",
llm=llm,
verbose=True,
)
result = summarizer.kickoff(
f"Summarize this research:\n{self.state.raw_research}"
)
self.state.summary = str(result)
return self.state.summary
@listen(write_summary)
def format_output(self, summary_output):
# Option 3: a complete crew (with one or more agents)
formatter = Agent(
role="Content Formatter",
goal="Transform research summaries into polished, publication-ready article sections",
backstory="You are a skilled editor with expertise in structuring and "
"presenting technical content for a general audience.",
llm=llm,
verbose=True,
)
format_task = Task(
description=f"Format this summary as a polished article section:\n{self.state.summary}",
expected_output="A well-structured, polished article section ready for publication.",
agent=formatter,
)
crew = Crew(
agents=[formatter],
tasks=[format_task],
process=Process.sequential,
verbose=True,
)
result = crew.kickoff()
self.state.formatted_output = str(result)
return self.state.formatted_output
# Run the flow
flow = ResearchFlow()
flow.state.topic = "quantum computing advances in 2026"
result = flow.kickoff()
print(flow.state.formatted_output)
```
Notice what's different: no graph construction, no edge wiring, no compile step. The execution order is declared right where the logic lives. `@start()` marks the entry point, and `@listen(method_name)` chains steps together. The state is a proper Pydantic model with type safety, validation, and IDE auto-completion.
---
## Demo 2: Conditional Routing
This is where things get interesting. Say you're building a content pipeline that routes to different processing paths based on the type of content detected.
### LangGraph Approach
```python
from typing import TypedDict, Literal
from langgraph.graph import StateGraph, START, END
class ContentState(TypedDict):
input_text: str
content_type: str
result: str
def classify_content(state: ContentState) -> dict:
content_type = llm.invoke(
f"Classify this content as 'technical', 'creative', or 'business':\n{state['input_text']}"
)
return {"content_type": content_type.strip().lower()}
def process_technical(state: ContentState) -> dict:
result = llm.invoke(f"Process as technical doc:\n{state['input_text']}")
return {"result": result}
def process_creative(state: ContentState) -> dict:
result = llm.invoke(f"Process as creative writing:\n{state['input_text']}")
return {"result": result}
def process_business(state: ContentState) -> dict:
result = llm.invoke(f"Process as business content:\n{state['input_text']}")
return {"result": result}
# Routing function
def route_content(state: ContentState) -> Literal["technical", "creative", "business"]:
return state["content_type"]
# Build the graph
graph = StateGraph(ContentState)
graph.add_node("classify", classify_content)
graph.add_node("technical", process_technical)
graph.add_node("creative", process_creative)
graph.add_node("business", process_business)
graph.add_edge(START, "classify")
graph.add_conditional_edges(
"classify",
route_content,
{
"technical": "technical",
"creative": "creative",
"business": "business",
}
)
graph.add_edge("technical", END)
graph.add_edge("creative", END)
graph.add_edge("business", END)
app = graph.compile()
result = app.invoke({"input_text": "Explain how TCP handshakes work"})
```
You need a separate routing function, explicit conditional edge mapping, and termination edges for every branch. The routing logic is decoupled from the node that produces the routing decision.
### CrewAI Flows Approach
```python
from crewai import LLM, Agent
from crewai.flow.flow import Flow, listen, router, start
from pydantic import BaseModel
llm = LLM(model="openai/gpt-5.2")
class ContentState(BaseModel):
input_text: str = ""
content_type: str = ""
result: str = ""
class ContentFlow(Flow[ContentState]):
@start()
def classify_content(self):
self.state.content_type = (
llm.call(
f"Classify this content as 'technical', 'creative', or 'business':\n"
f"{self.state.input_text}"
)
.strip()
.lower()
)
return self.state.content_type
@router(classify_content)
def route_content(self, classification):
if classification == "technical":
return "process_technical"
elif classification == "creative":
return "process_creative"
else:
return "process_business"
@listen("process_technical")
def handle_technical(self):
agent = Agent(
role="Technical Writer",
goal="Produce clear, accurate technical documentation",
backstory="You are an expert technical writer who specializes in "
"explaining complex technical concepts precisely.",
llm=llm,
verbose=True,
)
self.state.result = str(
agent.kickoff(f"Process as technical doc:\n{self.state.input_text}")
)
@listen("process_creative")
def handle_creative(self):
agent = Agent(
role="Creative Writer",
goal="Craft engaging and imaginative creative content",
backstory="You are a talented creative writer with a flair for "
"compelling storytelling and vivid expression.",
llm=llm,
verbose=True,
)
self.state.result = str(
agent.kickoff(f"Process as creative writing:\n{self.state.input_text}")
)
@listen("process_business")
def handle_business(self):
agent = Agent(
role="Business Writer",
goal="Produce professional, results-oriented business content",
backstory="You are an experienced business writer who communicates "
"strategy and value clearly to professional audiences.",
llm=llm,
verbose=True,
)
self.state.result = str(
agent.kickoff(f"Process as business content:\n{self.state.input_text}")
)
flow = ContentFlow()
flow.state.input_text = "Explain how TCP handshakes work"
flow.kickoff()
print(flow.state.result)
```
The `@router()` decorator turns a method into a decision point. It returns a string that matches a listener — no mapping dictionaries, no separate routing functions. The branching logic reads like a Python `if` statement because it *is* one.
---
## Demo 3: Integrating AI Agent Crews into Flows
Here's where CrewAI's real power shines. Flows aren't just for chaining LLM calls — they orchestrate full **Crews** of autonomous agents. This is something LangGraph simply doesn't have a native equivalent for.
```python
from crewai import Agent, Task, Crew
from crewai.flow.flow import Flow, listen, start
from pydantic import BaseModel
class ArticleState(BaseModel):
topic: str = ""
research: str = ""
draft: str = ""
final_article: str = ""
class ArticleFlow(Flow[ArticleState]):
@start()
def run_research_crew(self):
"""A full Crew of agents handles research."""
researcher = Agent(
role="Senior Research Analyst",
goal=f"Produce comprehensive research on: {self.state.topic}",
backstory="You're a veteran analyst known for thorough, "
"well-sourced research reports.",
llm="gpt-4o"
)
research_task = Task(
description=f"Research '{self.state.topic}' thoroughly. "
"Cover key trends, data points, and expert opinions.",
expected_output="A detailed research brief with sources.",
agent=researcher
)
crew = Crew(agents=[researcher], tasks=[research_task])
result = crew.kickoff()
self.state.research = result.raw
return result.raw
@listen(run_research_crew)
def run_writing_crew(self, research_output):
"""A different Crew handles writing."""
writer = Agent(
role="Technical Writer",
goal="Write a compelling article based on provided research.",
backstory="You turn complex research into engaging, clear prose.",
llm="gpt-4o"
)
editor = Agent(
role="Senior Editor",
goal="Review and polish articles for publication quality.",
backstory="20 years of editorial experience at top tech publications.",
llm="gpt-4o"
)
write_task = Task(
description=f"Write an article based on this research:\n{self.state.research}",
expected_output="A well-structured draft article.",
agent=writer
)
edit_task = Task(
description="Review, fact-check, and polish the draft article.",
expected_output="A publication-ready article.",
agent=editor
)
crew = Crew(agents=[writer, editor], tasks=[write_task, edit_task])
result = crew.kickoff()
self.state.final_article = result.raw
return result.raw
# Run the full pipeline
flow = ArticleFlow()
flow.state.topic = "The Future of Edge AI"
flow.kickoff()
print(flow.state.final_article)
```
This is the key insight: **Flows provide the orchestration layer, and Crews provide the intelligence layer.** Each step in a Flow can spin up a full team of collaborating agents, each with their own roles, goals, and tools. You get structured, predictable control flow *and* autonomous agent collaboration — the best of both worlds.
In LangGraph, achieving something similar means manually implementing agent communication protocols, tool-calling loops, and delegation logic inside your node functions. It's possible, but it's plumbing you're building from scratch every time.
---
## Demo 4: Parallel Execution and Synchronization
Real-world pipelines often need to fan out work and join the results. CrewAI Flows handles this elegantly with `and_` and `or_` operators.
```python
from crewai import LLM
from crewai.flow.flow import Flow, and_, listen, start
from pydantic import BaseModel
llm = LLM(model="openai/gpt-5.2")
class AnalysisState(BaseModel):
topic: str = ""
market_data: str = ""
tech_analysis: str = ""
competitor_intel: str = ""
final_report: str = ""
class ParallelAnalysisFlow(Flow[AnalysisState]):
@start()
def start_method(self):
pass
@listen(start_method)
def gather_market_data(self):
# Your agentic or deterministic code
pass
@listen(start_method)
def run_tech_analysis(self):
# Your agentic or deterministic code
pass
@listen(start_method)
def gather_competitor_intel(self):
# Your agentic or deterministic code
pass
@listen(and_(gather_market_data, run_tech_analysis, gather_competitor_intel))
def synthesize_report(self):
# Your agentic or deterministic code
pass
flow = ParallelAnalysisFlow()
flow.state.topic = "AI-powered developer tools"
flow.kickoff()
```
Multiple `@start()` decorators fire in parallel. The `and_()` combinator on the `@listen` decorator ensures `synthesize_report` only executes after *all three* upstream methods complete. There's also `or_()` for when you want to proceed as soon as *any* upstream task finishes.
In LangGraph, you'd need to build a fan-out/fan-in pattern with parallel branches, a synchronization node, and careful state merging — all wired explicitly through edges.
---
## Why CrewAI Flows for Production
Beyond cleaner syntax, Flows deliver several production-critical advantages:
**Built-in state persistence.** Flow state is backed by LanceDB, meaning your workflows can survive crashes, be resumed, and accumulate knowledge across runs. LangGraph requires you to configure a separate checkpointer.
**Type-safe state management.** Pydantic models give you validation, serialization, and IDE support out of the box. LangGraph's `TypedDict` states don't validate at runtime.
**First-class agent orchestration.** Crews are a native primitive. You define agents with roles, goals, backstories, and tools — and they collaborate autonomously within the structured envelope of a Flow. No need to reinvent multi-agent coordination.
**Simpler mental model.** Decorators declare intent. `@start` means "begin here." `@listen(x)` means "run after x." `@router(x)` means "decide where to go after x." The code reads like the workflow it describes.
**CLI integration.** Run flows with `crewai run`. No separate compilation step, no graph serialization. Your Flow is a Python class, and it runs like one.
---
## Migration Cheat Sheet
If you're sitting on a LangGraph codebase and want to move to CrewAI Flows, here's a practical conversion guide:
1. **Map your state.** Convert your `TypedDict` to a Pydantic `BaseModel`. Add default values for all fields.
2. **Convert nodes to methods.** Each `add_node` function becomes a method on your `Flow` subclass. Replace `state["field"]` reads with `self.state.field`.
3. **Replace edges with decorators.** Your `add_edge(START, "first_node")` becomes `@start()` on the first method. Sequential `add_edge("a", "b")` becomes `@listen(a)` on method `b`.
4. **Replace conditional edges with `@router`.** Your routing function and `add_conditional_edges()` mapping become a single `@router()` method that returns a route string.
5. **Replace compile + invoke with kickoff.** Drop `graph.compile()`. Call `flow.kickoff()` instead.
6. **Consider where Crews fit.** Any node where you have complex multi-step agent logic is a candidate for extraction into a Crew. This is where you'll see the biggest quality improvement.
---
## Getting Started
Install CrewAI and scaffold a new Flow project:
```bash
pip install crewai
crewai create flow my_first_flow
cd my_first_flow
```
This generates a project structure with a ready-to-edit Flow class, configuration files, and a `pyproject.toml` with `type = "flow"` already set. Run it with:
```bash
crewai run
```
From there, add your agents, wire up your listeners, and ship it.
---
## Final Thoughts
LangGraph taught the ecosystem that AI workflows need structure. That was an important lesson. But CrewAI Flows takes that lesson and delivers it in a form that's faster to write, easier to read, and more powerful in production — especially when your workflows involve multiple collaborating agents.
If you're building anything beyond a single-agent chain, give Flows a serious look. The decorator-driven model, native Crew integration, and built-in state management mean you'll spend less time on plumbing and more time on the problems that matter.
Start with `crewai create flow`. You won't look back.

View File

@@ -1,97 +1,316 @@
--- ---
title: Brave Search title: Brave Search Tools
description: The `BraveSearchTool` is designed to search the internet using the Brave Search API. description: A suite of tools for querying the Brave Search API — covering web, news, image, and video search.
icon: searchengin icon: searchengin
mode: "wide" mode: "wide"
--- ---
# `BraveSearchTool` # Brave Search Tools
## Description ## Description
This tool is designed to perform web searches using the Brave Search API. It allows you to search the internet with a specified query and retrieve relevant results. The tool supports customizable result counts and country-specific searches. CrewAI offers a family of Brave Search tools, each targeting a specific [Brave Search API](https://brave.com/search/api/) endpoint.
Rather than a single catch-all tool, you can pick exactly the tool that matches the kind of results your agent needs:
| Tool | Endpoint | Use case |
| --- | --- | --- |
| `BraveWebSearchTool` | Web Search | General web results, snippets, and URLs |
| `BraveNewsSearchTool` | News Search | Recent news articles and headlines |
| `BraveImageSearchTool` | Image Search | Image results with dimensions and source URLs |
| `BraveVideoSearchTool` | Video Search | Video results from across the web |
| `BraveLocalPOIsTool` | Local POIs | Find points of interest (e.g., restaurants) |
| `BraveLocalPOIsDescriptionTool` | Local POIs | Retrieve AI-generated location descriptions |
| `BraveLLMContextTool` | LLM Context | Pre-extracted web content optimized for AI agents, LLM grounding, and RAG pipelines. |
All tools share a common base class (`BraveSearchToolBase`) that provides consistent behavior — rate limiting, automatic retries on `429` responses, header and parameter validation, and optional file saving.
<Note>
The older `BraveSearchTool` class is still available for backwards compatibility, but it is considered **legacy** and will not receive the same level of attention going forward. We recommend migrating to the specific tools listed above, which offer richer configuration and a more focused interface.
</Note>
<Note>
While many tools (e.g., _BraveWebSearchTool_, _BraveNewsSearchTool_, _BraveImageSearchTool_, and _BraveVideoSearchTool_) can be used with a free Brave Search API subscription/plan, some parameters (e.g., `enable_snippets`) and tools (e.g., _BraveLocalPOIsTool_ and _BraveLocalPOIsDescriptionTool_) require a paid plan. Consult your subscription plan's capabilities for clarification.
</Note>
## Installation ## Installation
To incorporate this tool into your project, follow the installation instructions below:
```shell ```shell
pip install 'crewai[tools]' pip install 'crewai[tools]'
``` ```
## Steps to Get Started ## Getting Started
To effectively use the `BraveSearchTool`, follow these steps: 1. **Install the package** — confirm that `crewai[tools]` is installed in your Python environment.
2. **Get an API key** — sign up at [api-dashboard.search.brave.com/login](https://api-dashboard.search.brave.com/login) to generate a key.
3. **Set the environment variable** — store your key as `BRAVE_API_KEY`, or pass it directly via the `api_key` parameter.
1. **Package Installation**: Confirm that the `crewai[tools]` package is installed in your Python environment. ## Quick Examples
2. **API Key Acquisition**: Acquire a Brave Search API key at https://api.search.brave.com/app/keys (sign in to generate a key).
3. **Environment Configuration**: Store your obtained API key in an environment variable named `BRAVE_API_KEY` to facilitate its use by the tool.
## Example ### Web Search
The following example demonstrates how to initialize the tool and execute a search with a given query:
```python Code ```python Code
from crewai_tools import BraveSearchTool from crewai_tools import BraveWebSearchTool
# Initialize the tool for internet searching capabilities tool = BraveWebSearchTool()
tool = BraveSearchTool() results = tool.run(q="CrewAI agent framework")
# Execute a search
results = tool.run(search_query="CrewAI agent framework")
print(results) print(results)
``` ```
## Parameters ### News Search
The `BraveSearchTool` accepts the following parameters:
- **search_query**: Mandatory. The search query you want to use to search the internet.
- **country**: Optional. Specify the country for the search results. Default is empty string.
- **n_results**: Optional. Number of search results to return. Default is `10`.
- **save_file**: Optional. Whether to save the search results to a file. Default is `False`.
## Example with Parameters
Here is an example demonstrating how to use the tool with additional parameters:
```python Code ```python Code
from crewai_tools import BraveSearchTool from crewai_tools import BraveNewsSearchTool
# Initialize the tool with custom parameters tool = BraveNewsSearchTool()
tool = BraveSearchTool( results = tool.run(q="latest AI breakthroughs")
country="US", print(results)
n_results=5, ```
save_file=True
### Image Search
```python Code
from crewai_tools import BraveImageSearchTool
tool = BraveImageSearchTool()
results = tool.run(q="northern lights photography")
print(results)
```
### Video Search
```python Code
from crewai_tools import BraveVideoSearchTool
tool = BraveVideoSearchTool()
results = tool.run(q="how to build AI agents")
print(results)
```
### Location POI Descriptions
```python Code
from crewai_tools import (
BraveWebSearchTool,
BraveLocalPOIsDescriptionTool,
) )
# Execute a search web_search = BraveWebSearchTool(raw=True)
results = tool.run(search_query="Latest AI developments") poi_details = BraveLocalPOIsDescriptionTool()
print(results)
results = web_search.run(q="italian restaurants in pensacola, florida")
if "locations" in results:
location_ids = [ loc["id"] for loc in results["locations"]["results"] ]
if location_ids:
descriptions = poi_details.run(ids=location_ids)
print(descriptions)
```
## Common Constructor Parameters
Every Brave Search tool accepts the following parameters at initialization:
| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `api_key` | `str \| None` | `None` | Brave API key. Falls back to the `BRAVE_API_KEY` environment variable. |
| `headers` | `dict \| None` | `None` | Additional HTTP headers to send with every request (e.g., `api-version`, geolocation headers). |
| `requests_per_second` | `float` | `1.0` | Maximum request rate. The tool will sleep between calls to stay within this limit. |
| `save_file` | `bool` | `False` | When `True`, each response is written to a timestamped `.txt` file. |
| `raw` | `bool` | `False` | When `True`, the full API JSON response is returned without any refinement. |
| `timeout` | `int` | `30` | HTTP request timeout in seconds. |
| `country` | `str \| None` | `None` | Legacy shorthand for geo-targeting (e.g., `"US"`). Prefer using the `country` query parameter directly. |
| `n_results` | `int` | `10` | Legacy shorthand for result count. Prefer using the `count` query parameter directly. |
<Warning>
The `country` and `n_results` constructor parameters exist for backwards compatibility. They are applied as defaults when the corresponding query parameters (`country`, `count`) are not provided at call time. For new code, we recommend passing `country` and `count` directly as query parameters instead.
</Warning>
## Query Parameters
Each tool validates its query parameters against a Pydantic schema before sending the request.
The parameters vary slightly per endpoint — here is a summary of the most commonly used ones:
### BraveWebSearchTool
| Parameter | Description |
| --- | --- |
| `q` | **(required)** Search query string (max 400 chars). |
| `country` | Two-letter country code for geo-targeting (e.g., `"US"`). |
| `search_lang` | Two-letter language code for results (e.g., `"en"`). |
| `count` | Max number of results to return (120). |
| `offset` | Skip the first N pages of results (09). |
| `safesearch` | Content filter: `"off"`, `"moderate"`, or `"strict"`. |
| `freshness` | Recency filter: `"pd"` (past day), `"pw"` (past week), `"pm"` (past month), `"py"` (past year), or a date range like `"2025-01-01to2025-06-01"`. |
| `extra_snippets` | Include up to 5 additional text snippets per result. |
| `goggles` | Brave Goggles URL(s) and/or source for custom re-ranking. |
For the complete parameter and header reference, see the [Brave Web Search API documentation](https://api-dashboard.search.brave.com/api-reference/web/search/get).
### BraveNewsSearchTool
| Parameter | Description |
| --- | --- |
| `q` | **(required)** Search query string (max 400 chars). |
| `country` | Two-letter country code for geo-targeting. |
| `search_lang` | Two-letter language code for results. |
| `count` | Max number of results to return (150). |
| `offset` | Skip the first N pages of results (09). |
| `safesearch` | Content filter: `"off"`, `"moderate"`, or `"strict"`. |
| `freshness` | Recency filter (same options as Web Search). |
| `goggles` | Brave Goggles URL(s) and/or source for custom re-ranking. |
For the complete parameter and header reference, see the [Brave News Search API documentation](https://api-dashboard.search.brave.com/api-reference/news/news_search/get).
### BraveImageSearchTool
| Parameter | Description |
| --- | --- |
| `q` | **(required)** Search query string (max 400 chars). |
| `country` | Two-letter country code for geo-targeting. |
| `search_lang` | Two-letter language code for results. |
| `count` | Max number of results to return (1200). |
| `safesearch` | Content filter: `"off"` or `"strict"`. |
| `spellcheck` | Attempt to correct spelling errors in the query. |
For the complete parameter and header reference, see the [Brave Image Search API documentation](https://api-dashboard.search.brave.com/api-reference/images/image_search).
### BraveVideoSearchTool
| Parameter | Description |
| --- | --- |
| `q` | **(required)** Search query string (max 400 chars). |
| `country` | Two-letter country code for geo-targeting. |
| `search_lang` | Two-letter language code for results. |
| `count` | Max number of results to return (150). |
| `offset` | Skip the first N pages of results (09). |
| `safesearch` | Content filter: `"off"`, `"moderate"`, or `"strict"`. |
| `freshness` | Recency filter (same options as Web Search). |
For the complete parameter and header reference, see the [Brave Video Search API documentation](https://api-dashboard.search.brave.com/api-reference/videos/video_search/get).
### BraveLocalPOIsTool
| Parameter | Description |
| --- | --- |
| `ids` | **(required)** A list of unique identifiers for the desired locations. |
| `search_lang` | Two-letter language code for results. |
For the complete parameter and header reference, see [Brave Local POIs API documentation](https://api-dashboard.search.brave.com/api-reference/web/local_pois).
### BraveLocalPOIsDescriptionTool
| Parameter | Description |
| --- | --- |
| `ids` | **(required)** A list of unique identifiers for the desired locations. |
For the complete parameter and header reference, see [Brave POI Descriptions API documentation](https://api-dashboard.search.brave.com/api-reference/web/poi_descriptions).
## Custom Headers
All tools support custom HTTP request headers. The Web Search tool, for example, accepts geolocation headers for location-aware results:
```python Code
from crewai_tools import BraveWebSearchTool
tool = BraveWebSearchTool(
headers={
"x-loc-lat": "37.7749",
"x-loc-long": "-122.4194",
"x-loc-city": "San Francisco",
"x-loc-state": "CA",
"x-loc-country": "US",
}
)
results = tool.run(q="best coffee shops nearby")
```
You can also update headers after initialization using the `set_headers()` method:
```python Code
tool.set_headers({"api-version": "2025-01-01"})
```
## Raw Mode
By default, each tool refines the API response into a concise list of results. If you need the full, unprocessed API response, enable raw mode:
```python Code
from crewai_tools import BraveWebSearchTool
tool = BraveWebSearchTool(raw=True)
full_response = tool.run(q="Brave Search API")
``` ```
## Agent Integration Example ## Agent Integration Example
Here's how to integrate the `BraveSearchTool` with a CrewAI agent: Here's how to equip a CrewAI agent with multiple Brave Search tools:
```python Code ```python Code
from crewai import Agent from crewai import Agent
from crewai.project import agent from crewai.project import agent
from crewai_tools import BraveSearchTool from crewai_tools import BraveWebSearchTool, BraveNewsSearchTool
# Initialize the tool web_search = BraveWebSearchTool()
brave_search_tool = BraveSearchTool() news_search = BraveNewsSearchTool()
# Define an agent with the BraveSearchTool
@agent @agent
def researcher(self) -> Agent: def researcher(self) -> Agent:
return Agent( return Agent(
config=self.agents_config["researcher"], config=self.agents_config["researcher"],
allow_delegation=False, tools=[web_search, news_search],
tools=[brave_search_tool]
) )
``` ```
## Advanced Example
Combining multiple parameters for a targeted search:
```python Code
from crewai_tools import BraveWebSearchTool
tool = BraveWebSearchTool(
requests_per_second=0.5, # conservative rate limit
save_file=True,
)
results = tool.run(
q="artificial intelligence news",
country="US",
search_lang="en",
count=5,
freshness="pm", # past month only
extra_snippets=True,
)
print(results)
```
## Migrating from `BraveSearchTool` (Legacy)
If you are currently using `BraveSearchTool`, switching to the new tools is straightforward:
```python Code
# Before (legacy)
from crewai_tools import BraveSearchTool
tool = BraveSearchTool(country="US", n_results=5, save_file=True)
results = tool.run(search_query="AI agents")
# After (recommended)
from crewai_tools import BraveWebSearchTool
tool = BraveWebSearchTool(save_file=True)
results = tool.run(q="AI agents", country="US", count=5)
```
Key differences:
- **Import**: Use `BraveWebSearchTool` (or the news/image/video variant) instead of `BraveSearchTool`.
- **Query parameter**: Use `q` instead of `search_query`. (Both `search_query` and `query` are still accepted for convenience, but `q` is the preferred parameter.)
- **Result count**: Pass `count` as a query parameter instead of `n_results` at init time.
- **Country**: Pass `country` as a query parameter instead of at init time.
- **API key**: Can now be passed directly via `api_key=` in addition to the `BRAVE_API_KEY` environment variable.
- **Rate limiting**: Configurable via `requests_per_second` with automatic retry on `429` responses.
## Conclusion ## Conclusion
By integrating the `BraveSearchTool` into Python projects, users gain the ability to conduct real-time, relevant searches across the internet directly from their applications. The tool provides a simple interface to the powerful Brave Search API, making it easy to retrieve and process search results programmatically. By adhering to the setup and usage guidelines provided, incorporating this tool into projects is streamlined and straightforward. The Brave Search tool suite gives your CrewAI agents flexible, endpoint-specific access to the Brave Search API. Whether you need web pages, breaking news, images, or videos, there is a dedicated tool with validated parameters and built-in resilience. Pick the tool that fits your use case, and refer to the [Brave Search API documentation](https://brave.com/search/api/) for the full details on available parameters and response formats.

View File

@@ -4,6 +4,114 @@ description: "CrewAI의 제품 업데이트, 개선 사항 및 버그 수정"
icon: "clock" icon: "clock"
mode: "wide" mode: "wide"
--- ---
<Update label="2026년 3월 14일">
## v1.10.2rc2
[GitHub 릴리스 보기](https://github.com/crewAIInc/crewAI/releases/tag/1.10.2rc2)
## 변경 사항
### 버그 수정
- 읽기 전용 스토리지 작업에서 독점 잠금 제거
### 문서
- v1.10.2rc1에 대한 변경 로그 및 버전 업데이트
## 기여자
@greysonlalonde
</Update>
<Update label="2026년 3월 13일">
## v1.10.2rc1
[GitHub 릴리스 보기](https://github.com/crewAIInc/crewAI/releases/tag/1.10.2rc1)
## 변경 사항
### 기능
- 릴리스 명령 추가 및 PyPI 게시 트리거
### 버그 수정
- 보호되지 않은 I/O에 대한 프로세스 간 및 스레드 안전 잠금 수정
- 모든 스레드 및 실행기 경계를 넘는 contextvars 전파
- async 작업 스레드로 ContextVars 전파
### 문서
- v1.10.2a1에 대한 변경 로그 및 버전 업데이트
## 기여자
@danglies007, @greysonlalonde
</Update>
<Update label="2026년 3월 11일">
## v1.10.2a1
[GitHub 릴리스 보기](https://github.com/crewAIInc/crewAI/releases/tag/1.10.2a1)
## 변경 사항
### 기능
- Anthropics에 대한 도구 검색 지원 추가, 토큰 저장, 실행 중 적절한 도구를 동적으로 주입하는 기능 추가.
- 더 많은 Brave Search 도구 도입.
- 야간 릴리스를 위한 액션 생성.
### 버그 수정
- 동시 다중 프로세스 실행 중 LockException 수정.
- 단일 사용자 메시지에서 병렬 도구 결과 그룹화 문제 해결.
- MCP 도구 해상도 문제 해결 및 모든 공유 가변 연결 제거.
- human_feedback 함수에서 LLM 매개변수 처리 업데이트.
- LockedListProxy 및 LockedDictProxy에 누락된 list/dict 메서드 추가.
- 병렬 도구 호출 스레드에 contextvars 컨텍스트 전파.
- CVE 경로 탐색 취약점을 해결하기 위해 gitpython 의존성을 >=3.1.41로 업데이트.
### 리팩토링
- 메모리 클래스를 직렬화 가능하도록 리팩토링.
### 문서
- v1.10.1에 대한 변경 로그 및 버전 업데이트.
## 기여자
@akaKuruma, @github-actions[bot], @giulio-leone, @greysonlalonde, @joaomdmoura, @jonathansampson, @lorenzejay, @lucasgomide, @mattatcha
</Update>
<Update label="2026년 3월 4일">
## v1.10.1
[GitHub 릴리스 보기](https://github.com/crewAIInc/crewAI/releases/tag/1.10.1)
## 변경 사항
### 기능
- Gemini GenAI 업그레이드
### 버그 수정
- 재귀를 피하기 위해 실행기 리스너 값을 조정
- Gemini에서 병렬 함수 응답 부분을 단일 Content 객체로 그룹화
- Gemini에서 사고 모델의 사고 출력을 표시
- 에이전트 도구가 None일 때 MCP 및 플랫폼 도구 로드
- A2A에서 실행 이벤트 루프가 있는 Jupyter 환경 지원
- 일시적인 추적을 위해 익명 ID 사용
- 조건부로 플러스 헤더 전달
- 원격 측정을 위해 비주 스레드에서 신호 처리기 등록 건너뛰기
- 도구 오류를 관찰로 주입하고 이름 충돌 해결
- Dependabot 경고를 해결하기 위해 pypdf를 4.x에서 6.7.4로 업그레이드
- 심각 및 높은 Dependabot 보안 경고 해결
### 문서
- Composio 도구 문서를 지역별로 동기화
## 기여자
@giulio-leone, @greysonlalonde, @haxzie, @joaomdmoura, @lorenzejay, @mattatcha, @mplachta, @nicoferdi96
</Update>
<Update label="2026년 2월 27일"> <Update label="2026년 2월 27일">
## v1.10.1a1 ## v1.10.1a1

View File

@@ -0,0 +1,518 @@
---
title: "LangGraph에서 CrewAI로 옮기기: 엔지니어를 위한 실전 가이드"
description: LangGraph로 이미 구축했다면, 프로젝트를 CrewAI로 빠르게 옮기는 방법을 알아보세요
icon: switch
mode: "wide"
---
LangGraph로 에이전트를 구축해 왔습니다. `StateGraph`와 씨름하고, 조건부 에지를 연결하고, 새벽 2시에 상태 딕셔너리를 디버깅해 본 적도 있죠. 동작은 하지만 — 어느 순간부터 프로덕션으로 가는 더 나은 길이 없을까 고민하게 됩니다.
있습니다. **CrewAI Flows**는 이벤트 기반 오케스트레이션, 조건부 라우팅, 공유 상태라는 동일한 힘을 훨씬 적은 보일러플레이트와 실제로 다단계 AI 워크플로우를 생각하는 방식에 잘 맞는 정신적 모델로 제공합니다.
이 글은 핵심 개념을 나란히 비교하고 실제 코드 비교를 보여주며, 다음으로 손이 갈 프레임워크가 왜 CrewAI Flows인지 설명합니다.
---
## 정신적 모델의 전환
LangGraph는 **그래프**로 생각하라고 요구합니다: 노드, 에지, 그리고 상태 딕셔너리. 모든 워크플로우는 계산 단계 사이의 전이를 명시적으로 연결하는 방향 그래프입니다. 강력하지만, 특히 워크플로우가 몇 개의 결정 지점이 있는 순차적 흐름일 때 이 추상화는 오버헤드를 가져옵니다.
CrewAI Flows는 **이벤트**로 생각하라고 요구합니다: 시작하는 메서드, 결과를 듣는 메서드, 실행을 라우팅하는 메서드. 워크플로우의 토폴로지는 명시적 그래프 구성 대신 데코레이터 어노테이션에서 드러납니다. 이것은 단순한 문법 설탕이 아니라 — 파이프라인을 설계하고 읽고 유지하는 방식을 바꿉니다.
핵심 매핑은 다음과 같습니다:
| LangGraph 개념 | CrewAI Flows 대응 |
| --- | --- |
| `StateGraph` class | `Flow` class |
| `add_node()` | Methods decorated with `@start`, `@listen` |
| `add_edge()` / `add_conditional_edges()` | `@listen()` / `@router()` decorators |
| `TypedDict` state | Pydantic `BaseModel` state |
| `START` / `END` constants | `@start()` decorator / natural method return |
| `graph.compile()` | `flow.kickoff()` |
| Checkpointer / persistence | Built-in memory (LanceDB-backed) |
실제로 어떻게 보이는지 살펴보겠습니다.
---
## 데모 1: 간단한 순차 파이프라인
주제를 받아 조사하고, 요약을 작성한 뒤, 결과를 포맷팅하는 파이프라인을 만든다고 해봅시다. 각 프레임워크는 이렇게 처리합니다.
### LangGraph 방식
```python
from typing import TypedDict
from langgraph.graph import StateGraph, START, END
class ResearchState(TypedDict):
topic: str
raw_research: str
summary: str
formatted_output: str
def research_topic(state: ResearchState) -> dict:
# Call an LLM or search API
result = llm.invoke(f"Research the topic: {state['topic']}")
return {"raw_research": result}
def write_summary(state: ResearchState) -> dict:
result = llm.invoke(
f"Summarize this research:\n{state['raw_research']}"
)
return {"summary": result}
def format_output(state: ResearchState) -> dict:
result = llm.invoke(
f"Format this summary as a polished article section:\n{state['summary']}"
)
return {"formatted_output": result}
# Build the graph
graph = StateGraph(ResearchState)
graph.add_node("research", research_topic)
graph.add_node("summarize", write_summary)
graph.add_node("format", format_output)
graph.add_edge(START, "research")
graph.add_edge("research", "summarize")
graph.add_edge("summarize", "format")
graph.add_edge("format", END)
# Compile and run
app = graph.compile()
result = app.invoke({"topic": "quantum computing advances in 2026"})
print(result["formatted_output"])
```
함수를 정의하고 노드로 등록한 다음, 모든 전이를 수동으로 연결합니다. 이렇게 단순한 순서인데도 의례처럼 해야 할 작업이 많습니다.
### CrewAI Flows 방식
```python
from crewai import LLM, Agent, Crew, Process, Task
from crewai.flow.flow import Flow, listen, start
from pydantic import BaseModel
llm = LLM(model="openai/gpt-5.2")
class ResearchState(BaseModel):
topic: str = ""
raw_research: str = ""
summary: str = ""
formatted_output: str = ""
class ResearchFlow(Flow[ResearchState]):
@start()
def research_topic(self):
# Option 1: Direct LLM call
result = llm.call(f"Research the topic: {self.state.topic}")
self.state.raw_research = result
return result
@listen(research_topic)
def write_summary(self, research_output):
# Option 2: A single agent
summarizer = Agent(
role="Research Summarizer",
goal="Produce concise, accurate summaries of research content",
backstory="You are an expert at distilling complex research into clear, "
"digestible summaries.",
llm=llm,
verbose=True,
)
result = summarizer.kickoff(
f"Summarize this research:\n{self.state.raw_research}"
)
self.state.summary = str(result)
return self.state.summary
@listen(write_summary)
def format_output(self, summary_output):
# Option 3: a complete crew (with one or more agents)
formatter = Agent(
role="Content Formatter",
goal="Transform research summaries into polished, publication-ready article sections",
backstory="You are a skilled editor with expertise in structuring and "
"presenting technical content for a general audience.",
llm=llm,
verbose=True,
)
format_task = Task(
description=f"Format this summary as a polished article section:\n{self.state.summary}",
expected_output="A well-structured, polished article section ready for publication.",
agent=formatter,
)
crew = Crew(
agents=[formatter],
tasks=[format_task],
process=Process.sequential,
verbose=True,
)
result = crew.kickoff()
self.state.formatted_output = str(result)
return self.state.formatted_output
# Run the flow
flow = ResearchFlow()
flow.state.topic = "quantum computing advances in 2026"
result = flow.kickoff()
print(flow.state.formatted_output)
```
눈에 띄는 차이점이 있습니다: 그래프 구성 없음, 에지 연결 없음, 컴파일 단계 없음. 실행 순서는 로직이 있는 곳에서 바로 선언됩니다. `@start()`는 진입점을 표시하고, `@listen(method_name)`은 단계들을 연결합니다. 상태는 타입 안전성, 검증, IDE 자동 완성까지 제공하는 제대로 된 Pydantic 모델입니다.
---
## 데모 2: 조건부 라우팅
여기서 흥미로워집니다. 콘텐츠 유형에 따라 서로 다른 처리 경로로 라우팅하는 파이프라인을 만든다고 해봅시다.
### LangGraph 방식
```python
from typing import TypedDict, Literal
from langgraph.graph import StateGraph, START, END
class ContentState(TypedDict):
input_text: str
content_type: str
result: str
def classify_content(state: ContentState) -> dict:
content_type = llm.invoke(
f"Classify this content as 'technical', 'creative', or 'business':\n{state['input_text']}"
)
return {"content_type": content_type.strip().lower()}
def process_technical(state: ContentState) -> dict:
result = llm.invoke(f"Process as technical doc:\n{state['input_text']}")
return {"result": result}
def process_creative(state: ContentState) -> dict:
result = llm.invoke(f"Process as creative writing:\n{state['input_text']}")
return {"result": result}
def process_business(state: ContentState) -> dict:
result = llm.invoke(f"Process as business content:\n{state['input_text']}")
return {"result": result}
# Routing function
def route_content(state: ContentState) -> Literal["technical", "creative", "business"]:
return state["content_type"]
# Build the graph
graph = StateGraph(ContentState)
graph.add_node("classify", classify_content)
graph.add_node("technical", process_technical)
graph.add_node("creative", process_creative)
graph.add_node("business", process_business)
graph.add_edge(START, "classify")
graph.add_conditional_edges(
"classify",
route_content,
{
"technical": "technical",
"creative": "creative",
"business": "business",
}
)
graph.add_edge("technical", END)
graph.add_edge("creative", END)
graph.add_edge("business", END)
app = graph.compile()
result = app.invoke({"input_text": "Explain how TCP handshakes work"})
```
별도의 라우팅 함수, 명시적 조건부 에지 매핑, 그리고 모든 분기에 대한 종료 에지가 필요합니다. 라우팅 결정 로직이 그 결정을 만들어 내는 노드와 분리됩니다.
### CrewAI Flows 방식
```python
from crewai import LLM, Agent
from crewai.flow.flow import Flow, listen, router, start
from pydantic import BaseModel
llm = LLM(model="openai/gpt-5.2")
class ContentState(BaseModel):
input_text: str = ""
content_type: str = ""
result: str = ""
class ContentFlow(Flow[ContentState]):
@start()
def classify_content(self):
self.state.content_type = (
llm.call(
f"Classify this content as 'technical', 'creative', or 'business':\n"
f"{self.state.input_text}"
)
.strip()
.lower()
)
return self.state.content_type
@router(classify_content)
def route_content(self, classification):
if classification == "technical":
return "process_technical"
elif classification == "creative":
return "process_creative"
else:
return "process_business"
@listen("process_technical")
def handle_technical(self):
agent = Agent(
role="Technical Writer",
goal="Produce clear, accurate technical documentation",
backstory="You are an expert technical writer who specializes in "
"explaining complex technical concepts precisely.",
llm=llm,
verbose=True,
)
self.state.result = str(
agent.kickoff(f"Process as technical doc:\n{self.state.input_text}")
)
@listen("process_creative")
def handle_creative(self):
agent = Agent(
role="Creative Writer",
goal="Craft engaging and imaginative creative content",
backstory="You are a talented creative writer with a flair for "
"compelling storytelling and vivid expression.",
llm=llm,
verbose=True,
)
self.state.result = str(
agent.kickoff(f"Process as creative writing:\n{self.state.input_text}")
)
@listen("process_business")
def handle_business(self):
agent = Agent(
role="Business Writer",
goal="Produce professional, results-oriented business content",
backstory="You are an experienced business writer who communicates "
"strategy and value clearly to professional audiences.",
llm=llm,
verbose=True,
)
self.state.result = str(
agent.kickoff(f"Process as business content:\n{self.state.input_text}")
)
flow = ContentFlow()
flow.state.input_text = "Explain how TCP handshakes work"
flow.kickoff()
print(flow.state.result)
```
`@router()` 데코레이터는 메서드를 결정 지점으로 만듭니다. 리스너와 매칭되는 문자열을 반환하므로, 매핑 딕셔너리도, 별도의 라우팅 함수도 필요 없습니다. 분기 로직이 Python `if` 문처럼 읽히는 이유는, 실제로 `if` 문이기 때문입니다.
---
## 데모 3: AI 에이전트 Crew를 Flow에 통합하기
여기서 CrewAI의 진짜 힘이 드러납니다. Flows는 LLM 호출을 연결하는 것에 그치지 않고 자율적인 에이전트 **Crew** 전체를 오케스트레이션합니다. 이는 LangGraph에 기본으로 대응되는 개념이 없습니다.
```python
from crewai import Agent, Task, Crew
from crewai.flow.flow import Flow, listen, start
from pydantic import BaseModel
class ArticleState(BaseModel):
topic: str = ""
research: str = ""
draft: str = ""
final_article: str = ""
class ArticleFlow(Flow[ArticleState]):
@start()
def run_research_crew(self):
"""A full Crew of agents handles research."""
researcher = Agent(
role="Senior Research Analyst",
goal=f"Produce comprehensive research on: {self.state.topic}",
backstory="You're a veteran analyst known for thorough, "
"well-sourced research reports.",
llm="gpt-4o"
)
research_task = Task(
description=f"Research '{self.state.topic}' thoroughly. "
"Cover key trends, data points, and expert opinions.",
expected_output="A detailed research brief with sources.",
agent=researcher
)
crew = Crew(agents=[researcher], tasks=[research_task])
result = crew.kickoff()
self.state.research = result.raw
return result.raw
@listen(run_research_crew)
def run_writing_crew(self, research_output):
"""A different Crew handles writing."""
writer = Agent(
role="Technical Writer",
goal="Write a compelling article based on provided research.",
backstory="You turn complex research into engaging, clear prose.",
llm="gpt-4o"
)
editor = Agent(
role="Senior Editor",
goal="Review and polish articles for publication quality.",
backstory="20 years of editorial experience at top tech publications.",
llm="gpt-4o"
)
write_task = Task(
description=f"Write an article based on this research:\n{self.state.research}",
expected_output="A well-structured draft article.",
agent=writer
)
edit_task = Task(
description="Review, fact-check, and polish the draft article.",
expected_output="A publication-ready article.",
agent=editor
)
crew = Crew(agents=[writer, editor], tasks=[write_task, edit_task])
result = crew.kickoff()
self.state.final_article = result.raw
return result.raw
# Run the full pipeline
flow = ArticleFlow()
flow.state.topic = "The Future of Edge AI"
flow.kickoff()
print(flow.state.final_article)
```
핵심 인사이트는 다음과 같습니다: **Flows는 오케스트레이션 레이어를, Crews는 지능 레이어를 제공합니다.** Flow의 각 단계는 각자의 역할, 목표, 도구를 가진 협업 에이전트 팀을 띄울 수 있습니다. 구조화되고 예측 가능한 제어 흐름 *그리고* 자율적 에이전트 협업 — 두 세계의 장점을 모두 얻습니다.
LangGraph에서 비슷한 것을 하려면 노드 함수 안에 에이전트 통신 프로토콜, 도구 호출 루프, 위임 로직을 직접 구현해야 합니다. 가능하긴 하지만, 매번 처음부터 배관을 만드는 셈입니다.
---
## 데모 4: 병렬 실행과 동기화
실제 파이프라인은 종종 작업을 병렬로 분기하고 결과를 합쳐야 합니다. CrewAI Flows는 `and_`와 `or_` 연산자로 이를 우아하게 처리합니다.
```python
from crewai import LLM
from crewai.flow.flow import Flow, and_, listen, start
from pydantic import BaseModel
llm = LLM(model="openai/gpt-5.2")
class AnalysisState(BaseModel):
topic: str = ""
market_data: str = ""
tech_analysis: str = ""
competitor_intel: str = ""
final_report: str = ""
class ParallelAnalysisFlow(Flow[AnalysisState]):
@start()
def start_method(self):
pass
@listen(start_method)
def gather_market_data(self):
# Your agentic or deterministic code
pass
@listen(start_method)
def run_tech_analysis(self):
# Your agentic or deterministic code
pass
@listen(start_method)
def gather_competitor_intel(self):
# Your agentic or deterministic code
pass
@listen(and_(gather_market_data, run_tech_analysis, gather_competitor_intel))
def synthesize_report(self):
# Your agentic or deterministic code
pass
flow = ParallelAnalysisFlow()
flow.state.topic = "AI-powered developer tools"
flow.kickoff()
```
여러 `@start()` 데코레이터는 병렬로 실행됩니다. `@listen` 데코레이터의 `and_()` 결합자는 `synthesize_report`가 *세 가지* 상위 메서드가 모두 완료된 뒤에만 실행되도록 보장합니다. *어떤* 상위 작업이든 끝나는 즉시 진행하고 싶다면 `or_()`도 사용할 수 있습니다.
LangGraph에서는 병렬 분기, 동기화 노드, 신중한 상태 병합이 포함된 fan-out/fan-in 패턴을 만들어야 하며 — 모든 것을 에지로 명시적으로 연결해야 합니다.
---
## 프로덕션에서 CrewAI Flows를 쓰는 이유
깔끔한 문법을 넘어, Flows는 여러 프로덕션 핵심 이점을 제공합니다:
**내장 상태 지속성.** Flow 상태는 LanceDB에 의해 백업되므로 워크플로우가 크래시에서 살아남고, 재개될 수 있으며, 실행 간에 지식을 축적할 수 있습니다. LangGraph는 별도의 체크포인터를 구성해야 합니다.
**타입 안전한 상태 관리.** Pydantic 모델은 즉시 검증, 직렬화, IDE 지원을 제공합니다. LangGraph의 `TypedDict` 상태는 런타임 검증을 하지 않습니다.
**일급 에이전트 오케스트레이션.** Crews는 기본 프리미티브입니다. 역할, 목표, 배경, 도구를 가진 에이전트를 정의하고, Flow의 구조적 틀 안에서 자율적으로 협업하게 합니다. 다중 에이전트 조율을 다시 만들 필요가 없습니다.
**더 단순한 정신적 모델.** 데코레이터는 의도를 선언합니다. `@start`는 "여기서 시작", `@listen(x)`는 "x 이후 실행", `@router(x)`는 "x 이후 어디로 갈지 결정"을 의미합니다. 코드는 자신이 설명하는 워크플로우처럼 읽힙니다.
**CLI 통합.** `crewai run`으로 Flows를 실행합니다. 별도의 컴파일 단계나 그래프 직렬화가 없습니다. Flow는 Python 클래스이며, 그대로 실행됩니다.
---
## 마이그레이션 치트 시트
LangGraph 코드베이스를 CrewAI Flows로 옮기고 싶다면, 다음의 실전 변환 가이드를 참고하세요:
1. **상태를 매핑하세요.** `TypedDict`를 Pydantic `BaseModel`로 변환하고 모든 필드에 기본값을 추가하세요.
2. **노드를 메서드로 변환하세요.** 각 `add_node` 함수는 `Flow` 서브클래스의 메서드가 됩니다. `state["field"]` 읽기는 `self.state.field`로 바꾸세요.
3. **에지를 데코레이터로 교체하세요.** `add_edge(START, "first_node")`는 첫 메서드의 `@start()`가 됩니다. 순차적인 `add_edge("a", "b")`는 `b` 메서드의 `@listen(a)`가 됩니다.
4. **조건부 에지는 `@router`로 교체하세요.** 라우팅 함수와 `add_conditional_edges()` 매핑은 하나의 `@router()` 메서드로 통합하고, 라우트 문자열을 반환하세요.
5. **compile + invoke를 kickoff으로 교체하세요.** `graph.compile()`를 제거하고 `flow.kickoff()`를 호출하세요.
6. **Crew가 들어갈 지점을 고려하세요.** 복잡한 다단계 에이전트 로직이 있는 노드는 Crew로 분리할 후보입니다. 이 부분에서 가장 큰 품질 향상을 체감할 수 있습니다.
---
## 시작하기
CrewAI를 설치하고 새 Flow 프로젝트를 스캐폴딩하세요:
```bash
pip install crewai
crewai create flow my_first_flow
cd my_first_flow
```
이렇게 하면 바로 편집 가능한 Flow 클래스, 설정 파일, 그리고 `type = "flow"`가 이미 설정된 `pyproject.toml`이 포함된 프로젝트 구조가 생성됩니다. 다음으로 실행하세요:
```bash
crewai run
```
그 다음부터는 에이전트를 추가하고 리스너를 연결한 뒤, 배포하면 됩니다.
---
## 마무리
LangGraph는 AI 워크플로우에 구조가 필요하다는 사실을 생태계에 일깨워 주었습니다. 중요한 교훈이었습니다. 하지만 CrewAI Flows는 그 교훈을 더 빠르게 쓰고, 더 쉽게 읽으며, 프로덕션에서 더 강력한 형태로 제공합니다 — 특히 워크플로우에 여러 에이전트의 협업이 포함될 때 그렇습니다.
단일 에이전트 체인을 넘는 무엇인가를 만들고 있다면, Flows를 진지하게 검토해 보세요. 데코레이터 기반 모델, Crews의 네이티브 통합, 내장 상태 관리를 통해 배관 작업에 쓰는 시간을 줄이고, 중요한 문제에 더 많은 시간을 쓸 수 있습니다.
`crewai create flow`로 시작하세요. 후회하지 않을 겁니다.

View File

@@ -4,6 +4,114 @@ description: "Atualizações de produto, melhorias e correções do CrewAI"
icon: "clock" icon: "clock"
mode: "wide" mode: "wide"
--- ---
<Update label="14 mar 2026">
## v1.10.2rc2
[Ver release no GitHub](https://github.com/crewAIInc/crewAI/releases/tag/1.10.2rc2)
## O que Mudou
### Correções de Bugs
- Remover bloqueios exclusivos de operações de armazenamento somente leitura
### Documentação
- Atualizar changelog e versão para v1.10.2rc1
## Contribuidores
@greysonlalonde
</Update>
<Update label="13 mar 2026">
## v1.10.2rc1
[Ver release no GitHub](https://github.com/crewAIInc/crewAI/releases/tag/1.10.2rc1)
## O que Mudou
### Funcionalidades
- Adicionar comando de lançamento e acionar publicação no PyPI
### Correções de Bugs
- Corrigir bloqueio seguro entre processos e threads para I/O não protegido
- Propagar contextvars através de todos os limites de thread e executor
- Propagar ContextVars para threads de tarefas assíncronas
### Documentação
- Atualizar changelog e versão para v1.10.2a1
## Contribuidores
@danglies007, @greysonlalonde
</Update>
<Update label="11 mar 2026">
## v1.10.2a1
[Ver release no GitHub](https://github.com/crewAIInc/crewAI/releases/tag/1.10.2a1)
## O que mudou
### Recursos
- Adicionar suporte para busca de ferramentas, salvamento de tokens e injeção dinâmica de ferramentas apropriadas durante a execução para Anthropics.
- Introduzir mais ferramentas de Busca Brave.
- Criar ação para lançamentos noturnos.
### Correções de Bugs
- Corrigir LockException durante a execução concorrente de múltiplos processos.
- Resolver problemas com a agrupação de resultados de ferramentas paralelas em uma única mensagem de usuário.
- Abordar resoluções de ferramentas MCP e eliminar todas as conexões mutáveis compartilhadas.
- Atualizar o manuseio de parâmetros LLM na função human_feedback.
- Adicionar métodos de lista/dicionário ausentes a LockedListProxy e LockedDictProxy.
- Propagar o contexto de contextvars para as threads de chamada de ferramentas paralelas.
- Atualizar a dependência gitpython para >=3.1.41 para resolver a vulnerabilidade de travessia de diretórios CVE.
### Refatoração
- Refatorar classes de memória para serem serializáveis.
### Documentação
- Atualizar o changelog e a versão para v1.10.1.
## Contribuidores
@akaKuruma, @github-actions[bot], @giulio-leone, @greysonlalonde, @joaomdmoura, @jonathansampson, @lorenzejay, @lucasgomide, @mattatcha
</Update>
<Update label="04 mar 2026">
## v1.10.1
[Ver release no GitHub](https://github.com/crewAIInc/crewAI/releases/tag/1.10.1)
## O que mudou
### Recursos
- Atualizar Gemini GenAI
### Correções de Bugs
- Ajustar o valor do listener do executor para evitar recursão
- Agrupar partes da resposta da função paralela em um único objeto Content no Gemini
- Exibir a saída de pensamento dos modelos de pensamento no Gemini
- Carregar ferramentas MCP e da plataforma quando as ferramentas do agente forem None
- Suportar ambientes Jupyter com loops de eventos em A2A
- Usar ID anônimo para rastreamentos efêmeros
- Passar condicionalmente o cabeçalho plus
- Ignorar o registro do manipulador de sinal em threads não principais para telemetria
- Injetar erros de ferramentas como observações e resolver colisões de nomes
- Atualizar pypdf de 4.x para 6.7.4 para resolver alertas do Dependabot
- Resolver alertas de segurança críticos e altos do Dependabot
### Documentação
- Sincronizar a documentação da ferramenta Composio entre locais
## Contribuidores
@giulio-leone, @greysonlalonde, @haxzie, @joaomdmoura, @lorenzejay, @mattatcha, @mplachta, @nicoferdi96
</Update>
<Update label="27 fev 2026"> <Update label="27 fev 2026">
## v1.10.1a1 ## v1.10.1a1

View File

@@ -0,0 +1,518 @@
---
title: "Migrando do LangGraph para o CrewAI: um guia prático para engenheiros"
description: Se você já construiu com LangGraph, saiba como portar rapidamente seus projetos para o CrewAI
icon: switch
mode: "wide"
---
Você construiu agentes com LangGraph. Já lutou com o `StateGraph`, ligou arestas condicionais e depurou dicionários de estado às 2 da manhã. Funciona — mas, em algum momento, você começou a se perguntar se existe um caminho melhor para produção.
Existe. **CrewAI Flows** entrega o mesmo poder — orquestração orientada a eventos, roteamento condicional, estado compartilhado — com muito menos boilerplate e um modelo mental que se alinha a como você realmente pensa sobre fluxos de trabalho de IA em múltiplas etapas.
Este artigo apresenta os conceitos principais lado a lado, mostra comparações reais de código e demonstra por que o CrewAI Flows é o framework que você vai querer usar a seguir.
---
## A Mudança de Modelo Mental
LangGraph pede que você pense em **grafos**: nós, arestas e dicionários de estado. Todo workflow é um grafo direcionado em que você conecta explicitamente as transições entre as etapas de computação. É poderoso, mas a abstração traz overhead — especialmente quando o seu fluxo é fundamentalmente sequencial com alguns pontos de decisão.
CrewAI Flows pede que você pense em **eventos**: métodos que iniciam, métodos que escutam resultados e métodos que roteiam a execução. A topologia do workflow emerge de anotações com decorators, em vez de construção explícita do grafo. Isso não é apenas açúcar sintático — muda como você projeta, lê e mantém seus pipelines.
Veja o mapeamento principal:
| Conceito no LangGraph | Equivalente no CrewAI Flows |
| --- | --- |
| `StateGraph` class | `Flow` class |
| `add_node()` | Methods decorated with `@start`, `@listen` |
| `add_edge()` / `add_conditional_edges()` | `@listen()` / `@router()` decorators |
| `TypedDict` state | Pydantic `BaseModel` state |
| `START` / `END` constants | `@start()` decorator / natural method return |
| `graph.compile()` | `flow.kickoff()` |
| Checkpointer / persistence | Built-in memory (LanceDB-backed) |
Vamos ver como isso fica na prática.
---
## Demo 1: Um Pipeline Sequencial Simples
Imagine que você está construindo um pipeline que recebe um tema, pesquisa, escreve um resumo e formata a saída. Veja como cada framework lida com isso.
### Abordagem com LangGraph
```python
from typing import TypedDict
from langgraph.graph import StateGraph, START, END
class ResearchState(TypedDict):
topic: str
raw_research: str
summary: str
formatted_output: str
def research_topic(state: ResearchState) -> dict:
# Call an LLM or search API
result = llm.invoke(f"Research the topic: {state['topic']}")
return {"raw_research": result}
def write_summary(state: ResearchState) -> dict:
result = llm.invoke(
f"Summarize this research:\n{state['raw_research']}"
)
return {"summary": result}
def format_output(state: ResearchState) -> dict:
result = llm.invoke(
f"Format this summary as a polished article section:\n{state['summary']}"
)
return {"formatted_output": result}
# Build the graph
graph = StateGraph(ResearchState)
graph.add_node("research", research_topic)
graph.add_node("summarize", write_summary)
graph.add_node("format", format_output)
graph.add_edge(START, "research")
graph.add_edge("research", "summarize")
graph.add_edge("summarize", "format")
graph.add_edge("format", END)
# Compile and run
app = graph.compile()
result = app.invoke({"topic": "quantum computing advances in 2026"})
print(result["formatted_output"])
```
Você define funções, registra-as como nós e conecta manualmente cada transição. Para uma sequência simples como essa, há muita cerimônia.
### Abordagem com CrewAI Flows
```python
from crewai import LLM, Agent, Crew, Process, Task
from crewai.flow.flow import Flow, listen, start
from pydantic import BaseModel
llm = LLM(model="openai/gpt-5.2")
class ResearchState(BaseModel):
topic: str = ""
raw_research: str = ""
summary: str = ""
formatted_output: str = ""
class ResearchFlow(Flow[ResearchState]):
@start()
def research_topic(self):
# Option 1: Direct LLM call
result = llm.call(f"Research the topic: {self.state.topic}")
self.state.raw_research = result
return result
@listen(research_topic)
def write_summary(self, research_output):
# Option 2: A single agent
summarizer = Agent(
role="Research Summarizer",
goal="Produce concise, accurate summaries of research content",
backstory="You are an expert at distilling complex research into clear, "
"digestible summaries.",
llm=llm,
verbose=True,
)
result = summarizer.kickoff(
f"Summarize this research:\n{self.state.raw_research}"
)
self.state.summary = str(result)
return self.state.summary
@listen(write_summary)
def format_output(self, summary_output):
# Option 3: a complete crew (with one or more agents)
formatter = Agent(
role="Content Formatter",
goal="Transform research summaries into polished, publication-ready article sections",
backstory="You are a skilled editor with expertise in structuring and "
"presenting technical content for a general audience.",
llm=llm,
verbose=True,
)
format_task = Task(
description=f"Format this summary as a polished article section:\n{self.state.summary}",
expected_output="A well-structured, polished article section ready for publication.",
agent=formatter,
)
crew = Crew(
agents=[formatter],
tasks=[format_task],
process=Process.sequential,
verbose=True,
)
result = crew.kickoff()
self.state.formatted_output = str(result)
return self.state.formatted_output
# Run the flow
flow = ResearchFlow()
flow.state.topic = "quantum computing advances in 2026"
result = flow.kickoff()
print(flow.state.formatted_output)
```
Repare a diferença: nada de construção de grafo, de ligação de arestas, nem de etapa de compilação. A ordem de execução é declarada exatamente onde a lógica vive. `@start()` marca o ponto de entrada, e `@listen(method_name)` encadeia as etapas. O estado é um modelo Pydantic de verdade, com segurança de tipos, validação e auto-complete na IDE.
---
## Demo 2: Roteamento Condicional
Aqui é que fica interessante. Digamos que você está construindo um pipeline de conteúdo que roteia para diferentes caminhos de processamento com base no tipo de conteúdo detectado.
### Abordagem com LangGraph
```python
from typing import TypedDict, Literal
from langgraph.graph import StateGraph, START, END
class ContentState(TypedDict):
input_text: str
content_type: str
result: str
def classify_content(state: ContentState) -> dict:
content_type = llm.invoke(
f"Classify this content as 'technical', 'creative', or 'business':\n{state['input_text']}"
)
return {"content_type": content_type.strip().lower()}
def process_technical(state: ContentState) -> dict:
result = llm.invoke(f"Process as technical doc:\n{state['input_text']}")
return {"result": result}
def process_creative(state: ContentState) -> dict:
result = llm.invoke(f"Process as creative writing:\n{state['input_text']}")
return {"result": result}
def process_business(state: ContentState) -> dict:
result = llm.invoke(f"Process as business content:\n{state['input_text']}")
return {"result": result}
# Routing function
def route_content(state: ContentState) -> Literal["technical", "creative", "business"]:
return state["content_type"]
# Build the graph
graph = StateGraph(ContentState)
graph.add_node("classify", classify_content)
graph.add_node("technical", process_technical)
graph.add_node("creative", process_creative)
graph.add_node("business", process_business)
graph.add_edge(START, "classify")
graph.add_conditional_edges(
"classify",
route_content,
{
"technical": "technical",
"creative": "creative",
"business": "business",
}
)
graph.add_edge("technical", END)
graph.add_edge("creative", END)
graph.add_edge("business", END)
app = graph.compile()
result = app.invoke({"input_text": "Explain how TCP handshakes work"})
```
Você precisa de uma função de roteamento separada, de um mapeamento explícito de arestas condicionais e de arestas de término para cada ramificação. A lógica de roteamento fica desacoplada do nó que produz a decisão.
### Abordagem com CrewAI Flows
```python
from crewai import LLM, Agent
from crewai.flow.flow import Flow, listen, router, start
from pydantic import BaseModel
llm = LLM(model="openai/gpt-5.2")
class ContentState(BaseModel):
input_text: str = ""
content_type: str = ""
result: str = ""
class ContentFlow(Flow[ContentState]):
@start()
def classify_content(self):
self.state.content_type = (
llm.call(
f"Classify this content as 'technical', 'creative', or 'business':\n"
f"{self.state.input_text}"
)
.strip()
.lower()
)
return self.state.content_type
@router(classify_content)
def route_content(self, classification):
if classification == "technical":
return "process_technical"
elif classification == "creative":
return "process_creative"
else:
return "process_business"
@listen("process_technical")
def handle_technical(self):
agent = Agent(
role="Technical Writer",
goal="Produce clear, accurate technical documentation",
backstory="You are an expert technical writer who specializes in "
"explaining complex technical concepts precisely.",
llm=llm,
verbose=True,
)
self.state.result = str(
agent.kickoff(f"Process as technical doc:\n{self.state.input_text}")
)
@listen("process_creative")
def handle_creative(self):
agent = Agent(
role="Creative Writer",
goal="Craft engaging and imaginative creative content",
backstory="You are a talented creative writer with a flair for "
"compelling storytelling and vivid expression.",
llm=llm,
verbose=True,
)
self.state.result = str(
agent.kickoff(f"Process as creative writing:\n{self.state.input_text}")
)
@listen("process_business")
def handle_business(self):
agent = Agent(
role="Business Writer",
goal="Produce professional, results-oriented business content",
backstory="You are an experienced business writer who communicates "
"strategy and value clearly to professional audiences.",
llm=llm,
verbose=True,
)
self.state.result = str(
agent.kickoff(f"Process as business content:\n{self.state.input_text}")
)
flow = ContentFlow()
flow.state.input_text = "Explain how TCP handshakes work"
flow.kickoff()
print(flow.state.result)
```
O decorator `@router()` transforma um método em um ponto de decisão. Ele retorna uma string que corresponde a um listener — sem dicionários de mapeamento, sem funções de roteamento separadas. A lógica de ramificação parece um `if` em Python porque *é* um.
---
## Demo 3: Integrando Crews de Agentes de IA em Flows
É aqui que o verdadeiro poder do CrewAI aparece. Flows não servem apenas para encadear chamadas de LLM — elas orquestram **Crews** completas de agentes autônomos. Isso é algo para o qual o LangGraph simplesmente não tem um equivalente nativo.
```python
from crewai import Agent, Task, Crew
from crewai.flow.flow import Flow, listen, start
from pydantic import BaseModel
class ArticleState(BaseModel):
topic: str = ""
research: str = ""
draft: str = ""
final_article: str = ""
class ArticleFlow(Flow[ArticleState]):
@start()
def run_research_crew(self):
"""A full Crew of agents handles research."""
researcher = Agent(
role="Senior Research Analyst",
goal=f"Produce comprehensive research on: {self.state.topic}",
backstory="You're a veteran analyst known for thorough, "
"well-sourced research reports.",
llm="gpt-4o"
)
research_task = Task(
description=f"Research '{self.state.topic}' thoroughly. "
"Cover key trends, data points, and expert opinions.",
expected_output="A detailed research brief with sources.",
agent=researcher
)
crew = Crew(agents=[researcher], tasks=[research_task])
result = crew.kickoff()
self.state.research = result.raw
return result.raw
@listen(run_research_crew)
def run_writing_crew(self, research_output):
"""A different Crew handles writing."""
writer = Agent(
role="Technical Writer",
goal="Write a compelling article based on provided research.",
backstory="You turn complex research into engaging, clear prose.",
llm="gpt-4o"
)
editor = Agent(
role="Senior Editor",
goal="Review and polish articles for publication quality.",
backstory="20 years of editorial experience at top tech publications.",
llm="gpt-4o"
)
write_task = Task(
description=f"Write an article based on this research:\n{self.state.research}",
expected_output="A well-structured draft article.",
agent=writer
)
edit_task = Task(
description="Review, fact-check, and polish the draft article.",
expected_output="A publication-ready article.",
agent=editor
)
crew = Crew(agents=[writer, editor], tasks=[write_task, edit_task])
result = crew.kickoff()
self.state.final_article = result.raw
return result.raw
# Run the full pipeline
flow = ArticleFlow()
flow.state.topic = "The Future of Edge AI"
flow.kickoff()
print(flow.state.final_article)
```
Este é o insight-chave: **Flows fornecem a camada de orquestração, e Crews fornecem a camada de inteligência.** Cada etapa em um Flow pode subir uma equipe completa de agentes colaborativos, cada um com seus próprios papéis, objetivos e ferramentas. Você obtém fluxo de controle estruturado e previsível *e* colaboração autônoma de agentes — o melhor dos dois mundos.
No LangGraph, alcançar algo similar significa implementar manualmente protocolos de comunicação entre agentes, loops de chamada de ferramentas e lógica de delegação dentro das funções dos nós. É possível, mas é encanamento que você constrói do zero todas as vezes.
---
## Demo 4: Execução Paralela e Sincronização
Pipelines do mundo real frequentemente precisam dividir o trabalho e juntar os resultados. O CrewAI Flows lida com isso de forma elegante com os operadores `and_` e `or_`.
```python
from crewai import LLM
from crewai.flow.flow import Flow, and_, listen, start
from pydantic import BaseModel
llm = LLM(model="openai/gpt-5.2")
class AnalysisState(BaseModel):
topic: str = ""
market_data: str = ""
tech_analysis: str = ""
competitor_intel: str = ""
final_report: str = ""
class ParallelAnalysisFlow(Flow[AnalysisState]):
@start()
def start_method(self):
pass
@listen(start_method)
def gather_market_data(self):
# Your agentic or deterministic code
pass
@listen(start_method)
def run_tech_analysis(self):
# Your agentic or deterministic code
pass
@listen(start_method)
def gather_competitor_intel(self):
# Your agentic or deterministic code
pass
@listen(and_(gather_market_data, run_tech_analysis, gather_competitor_intel))
def synthesize_report(self):
# Your agentic or deterministic code
pass
flow = ParallelAnalysisFlow()
flow.state.topic = "AI-powered developer tools"
flow.kickoff()
```
Vários decorators `@start()` disparam em paralelo. O combinador `and_()` no decorator `@listen` garante que `synthesize_report` só execute depois que *todos os três* métodos upstream forem concluídos. Também existe `or_()` para quando você quer prosseguir assim que *qualquer* tarefa upstream terminar.
No LangGraph, você precisaria construir um padrão fan-out/fan-in com ramificações paralelas, um nó de sincronização e uma mesclagem de estado cuidadosa — tudo conectado explicitamente por arestas.
---
## Por que CrewAI Flows em Produção
Além de uma sintaxe mais limpa, Flows entrega várias vantagens críticas para produção:
**Persistência de estado integrada.** O estado do Flow é respaldado pelo LanceDB, o que significa que seus workflows podem sobreviver a falhas, ser retomados e acumular conhecimento entre execuções. No LangGraph, você precisa configurar um checkpointer separado.
**Gerenciamento de estado com segurança de tipos.** Modelos Pydantic oferecem validação, serialização e suporte de IDE prontos para uso. Estados `TypedDict` do LangGraph não validam em runtime.
**Orquestração de agentes de primeira classe.** Crews são um primitivo nativo. Você define agentes com papéis, objetivos, histórias e ferramentas — e eles colaboram de forma autônoma dentro do envelope estruturado de um Flow. Não é preciso reinventar a coordenação multiagente.
**Modelo mental mais simples.** Decorators declaram intenção. `@start` significa "comece aqui". `@listen(x)` significa "execute depois de x". `@router(x)` significa "decida para onde ir depois de x". O código lê como o workflow que ele descreve.
**Integração com CLI.** Execute flows com `crewai run`. Sem etapa de compilação separada, sem serialização de grafo. Seu Flow é uma classe Python, e ele roda como tal.
---
## Cheat Sheet de Migração
Se você está com uma base de código LangGraph e quer migrar para o CrewAI Flows, aqui vai um guia prático de conversão:
1. **Mapeie seu estado.** Converta seu `TypedDict` para um `BaseModel` do Pydantic. Adicione valores padrão para todos os campos.
2. **Converta nós em métodos.** Cada função de `add_node` vira um método na sua subclasse de `Flow`. Substitua leituras `state["field"]` por `self.state.field`.
3. **Substitua arestas por decorators.** `add_edge(START, "first_node")` vira `@start()` no primeiro método. A sequência `add_edge("a", "b")` vira `@listen(a)` no método `b`.
4. **Substitua arestas condicionais por `@router`.** A função de roteamento e o mapeamento do `add_conditional_edges()` viram um único método `@router()` que retorna a string de rota.
5. **Troque compile + invoke por kickoff.** Remova `graph.compile()`. Chame `flow.kickoff()`.
6. **Considere onde as Crews se encaixam.** Qualquer nó com lógica complexa de agentes em múltiplas etapas é um candidato a extração para uma Crew. É aqui que você verá a maior melhoria de qualidade.
---
## Primeiros Passos
Instale o CrewAI e crie o scaffold de um novo projeto Flow:
```bash
pip install crewai
crewai create flow my_first_flow
cd my_first_flow
```
Isso gera uma estrutura de projeto com uma classe Flow pronta para edição, arquivos de configuração e um `pyproject.toml` com `type = "flow"` já definido. Execute com:
```bash
crewai run
```
A partir daí, adicione seus agentes, conecte seus listeners e publique.
---
## Considerações Finais
O LangGraph ensinou ao ecossistema que workflows de IA precisam de estrutura. Essa foi uma lição importante. Mas o CrewAI Flows pega essa lição e a entrega de um jeito mais rápido de escrever, mais fácil de ler e mais poderoso em produção — especialmente quando seus workflows envolvem múltiplos agentes colaborando.
Se você está construindo algo além de uma cadeia de agente único, dê uma olhada séria no Flows. O modelo baseado em decorators, a integração nativa com Crews e o gerenciamento de estado embutido significam menos tempo com encanamento e mais tempo nos problemas que importam.
Comece com `crewai create flow`. Você não vai olhar para trás.

View File

@@ -9,7 +9,7 @@ authors = [
requires-python = ">=3.10, <3.14" requires-python = ">=3.10, <3.14"
dependencies = [ dependencies = [
"Pillow~=12.1.1", "Pillow~=12.1.1",
"pypdf~=6.7.4", "pypdf~=6.7.5",
"python-magic>=0.4.27", "python-magic>=0.4.27",
"aiocache~=0.12.3", "aiocache~=0.12.3",
"aiofiles~=24.1.0", "aiofiles~=24.1.0",

View File

@@ -152,4 +152,4 @@ __all__ = [
"wrap_file_source", "wrap_file_source",
] ]
__version__ = "1.10.1a1" __version__ = "1.10.2rc2"

View File

@@ -11,7 +11,7 @@ dependencies = [
"pytube~=15.0.0", "pytube~=15.0.0",
"requests~=2.32.5", "requests~=2.32.5",
"docker~=7.1.0", "docker~=7.1.0",
"crewai==1.10.1a1", "crewai==1.10.2rc2",
"tiktoken~=0.8.0", "tiktoken~=0.8.0",
"beautifulsoup4~=4.13.4", "beautifulsoup4~=4.13.4",
"python-docx~=1.2.0", "python-docx~=1.2.0",
@@ -108,7 +108,7 @@ stagehand = [
"stagehand>=0.4.1", "stagehand>=0.4.1",
] ]
github = [ github = [
"gitpython==3.1.38", "gitpython>=3.1.41,<4",
"PyGithub==1.59.1", "PyGithub==1.59.1",
] ]
rag = [ rag = [

View File

@@ -10,7 +10,18 @@ from crewai_tools.aws.s3.writer_tool import S3WriterTool
from crewai_tools.tools.ai_mind_tool.ai_mind_tool import AIMindTool from crewai_tools.tools.ai_mind_tool.ai_mind_tool import AIMindTool
from crewai_tools.tools.apify_actors_tool.apify_actors_tool import ApifyActorsTool from crewai_tools.tools.apify_actors_tool.apify_actors_tool import ApifyActorsTool
from crewai_tools.tools.arxiv_paper_tool.arxiv_paper_tool import ArxivPaperTool from crewai_tools.tools.arxiv_paper_tool.arxiv_paper_tool import ArxivPaperTool
from crewai_tools.tools.brave_search_tool.brave_image_tool import BraveImageSearchTool
from crewai_tools.tools.brave_search_tool.brave_llm_context_tool import (
BraveLLMContextTool,
)
from crewai_tools.tools.brave_search_tool.brave_local_pois_tool import (
BraveLocalPOIsDescriptionTool,
BraveLocalPOIsTool,
)
from crewai_tools.tools.brave_search_tool.brave_news_tool import BraveNewsSearchTool
from crewai_tools.tools.brave_search_tool.brave_search_tool import BraveSearchTool from crewai_tools.tools.brave_search_tool.brave_search_tool import BraveSearchTool
from crewai_tools.tools.brave_search_tool.brave_video_tool import BraveVideoSearchTool
from crewai_tools.tools.brave_search_tool.brave_web_tool import BraveWebSearchTool
from crewai_tools.tools.brightdata_tool.brightdata_dataset import ( from crewai_tools.tools.brightdata_tool.brightdata_dataset import (
BrightDataDatasetTool, BrightDataDatasetTool,
) )
@@ -200,7 +211,14 @@ __all__ = [
"ArxivPaperTool", "ArxivPaperTool",
"BedrockInvokeAgentTool", "BedrockInvokeAgentTool",
"BedrockKBRetrieverTool", "BedrockKBRetrieverTool",
"BraveImageSearchTool",
"BraveLLMContextTool",
"BraveLocalPOIsDescriptionTool",
"BraveLocalPOIsTool",
"BraveNewsSearchTool",
"BraveSearchTool", "BraveSearchTool",
"BraveVideoSearchTool",
"BraveWebSearchTool",
"BrightDataDatasetTool", "BrightDataDatasetTool",
"BrightDataSearchTool", "BrightDataSearchTool",
"BrightDataWebUnlockerTool", "BrightDataWebUnlockerTool",
@@ -291,4 +309,4 @@ __all__ = [
"ZapierActionTools", "ZapierActionTools",
] ]
__version__ = "1.10.1a1" __version__ = "1.10.2rc2"

View File

@@ -1,7 +1,9 @@
from collections.abc import Callable from collections.abc import Callable
import os
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
from crewai.utilities.lock_store import lock as store_lock
from lancedb import ( # type: ignore[import-untyped] from lancedb import ( # type: ignore[import-untyped]
DBConnection as LanceDBConnection, DBConnection as LanceDBConnection,
connect as lancedb_connect, connect as lancedb_connect,
@@ -33,10 +35,12 @@ class LanceDBAdapter(Adapter):
_db: LanceDBConnection = PrivateAttr() _db: LanceDBConnection = PrivateAttr()
_table: LanceDBTable = PrivateAttr() _table: LanceDBTable = PrivateAttr()
_lock_name: str = PrivateAttr(default="")
def model_post_init(self, __context: Any) -> None: def model_post_init(self, __context: Any) -> None:
self._db = lancedb_connect(self.uri) self._db = lancedb_connect(self.uri)
self._table = self._db.open_table(self.table_name) self._table = self._db.open_table(self.table_name)
self._lock_name = f"lancedb:{os.path.realpath(str(self.uri))}"
super().model_post_init(__context) super().model_post_init(__context)
@@ -56,4 +60,5 @@ class LanceDBAdapter(Adapter):
*args: Any, *args: Any,
**kwargs: Any, **kwargs: Any,
) -> None: ) -> None:
self._table.add(*args, **kwargs) with store_lock(self._lock_name):
self._table.add(*args, **kwargs)

View File

@@ -1,6 +1,9 @@
from __future__ import annotations from __future__ import annotations
import asyncio
import contextvars
import logging import logging
import threading
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
@@ -18,6 +21,9 @@ class BrowserSessionManager:
This class maintains separate browser sessions for different threads, This class maintains separate browser sessions for different threads,
enabling concurrent usage of browsers in multi-threaded environments. enabling concurrent usage of browsers in multi-threaded environments.
Browsers are created lazily only when needed by tools. Browsers are created lazily only when needed by tools.
Uses per-key events to serialize creation for the same thread_id without
blocking unrelated callers or wasting resources on duplicate sessions.
""" """
def __init__(self, region: str = "us-west-2"): def __init__(self, region: str = "us-west-2"):
@@ -27,8 +33,10 @@ class BrowserSessionManager:
region: AWS region for browser client region: AWS region for browser client
""" """
self.region = region self.region = region
self._lock = threading.Lock()
self._async_sessions: dict[str, tuple[BrowserClient, AsyncBrowser]] = {} self._async_sessions: dict[str, tuple[BrowserClient, AsyncBrowser]] = {}
self._sync_sessions: dict[str, tuple[BrowserClient, SyncBrowser]] = {} self._sync_sessions: dict[str, tuple[BrowserClient, SyncBrowser]] = {}
self._creating: dict[str, threading.Event] = {}
async def get_async_browser(self, thread_id: str) -> AsyncBrowser: async def get_async_browser(self, thread_id: str) -> AsyncBrowser:
"""Get or create an async browser for the specified thread. """Get or create an async browser for the specified thread.
@@ -39,10 +47,29 @@ class BrowserSessionManager:
Returns: Returns:
An async browser instance specific to the thread An async browser instance specific to the thread
""" """
if thread_id in self._async_sessions: loop = asyncio.get_event_loop()
return self._async_sessions[thread_id][1] while True:
with self._lock:
if thread_id in self._async_sessions:
return self._async_sessions[thread_id][1]
if thread_id not in self._creating:
self._creating[thread_id] = threading.Event()
break
event = self._creating[thread_id]
ctx = contextvars.copy_context()
await loop.run_in_executor(None, ctx.run, event.wait)
return await self._create_async_browser_session(thread_id) try:
browser_client, browser = await self._create_async_browser_session(
thread_id
)
with self._lock:
self._async_sessions[thread_id] = (browser_client, browser)
return browser
finally:
with self._lock:
evt = self._creating.pop(thread_id)
evt.set()
def get_sync_browser(self, thread_id: str) -> SyncBrowser: def get_sync_browser(self, thread_id: str) -> SyncBrowser:
"""Get or create a sync browser for the specified thread. """Get or create a sync browser for the specified thread.
@@ -53,19 +80,33 @@ class BrowserSessionManager:
Returns: Returns:
A sync browser instance specific to the thread A sync browser instance specific to the thread
""" """
if thread_id in self._sync_sessions: while True:
return self._sync_sessions[thread_id][1] with self._lock:
if thread_id in self._sync_sessions:
return self._sync_sessions[thread_id][1]
if thread_id not in self._creating:
self._creating[thread_id] = threading.Event()
break
event = self._creating[thread_id]
event.wait()
return self._create_sync_browser_session(thread_id) try:
return self._create_sync_browser_session(thread_id)
finally:
with self._lock:
evt = self._creating.pop(thread_id)
evt.set()
async def _create_async_browser_session(self, thread_id: str) -> AsyncBrowser: async def _create_async_browser_session(
self, thread_id: str
) -> tuple[BrowserClient, AsyncBrowser]:
"""Create a new async browser session for the specified thread. """Create a new async browser session for the specified thread.
Args: Args:
thread_id: Unique identifier for the thread thread_id: Unique identifier for the thread
Returns: Returns:
The newly created async browser instance Tuple of (BrowserClient, AsyncBrowser).
Raises: Raises:
Exception: If browser session creation fails Exception: If browser session creation fails
@@ -75,10 +116,8 @@ class BrowserSessionManager:
browser_client = BrowserClient(region=self.region) browser_client = BrowserClient(region=self.region)
try: try:
# Start browser session
browser_client.start() browser_client.start()
# Get WebSocket connection info
ws_url, headers = browser_client.generate_ws_headers() ws_url, headers = browser_client.generate_ws_headers()
logger.info( logger.info(
@@ -87,7 +126,6 @@ class BrowserSessionManager:
from playwright.async_api import async_playwright from playwright.async_api import async_playwright
# Connect to browser using Playwright
playwright = await async_playwright().start() playwright = await async_playwright().start()
browser = await playwright.chromium.connect_over_cdp( browser = await playwright.chromium.connect_over_cdp(
endpoint_url=ws_url, headers=headers, timeout=30000 endpoint_url=ws_url, headers=headers, timeout=30000
@@ -96,17 +134,13 @@ class BrowserSessionManager:
f"Successfully connected to async browser for thread {thread_id}" f"Successfully connected to async browser for thread {thread_id}"
) )
# Store session resources return browser_client, browser
self._async_sessions[thread_id] = (browser_client, browser)
return browser
except Exception as e: except Exception as e:
logger.error( logger.error(
f"Failed to create async browser session for thread {thread_id}: {e}" f"Failed to create async browser session for thread {thread_id}: {e}"
) )
# Clean up resources if session creation fails
if browser_client: if browser_client:
try: try:
browser_client.stop() browser_client.stop()
@@ -132,10 +166,8 @@ class BrowserSessionManager:
browser_client = BrowserClient(region=self.region) browser_client = BrowserClient(region=self.region)
try: try:
# Start browser session
browser_client.start() browser_client.start()
# Get WebSocket connection info
ws_url, headers = browser_client.generate_ws_headers() ws_url, headers = browser_client.generate_ws_headers()
logger.info( logger.info(
@@ -144,7 +176,6 @@ class BrowserSessionManager:
from playwright.sync_api import sync_playwright from playwright.sync_api import sync_playwright
# Connect to browser using Playwright
playwright = sync_playwright().start() playwright = sync_playwright().start()
browser = playwright.chromium.connect_over_cdp( browser = playwright.chromium.connect_over_cdp(
endpoint_url=ws_url, headers=headers, timeout=30000 endpoint_url=ws_url, headers=headers, timeout=30000
@@ -153,8 +184,8 @@ class BrowserSessionManager:
f"Successfully connected to sync browser for thread {thread_id}" f"Successfully connected to sync browser for thread {thread_id}"
) )
# Store session resources with self._lock:
self._sync_sessions[thread_id] = (browser_client, browser) self._sync_sessions[thread_id] = (browser_client, browser)
return browser return browser
@@ -163,7 +194,6 @@ class BrowserSessionManager:
f"Failed to create sync browser session for thread {thread_id}: {e}" f"Failed to create sync browser session for thread {thread_id}: {e}"
) )
# Clean up resources if session creation fails
if browser_client: if browser_client:
try: try:
browser_client.stop() browser_client.stop()
@@ -178,13 +208,13 @@ class BrowserSessionManager:
Args: Args:
thread_id: Unique identifier for the thread thread_id: Unique identifier for the thread
""" """
if thread_id not in self._async_sessions: with self._lock:
logger.warning(f"No async browser session found for thread {thread_id}") if thread_id not in self._async_sessions:
return logger.warning(f"No async browser session found for thread {thread_id}")
return
browser_client, browser = self._async_sessions[thread_id] browser_client, browser = self._async_sessions.pop(thread_id)
# Close browser
if browser: if browser:
try: try:
await browser.close() await browser.close()
@@ -193,7 +223,6 @@ class BrowserSessionManager:
f"Error closing async browser for thread {thread_id}: {e}" f"Error closing async browser for thread {thread_id}: {e}"
) )
# Stop browser client
if browser_client: if browser_client:
try: try:
browser_client.stop() browser_client.stop()
@@ -202,8 +231,6 @@ class BrowserSessionManager:
f"Error stopping browser client for thread {thread_id}: {e}" f"Error stopping browser client for thread {thread_id}: {e}"
) )
# Remove session from dictionary
del self._async_sessions[thread_id]
logger.info(f"Async browser session cleaned up for thread {thread_id}") logger.info(f"Async browser session cleaned up for thread {thread_id}")
def close_sync_browser(self, thread_id: str) -> None: def close_sync_browser(self, thread_id: str) -> None:
@@ -212,13 +239,13 @@ class BrowserSessionManager:
Args: Args:
thread_id: Unique identifier for the thread thread_id: Unique identifier for the thread
""" """
if thread_id not in self._sync_sessions: with self._lock:
logger.warning(f"No sync browser session found for thread {thread_id}") if thread_id not in self._sync_sessions:
return logger.warning(f"No sync browser session found for thread {thread_id}")
return
browser_client, browser = self._sync_sessions[thread_id] browser_client, browser = self._sync_sessions.pop(thread_id)
# Close browser
if browser: if browser:
try: try:
browser.close() browser.close()
@@ -227,7 +254,6 @@ class BrowserSessionManager:
f"Error closing sync browser for thread {thread_id}: {e}" f"Error closing sync browser for thread {thread_id}: {e}"
) )
# Stop browser client
if browser_client: if browser_client:
try: try:
browser_client.stop() browser_client.stop()
@@ -236,19 +262,17 @@ class BrowserSessionManager:
f"Error stopping browser client for thread {thread_id}: {e}" f"Error stopping browser client for thread {thread_id}: {e}"
) )
# Remove session from dictionary
del self._sync_sessions[thread_id]
logger.info(f"Sync browser session cleaned up for thread {thread_id}") logger.info(f"Sync browser session cleaned up for thread {thread_id}")
async def close_all_browsers(self) -> None: async def close_all_browsers(self) -> None:
"""Close all browser sessions.""" """Close all browser sessions."""
# Close all async browsers with self._lock:
async_thread_ids = list(self._async_sessions.keys()) async_thread_ids = list(self._async_sessions.keys())
sync_thread_ids = list(self._sync_sessions.keys())
for thread_id in async_thread_ids: for thread_id in async_thread_ids:
await self.close_async_browser(thread_id) await self.close_async_browser(thread_id)
# Close all sync browsers
sync_thread_ids = list(self._sync_sessions.keys())
for thread_id in sync_thread_ids: for thread_id in sync_thread_ids:
self.close_sync_browser(thread_id) self.close_sync_browser(thread_id)

View File

@@ -1,9 +1,11 @@
import logging import logging
import os
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
from uuid import uuid4 from uuid import uuid4
import chromadb import chromadb
from crewai.utilities.lock_store import lock as store_lock
from pydantic import BaseModel, Field, PrivateAttr from pydantic import BaseModel, Field, PrivateAttr
from crewai_tools.rag.base_loader import BaseLoader from crewai_tools.rag.base_loader import BaseLoader
@@ -38,22 +40,32 @@ class RAG(Adapter):
_client: Any = PrivateAttr() _client: Any = PrivateAttr()
_collection: Any = PrivateAttr() _collection: Any = PrivateAttr()
_embedding_service: EmbeddingService = PrivateAttr() _embedding_service: EmbeddingService = PrivateAttr()
_lock_name: str = PrivateAttr(default="")
def model_post_init(self, __context: Any) -> None: def model_post_init(self, __context: Any) -> None:
try: try:
if self.persist_directory: self._lock_name = (
self._client = chromadb.PersistentClient(path=self.persist_directory) f"chromadb:{os.path.realpath(self.persist_directory)}"
else: if self.persist_directory
self._client = chromadb.Client() else "chromadb:ephemeral"
self._collection = self._client.get_or_create_collection(
name=self.collection_name,
metadata={
"hnsw:space": "cosine",
"description": "CrewAI Knowledge Base",
},
) )
with store_lock(self._lock_name):
if self.persist_directory:
self._client = chromadb.PersistentClient(
path=self.persist_directory
)
else:
self._client = chromadb.Client()
self._collection = self._client.get_or_create_collection(
name=self.collection_name,
metadata={
"hnsw:space": "cosine",
"description": "CrewAI Knowledge Base",
},
)
self._embedding_service = EmbeddingService( self._embedding_service = EmbeddingService(
provider=self.embedding_provider, provider=self.embedding_provider,
model=self.embedding_model, model=self.embedding_model,
@@ -87,29 +99,8 @@ class RAG(Adapter):
loader_result = loader.load(source_content) loader_result = loader.load(source_content)
doc_id = loader_result.doc_id doc_id = loader_result.doc_id
existing_doc = self._collection.get(
where={"source": source_content.source_ref}, limit=1
)
existing_doc_id = (
existing_doc and existing_doc["metadatas"][0]["doc_id"]
if existing_doc["metadatas"]
else None
)
if existing_doc_id == doc_id:
logger.warning(
f"Document with source {loader_result.source} already exists"
)
return
# Document with same source ref does exists but the content has changed, deleting the oldest reference
if existing_doc_id and existing_doc_id != loader_result.doc_id:
logger.warning(f"Deleting old document with doc_id {existing_doc_id}")
self._collection.delete(where={"doc_id": existing_doc_id})
documents = []
chunks = chunker.chunk(loader_result.content) chunks = chunker.chunk(loader_result.content)
documents = []
for i, chunk in enumerate(chunks): for i, chunk in enumerate(chunks):
doc_metadata = (metadata or {}).copy() doc_metadata = (metadata or {}).copy()
doc_metadata["chunk_index"] = i doc_metadata["chunk_index"] = i
@@ -136,7 +127,6 @@ class RAG(Adapter):
ids = [doc.id for doc in documents] ids = [doc.id for doc in documents]
metadatas = [] metadatas = []
for doc in documents: for doc in documents:
doc_metadata = doc.metadata.copy() doc_metadata = doc.metadata.copy()
doc_metadata.update( doc_metadata.update(
@@ -148,16 +138,36 @@ class RAG(Adapter):
) )
metadatas.append(doc_metadata) metadatas.append(doc_metadata)
try: with store_lock(self._lock_name):
self._collection.add( existing_doc = self._collection.get(
ids=ids, where={"source": source_content.source_ref}, limit=1
embeddings=embeddings,
documents=contents,
metadatas=metadatas,
) )
logger.info(f"Added {len(documents)} documents to knowledge base") existing_doc_id = (
except Exception as e: existing_doc and existing_doc["metadatas"][0]["doc_id"]
logger.error(f"Failed to add documents to ChromaDB: {e}") if existing_doc["metadatas"]
else None
)
if existing_doc_id == doc_id:
logger.warning(
f"Document with source {loader_result.source} already exists"
)
return
if existing_doc_id and existing_doc_id != loader_result.doc_id:
logger.warning(f"Deleting old document with doc_id {existing_doc_id}")
self._collection.delete(where={"doc_id": existing_doc_id})
try:
self._collection.add(
ids=ids,
embeddings=embeddings,
documents=contents,
metadatas=metadatas,
)
logger.info(f"Added {len(documents)} documents to knowledge base")
except Exception as e:
logger.error(f"Failed to add documents to ChromaDB: {e}")
def query(self, question: str, where: dict[str, Any] | None = None) -> str: # type: ignore def query(self, question: str, where: dict[str, Any] | None = None) -> str: # type: ignore
try: try:
@@ -201,7 +211,8 @@ class RAG(Adapter):
def delete_collection(self) -> None: def delete_collection(self) -> None:
try: try:
self._client.delete_collection(self.collection_name) with store_lock(self._lock_name):
self._client.delete_collection(self.collection_name)
logger.info(f"Deleted collection: {self.collection_name}") logger.info(f"Deleted collection: {self.collection_name}")
except Exception as e: except Exception as e:
logger.error(f"Failed to delete collection: {e}") logger.error(f"Failed to delete collection: {e}")

View File

@@ -1,7 +1,18 @@
from crewai_tools.tools.ai_mind_tool.ai_mind_tool import AIMindTool from crewai_tools.tools.ai_mind_tool.ai_mind_tool import AIMindTool
from crewai_tools.tools.apify_actors_tool.apify_actors_tool import ApifyActorsTool from crewai_tools.tools.apify_actors_tool.apify_actors_tool import ApifyActorsTool
from crewai_tools.tools.arxiv_paper_tool.arxiv_paper_tool import ArxivPaperTool from crewai_tools.tools.arxiv_paper_tool.arxiv_paper_tool import ArxivPaperTool
from crewai_tools.tools.brave_search_tool.brave_image_tool import BraveImageSearchTool
from crewai_tools.tools.brave_search_tool.brave_llm_context_tool import (
BraveLLMContextTool,
)
from crewai_tools.tools.brave_search_tool.brave_local_pois_tool import (
BraveLocalPOIsDescriptionTool,
BraveLocalPOIsTool,
)
from crewai_tools.tools.brave_search_tool.brave_news_tool import BraveNewsSearchTool
from crewai_tools.tools.brave_search_tool.brave_search_tool import BraveSearchTool from crewai_tools.tools.brave_search_tool.brave_search_tool import BraveSearchTool
from crewai_tools.tools.brave_search_tool.brave_video_tool import BraveVideoSearchTool
from crewai_tools.tools.brave_search_tool.brave_web_tool import BraveWebSearchTool
from crewai_tools.tools.brightdata_tool import ( from crewai_tools.tools.brightdata_tool import (
BrightDataDatasetTool, BrightDataDatasetTool,
BrightDataSearchTool, BrightDataSearchTool,
@@ -185,7 +196,14 @@ __all__ = [
"AIMindTool", "AIMindTool",
"ApifyActorsTool", "ApifyActorsTool",
"ArxivPaperTool", "ArxivPaperTool",
"BraveImageSearchTool",
"BraveLLMContextTool",
"BraveLocalPOIsDescriptionTool",
"BraveLocalPOIsTool",
"BraveNewsSearchTool",
"BraveSearchTool", "BraveSearchTool",
"BraveVideoSearchTool",
"BraveWebSearchTool",
"BrightDataDatasetTool", "BrightDataDatasetTool",
"BrightDataSearchTool", "BrightDataSearchTool",
"BrightDataWebUnlockerTool", "BrightDataWebUnlockerTool",

View File

@@ -0,0 +1,322 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from datetime import datetime
import json
import logging
import os
import threading
import time
from typing import Any, ClassVar
from crewai.tools import BaseTool, EnvVar
from pydantic import BaseModel, Field
import requests
logger = logging.getLogger(__name__)
# Brave API error codes that indicate non-retryable quota/usage exhaustion.
_QUOTA_CODES = frozenset({"QUOTA_LIMITED", "USAGE_LIMIT_EXCEEDED"})
def _save_results_to_file(content: str) -> None:
"""Saves the search results to a file."""
filename = f"search_results_{datetime.now().strftime('%Y-%m-%d_%H-%M-%S')}.txt"
with open(filename, "w") as file:
file.write(content)
def _parse_error_body(resp: requests.Response) -> dict[str, Any] | None:
"""Extract the structured "error" object from a Brave API error response."""
try:
body = resp.json()
error = body.get("error")
return error if isinstance(error, dict) else None
except (ValueError, KeyError):
return None
def _raise_for_error(resp: requests.Response) -> None:
"""Brave Search API error responses contain helpful JSON payloads"""
status = resp.status_code
try:
body = json.dumps(resp.json())
except (ValueError, KeyError):
body = resp.text[:500]
raise RuntimeError(f"Brave Search API error (HTTP {status}): {body}")
def _is_retryable(resp: requests.Response) -> bool:
"""Return True for transient failures that are worth retrying.
* 429 + RATE_LIMITED — the per-second sliding window is full.
* 5xx — transient server-side errors.
Quota exhaustion (QUOTA_LIMITED, USAGE_LIMIT_EXCEEDED) is
explicitly excluded: retrying will never succeed until the billing
period resets.
"""
if resp.status_code == 429:
error = _parse_error_body(resp) or {}
return error.get("code") not in _QUOTA_CODES
return 500 <= resp.status_code < 600
def _retry_delay(resp: requests.Response, attempt: int) -> float:
"""Compute wait time before the next retry attempt.
Prefers the server-supplied Retry-After header when available;
falls back to exponential backoff (1s, 2s, 4s, ...).
"""
retry_after = resp.headers.get("Retry-After")
if retry_after is not None:
try:
return max(0.0, float(retry_after))
except (ValueError, TypeError):
pass
return float(2**attempt)
class BraveSearchToolBase(BaseTool, ABC):
"""
Base class for Brave Search API interactions.
Individual tool subclasses must provide the following:
- search_url
- header_schema (pydantic model)
- args_schema (pydantic model)
- _refine_payload() -> dict[str, Any]
"""
search_url: str
raw: bool = False
args_schema: type[BaseModel]
header_schema: type[BaseModel]
# Tool options (legacy parameters)
country: str | None = None
save_file: bool = False
n_results: int = 10
env_vars: list[EnvVar] = Field(
default_factory=lambda: [
EnvVar(
name="BRAVE_API_KEY",
description="API key for Brave Search",
required=True,
),
]
)
def __init__(
self,
*,
api_key: str | None = None,
headers: dict[str, Any] | None = None,
requests_per_second: float = 1.0,
save_file: bool = False,
raw: bool = False,
timeout: int = 30,
**kwargs: Any,
):
super().__init__(**kwargs)
self._api_key = api_key or os.environ.get("BRAVE_API_KEY")
if not self._api_key:
raise ValueError("BRAVE_API_KEY environment variable is required")
self.raw = bool(raw)
self._timeout = int(timeout)
self.save_file = bool(save_file)
self._requests_per_second = float(requests_per_second)
self._headers = self._build_and_validate_headers(headers or {})
# Per-instance rate limiting: each instance has its own clock and lock.
# Total process rate is the sum of limits of instances you create.
self._last_request_time: float = 0
self._rate_limit_lock = threading.Lock()
@property
def api_key(self) -> str:
return self._api_key
@property
def headers(self) -> dict[str, Any]:
return self._headers
def set_headers(self, headers: dict[str, Any]) -> BraveSearchToolBase:
merged = {**self._headers, **{k.lower(): v for k, v in headers.items()}}
self._headers = self._build_and_validate_headers(merged)
return self
def _build_and_validate_headers(self, headers: dict[str, Any]) -> dict[str, Any]:
normalized = {k.lower(): v for k, v in headers.items()}
normalized.setdefault("x-subscription-token", self._api_key)
normalized.setdefault("accept", "application/json")
try:
self.header_schema(**normalized)
except Exception as e:
raise ValueError(f"Invalid headers: {e}") from e
return normalized
def _rate_limit(self) -> None:
"""Enforce minimum interval between requests for this instance. Thread-safe."""
if self._requests_per_second <= 0:
return
min_interval = 1.0 / self._requests_per_second
with self._rate_limit_lock:
now = time.time()
next_allowed = self._last_request_time + min_interval
if now < next_allowed:
time.sleep(next_allowed - now)
now = time.time()
self._last_request_time = now
def _make_request(
self, params: dict[str, Any], *, _max_retries: int = 3
) -> dict[str, Any]:
"""Execute an HTTP GET against the Brave Search API with retry logic."""
last_resp: requests.Response | None = None
# Retry the request up to _max_retries times
for attempt in range(_max_retries):
self._rate_limit()
# Make the request
try:
resp = requests.get(
self.search_url,
headers=self._headers,
params=params,
timeout=self._timeout,
)
except requests.ConnectionError as exc:
raise RuntimeError(
f"Brave Search API connection failed: {exc}"
) from exc
except requests.Timeout as exc:
raise RuntimeError(
f"Brave Search API request timed out after {self._timeout}s: {exc}"
) from exc
# Log the rate limit headers and request details
logger.debug(
"Brave Search API request: %s %s -> %d",
"GET",
resp.url,
resp.status_code,
)
# Response was OK, return the JSON body
if resp.ok:
try:
return resp.json()
except ValueError as exc:
raise RuntimeError(
f"Brave Search API returned invalid JSON (HTTP {resp.status_code}): {exc}"
) from exc
# Response was not OK, but is retryable
# (e.g., 429 Too Many Requests, 500 Internal Server Error)
if _is_retryable(resp) and attempt < _max_retries - 1:
delay = _retry_delay(resp, attempt)
logger.warning(
"Brave Search API returned %d. Retrying in %.1fs (attempt %d/%d)",
resp.status_code,
delay,
attempt + 1,
_max_retries,
)
time.sleep(delay)
last_resp = resp
continue
# Response was not OK, nor was it retryable
# (e.g., 422 Unprocessable Entity, 400 Bad Request (OPTION_NOT_IN_PLAN))
_raise_for_error(resp)
# All retries exhausted
_raise_for_error(last_resp or resp) # type: ignore[possibly-undefined]
return {} # unreachable (here to satisfy the type checker and linter)
def _run(self, q: str | None = None, **params: Any) -> Any:
# Allow positional usage: tool.run("latest Brave browser features")
if q is not None:
params["q"] = q
params = self._common_payload_refinement(params)
# Validate only schema fields
schema_keys = self.args_schema.model_fields
payload_in = {k: v for k, v in params.items() if k in schema_keys}
try:
validated = self.args_schema(**payload_in)
except Exception as e:
raise ValueError(f"Invalid parameters: {e}") from e
# The subclass may have additional refinements to apply to the payload, such as goggles or other parameters
payload = self._refine_request_payload(validated.model_dump(exclude_none=True))
response = self._make_request(payload)
if not self.raw:
response = self._refine_response(response)
if self.save_file:
_save_results_to_file(json.dumps(response, indent=2))
return response
@abstractmethod
def _refine_request_payload(self, params: dict[str, Any]) -> dict[str, Any]:
"""Subclass must implement: transform validated params dict into API request params."""
raise NotImplementedError
@abstractmethod
def _refine_response(self, response: dict[str, Any]) -> Any:
"""Subclass must implement: transform response dict into a more useful format."""
raise NotImplementedError
_EMPTY_VALUES: ClassVar[tuple[None, str, str, list[Any]]] = (None, "", "null", [])
def _common_payload_refinement(self, params: dict[str, Any]) -> dict[str, Any]:
"""Common payload refinement for all tools."""
# crewAI's schema pipeline (ensure_all_properties_required in
# pydantic_schema_utils.py) marks every property as required so
# that OpenAI strict-mode structured outputs work correctly.
# The side-effect is that the LLM fills in *every* parameter —
# even truly optional ones — using placeholder values such as
# None, "", "null", or []. Only optional fields are affected,
# so we limit the check to those.
fields = self.args_schema.model_fields
params = {
k: v
for k, v in params.items()
# Permit custom and required fields, and fields with non-empty values
if k not in fields or fields[k].is_required() or v not in self._EMPTY_VALUES
}
# Make sure params has "q" for query instead of "query" or "search_query"
query = params.get("query") or params.get("search_query")
if query is not None and "q" not in params:
params["q"] = query
params.pop("query", None)
params.pop("search_query", None)
# If "count" was not explicitly provided, use n_results
# (only when the schema actually supports a "count" field)
if "count" in self.args_schema.model_fields:
if "count" not in params and self.n_results is not None:
params["count"] = self.n_results
# If "country" was not explicitly provided, but self.country is set, use it
# (only when the schema actually supports a "country" field)
if "country" in self.args_schema.model_fields:
if "country" not in params and self.country is not None:
params["country"] = self.country
return params

View File

@@ -0,0 +1,42 @@
from typing import Any
from pydantic import BaseModel
from crewai_tools.tools.brave_search_tool.base import BraveSearchToolBase
from crewai_tools.tools.brave_search_tool.schemas import (
ImageSearchHeaders,
ImageSearchParams,
)
class BraveImageSearchTool(BraveSearchToolBase):
"""A tool that performs image searches using the Brave Search API."""
name: str = "Brave Image Search"
args_schema: type[BaseModel] = ImageSearchParams
header_schema: type[BaseModel] = ImageSearchHeaders
description: str = (
"A tool that performs image searches using the Brave Search API. "
"Results are returned as structured JSON data."
)
search_url: str = "https://api.search.brave.com/res/v1/images/search"
def _refine_request_payload(self, params: dict[str, Any]) -> dict[str, Any]:
return params
def _refine_response(self, response: dict[str, Any]) -> list[dict[str, Any]]:
# Make the response more concise, and easier to consume
results = response.get("results", [])
return [
{
"title": result.get("title"),
"url": result.get("properties", {}).get("url"),
"dimensions": f"{w}x{h}"
if (w := result.get("properties", {}).get("width"))
and (h := result.get("properties", {}).get("height"))
else None,
}
for result in results
]

View File

@@ -0,0 +1,32 @@
from typing import Any
from pydantic import BaseModel
from crewai_tools.tools.brave_search_tool.base import BraveSearchToolBase
from crewai_tools.tools.brave_search_tool.response_types import LLMContext
from crewai_tools.tools.brave_search_tool.schemas import (
LLMContextHeaders,
LLMContextParams,
)
class BraveLLMContextTool(BraveSearchToolBase):
"""A tool that retrieves context for LLM usage from the Brave Search API."""
name: str = "Brave LLM Context"
args_schema: type[BaseModel] = LLMContextParams
header_schema: type[BaseModel] = LLMContextHeaders
description: str = (
"A tool that retrieves context for LLM usage from the Brave Search API. "
"Results are returned as structured JSON data."
)
search_url: str = "https://api.search.brave.com/res/v1/llm/context"
def _refine_request_payload(self, params: dict[str, Any]) -> dict[str, Any]:
return params
def _refine_response(self, response: LLMContext.Response) -> LLMContext.Response:
"""The LLM Context response schema is fairly simple. Return as is."""
return response

View File

@@ -0,0 +1,109 @@
from typing import Any
from pydantic import BaseModel
from crewai_tools.tools.brave_search_tool.base import BraveSearchToolBase
from crewai_tools.tools.brave_search_tool.response_types import LocalPOIs
from crewai_tools.tools.brave_search_tool.schemas import (
LocalPOIsDescriptionHeaders,
LocalPOIsDescriptionParams,
LocalPOIsHeaders,
LocalPOIsParams,
)
DayOpeningHours = LocalPOIs.DayOpeningHours
OpeningHours = LocalPOIs.OpeningHours
LocationResult = LocalPOIs.LocationResult
LocalPOIsResponse = LocalPOIs.Response
def _flatten_slots(slots: list[DayOpeningHours]) -> list[dict[str, str]]:
"""Convert a list of DayOpeningHours dicts into simplified entries."""
return [
{
"day": slot["full_name"].lower(),
"opens": slot["opens"],
"closes": slot["closes"],
}
for slot in slots
]
def _simplify_opening_hours(result: LocationResult) -> list[dict[str, str]] | None:
"""Collapse opening_hours into a flat list of {day, opens, closes} dicts."""
hours = result.get("opening_hours")
if not hours:
return None
entries: list[dict[str, str]] = []
current = hours.get("current_day")
if current:
entries.extend(_flatten_slots(current))
days = hours.get("days")
if days:
for day_slots in days:
entries.extend(_flatten_slots(day_slots))
return entries or None
class BraveLocalPOIsTool(BraveSearchToolBase):
"""A tool that retrieves local POIs using the Brave Search API."""
name: str = "Brave Local POIs"
args_schema: type[BaseModel] = LocalPOIsParams
header_schema: type[BaseModel] = LocalPOIsHeaders
description: str = (
"A tool that retrieves local POIs using the Brave Search API. "
"Results are returned as structured JSON data."
)
search_url: str = "https://api.search.brave.com/res/v1/local/pois"
def _refine_request_payload(self, params: dict[str, Any]) -> dict[str, Any]:
return params
def _refine_response(self, response: LocalPOIsResponse) -> list[dict[str, Any]]:
results = response.get("results", [])
return [
{
"title": result.get("title"),
"url": result.get("url"),
"description": result.get("description"),
"address": result.get("postal_address", {}).get("displayAddress"),
"contact": result.get("contact", {}).get("telephone")
or result.get("contact", {}).get("email")
or None,
"opening_hours": _simplify_opening_hours(result),
}
for result in results
]
class BraveLocalPOIsDescriptionTool(BraveSearchToolBase):
"""A tool that retrieves AI-generated descriptions for local POIs using the Brave Search API."""
name: str = "Brave Local POI Descriptions"
args_schema: type[BaseModel] = LocalPOIsDescriptionParams
header_schema: type[BaseModel] = LocalPOIsDescriptionHeaders
description: str = (
"A tool that retrieves AI-generated descriptions for local POIs using the Brave Search API. "
"Results are returned as structured JSON data."
)
search_url: str = "https://api.search.brave.com/res/v1/local/descriptions"
def _refine_request_payload(self, params: dict[str, Any]) -> dict[str, Any]:
return params
def _refine_response(self, response: LocalPOIsResponse) -> list[dict[str, Any]]:
# Make the response more concise, and easier to consume
results = response.get("results", [])
return [
{
"id": result.get("id"),
"description": result.get("description"),
}
for result in results
]

View File

@@ -0,0 +1,39 @@
from typing import Any
from pydantic import BaseModel
from crewai_tools.tools.brave_search_tool.base import BraveSearchToolBase
from crewai_tools.tools.brave_search_tool.schemas import (
NewsSearchHeaders,
NewsSearchParams,
)
class BraveNewsSearchTool(BraveSearchToolBase):
"""A tool that performs news searches using the Brave Search API."""
name: str = "Brave News Search"
args_schema: type[BaseModel] = NewsSearchParams
header_schema: type[BaseModel] = NewsSearchHeaders
description: str = (
"A tool that performs news searches using the Brave Search API. "
"Results are returned as structured JSON data."
)
search_url: str = "https://api.search.brave.com/res/v1/news/search"
def _refine_request_payload(self, params: dict[str, Any]) -> dict[str, Any]:
return params
def _refine_response(self, response: dict[str, Any]) -> list[dict[str, Any]]:
# Make the response more concise, and easier to consume
results = response.get("results", [])
return [
{
"url": result.get("url"),
"title": result.get("title"),
"description": result.get("description"),
}
for result in results
]

View File

@@ -1,4 +1,3 @@
from datetime import datetime
import json import json
import os import os
import time import time
@@ -10,16 +9,13 @@ from pydantic import BaseModel, Field
from pydantic.types import StringConstraints from pydantic.types import StringConstraints
import requests import requests
from crewai_tools.tools.brave_search_tool.base import _save_results_to_file
from crewai_tools.tools.brave_search_tool.schemas import WebSearchParams
load_dotenv() load_dotenv()
def _save_results_to_file(content: str) -> None:
"""Saves the search results to a file."""
filename = f"search_results_{datetime.now().strftime('%Y-%m-%d_%H-%M-%S')}.txt"
with open(filename, "w") as file:
file.write(content)
FreshnessPreset = Literal["pd", "pw", "pm", "py"] FreshnessPreset = Literal["pd", "pw", "pm", "py"]
FreshnessRange = Annotated[ FreshnessRange = Annotated[
str, StringConstraints(pattern=r"^\d{4}-\d{2}-\d{2}to\d{4}-\d{2}-\d{2}$") str, StringConstraints(pattern=r"^\d{4}-\d{2}-\d{2}to\d{4}-\d{2}-\d{2}$")
@@ -28,51 +24,6 @@ Freshness = FreshnessPreset | FreshnessRange
SafeSearch = Literal["off", "moderate", "strict"] SafeSearch = Literal["off", "moderate", "strict"]
class BraveSearchToolSchema(BaseModel):
"""Input for BraveSearchTool"""
query: str = Field(..., description="Search query to perform")
country: str | None = Field(
default=None,
description="Country code for geo-targeting (e.g., 'US', 'BR').",
)
search_language: str | None = Field(
default=None,
description="Language code for the search results (e.g., 'en', 'es').",
)
count: int | None = Field(
default=None,
description="The maximum number of results to return. Actual number may be less.",
)
offset: int | None = Field(
default=None, description="Skip the first N result sets/pages. Max is 9."
)
safesearch: SafeSearch | None = Field(
default=None,
description="Filter out explicit content. Options: off/moderate/strict",
)
spellcheck: bool | None = Field(
default=None,
description="Attempt to correct spelling errors in the search query.",
)
freshness: Freshness | None = Field(
default=None,
description="Enforce freshness of results. Options: pd/pw/pm/py, or YYYY-MM-DDtoYYYY-MM-DD",
)
text_decorations: bool | None = Field(
default=None,
description="Include markup to highlight search terms in the results.",
)
extra_snippets: bool | None = Field(
default=None,
description="Include up to 5 text snippets for each page if possible.",
)
operators: bool | None = Field(
default=None,
description="Whether to apply search operators (e.g., site:example.com).",
)
# TODO: Extend support to additional endpoints (e.g., /images, /news, etc.) # TODO: Extend support to additional endpoints (e.g., /images, /news, etc.)
class BraveSearchTool(BaseTool): class BraveSearchTool(BaseTool):
"""A tool that performs web searches using the Brave Search API.""" """A tool that performs web searches using the Brave Search API."""
@@ -82,7 +33,7 @@ class BraveSearchTool(BaseTool):
"A tool that performs web searches using the Brave Search API. " "A tool that performs web searches using the Brave Search API. "
"Results are returned as structured JSON data." "Results are returned as structured JSON data."
) )
args_schema: type[BaseModel] = BraveSearchToolSchema args_schema: type[BaseModel] = WebSearchParams
search_url: str = "https://api.search.brave.com/res/v1/web/search" search_url: str = "https://api.search.brave.com/res/v1/web/search"
n_results: int = 10 n_results: int = 10
save_file: bool = False save_file: bool = False
@@ -119,8 +70,8 @@ class BraveSearchTool(BaseTool):
# Construct and send the request # Construct and send the request
try: try:
# Maintain both "search_query" and "query" for backwards compatibility # Fallback to "query" or "search_query" for backwards compatibility
query = kwargs.get("search_query") or kwargs.get("query") query = kwargs.get("q") or kwargs.get("query") or kwargs.get("search_query")
if not query: if not query:
raise ValueError("Query is required") raise ValueError("Query is required")
@@ -129,8 +80,11 @@ class BraveSearchTool(BaseTool):
if country := kwargs.get("country"): if country := kwargs.get("country"):
payload["country"] = country payload["country"] = country
if search_language := kwargs.get("search_language"): # Fallback to "search_language" for backwards compatibility
payload["search_language"] = search_language if search_lang := kwargs.get("search_lang") or kwargs.get(
"search_language"
):
payload["search_lang"] = search_lang
# Fallback to deprecated n_results parameter if no count is provided # Fallback to deprecated n_results parameter if no count is provided
count = kwargs.get("count") count = kwargs.get("count")

View File

@@ -0,0 +1,39 @@
from typing import Any
from pydantic import BaseModel
from crewai_tools.tools.brave_search_tool.base import BraveSearchToolBase
from crewai_tools.tools.brave_search_tool.schemas import (
VideoSearchHeaders,
VideoSearchParams,
)
class BraveVideoSearchTool(BraveSearchToolBase):
"""A tool that performs video searches using the Brave Search API."""
name: str = "Brave Video Search"
args_schema: type[BaseModel] = VideoSearchParams
header_schema: type[BaseModel] = VideoSearchHeaders
description: str = (
"A tool that performs video searches using the Brave Search API. "
"Results are returned as structured JSON data."
)
search_url: str = "https://api.search.brave.com/res/v1/videos/search"
def _refine_request_payload(self, params: dict[str, Any]) -> dict[str, Any]:
return params
def _refine_response(self, response: dict[str, Any]) -> list[dict[str, Any]]:
# Make the response more concise, and easier to consume
results = response.get("results", [])
return [
{
"url": result.get("url"),
"title": result.get("title"),
"description": result.get("description"),
}
for result in results
]

View File

@@ -0,0 +1,45 @@
from typing import Any
from pydantic import BaseModel
from crewai_tools.tools.brave_search_tool.base import BraveSearchToolBase
from crewai_tools.tools.brave_search_tool.schemas import (
WebSearchHeaders,
WebSearchParams,
)
class BraveWebSearchTool(BraveSearchToolBase):
"""A tool that performs web searches using the Brave Search API."""
name: str = "Brave Web Search"
args_schema: type[BaseModel] = WebSearchParams
header_schema: type[BaseModel] = WebSearchHeaders
description: str = (
"A tool that performs web searches using the Brave Search API. "
"Results are returned as structured JSON data."
)
search_url: str = "https://api.search.brave.com/res/v1/web/search"
def _refine_request_payload(self, params: dict[str, Any]) -> dict[str, Any]:
return params
def _refine_response(self, response: dict[str, Any]) -> list[dict[str, Any]]:
results = response.get("web", {}).get("results", [])
refined = []
for result in results:
snippets = result.get("extra_snippets") or []
if not snippets:
desc = result.get("description")
if desc:
snippets = [desc]
refined.append(
{
"url": result.get("url"),
"title": result.get("title"),
"snippets": snippets,
}
)
return refined

View File

@@ -0,0 +1,67 @@
from __future__ import annotations
from typing import Literal, TypedDict
class LocalPOIs:
class PostalAddress(TypedDict, total=False):
type: Literal["PostalAddress"]
country: str
postalCode: str
streetAddress: str
addressRegion: str
addressLocality: str
displayAddress: str
class DayOpeningHours(TypedDict):
abbr_name: str
full_name: str
opens: str
closes: str
class OpeningHours(TypedDict, total=False):
current_day: list[LocalPOIs.DayOpeningHours]
days: list[list[LocalPOIs.DayOpeningHours]]
class LocationResult(TypedDict, total=False):
provider_url: str
title: str
url: str
id: str | None
opening_hours: LocalPOIs.OpeningHours | None
postal_address: LocalPOIs.PostalAddress | None
class Response(TypedDict, total=False):
type: Literal["local_pois"]
results: list[LocalPOIs.LocationResult]
class LLMContext:
class LLMContextItem(TypedDict, total=False):
snippets: list[str]
title: str
url: str
class LLMContextMapItem(TypedDict, total=False):
name: str
snippets: list[str]
title: str
url: str
class LLMContextPOIItem(TypedDict, total=False):
name: str
snippets: list[str]
title: str
url: str
class Grounding(TypedDict, total=False):
generic: list[LLMContext.LLMContextItem]
poi: LLMContext.LLMContextPOIItem
map: list[LLMContext.LLMContextMapItem]
class Sources(TypedDict, total=False):
pass
class Response(TypedDict, total=False):
grounding: LLMContext.Grounding
sources: LLMContext.Sources

View File

@@ -0,0 +1,525 @@
from typing import Annotated, Literal
from pydantic import BaseModel, Field
from pydantic.types import StringConstraints
# Common types
Units = Literal["metric", "imperial"]
SafeSearch = Literal["off", "moderate", "strict"]
Freshness = (
Literal["pd", "pw", "pm", "py"]
| Annotated[
str, StringConstraints(pattern=r"^\d{4}-\d{2}-\d{2}to\d{4}-\d{2}-\d{2}$")
]
)
ResultFilter = list[
Literal[
"discussions",
"faq",
"infobox",
"news",
"query",
"summarizer",
"videos",
"web",
"locations",
]
]
class LLMContextParams(BaseModel):
"""Parameters for Brave LLM Context endpoint."""
q: str = Field(
description="Search query to perform",
min_length=1,
max_length=400,
)
country: str | None = Field(
default=None,
description="Country code for geo-targeting (e.g., 'US', 'BR').",
pattern=r"^[A-Z]{2}$",
)
search_lang: str | None = Field(
default=None,
description="Language code for the search results (e.g., 'en', 'es').",
pattern=r"^[a-z]{2}$",
)
count: int | None = Field(
default=None,
description="The maximum number of results to return. Actual number may be less.",
ge=1,
le=50,
)
maximum_number_of_urls: int | None = Field(
default=None,
description="The maximum number of URLs to include in the context.",
ge=1,
le=50,
)
maximum_number_of_tokens: int | None = Field(
default=None,
description="The approximate maximum number of tokens to include in the context.",
ge=1,
le=32768,
)
maximum_number_of_snippets: int | None = Field(
default=None,
description="The maximum number of different snippets to include in the context.",
ge=1,
le=100,
)
context_threshold_mode: (
Literal["disabled", "strict", "lenient", "balanced"] | None
) = Field(
default=None,
description="The mode to use for the context thresholding.",
)
maximum_number_of_tokens_per_url: int | None = Field(
default=None,
description="The maximum number of tokens to include for each URL in the context.",
ge=1,
le=8192,
)
maximum_number_of_snippets_per_url: int | None = Field(
default=None,
description="The maximum number of snippets to include per URL.",
ge=1,
le=100,
)
goggles: str | list[str] | None = Field(
default=None,
description="Goggles act as a custom re-ranking mechanism. Goggle source or URLs.",
)
enable_local: bool | None = Field(
default=None,
description="Whether to enable local recall. Not setting this value means auto-detect and uses local recall if any of the localization headers are provided.",
)
class WebSearchParams(BaseModel):
"""Parameters for Brave Web Search endpoint."""
q: str = Field(
description="Search query to perform",
min_length=1,
max_length=400,
)
country: str | None = Field(
default=None,
description="Country code for geo-targeting (e.g., 'US', 'BR').",
pattern=r"^[A-Z]{2}$",
)
search_lang: str | None = Field(
default=None,
description="Language code for the search results (e.g., 'en', 'es').",
pattern=r"^[a-z]{2}$",
)
ui_lang: str | None = Field(
default=None,
description="Language code for the user interface (e.g., 'en-US', 'es-AR').",
pattern=r"^[a-z]{2}-[A-Z]{2}$",
)
count: int | None = Field(
default=None,
description="The maximum number of results to return. Actual number may be less.",
ge=1,
le=20,
)
offset: int | None = Field(
default=None,
description="Skip the first N result sets/pages. Max is 9.",
ge=0,
le=9,
)
safesearch: Literal["off", "moderate", "strict"] | None = Field(
default=None,
description="Filter out explicit content. Options: off/moderate/strict",
)
spellcheck: bool | None = Field(
default=None,
description="Attempt to correct spelling errors in the search query.",
)
freshness: Freshness | None = Field(
default=None,
description="Enforce freshness of results. Options: pd/pw/pm/py, or YYYY-MM-DDtoYYYY-MM-DD",
)
text_decorations: bool | None = Field(
default=None,
description="Include markup to highlight search terms in the results.",
)
extra_snippets: bool | None = Field(
default=None,
description="Include up to 5 text snippets for each page if possible.",
)
result_filter: ResultFilter | None = Field(
default=None,
description="Filter the results by type. Options: discussions/faq/infobox/news/query/summarizer/videos/web/locations. Note: The `count` parameter is applied only to the `web` results.",
)
units: Units | None = Field(
default=None,
description="The units to use for the results. Options: metric/imperial",
)
goggles: str | list[str] | None = Field(
default=None,
description="Goggles act as a custom re-ranking mechanism. Goggle source or URLs.",
)
summary: bool | None = Field(
default=None,
description="Whether to generate a summarizer ID for the results.",
)
enable_rich_callback: bool | None = Field(
default=None,
description="Whether to enable rich callbacks for the results. Requires Pro level subscription.",
)
include_fetch_metadata: bool | None = Field(
default=None,
description="Whether to include fetch metadata (e.g., last fetch time) in the results.",
)
operators: bool | None = Field(
default=None,
description="Whether to apply search operators (e.g., site:example.com).",
)
class LocalPOIsParams(BaseModel):
"""Parameters for Brave Local POIs endpoint."""
ids: list[str] = Field(
description="List of POI IDs to retrieve. Maximum of 20. IDs are valid for 8 hours.",
min_length=1,
max_length=20,
)
search_lang: str | None = Field(
default=None,
description="Language code for the search results (e.g., 'en', 'es').",
pattern=r"^[a-z]{2}$",
)
ui_lang: str | None = Field(
default=None,
description="Language code for the user interface (e.g., 'en-US', 'es-AR').",
pattern=r"^[a-z]{2}-[A-Z]{2}$",
)
units: Units | None = Field(
default=None,
description="The units to use for the results. Options: metric/imperial",
)
class LocalPOIsDescriptionParams(BaseModel):
"""Parameters for Brave Local POI Descriptions endpoint."""
ids: list[str] = Field(
description="List of POI IDs to retrieve. Maximum of 20. IDs are valid for 8 hours.",
min_length=1,
max_length=20,
)
class ImageSearchParams(BaseModel):
"""Parameters for Brave Image Search endpoint."""
q: str = Field(
description="Search query to perform",
min_length=1,
max_length=400,
)
search_lang: str | None = Field(
default=None,
description="Language code for the search results (e.g., 'en', 'es').",
pattern=r"^[a-z]{2}$",
)
country: str | None = Field(
default=None,
description="Country code for geo-targeting (e.g., 'US', 'BR').",
pattern=r"^[A-Z]{2}$",
)
safesearch: Literal["off", "strict"] | None = Field(
default=None,
description="Filter out explicit content. Default is strict.",
)
count: int | None = Field(
default=None,
description="The maximum number of results to return.",
ge=1,
le=200,
)
spellcheck: bool | None = Field(
default=None,
description="Attempt to correct spelling errors in the search query.",
)
class VideoSearchParams(BaseModel):
"""Parameters for Brave Video Search endpoint."""
q: str = Field(
description="Search query to perform",
min_length=1,
max_length=400,
)
search_lang: str | None = Field(
default=None,
description="Language code for the search results (e.g., 'en', 'es').",
pattern=r"^[a-z]{2}$",
)
ui_lang: str | None = Field(
default=None,
description="Language code for the user interface (e.g., 'en-US', 'es-AR').",
pattern=r"^[a-z]{2}-[A-Z]{2}$",
)
country: str | None = Field(
default=None,
description="Country code for geo-targeting (e.g., 'US', 'BR').",
pattern=r"^[A-Z]{2}$",
)
safesearch: SafeSearch | None = Field(
default=None,
description="Filter out explicit content. Options: off/moderate/strict",
)
count: int | None = Field(
default=None,
description="The maximum number of results to return.",
ge=1,
le=50,
)
offset: int | None = Field(
default=None,
description="Skip the first N result sets/pages. Max is 9.",
ge=0,
le=9,
)
spellcheck: bool | None = Field(
default=None,
description="Attempt to correct spelling errors in the search query.",
)
freshness: Freshness | None = Field(
default=None,
description="Enforce freshness of results. Options: pd/pw/pm/py, or YYYY-MM-DDtoYYYY-MM-DD",
)
include_fetch_metadata: bool | None = Field(
default=None,
description="Whether to include fetch metadata (e.g., last fetch time) in the results.",
)
operators: bool | None = Field(
default=None,
description="Whether to apply search operators (e.g., site:example.com).",
)
class NewsSearchParams(BaseModel):
"""Parameters for Brave News Search endpoint."""
q: str = Field(
description="Search query to perform",
min_length=1,
max_length=400,
)
search_lang: str | None = Field(
default=None,
description="Language code for the search results (e.g., 'en', 'es').",
pattern=r"^[a-z]{2}$",
)
ui_lang: str | None = Field(
default=None,
description="Language code for the user interface (e.g., 'en-US', 'es-AR').",
pattern=r"^[a-z]{2}-[A-Z]{2}$",
)
country: str | None = Field(
default=None,
description="Country code for geo-targeting (e.g., 'US', 'BR').",
pattern=r"^[A-Z]{2}$",
)
safesearch: Literal["off", "moderate", "strict"] | None = Field(
default=None,
description="Filter out explicit content. Options: off/moderate/strict",
)
count: int | None = Field(
default=None,
description="The maximum number of results to return.",
ge=1,
le=50,
)
offset: int | None = Field(
default=None,
description="Skip the first N result sets/pages. Max is 9.",
ge=0,
le=9,
)
spellcheck: bool | None = Field(
default=None,
description="Attempt to correct spelling errors in the search query.",
)
freshness: Freshness | None = Field(
default=None,
description="Enforce freshness of results. Options: pd/pw/pm/py, or YYYY-MM-DDtoYYYY-MM-DD",
)
extra_snippets: bool | None = Field(
default=None,
description="Include up to 5 text snippets for each page if possible.",
)
goggles: str | list[str] | None = Field(
default=None,
description="Goggles act as a custom re-ranking mechanism. Goggle source or URLs.",
)
include_fetch_metadata: bool | None = Field(
default=None,
description="Whether to include fetch metadata in the results.",
)
operators: bool | None = Field(
default=None,
description="Whether to apply search operators (e.g., site:example.com).",
)
class BaseSearchHeaders(BaseModel):
"""Common headers for Brave Search endpoints."""
x_subscription_token: str = Field(
alias="x-subscription-token",
description="API key for Brave Search",
)
api_version: str | None = Field(
alias="api-version",
default=None,
description="API version to use. Default is latest available.",
pattern=r"^\d{4}-\d{2}-\d{2}$", # YYYY-MM-DD
)
accept: Literal["application/json"] | Literal["*/*"] | None = Field(
default=None,
description="Accept header for the request.",
)
cache_control: Literal["no-cache"] | None = Field(
alias="cache-control",
default=None,
description="Cache control header for the request.",
)
user_agent: str | None = Field(
alias="user-agent",
default=None,
description="User agent for the request.",
)
class LLMContextHeaders(BaseSearchHeaders):
"""Headers for Brave LLM Context endpoint."""
x_loc_lat: float | None = Field(
alias="x-loc-lat",
default=None,
description="Latitude of the user's location.",
ge=-90.0,
le=90.0,
)
x_loc_long: float | None = Field(
alias="x-loc-long",
default=None,
description="Longitude of the user's location.",
ge=-180.0,
le=180.0,
)
x_loc_city: str | None = Field(
alias="x-loc-city",
default=None,
description="City of the user's location.",
)
x_loc_state: str | None = Field(
alias="x-loc-state",
default=None,
description="State of the user's location.",
)
x_loc_state_name: str | None = Field(
alias="x-loc-state-name",
default=None,
description="Name of the state of the user's location.",
)
x_loc_country: str | None = Field(
alias="x-loc-country",
default=None,
description="The ISO 3166-1 alpha-2 country code of the user's location.",
)
class LocalPOIsHeaders(BaseSearchHeaders):
"""Headers for Brave Local POIs endpoint."""
x_loc_lat: float | None = Field(
alias="x-loc-lat",
default=None,
description="Latitude of the user's location.",
ge=-90.0,
le=90.0,
)
x_loc_long: float | None = Field(
alias="x-loc-long",
default=None,
description="Longitude of the user's location.",
ge=-180.0,
le=180.0,
)
class LocalPOIsDescriptionHeaders(BaseSearchHeaders):
"""Headers for Brave Local POI Descriptions endpoint."""
class VideoSearchHeaders(BaseSearchHeaders):
"""Headers for Brave Video Search endpoint."""
class ImageSearchHeaders(BaseSearchHeaders):
"""Headers for Brave Image Search endpoint."""
class NewsSearchHeaders(BaseSearchHeaders):
"""Headers for Brave News Search endpoint."""
class WebSearchHeaders(BaseSearchHeaders):
"""Headers for Brave Web Search endpoint."""
x_loc_lat: float | None = Field(
alias="x-loc-lat",
default=None,
description="Latitude of the user's location.",
ge=-90.0,
le=90.0,
)
x_loc_long: float | None = Field(
alias="x-loc-long",
default=None,
description="Longitude of the user's location.",
ge=-180.0,
le=180.0,
)
x_loc_timezone: str | None = Field(
alias="x-loc-timezone",
default=None,
description="Timezone of the user's location.",
)
x_loc_city: str | None = Field(
alias="x-loc-city",
default=None,
description="City of the user's location.",
)
x_loc_state: str | None = Field(
alias="x-loc-state",
default=None,
description="State of the user's location.",
)
x_loc_state_name: str | None = Field(
alias="x-loc-state-name",
default=None,
description="Name of the state of the user's location.",
)
x_loc_country: str | None = Field(
alias="x-loc-country",
default=None,
description="The ISO 3166-1 alpha-2 country code of the user's location.",
)
x_loc_postal_code: str | None = Field(
alias="x-loc-postal-code",
default=None,
description="The postal code of the user's location.",
)

View File

@@ -30,9 +30,8 @@ class FileWriterTool(BaseTool):
def _run(self, **kwargs: Any) -> str: def _run(self, **kwargs: Any) -> str:
try: try:
# Create the directory if it doesn't exist if kwargs.get("directory"):
if kwargs.get("directory") and not os.path.exists(kwargs["directory"]): os.makedirs(kwargs["directory"], exist_ok=True)
os.makedirs(kwargs["directory"])
# Construct the full path # Construct the full path
filepath = os.path.join(kwargs.get("directory") or "", kwargs["filename"]) filepath = os.path.join(kwargs.get("directory") or "", kwargs["filename"])

View File

@@ -99,8 +99,8 @@ class FileCompressorTool(BaseTool):
def _prepare_output(output_path: str, overwrite: bool) -> bool: def _prepare_output(output_path: str, overwrite: bool) -> bool:
"""Ensures output path is ready for writing.""" """Ensures output path is ready for writing."""
output_dir = os.path.dirname(output_path) output_dir = os.path.dirname(output_path)
if output_dir and not os.path.exists(output_dir): if output_dir:
os.makedirs(output_dir) os.makedirs(output_dir, exist_ok=True)
if os.path.exists(output_path) and not overwrite: if os.path.exists(output_path) and not overwrite:
return False return False
return True return True

View File

@@ -18,7 +18,6 @@ class MergeAgentHandlerToolError(Exception):
"""Base exception for Merge Agent Handler tool errors.""" """Base exception for Merge Agent Handler tool errors."""
class MergeAgentHandlerTool(BaseTool): class MergeAgentHandlerTool(BaseTool):
""" """
Wrapper for Merge Agent Handler tools. Wrapper for Merge Agent Handler tools.
@@ -174,7 +173,7 @@ class MergeAgentHandlerTool(BaseTool):
>>> tool = MergeAgentHandlerTool.from_tool_name( >>> tool = MergeAgentHandlerTool.from_tool_name(
... tool_name="linear__create_issue", ... tool_name="linear__create_issue",
... tool_pack_id="134e0111-0f67-44f6-98f0-597000290bb3", ... tool_pack_id="134e0111-0f67-44f6-98f0-597000290bb3",
... registered_user_id="91b2b905-e866-40c8-8be2-efe53827a0aa" ... registered_user_id="91b2b905-e866-40c8-8be2-efe53827a0aa",
... ) ... )
""" """
# Create an empty args schema model (proper BaseModel subclass) # Create an empty args schema model (proper BaseModel subclass)
@@ -210,7 +209,10 @@ class MergeAgentHandlerTool(BaseTool):
if "parameters" in tool_schema: if "parameters" in tool_schema:
try: try:
params = tool_schema["parameters"] params = tool_schema["parameters"]
if params.get("type") == "object" and "properties" in params: if (
params.get("type") == "object"
and "properties" in params
):
# Build field definitions for Pydantic # Build field definitions for Pydantic
fields = {} fields = {}
properties = params["properties"] properties = params["properties"]
@@ -298,7 +300,7 @@ class MergeAgentHandlerTool(BaseTool):
>>> tools = MergeAgentHandlerTool.from_tool_pack( >>> tools = MergeAgentHandlerTool.from_tool_pack(
... tool_pack_id="134e0111-0f67-44f6-98f0-597000290bb3", ... tool_pack_id="134e0111-0f67-44f6-98f0-597000290bb3",
... registered_user_id="91b2b905-e866-40c8-8be2-efe53827a0aa", ... registered_user_id="91b2b905-e866-40c8-8be2-efe53827a0aa",
... tool_names=["linear__create_issue", "linear__get_issues"] ... tool_names=["linear__create_issue", "linear__get_issues"],
... ) ... )
""" """
# Create a temporary instance to fetch the tool list # Create a temporary instance to fetch the tool list

View File

@@ -1,7 +1,7 @@
import os import os
from crewai import Agent, Crew, Task from crewai import Agent, Crew, Task
from multion_tool import MultiOnTool # type: ignore[import-not-found] from multion_tool import MultiOnTool # type: ignore[import-not-found]
os.environ["OPENAI_API_KEY"] = "Your Key" os.environ["OPENAI_API_KEY"] = "Your Key"

View File

@@ -110,11 +110,13 @@ class QdrantVectorSearchTool(BaseTool):
self.custom_embedding_fn(query) self.custom_embedding_fn(query)
if self.custom_embedding_fn if self.custom_embedding_fn
else ( else (
lambda: __import__("openai") lambda: (
.Client(api_key=os.getenv("OPENAI_API_KEY")) __import__("openai")
.embeddings.create(input=[query], model="text-embedding-3-large") .Client(api_key=os.getenv("OPENAI_API_KEY"))
.data[0] .embeddings.create(input=[query], model="text-embedding-3-large")
.embedding .data[0]
.embedding
)
)() )()
) )
results = self.client.query_points( results = self.client.query_points(

View File

@@ -3,6 +3,7 @@ from __future__ import annotations
import asyncio import asyncio
from concurrent.futures import ThreadPoolExecutor from concurrent.futures import ThreadPoolExecutor
import logging import logging
import threading
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
from crewai.tools.base_tool import BaseTool from crewai.tools.base_tool import BaseTool
@@ -33,6 +34,7 @@ logger = logging.getLogger(__name__)
# Cache for query results # Cache for query results
_query_cache: dict[str, list[dict[str, Any]]] = {} _query_cache: dict[str, list[dict[str, Any]]] = {}
_cache_lock = threading.Lock()
class SnowflakeConfig(BaseModel): class SnowflakeConfig(BaseModel):
@@ -102,7 +104,7 @@ class SnowflakeSearchTool(BaseTool):
) )
_connection_pool: list[SnowflakeConnection] | None = None _connection_pool: list[SnowflakeConnection] | None = None
_pool_lock: asyncio.Lock | None = None _pool_lock: threading.Lock | None = None
_thread_pool: ThreadPoolExecutor | None = None _thread_pool: ThreadPoolExecutor | None = None
_model_rebuilt: bool = False _model_rebuilt: bool = False
package_dependencies: list[str] = Field( package_dependencies: list[str] = Field(
@@ -122,7 +124,7 @@ class SnowflakeSearchTool(BaseTool):
try: try:
if SNOWFLAKE_AVAILABLE: if SNOWFLAKE_AVAILABLE:
self._connection_pool = [] self._connection_pool = []
self._pool_lock = asyncio.Lock() self._pool_lock = threading.Lock()
self._thread_pool = ThreadPoolExecutor(max_workers=self.pool_size) self._thread_pool = ThreadPoolExecutor(max_workers=self.pool_size)
else: else:
raise ImportError raise ImportError
@@ -147,7 +149,7 @@ class SnowflakeSearchTool(BaseTool):
) )
self._connection_pool = [] self._connection_pool = []
self._pool_lock = asyncio.Lock() self._pool_lock = threading.Lock()
self._thread_pool = ThreadPoolExecutor(max_workers=self.pool_size) self._thread_pool = ThreadPoolExecutor(max_workers=self.pool_size)
except subprocess.CalledProcessError as e: except subprocess.CalledProcessError as e:
raise ImportError("Failed to install Snowflake dependencies") from e raise ImportError("Failed to install Snowflake dependencies") from e
@@ -163,13 +165,12 @@ class SnowflakeSearchTool(BaseTool):
raise RuntimeError("Pool lock not initialized") raise RuntimeError("Pool lock not initialized")
if self._connection_pool is None: if self._connection_pool is None:
raise RuntimeError("Connection pool not initialized") raise RuntimeError("Connection pool not initialized")
async with self._pool_lock: with self._pool_lock:
if not self._connection_pool: if self._connection_pool:
conn = await asyncio.get_event_loop().run_in_executor( return self._connection_pool.pop()
self._thread_pool, self._create_connection return await asyncio.get_event_loop().run_in_executor(
) self._thread_pool, self._create_connection
self._connection_pool.append(conn) )
return self._connection_pool.pop()
def _create_connection(self) -> SnowflakeConnection: def _create_connection(self) -> SnowflakeConnection:
"""Create a new Snowflake connection.""" """Create a new Snowflake connection."""
@@ -204,9 +205,10 @@ class SnowflakeSearchTool(BaseTool):
"""Execute a query with retries and return results.""" """Execute a query with retries and return results."""
if self.enable_caching: if self.enable_caching:
cache_key = self._get_cache_key(query, timeout) cache_key = self._get_cache_key(query, timeout)
if cache_key in _query_cache: with _cache_lock:
logger.info("Returning cached result") if cache_key in _query_cache:
return _query_cache[cache_key] logger.info("Returning cached result")
return _query_cache[cache_key]
for attempt in range(self.max_retries): for attempt in range(self.max_retries):
try: try:
@@ -225,7 +227,8 @@ class SnowflakeSearchTool(BaseTool):
] ]
if self.enable_caching: if self.enable_caching:
_query_cache[self._get_cache_key(query, timeout)] = results with _cache_lock:
_query_cache[self._get_cache_key(query, timeout)] = results
return results return results
finally: finally:
@@ -234,7 +237,7 @@ class SnowflakeSearchTool(BaseTool):
self._pool_lock is not None self._pool_lock is not None
and self._connection_pool is not None and self._connection_pool is not None
): ):
async with self._pool_lock: with self._pool_lock:
self._connection_pool.append(conn) self._connection_pool.append(conn)
except (DatabaseError, OperationalError) as e: # noqa: PERF203 except (DatabaseError, OperationalError) as e: # noqa: PERF203
if attempt == self.max_retries - 1: if attempt == self.max_retries - 1:

View File

@@ -17,11 +17,11 @@ Usage:
import os import os
from crewai import Agent, Crew, Process, Task
from crewai.utilities.printer import Printer from crewai.utilities.printer import Printer
from dotenv import load_dotenv from dotenv import load_dotenv
from stagehand.schemas import AvailableModel # type: ignore[import-untyped] from stagehand.schemas import AvailableModel # type: ignore[import-untyped]
from crewai import Agent, Crew, Process, Task
from crewai_tools import StagehandTool from crewai_tools import StagehandTool

View File

@@ -1,4 +1,5 @@
import asyncio import asyncio
import contextvars
import json import json
import os import os
import re import re
@@ -137,7 +138,9 @@ class StagehandTool(BaseTool):
- 'observe': For finding elements in a specific area - 'observe': For finding elements in a specific area
""" """
args_schema: type[BaseModel] = StagehandToolSchema args_schema: type[BaseModel] = StagehandToolSchema
package_dependencies: list[str] = Field(default_factory=lambda: ["stagehand<=0.5.9"]) package_dependencies: list[str] = Field(
default_factory=lambda: ["stagehand<=0.5.9"]
)
env_vars: list[EnvVar] = Field( env_vars: list[EnvVar] = Field(
default_factory=lambda: [ default_factory=lambda: [
EnvVar( EnvVar(
@@ -620,9 +623,12 @@ class StagehandTool(BaseTool):
# We're in an existing event loop, use it # We're in an existing event loop, use it
import concurrent.futures import concurrent.futures
ctx = contextvars.copy_context()
with concurrent.futures.ThreadPoolExecutor() as executor: with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit( future = executor.submit(
asyncio.run, self._async_run(instruction, url, command_type) ctx.run,
asyncio.run,
self._async_run(instruction, url, command_type),
) )
result = future.result() result = future.result()
else: else:
@@ -706,11 +712,12 @@ class StagehandTool(BaseTool):
if loop.is_running(): if loop.is_running():
import concurrent.futures import concurrent.futures
ctx = contextvars.copy_context()
with ( with (
concurrent.futures.ThreadPoolExecutor() as executor concurrent.futures.ThreadPoolExecutor() as executor
): ):
future = executor.submit( future = executor.submit(
asyncio.run, self._async_close() ctx.run, asyncio.run, self._async_close()
) )
future.result() future.result()
else: else:

View File

@@ -1,80 +1,777 @@
import json import os
from unittest.mock import patch from unittest.mock import MagicMock, patch
import pytest import pytest
import requests as requests_lib
from crewai_tools.tools.brave_search_tool.brave_search_tool import BraveSearchTool from crewai_tools.tools.brave_search_tool.base import BraveSearchToolBase
from crewai_tools.tools.brave_search_tool.brave_web_tool import BraveWebSearchTool
from crewai_tools.tools.brave_search_tool.brave_image_tool import BraveImageSearchTool
from crewai_tools.tools.brave_search_tool.brave_news_tool import BraveNewsSearchTool
from crewai_tools.tools.brave_search_tool.brave_video_tool import BraveVideoSearchTool
from crewai_tools.tools.brave_search_tool.brave_llm_context_tool import (
BraveLLMContextTool,
)
from crewai_tools.tools.brave_search_tool.brave_local_pois_tool import (
BraveLocalPOIsTool,
BraveLocalPOIsDescriptionTool,
)
from crewai_tools.tools.brave_search_tool.schemas import (
WebSearchParams,
WebSearchHeaders,
ImageSearchParams,
ImageSearchHeaders,
NewsSearchParams,
NewsSearchHeaders,
VideoSearchParams,
VideoSearchHeaders,
LLMContextParams,
LLMContextHeaders,
LocalPOIsParams,
LocalPOIsHeaders,
LocalPOIsDescriptionParams,
LocalPOIsDescriptionHeaders,
)
def _mock_response(
status_code: int = 200,
json_data: dict | None = None,
headers: dict | None = None,
text: str = "",
) -> MagicMock:
"""Build a ``requests.Response``-like mock with the attributes used by ``_make_request``."""
resp = MagicMock(spec=requests_lib.Response)
resp.status_code = status_code
resp.ok = 200 <= status_code < 400
resp.url = "https://api.search.brave.com/res/v1/web/search?q=test"
resp.text = text or (str(json_data) if json_data else "")
resp.headers = headers or {}
resp.json.return_value = json_data if json_data is not None else {}
return resp
# Fixtures
@pytest.fixture(autouse=True)
def _brave_env_and_rate_limit():
"""Set BRAVE_API_KEY for every test. Rate limiting is per-instance (each tool starts with a fresh clock)."""
with patch.dict(os.environ, {"BRAVE_API_KEY": "test-api-key"}):
yield
@pytest.fixture @pytest.fixture
def brave_tool(): def web_tool():
return BraveSearchTool(n_results=2) return BraveWebSearchTool()
def test_brave_tool_initialization(): @pytest.fixture
tool = BraveSearchTool() def image_tool():
assert tool.n_results == 10 return BraveImageSearchTool()
@pytest.fixture
def news_tool():
return BraveNewsSearchTool()
@pytest.fixture
def video_tool():
return BraveVideoSearchTool()
# Initialization
ALL_TOOL_CLASSES = [
BraveWebSearchTool,
BraveImageSearchTool,
BraveNewsSearchTool,
BraveVideoSearchTool,
BraveLLMContextTool,
BraveLocalPOIsTool,
BraveLocalPOIsDescriptionTool,
]
@pytest.mark.parametrize("tool_cls", ALL_TOOL_CLASSES)
def test_instantiation_with_env_var(tool_cls):
"""Each tool can be created when BRAVE_API_KEY is in the environment."""
tool = tool_cls()
assert tool.api_key == "test-api-key"
@pytest.mark.parametrize("tool_cls", ALL_TOOL_CLASSES)
def test_instantiation_with_explicit_key(tool_cls):
"""An explicit api_key takes precedence over the environment."""
tool = tool_cls(api_key="explicit-key")
assert tool.api_key == "explicit-key"
def test_missing_api_key_raises():
with patch.dict(os.environ, {}, clear=True):
with pytest.raises(ValueError, match="BRAVE_API_KEY"):
BraveWebSearchTool()
def test_default_attributes():
tool = BraveWebSearchTool()
assert tool.save_file is False assert tool.save_file is False
assert tool.n_results == 10
assert tool._timeout == 30
assert tool._requests_per_second == 1.0
assert tool.raw is False
@patch("requests.get") def test_custom_constructor_args():
def test_brave_tool_search(mock_get, brave_tool): tool = BraveWebSearchTool(
mock_response = { save_file=True,
timeout=60,
n_results=5,
requests_per_second=0.5,
raw=True,
)
assert tool.save_file is True
assert tool._timeout == 60
assert tool.n_results == 5
assert tool._requests_per_second == 0.5
assert tool.raw is True
# Headers
def test_default_headers():
tool = BraveWebSearchTool()
assert tool.headers["x-subscription-token"] == "test-api-key"
assert tool.headers["accept"] == "application/json"
def test_set_headers_merges_and_normalizes():
tool = BraveWebSearchTool()
tool.set_headers({"Cache-Control": "no-cache"})
assert tool.headers["cache-control"] == "no-cache"
assert tool.headers["x-subscription-token"] == "test-api-key"
def test_set_headers_returns_self_for_chaining():
tool = BraveWebSearchTool()
assert tool.set_headers({"Cache-Control": "no-cache"}) is tool
def test_invalid_header_value_raises():
tool = BraveImageSearchTool()
with pytest.raises(ValueError, match="Invalid headers"):
tool.set_headers({"Accept": "text/xml"})
# Endpoint & Schema Wiring
@pytest.mark.parametrize(
"tool_cls, expected_url, expected_params, expected_headers",
[
(
BraveWebSearchTool,
"https://api.search.brave.com/res/v1/web/search",
WebSearchParams,
WebSearchHeaders,
),
(
BraveImageSearchTool,
"https://api.search.brave.com/res/v1/images/search",
ImageSearchParams,
ImageSearchHeaders,
),
(
BraveNewsSearchTool,
"https://api.search.brave.com/res/v1/news/search",
NewsSearchParams,
NewsSearchHeaders,
),
(
BraveVideoSearchTool,
"https://api.search.brave.com/res/v1/videos/search",
VideoSearchParams,
VideoSearchHeaders,
),
(
BraveLLMContextTool,
"https://api.search.brave.com/res/v1/llm/context",
LLMContextParams,
LLMContextHeaders,
),
(
BraveLocalPOIsTool,
"https://api.search.brave.com/res/v1/local/pois",
LocalPOIsParams,
LocalPOIsHeaders,
),
(
BraveLocalPOIsDescriptionTool,
"https://api.search.brave.com/res/v1/local/descriptions",
LocalPOIsDescriptionParams,
LocalPOIsDescriptionHeaders,
),
],
)
def test_tool_wiring(tool_cls, expected_url, expected_params, expected_headers):
tool = tool_cls()
assert tool.search_url == expected_url
assert tool.args_schema is expected_params
assert tool.header_schema is expected_headers
# Payload Refinement (e.g., `query` -> `q`, `count` fallback, param pass-through)
def test_web_refine_request_payload_passes_all_params(web_tool):
params = web_tool._common_payload_refinement(
{
"query": "test",
"country": "US",
"search_lang": "en",
"count": 5,
"offset": 2,
"safesearch": "moderate",
"freshness": "pw",
}
)
refined_params = web_tool._refine_request_payload(params)
assert refined_params["q"] == "test"
assert "query" not in refined_params
assert refined_params["count"] == 5
assert refined_params["country"] == "US"
assert refined_params["search_lang"] == "en"
assert refined_params["offset"] == 2
assert refined_params["safesearch"] == "moderate"
assert refined_params["freshness"] == "pw"
def test_image_refine_request_payload_passes_all_params(image_tool):
params = image_tool._common_payload_refinement(
{
"query": "cat photos",
"country": "US",
"search_lang": "en",
"safesearch": "strict",
"count": 50,
"spellcheck": True,
}
)
refined_params = image_tool._refine_request_payload(params)
assert refined_params["q"] == "cat photos"
assert "query" not in refined_params
assert refined_params["country"] == "US"
assert refined_params["safesearch"] == "strict"
assert refined_params["count"] == 50
assert refined_params["spellcheck"] is True
def test_news_refine_request_payload_passes_all_params(news_tool):
params = news_tool._common_payload_refinement(
{
"query": "breaking news",
"country": "US",
"count": 10,
"offset": 1,
"freshness": "pd",
"extra_snippets": True,
}
)
refined_params = news_tool._refine_request_payload(params)
assert refined_params["q"] == "breaking news"
assert "query" not in refined_params
assert refined_params["country"] == "US"
assert refined_params["offset"] == 1
assert refined_params["freshness"] == "pd"
assert refined_params["extra_snippets"] is True
def test_video_refine_request_payload_passes_all_params(video_tool):
params = video_tool._common_payload_refinement(
{
"query": "tutorial",
"country": "US",
"count": 25,
"offset": 0,
"safesearch": "strict",
"freshness": "pm",
}
)
refined_params = video_tool._refine_request_payload(params)
assert refined_params["q"] == "tutorial"
assert "query" not in refined_params
assert refined_params["country"] == "US"
assert refined_params["offset"] == 0
assert refined_params["freshness"] == "pm"
def test_legacy_constructor_params_flow_into_query_params():
"""The legacy n_results and country constructor params are applied as defaults
when count/country are not explicitly provided at call time."""
tool = BraveWebSearchTool(n_results=3, country="BR")
params = tool._common_payload_refinement({"query": "test"})
assert params["count"] == 3
assert params["country"] == "BR"
def test_legacy_constructor_params_do_not_override_explicit_query_params():
"""Explicit query-time count/country take precedence over constructor defaults."""
tool = BraveWebSearchTool(n_results=3, country="BR")
params = tool._common_payload_refinement(
{"query": "test", "count": 10, "country": "US"}
)
assert params["count"] == 10
assert params["country"] == "US"
def test_refine_request_payload_passes_multiple_goggles_as_multiple_params(web_tool):
result = web_tool._refine_request_payload(
{
"query": "test",
"goggles": ["goggle1", "goggle2"],
}
)
assert result["goggles"] == ["goggle1", "goggle2"]
# Null-like / empty value stripping
#
# crewAI's ensure_all_properties_required (pydantic_schema_utils.py) marks
# every schema property as required for OpenAI strict-mode compatibility.
# Because optional Brave API parameters look required to the LLM, it fills
# them with placeholder junk — None, "", "null", or []. The test below
# verifies that _common_payload_refinement strips these from optional fields.
def test_common_refinement_strips_null_like_values(web_tool):
"""_common_payload_refinement drops optional keys with None / '' / 'null' / []."""
params = web_tool._common_payload_refinement(
{
"query": "test",
"country": "US",
"search_lang": "",
"freshness": "null",
"count": 5,
"goggles": [],
}
)
assert params["q"] == "test"
assert params["country"] == "US"
assert params["count"] == 5
assert "search_lang" not in params
assert "freshness" not in params
assert "goggles" not in params
# End-to-End _run() with Mocked HTTP Response
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_web_search_end_to_end(mock_get, web_tool):
web_tool.raw = True
data = {"web": {"results": [{"title": "R", "url": "http://r.co"}]}}
mock_get.return_value = _mock_response(json_data=data)
result = web_tool._run(query="test")
mock_get.assert_called_once()
call_args = mock_get.call_args.kwargs
assert call_args["params"]["q"] == "test"
assert call_args["headers"]["x-subscription-token"] == "test-api-key"
assert result == data
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_image_search_end_to_end(mock_get, image_tool):
image_tool.raw = True
data = {"results": [{"url": "http://img.co/a.jpg"}]}
mock_get.return_value = _mock_response(json_data=data)
assert image_tool._run(query="cats") == data
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_news_search_end_to_end(mock_get, news_tool):
news_tool.raw = True
data = {"results": [{"title": "News", "url": "http://n.co"}]}
mock_get.return_value = _mock_response(json_data=data)
assert news_tool._run(query="headlines") == data
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_video_search_end_to_end(mock_get, video_tool):
video_tool.raw = True
data = {"results": [{"title": "Vid", "url": "http://v.co"}]}
mock_get.return_value = _mock_response(json_data=data)
assert video_tool._run(query="python tutorial") == data
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_raw_false_calls_refine_response(mock_get, web_tool):
"""With raw=False (the default), _refine_response transforms the API response."""
api_response = {
"web": { "web": {
"results": [ "results": [
{ {
"title": "Test Title", "title": "CrewAI",
"url": "http://test.com", "url": "https://crewai.com",
"description": "Test Description", "description": "AI agent framework",
} }
] ]
} }
} }
mock_get.return_value.json.return_value = mock_response mock_get.return_value = _mock_response(json_data=api_response)
result = brave_tool.run(query="test") assert web_tool.raw is False
data = json.loads(result) result = web_tool._run(query="crewai")
assert isinstance(data, list)
assert len(data) >= 1 # The web tool's _refine_response extracts and reshapes results.
assert data[0]["title"] == "Test Title" # The key assertion: we should NOT get back the raw API envelope.
assert data[0]["url"] == "http://test.com" assert result != api_response
@patch("requests.get") # Backward Compatibility & Legacy Parameter Support
def test_brave_tool(mock_get):
mock_response = {
"web": {
"results": [
{
"title": "Brave Browser",
"url": "https://brave.com",
"description": "Brave Browser description",
}
]
}
}
mock_get.return_value.json.return_value = mock_response
tool = BraveSearchTool(n_results=2)
result = tool.run(query="Brave Browser")
assert result is not None
# Parse JSON so we can examine the structure
data = json.loads(result)
assert isinstance(data, list)
assert len(data) >= 1
# First item should have expected fields: title, url, and description
first = data[0]
assert "title" in first
assert first["title"] == "Brave Browser"
assert "url" in first
assert first["url"] == "https://brave.com"
assert "description" in first
assert first["description"] == "Brave Browser description"
if __name__ == "__main__": @patch("crewai_tools.tools.brave_search_tool.base.requests.get")
test_brave_tool() def test_positional_query_argument(mock_get, web_tool):
test_brave_tool_initialization() """tool.run('my query') works as a positional argument."""
# test_brave_tool_search(brave_tool) mock_get.return_value = _mock_response(json_data={})
web_tool._run("positional test")
assert mock_get.call_args.kwargs["params"]["q"] == "positional test"
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_search_query_backward_compat(mock_get, web_tool):
"""The legacy 'search_query' param is mapped to 'query'."""
mock_get.return_value = _mock_response(json_data={})
web_tool._run(search_query="legacy test")
assert mock_get.call_args.kwargs["params"]["q"] == "legacy test"
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
@patch("crewai_tools.tools.brave_search_tool.base._save_results_to_file")
def test_save_file_called_when_enabled(mock_save, mock_get):
mock_get.return_value = _mock_response(json_data={"results": []})
tool = BraveWebSearchTool(save_file=True)
tool._run(query="test")
mock_save.assert_called_once()
# Error Handling
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_connection_error_raises_runtime_error(mock_get, web_tool):
mock_get.side_effect = requests_lib.exceptions.ConnectionError("refused")
with pytest.raises(RuntimeError, match="Brave Search API connection failed"):
web_tool._run(query="test")
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_timeout_raises_runtime_error(mock_get, web_tool):
mock_get.side_effect = requests_lib.exceptions.Timeout("timed out")
with pytest.raises(RuntimeError, match="timed out"):
web_tool._run(query="test")
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_invalid_params_raises_value_error(mock_get, web_tool):
"""count=999 exceeds WebSearchParams.count le=20."""
with pytest.raises(ValueError, match="Invalid parameters"):
web_tool._run(query="test", count=999)
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_4xx_error_raises_with_api_detail(mock_get, web_tool):
"""A 422 with a structured error body includes code and detail in the message."""
mock_get.return_value = _mock_response(
status_code=422,
json_data={
"error": {
"id": "abc-123",
"status": 422,
"code": "OPTION_NOT_IN_PLAN",
"detail": "extra_snippets requires a Pro plan",
}
},
)
with pytest.raises(RuntimeError, match="OPTION_NOT_IN_PLAN") as exc_info:
web_tool._run(query="test")
assert "extra_snippets requires a Pro plan" in str(exc_info.value)
assert "HTTP 422" in str(exc_info.value)
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_auth_error_raises_immediately(mock_get, web_tool):
"""A 401 with SUBSCRIPTION_TOKEN_INVALID is not retried."""
mock_get.return_value = _mock_response(
status_code=401,
json_data={
"error": {
"id": "xyz",
"status": 401,
"code": "SUBSCRIPTION_TOKEN_INVALID",
"detail": "The subscription token is invalid",
}
},
)
with pytest.raises(RuntimeError, match="SUBSCRIPTION_TOKEN_INVALID"):
web_tool._run(query="test")
# Should NOT have retried — only one call.
assert mock_get.call_count == 1
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_quota_limited_429_raises_immediately(mock_get, web_tool):
"""A 429 with QUOTA_LIMITED is NOT retried — quota exhaustion is terminal."""
mock_get.return_value = _mock_response(
status_code=429,
json_data={
"error": {
"id": "ql-1",
"status": 429,
"code": "QUOTA_LIMITED",
"detail": "Monthly quota exceeded",
}
},
)
with pytest.raises(RuntimeError, match="QUOTA_LIMITED") as exc_info:
web_tool._run(query="test")
assert "Monthly quota exceeded" in str(exc_info.value)
# Terminal — only one HTTP call, no retries.
assert mock_get.call_count == 1
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_usage_limit_exceeded_429_raises_immediately(mock_get, web_tool):
"""USAGE_LIMIT_EXCEEDED is also non-retryable, just like QUOTA_LIMITED."""
mock_get.return_value = _mock_response(
status_code=429,
json_data={
"error": {
"id": "ule-1",
"status": 429,
"code": "USAGE_LIMIT_EXCEEDED",
}
},
text="usage limit exceeded",
)
with pytest.raises(RuntimeError, match="USAGE_LIMIT_EXCEEDED"):
web_tool._run(query="test")
assert mock_get.call_count == 1
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_error_body_is_fully_included_in_message(mock_get, web_tool):
"""The full JSON error body is included in the RuntimeError message."""
mock_get.return_value = _mock_response(
status_code=429,
json_data={
"error": {
"id": "x",
"status": 429,
"code": "QUOTA_LIMITED",
"detail": "Exceeded",
"meta": {"plan": "free", "limit": 1000},
}
},
)
with pytest.raises(RuntimeError) as exc_info:
web_tool._run(query="test")
msg = str(exc_info.value)
assert "HTTP 429" in msg
assert "QUOTA_LIMITED" in msg
assert "free" in msg
assert "1000" in msg
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_error_without_json_body_falls_back_to_text(mock_get, web_tool):
"""When the error response isn't valid JSON, resp.text is used as the detail."""
resp = _mock_response(status_code=500, text="Internal Server Error")
resp.json.side_effect = ValueError("No JSON")
mock_get.return_value = resp
with pytest.raises(RuntimeError, match="Internal Server Error"):
web_tool._run(query="test")
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
def test_invalid_json_on_success_raises_runtime_error(mock_get, web_tool):
"""A 200 OK with a non-JSON body raises RuntimeError."""
resp = _mock_response(status_code=200)
resp.json.side_effect = ValueError("Expecting value")
mock_get.return_value = resp
with pytest.raises(RuntimeError, match="invalid JSON"):
web_tool._run(query="test")
# Rate Limiting
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
@patch("crewai_tools.tools.brave_search_tool.base.time")
def test_rate_limit_sleeps_when_too_fast(mock_time, mock_get, web_tool):
"""Back-to-back calls within the interval trigger a sleep."""
mock_get.return_value = _mock_response(json_data={})
# Simulate: last request was at t=100, "now" is t=100.2 (only 0.2s elapsed).
# With default 1 req/s the min interval is 1.0s, so it should sleep ~0.8s.
mock_time.time.return_value = 100.2
web_tool._last_request_time = 100.0
web_tool._run(query="test")
mock_time.sleep.assert_called_once()
sleep_duration = mock_time.sleep.call_args[0][0]
assert 0.7 < sleep_duration < 0.9 # approximately 0.8s
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
@patch("crewai_tools.tools.brave_search_tool.base.time")
def test_rate_limit_skips_sleep_when_enough_time_passed(mock_time, mock_get, web_tool):
"""No sleep when the elapsed time already exceeds the interval."""
mock_get.return_value = _mock_response(json_data={})
# Last request was at t=100, "now" is t=102 (2s elapsed > 1s interval).
mock_time.time.return_value = 102.0
web_tool._last_request_time = 100.0
web_tool._run(query="test")
mock_time.sleep.assert_not_called()
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
@patch("crewai_tools.tools.brave_search_tool.base.time")
def test_rate_limit_disabled_when_zero(mock_time, mock_get, web_tool):
"""requests_per_second=0 disables rate limiting entirely."""
mock_get.return_value = _mock_response(json_data={})
web_tool._last_request_time = 100.0
mock_time.time.return_value = 100.0 # same instant
web_tool._run(query="test")
mock_time.sleep.assert_not_called()
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
@patch("crewai_tools.tools.brave_search_tool.base.time")
def test_rate_limit_per_instance_independent(mock_time, mock_get, web_tool, image_tool):
"""Each instance has its own rate-limit clock; a request on one does not delay the other."""
mock_get.return_value = _mock_response(json_data={})
# Web tool fires at t=100 (its clock goes 0 -> 100).
mock_time.time.return_value = 100.0
web_tool._run(query="test")
# Image tool fires at t=100.3. Its clock is still 0 (separate instance), so
# next_allowed = 1.0 and 100.3 > 1.0 — no sleep. Total process rate can be sum of instance limits.
mock_time.time.return_value = 100.3
image_tool._run(query="cats")
mock_time.sleep.assert_not_called()
# Retry Behavior
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
@patch("crewai_tools.tools.brave_search_tool.base.time")
def test_429_rate_limited_retries_then_succeeds(mock_time, mock_get, web_tool):
"""A transient RATE_LIMITED 429 is retried; success on the second attempt."""
mock_time.time.return_value = 200.0
resp_429 = _mock_response(
status_code=429,
json_data={"error": {"id": "r", "status": 429, "code": "RATE_LIMITED"}},
headers={"Retry-After": "2"},
)
resp_200 = _mock_response(status_code=200, json_data={"web": {"results": []}})
mock_get.side_effect = [resp_429, resp_200]
web_tool.raw = True
result = web_tool._run(query="test")
assert result == {"web": {"results": []}}
assert mock_get.call_count == 2
# Slept for the Retry-After value.
retry_sleeps = [c for c in mock_time.sleep.call_args_list if c[0][0] == 2.0]
assert len(retry_sleeps) == 1
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
@patch("crewai_tools.tools.brave_search_tool.base.time")
def test_5xx_is_retried(mock_time, mock_get, web_tool):
"""A 502 server error is retried; success on the second attempt."""
mock_time.time.return_value = 200.0
resp_502 = _mock_response(status_code=502, text="Bad Gateway")
resp_502.json.side_effect = ValueError("no json")
resp_200 = _mock_response(status_code=200, json_data={"web": {"results": []}})
mock_get.side_effect = [resp_502, resp_200]
web_tool.raw = True
result = web_tool._run(query="test")
assert result == {"web": {"results": []}}
assert mock_get.call_count == 2
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
@patch("crewai_tools.tools.brave_search_tool.base.time")
def test_429_rate_limited_exhausts_retries(mock_time, mock_get, web_tool):
"""Persistent RATE_LIMITED 429s exhaust retries and raise RuntimeError."""
mock_time.time.return_value = 200.0
resp_429 = _mock_response(
status_code=429,
json_data={"error": {"id": "r", "status": 429, "code": "RATE_LIMITED"}},
)
mock_get.return_value = resp_429
with pytest.raises(RuntimeError, match="RATE_LIMITED"):
web_tool._run(query="test")
# 3 attempts (default _max_retries).
assert mock_get.call_count == 3
@patch("crewai_tools.tools.brave_search_tool.base.requests.get")
@patch("crewai_tools.tools.brave_search_tool.base.time")
def test_retry_uses_exponential_backoff_when_no_retry_after(
mock_time, mock_get, web_tool
):
"""Without Retry-After, backoff is 2^attempt (1s, 2s, ...)."""
mock_time.time.return_value = 200.0
resp_503 = _mock_response(status_code=503, text="Service Unavailable")
resp_503.json.side_effect = ValueError("no json")
resp_200 = _mock_response(status_code=200, json_data={"ok": True})
mock_get.side_effect = [resp_503, resp_503, resp_200]
web_tool.raw = True
web_tool._run(query="test")
# Two retries: attempt 0 → sleep(1.0), attempt 1 → sleep(2.0).
retry_sleeps = [c[0][0] for c in mock_time.sleep.call_args_list]
assert 1.0 in retry_sleeps
assert 2.0 in retry_sleeps

File diff suppressed because it is too large Load Diff

View File

@@ -21,7 +21,7 @@ dependencies = [
"opentelemetry-exporter-otlp-proto-http~=1.34.0", "opentelemetry-exporter-otlp-proto-http~=1.34.0",
# Data Handling # Data Handling
"chromadb~=1.1.0", "chromadb~=1.1.0",
"tokenizers~=0.20.3", "tokenizers>=0.21,<1",
"openpyxl~=3.1.5", "openpyxl~=3.1.5",
# Authentication and Security # Authentication and Security
"python-dotenv~=1.1.1", "python-dotenv~=1.1.1",
@@ -53,7 +53,7 @@ Repository = "https://github.com/crewAIInc/crewAI"
[project.optional-dependencies] [project.optional-dependencies]
tools = [ tools = [
"crewai-tools==1.10.1a1", "crewai-tools==1.10.2rc2",
] ]
embeddings = [ embeddings = [
"tiktoken~=0.8.0" "tiktoken~=0.8.0"
@@ -88,7 +88,7 @@ bedrock = [
"boto3~=1.40.45", "boto3~=1.40.45",
] ]
google-genai = [ google-genai = [
"google-genai~=1.49.0", "google-genai~=1.65.0",
] ]
azure-ai-inference = [ azure-ai-inference = [
"azure-ai-inference~=1.0.0b9", "azure-ai-inference~=1.0.0b9",

View File

@@ -1,3 +1,4 @@
import contextvars
import threading import threading
from typing import Any from typing import Any
import urllib.request import urllib.request
@@ -40,7 +41,7 @@ def _suppress_pydantic_deprecation_warnings() -> None:
_suppress_pydantic_deprecation_warnings() _suppress_pydantic_deprecation_warnings()
__version__ = "1.10.1a1" __version__ = "1.10.2rc2"
_telemetry_submitted = False _telemetry_submitted = False
@@ -66,7 +67,8 @@ def _track_install() -> None:
def _track_install_async() -> None: def _track_install_async() -> None:
"""Track installation in background thread to avoid blocking imports.""" """Track installation in background thread to avoid blocking imports."""
if not Telemetry._is_telemetry_disabled(): if not Telemetry._is_telemetry_disabled():
thread = threading.Thread(target=_track_install, daemon=True) ctx = contextvars.copy_context()
thread = threading.Thread(target=ctx.run, args=(_track_install,), daemon=True)
thread.start() thread.start()

View File

@@ -4,6 +4,8 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import MutableMapping from collections.abc import MutableMapping
import concurrent.futures
import contextvars
from functools import lru_cache from functools import lru_cache
import ssl import ssl
import time import time
@@ -138,14 +140,18 @@ def fetch_agent_card(
ttl_hash = int(time.time() // cache_ttl) ttl_hash = int(time.time() // cache_ttl)
return _fetch_agent_card_cached(endpoint, auth_hash, timeout, ttl_hash) return _fetch_agent_card_cached(endpoint, auth_hash, timeout, ttl_hash)
loop = asyncio.new_event_loop() coro = afetch_agent_card(endpoint=endpoint, auth=auth, timeout=timeout)
asyncio.set_event_loop(loop)
try: try:
return loop.run_until_complete( asyncio.get_running_loop()
afetch_agent_card(endpoint=endpoint, auth=auth, timeout=timeout) has_running_loop = True
) except RuntimeError:
finally: has_running_loop = False
loop.close()
if has_running_loop:
ctx = contextvars.copy_context()
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
return pool.submit(ctx.run, asyncio.run, coro).result()
return asyncio.run(coro)
async def afetch_agent_card( async def afetch_agent_card(
@@ -203,14 +209,18 @@ def _fetch_agent_card_cached(
"""Cached sync version of fetch_agent_card.""" """Cached sync version of fetch_agent_card."""
auth = _auth_store.get(auth_hash) auth = _auth_store.get(auth_hash)
loop = asyncio.new_event_loop() coro = _afetch_agent_card_impl(endpoint=endpoint, auth=auth, timeout=timeout)
asyncio.set_event_loop(loop)
try: try:
return loop.run_until_complete( asyncio.get_running_loop()
_afetch_agent_card_impl(endpoint=endpoint, auth=auth, timeout=timeout) has_running_loop = True
) except RuntimeError:
finally: has_running_loop = False
loop.close()
if has_running_loop:
ctx = contextvars.copy_context()
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
return pool.submit(ctx.run, asyncio.run, coro).result()
return asyncio.run(coro)
@cached(ttl=300, serializer=PickleSerializer()) # type: ignore[untyped-decorator] @cached(ttl=300, serializer=PickleSerializer()) # type: ignore[untyped-decorator]

View File

@@ -5,7 +5,9 @@ from __future__ import annotations
import asyncio import asyncio
import base64 import base64
from collections.abc import AsyncIterator, Callable, MutableMapping from collections.abc import AsyncIterator, Callable, MutableMapping
import concurrent.futures
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
import contextvars
import logging import logging
from typing import TYPE_CHECKING, Any, Final, Literal from typing import TYPE_CHECKING, Any, Final, Literal
import uuid import uuid
@@ -194,56 +196,44 @@ def execute_a2a_delegation(
Returns: Returns:
TaskStateResult with status, result/error, history, and agent_card. TaskStateResult with status, result/error, history, and agent_card.
Raises:
RuntimeError: If called from an async context with a running event loop.
""" """
coro = aexecute_a2a_delegation(
endpoint=endpoint,
auth=auth,
timeout=timeout,
task_description=task_description,
context=context,
context_id=context_id,
task_id=task_id,
reference_task_ids=reference_task_ids,
metadata=metadata,
extensions=extensions,
conversation_history=conversation_history,
agent_id=agent_id,
agent_role=agent_role,
agent_branch=agent_branch,
response_model=response_model,
turn_number=turn_number,
updates=updates,
from_task=from_task,
from_agent=from_agent,
skill_id=skill_id,
client_extensions=client_extensions,
transport=transport,
accepted_output_modes=accepted_output_modes,
input_files=input_files,
)
try: try:
asyncio.get_running_loop() asyncio.get_running_loop()
raise RuntimeError( has_running_loop = True
"execute_a2a_delegation() cannot be called from an async context. " except RuntimeError:
"Use 'await aexecute_a2a_delegation()' instead." has_running_loop = False
)
except RuntimeError as e:
if "no running event loop" not in str(e).lower():
raise
loop = asyncio.new_event_loop() if has_running_loop:
asyncio.set_event_loop(loop) ctx = contextvars.copy_context()
try: with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
return loop.run_until_complete( return pool.submit(ctx.run, asyncio.run, coro).result()
aexecute_a2a_delegation( return asyncio.run(coro)
endpoint=endpoint,
auth=auth,
timeout=timeout,
task_description=task_description,
context=context,
context_id=context_id,
task_id=task_id,
reference_task_ids=reference_task_ids,
metadata=metadata,
extensions=extensions,
conversation_history=conversation_history,
agent_id=agent_id,
agent_role=agent_role,
agent_branch=agent_branch,
response_model=response_model,
turn_number=turn_number,
updates=updates,
from_task=from_task,
from_agent=from_agent,
skill_id=skill_id,
client_extensions=client_extensions,
transport=transport,
accepted_output_modes=accepted_output_modes,
input_files=input_files,
)
)
finally:
try:
loop.run_until_complete(loop.shutdown_asyncgens())
finally:
loop.close()
async def aexecute_a2a_delegation( async def aexecute_a2a_delegation(

View File

@@ -8,6 +8,7 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import Callable, Coroutine, Mapping from collections.abc import Callable, Coroutine, Mapping
from concurrent.futures import ThreadPoolExecutor, as_completed from concurrent.futures import ThreadPoolExecutor, as_completed
import contextvars
from functools import wraps from functools import wraps
import json import json
from types import MethodType from types import MethodType
@@ -278,7 +279,9 @@ def _fetch_agent_cards_concurrently(
max_workers = min(len(a2a_agents), 10) max_workers = min(len(a2a_agents), 10)
with ThreadPoolExecutor(max_workers=max_workers) as executor: with ThreadPoolExecutor(max_workers=max_workers) as executor:
futures = { futures = {
executor.submit(_fetch_card_from_config, config): config executor.submit(
contextvars.copy_context().run, _fetch_card_from_config, config
): config
for config in a2a_agents for config in a2a_agents
} }
for future in as_completed(futures): for future in as_completed(futures):

View File

@@ -2,6 +2,7 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import Callable, Coroutine, Sequence from collections.abc import Callable, Coroutine, Sequence
import contextvars
import shutil import shutil
import subprocess import subprocess
import time import time
@@ -513,9 +514,13 @@ class Agent(BaseAgent):
""" """
import concurrent.futures import concurrent.futures
ctx = contextvars.copy_context()
with concurrent.futures.ThreadPoolExecutor() as executor: with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit( future = executor.submit(
self._execute_without_timeout, task_prompt=task_prompt, task=task ctx.run,
self._execute_without_timeout,
task_prompt=task_prompt,
task=task,
) )
try: try:
@@ -1156,11 +1161,15 @@ class Agent(BaseAgent):
# Process platform apps and MCP tools # Process platform apps and MCP tools
if self.apps: if self.apps:
platform_tools = self.get_platform_tools(self.apps) platform_tools = self.get_platform_tools(self.apps)
if platform_tools and self.tools is not None: if platform_tools:
if self.tools is None:
self.tools = []
self.tools.extend(platform_tools) self.tools.extend(platform_tools)
if self.mcps: if self.mcps:
mcps = self.get_mcp_tools(self.mcps) mcps = self.get_mcp_tools(self.mcps)
if mcps and self.tools is not None: if mcps:
if self.tools is None:
self.tools = []
self.tools.extend(mcps) self.tools.extend(mcps)
# Prepare tools # Prepare tools
@@ -1264,7 +1273,7 @@ class Agent(BaseAgent):
), ),
) )
start_time = time.time() start_time = time.time()
matches = agent_memory.recall(formatted_messages, limit=5) matches = agent_memory.recall(formatted_messages, limit=20)
memory_block = "" memory_block = ""
if matches: if matches:
memory_block = "Relevant memories:\n" + "\n".join( memory_block = "Relevant memories:\n" + "\n".join(

View File

@@ -38,7 +38,7 @@ from crewai.utilities.string_utils import interpolate_only
_SLUG_RE: Final[re.Pattern[str]] = re.compile( _SLUG_RE: Final[re.Pattern[str]] = re.compile(
r"^(?:crewai-amp:)?[a-zA-Z0-9][a-zA-Z0-9_-]*(?:#\w+)?$" r"^(?:crewai-amp:)?[a-zA-Z0-9][a-zA-Z0-9_-]*(?:#[\w-]+)?$"
) )

View File

@@ -30,12 +30,9 @@ class CrewAgentExecutorMixin:
memory = getattr(self.agent, "memory", None) or ( memory = getattr(self.agent, "memory", None) or (
getattr(self.crew, "_memory", None) if self.crew else None getattr(self.crew, "_memory", None) if self.crew else None
) )
if memory is None or not self.task or getattr(memory, "_read_only", False): if memory is None or not self.task or memory.read_only:
return return
if ( if f"Action: {sanitize_tool_name('Delegate work to coworker')}" in output.text:
f"Action: {sanitize_tool_name('Delegate work to coworker')}"
in output.text
):
return return
try: try:
raw = ( raw = (
@@ -48,6 +45,4 @@ class CrewAgentExecutorMixin:
if extracted: if extracted:
memory.remember_many(extracted, agent_role=self.agent.role) memory.remember_many(extracted, agent_role=self.agent.role)
except Exception as e: except Exception as e:
self.agent._logger.log( self.agent._logger.log("error", f"Failed to save to memory: {e}")
"error", f"Failed to save to memory: {e}"
)

View File

@@ -1,5 +1,4 @@
from crewai.agents.cache.cache_handler import CacheHandler from crewai.agents.cache.cache_handler import CacheHandler
__all__ = ["CacheHandler"] __all__ = ["CacheHandler"]

View File

@@ -9,6 +9,7 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import Callable from collections.abc import Callable
from concurrent.futures import ThreadPoolExecutor, as_completed from concurrent.futures import ThreadPoolExecutor, as_completed
import contextvars
import inspect import inspect
import logging import logging
from typing import TYPE_CHECKING, Any, Literal, cast from typing import TYPE_CHECKING, Any, Literal, cast
@@ -755,6 +756,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
with ThreadPoolExecutor(max_workers=max_workers) as pool: with ThreadPoolExecutor(max_workers=max_workers) as pool:
futures = { futures = {
pool.submit( pool.submit(
contextvars.copy_context().run,
self._execute_single_native_tool_call, self._execute_single_native_tool_call,
call_id=call_id, call_id=call_id,
func_name=func_name, func_name=func_name,
@@ -893,7 +895,9 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
ToolUsageStartedEvent, ToolUsageStartedEvent,
) )
args_dict, parse_error = parse_tool_call_args(func_args, func_name, call_id, original_tool) args_dict, parse_error = parse_tool_call_args(
func_args, func_name, call_id, original_tool
)
if parse_error is not None: if parse_error is not None:
return parse_error return parse_error

View File

@@ -1,5 +1,4 @@
from crewai.cli.authentication.main import AuthenticationCommand from crewai.cli.authentication.main import AuthenticationCommand
__all__ = ["AuthenticationCommand"] __all__ = ["AuthenticationCommand"]

View File

@@ -182,15 +182,24 @@ def log_tasks_outputs() -> None:
@crewai.command() @crewai.command()
@click.option("-m", "--memory", is_flag=True, help="Reset MEMORY") @click.option("-m", "--memory", is_flag=True, help="Reset MEMORY")
@click.option( @click.option(
"-l", "--long", is_flag=True, hidden=True, "-l",
"--long",
is_flag=True,
hidden=True,
help="[Deprecated: use --memory] Reset memory", help="[Deprecated: use --memory] Reset memory",
) )
@click.option( @click.option(
"-s", "--short", is_flag=True, hidden=True, "-s",
"--short",
is_flag=True,
hidden=True,
help="[Deprecated: use --memory] Reset memory", help="[Deprecated: use --memory] Reset memory",
) )
@click.option( @click.option(
"-e", "--entities", is_flag=True, hidden=True, "-e",
"--entities",
is_flag=True,
hidden=True,
help="[Deprecated: use --memory] Reset memory", help="[Deprecated: use --memory] Reset memory",
) )
@click.option("-kn", "--knowledge", is_flag=True, help="Reset KNOWLEDGE storage") @click.option("-kn", "--knowledge", is_flag=True, help="Reset KNOWLEDGE storage")
@@ -218,7 +227,13 @@ def reset_memories(
# Treat legacy flags as --memory with a deprecation warning # Treat legacy flags as --memory with a deprecation warning
if long or short or entities: if long or short or entities:
legacy_used = [ legacy_used = [
f for f, v in [("--long", long), ("--short", short), ("--entities", entities)] if v f
for f, v in [
("--long", long),
("--short", short),
("--entities", entities),
]
if v
] ]
click.echo( click.echo(
f"Warning: {', '.join(legacy_used)} {'is' if len(legacy_used) == 1 else 'are'} " f"Warning: {', '.join(legacy_used)} {'is' if len(legacy_used) == 1 else 'are'} "
@@ -238,9 +253,7 @@ def reset_memories(
"Please specify at least one memory type to reset using the appropriate flags." "Please specify at least one memory type to reset using the appropriate flags."
) )
return return
reset_memories_command( reset_memories_command(memory, knowledge, agent_knowledge, kickoff_outputs, all)
memory, knowledge, agent_knowledge, kickoff_outputs, all
)
except Exception as e: except Exception as e:
click.echo(f"An error occurred while resetting memories: {e}", err=True) click.echo(f"An error occurred while resetting memories: {e}", err=True)
@@ -669,18 +682,11 @@ def traces_enable():
from rich.console import Console from rich.console import Console
from rich.panel import Panel from rich.panel import Panel
from crewai.events.listeners.tracing.utils import ( from crewai.events.listeners.tracing.utils import update_user_data
_load_user_data,
_save_user_data,
)
console = Console() console = Console()
# Update user data to enable traces update_user_data({"trace_consent": True, "first_execution_done": True})
user_data = _load_user_data()
user_data["trace_consent"] = True
user_data["first_execution_done"] = True
_save_user_data(user_data)
panel = Panel( panel = Panel(
"✅ Trace collection has been enabled!\n\n" "✅ Trace collection has been enabled!\n\n"
@@ -699,18 +705,11 @@ def traces_disable():
from rich.console import Console from rich.console import Console
from rich.panel import Panel from rich.panel import Panel
from crewai.events.listeners.tracing.utils import ( from crewai.events.listeners.tracing.utils import update_user_data
_load_user_data,
_save_user_data,
)
console = Console() console = Console()
# Update user data to disable traces update_user_data({"trace_consent": False, "first_execution_done": True})
user_data = _load_user_data()
user_data["trace_consent"] = False
user_data["first_execution_done"] = True
_save_user_data(user_data)
panel = Panel( panel = Panel(
"❌ Trace collection has been disabled!\n\n" "❌ Trace collection has been disabled!\n\n"

View File

@@ -143,7 +143,7 @@ def create_folder_structure(
(folder_path / "src" / folder_name).mkdir(parents=True) (folder_path / "src" / folder_name).mkdir(parents=True)
(folder_path / "src" / folder_name / "tools").mkdir(parents=True) (folder_path / "src" / folder_name / "tools").mkdir(parents=True)
(folder_path / "src" / folder_name / "config").mkdir(parents=True) (folder_path / "src" / folder_name / "config").mkdir(parents=True)
# Copy AGENTS.md to project root (top-level projects only) # Copy AGENTS.md to project root (top-level projects only)
package_dir = Path(__file__).parent package_dir = Path(__file__).parent
agents_md_src = package_dir / "templates" / "AGENTS.md" agents_md_src = package_dir / "templates" / "AGENTS.md"

View File

@@ -1,5 +1,5 @@
import shutil
from pathlib import Path from pathlib import Path
import shutil
import click import click

View File

@@ -1,3 +1,4 @@
import contextvars
import json import json
from pathlib import Path from pathlib import Path
import platform import platform
@@ -80,7 +81,10 @@ def run_chat() -> None:
# Start loading indicator # Start loading indicator
loading_complete = threading.Event() loading_complete = threading.Event()
loading_thread = threading.Thread(target=show_loading, args=(loading_complete,)) ctx = contextvars.copy_context()
loading_thread = threading.Thread(
target=ctx.run, args=(show_loading, loading_complete)
)
loading_thread.start() loading_thread.start()
try: try:

View File

@@ -125,13 +125,19 @@ class MemoryTUI(App[None]):
from crewai.memory.storage.lancedb_storage import LanceDBStorage from crewai.memory.storage.lancedb_storage import LanceDBStorage
from crewai.memory.unified_memory import Memory from crewai.memory.unified_memory import Memory
storage = LanceDBStorage(path=storage_path) if storage_path else LanceDBStorage() storage = (
LanceDBStorage(path=storage_path) if storage_path else LanceDBStorage()
)
embedder = None embedder = None
if embedder_config is not None: if embedder_config is not None:
from crewai.rag.embeddings.factory import build_embedder from crewai.rag.embeddings.factory import build_embedder
embedder = build_embedder(embedder_config) embedder = build_embedder(embedder_config)
self._memory = Memory(storage=storage, embedder=embedder) if embedder else Memory(storage=storage) self._memory = (
Memory(storage=storage, embedder=embedder)
if embedder
else Memory(storage=storage)
)
except Exception as e: except Exception as e:
self._init_error = str(e) self._init_error = str(e)
@@ -200,11 +206,7 @@ class MemoryTUI(App[None]):
if len(record.content) > 80 if len(record.content) > 80
else record.content else record.content
) )
label = ( label = f"{date_str} [bold]{record.importance:.1f}[/] {preview}"
f"{date_str} "
f"[bold]{record.importance:.1f}[/] "
f"{preview}"
)
option_list.add_option(label) option_list.add_option(label)
def _populate_recall_list(self) -> None: def _populate_recall_list(self) -> None:
@@ -220,9 +222,7 @@ class MemoryTUI(App[None]):
else m.record.content else m.record.content
) )
label = ( label = (
f"[bold]\\[{m.score:.2f}][/] " f"[bold]\\[{m.score:.2f}][/] {preview} [dim]scope={m.record.scope}[/]"
f"{preview} "
f"[dim]scope={m.record.scope}[/]"
) )
option_list.add_option(label) option_list.add_option(label)
@@ -251,8 +251,7 @@ class MemoryTUI(App[None]):
lines.append(f"[dim]Scope:[/] [bold]{record.scope}[/]") lines.append(f"[dim]Scope:[/] [bold]{record.scope}[/]")
lines.append(f"[dim]Importance:[/] [bold]{record.importance:.2f}[/]") lines.append(f"[dim]Importance:[/] [bold]{record.importance:.2f}[/]")
lines.append( lines.append(
f"[dim]Created:[/] " f"[dim]Created:[/] {record.created_at.strftime('%Y-%m-%d %H:%M:%S')}"
f"{record.created_at.strftime('%Y-%m-%d %H:%M:%S')}"
) )
lines.append( lines.append(
f"[dim]Last accessed:[/] " f"[dim]Last accessed:[/] "
@@ -362,17 +361,11 @@ class MemoryTUI(App[None]):
panel = self.query_one("#info-panel", Static) panel = self.query_one("#info-panel", Static)
panel.loading = True panel.loading = True
try: try:
scope = ( scope = self._selected_scope if self._selected_scope != "/" else None
self._selected_scope
if self._selected_scope != "/"
else None
)
loop = asyncio.get_event_loop() loop = asyncio.get_event_loop()
matches = await loop.run_in_executor( matches = await loop.run_in_executor(
None, None,
lambda: self._memory.recall( lambda: self._memory.recall(query, scope=scope, limit=10, depth="deep"),
query, scope=scope, limit=10, depth="deep"
),
) )
self._recall_matches = matches or [] self._recall_matches = matches or []
self._view_mode = "recall" self._view_mode = "recall"

View File

@@ -22,14 +22,15 @@ class PlusAPI:
EPHEMERAL_TRACING_RESOURCE = "/crewai_plus/api/v1/tracing/ephemeral" EPHEMERAL_TRACING_RESOURCE = "/crewai_plus/api/v1/tracing/ephemeral"
INTEGRATIONS_RESOURCE = "/crewai_plus/api/v1/integrations" INTEGRATIONS_RESOURCE = "/crewai_plus/api/v1/integrations"
def __init__(self, api_key: str) -> None: def __init__(self, api_key: str | None = None) -> None:
self.api_key = api_key self.api_key = api_key
self.headers = { self.headers = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json", "Content-Type": "application/json",
"User-Agent": f"CrewAI-CLI/{get_crewai_version()}", "User-Agent": f"CrewAI-CLI/{get_crewai_version()}",
"X-Crewai-Version": get_crewai_version(), "X-Crewai-Version": get_crewai_version(),
} }
if api_key:
self.headers["Authorization"] = f"Bearer {api_key}"
settings = Settings() settings = Settings()
if settings.org_uuid: if settings.org_uuid:
self.headers["X-Crewai-Organization-Id"] = settings.org_uuid self.headers["X-Crewai-Organization-Id"] = settings.org_uuid
@@ -48,8 +49,13 @@ class PlusAPI:
with httpx.Client(trust_env=False, verify=verify) as client: with httpx.Client(trust_env=False, verify=verify) as client:
return client.request(method, url, headers=self.headers, **kwargs) return client.request(method, url, headers=self.headers, **kwargs)
def login_to_tool_repository(self) -> httpx.Response: def login_to_tool_repository(
return self._make_request("POST", f"{self.TOOLS_RESOURCE}/login") self, user_identifier: str | None = None
) -> httpx.Response:
payload = {}
if user_identifier:
payload["user_identifier"] = user_identifier
return self._make_request("POST", f"{self.TOOLS_RESOURCE}/login", json=payload)
def get_tool(self, handle: str) -> httpx.Response: def get_tool(self, handle: str) -> httpx.Response:
return self._make_request("GET", f"{self.TOOLS_RESOURCE}/{handle}") return self._make_request("GET", f"{self.TOOLS_RESOURCE}/{handle}")

View File

@@ -95,9 +95,7 @@ def reset_memories_command(
continue continue
if memory: if memory:
_reset_flow_memory(flow) _reset_flow_memory(flow)
click.echo( click.echo(f"[Flow ({flow_name})] Memory has been reset.")
f"[Flow ({flow_name})] Memory has been reset."
)
except subprocess.CalledProcessError as e: except subprocess.CalledProcessError as e:
click.echo(f"An error occurred while resetting the memories: {e}", err=True) click.echo(f"An error occurred while resetting the memories: {e}", err=True)

View File

@@ -5,7 +5,7 @@ description = "{{name}} using crewAI"
authors = [{ name = "Your Name", email = "you@example.com" }] authors = [{ name = "Your Name", email = "you@example.com" }]
requires-python = ">=3.10,<3.14" requires-python = ">=3.10,<3.14"
dependencies = [ dependencies = [
"crewai[tools]==1.10.1a1" "crewai[tools]==1.10.2rc2"
] ]
[project.scripts] [project.scripts]

View File

@@ -5,7 +5,7 @@ description = "{{name}} using crewAI"
authors = [{ name = "Your Name", email = "you@example.com" }] authors = [{ name = "Your Name", email = "you@example.com" }]
requires-python = ">=3.10,<3.14" requires-python = ">=3.10,<3.14"
dependencies = [ dependencies = [
"crewai[tools]==1.10.1a1" "crewai[tools]==1.10.2rc2"
] ]
[project.scripts] [project.scripts]

View File

@@ -5,7 +5,7 @@ description = "Power up your crews with {{folder_name}}"
readme = "README.md" readme = "README.md"
requires-python = ">=3.10,<3.14" requires-python = ">=3.10,<3.14"
dependencies = [ dependencies = [
"crewai[tools]==1.10.1a1" "crewai[tools]==1.10.2rc2"
] ]
[tool.crewai] [tool.crewai]

View File

@@ -23,6 +23,7 @@ from crewai.cli.utils import (
tree_copy, tree_copy,
tree_find_and_replace, tree_find_and_replace,
) )
from crewai.events.listeners.tracing.utils import get_user_id
console = Console() console = Console()
@@ -169,7 +170,9 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
console.print(f"Successfully installed {handle}", style="bold green") console.print(f"Successfully installed {handle}", style="bold green")
def login(self) -> None: def login(self) -> None:
login_response = self.plus_api_client.login_to_tool_repository() login_response = self.plus_api_client.login_to_tool_repository(
user_identifier=get_user_id()
)
if login_response.status_code != 200: if login_response.status_code != 200:
console.print( console.print(

View File

@@ -442,9 +442,7 @@ def get_flows(flow_path: str = "main.py") -> list[Flow]:
for search_path in search_paths: for search_path in search_paths:
for root, dirs, files in os.walk(search_path): for root, dirs, files in os.walk(search_path):
dirs[:] = [ dirs[:] = [
d d for d in dirs if d not in _SKIP_DIRS and not d.startswith(".")
for d in dirs
if d not in _SKIP_DIRS and not d.startswith(".")
] ]
if flow_path in files and "cli/templates" not in root: if flow_path in files and "cli/templates" not in root:
file_os_path = os.path.join(root, flow_path) file_os_path = os.path.join(root, flow_path)
@@ -464,9 +462,7 @@ def get_flows(flow_path: str = "main.py") -> list[Flow]:
for attr_name in dir(module): for attr_name in dir(module):
module_attr = getattr(module, attr_name) module_attr = getattr(module, attr_name)
try: try:
if flow_instance := get_flow_instance( if flow_instance := get_flow_instance(module_attr):
module_attr
):
flow_instances.append(flow_instance) flow_instances.append(flow_instance)
except Exception: # noqa: S112 except Exception: # noqa: S112
continue continue

View File

@@ -438,18 +438,6 @@ class Crew(FlowTrackable, BaseModel):
agent.set_rpm_controller(self._rpm_controller) agent.set_rpm_controller(self._rpm_controller)
return self return self
@model_validator(mode="after")
def stamp_execution_order(self) -> Self:
"""Lock task execution order by stamping each task with its insertion index.
This guarantees deterministic, stable ordering for all tasks regardless
of any other attribute. Tasks are always dispatched in the order they
appear in ``self.tasks`` (i.e. insertion order).
"""
for idx, task in enumerate(self.tasks):
task._execution_index = idx
return self
@model_validator(mode="after") @model_validator(mode="after")
def validate_tasks(self) -> Self: def validate_tasks(self) -> Self:
if self.process == Process.sequential: if self.process == Process.sequential:
@@ -996,18 +984,13 @@ class Crew(FlowTrackable, BaseModel):
) -> CrewOutput: ) -> CrewOutput:
"""Executes tasks using native async and returns the final output. """Executes tasks using native async and returns the final output.
**Ordering contract**: tasks are dispatched in the exact order they
appear in *tasks* (i.e. their insertion / list order). Each task
carries an ``_execution_index`` stamped at crew-construction time
that locks this order deterministically.
Args: Args:
tasks: List of tasks to execute (preserves insertion order). tasks: List of tasks to execute
start_index: Index to start execution from (for replay). start_index: Index to start execution from (for replay)
was_replayed: Whether this is a replayed execution. was_replayed: Whether this is a replayed execution
Returns: Returns:
CrewOutput: Final output of the crew. CrewOutput: Final output of the crew
""" """
task_outputs: list[TaskOutput] = [] task_outputs: list[TaskOutput] = []
pending_tasks: list[tuple[Task, asyncio.Task[TaskOutput], int]] = [] pending_tasks: list[tuple[Task, asyncio.Task[TaskOutput], int]] = []
@@ -1200,18 +1183,13 @@ class Crew(FlowTrackable, BaseModel):
) -> CrewOutput: ) -> CrewOutput:
"""Executes tasks sequentially and returns the final output. """Executes tasks sequentially and returns the final output.
**Ordering contract**: tasks are dispatched in the exact order they
appear in *tasks* (i.e. their insertion / list order). Each task
carries an ``_execution_index`` stamped at crew-construction time
that locks this order deterministically.
Args: Args:
tasks: List of tasks to execute (preserves insertion order). tasks (List[Task]): List of tasks to execute
start_index: Index to resume execution from (for replay). manager (Optional[BaseAgent], optional): Manager agent to use for
was_replayed: Whether this is a replayed execution. delegation. Defaults to None.
Returns: Returns:
CrewOutput: Final output of the crew. CrewOutput: Final output of the crew
""" """
custom_start = self._get_execution_start_index(tasks) custom_start = self._get_execution_start_index(tasks)
if custom_start is not None: if custom_start is not None:
@@ -1432,9 +1410,7 @@ class Crew(FlowTrackable, BaseModel):
return self._merge_tools(tools, cast(list[BaseTool], code_tools)) return self._merge_tools(tools, cast(list[BaseTool], code_tools))
return tools return tools
def _add_memory_tools( def _add_memory_tools(self, tools: list[BaseTool], memory: Any) -> list[BaseTool]:
self, tools: list[BaseTool], memory: Any
) -> list[BaseTool]:
"""Add recall and remember tools when memory is available. """Add recall and remember tools when memory is available.
Args: Args:

View File

@@ -1,5 +1,4 @@
from crewai.crews.crew_output import CrewOutput from crewai.crews.crew_output import CrewOutput
__all__ = ["CrewOutput"] __all__ = ["CrewOutput"]

View File

@@ -23,4 +23,3 @@ class BaseEventListener(ABC):
Args: Args:
crewai_event_bus: The event bus to register listeners on. crewai_event_bus: The event bus to register listeners on.
""" """
pass

View File

@@ -15,6 +15,7 @@ from crewai.cli.plus_api import PlusAPI
from crewai.cli.version import get_crewai_version from crewai.cli.version import get_crewai_version
from crewai.events.listeners.tracing.types import TraceEvent from crewai.events.listeners.tracing.types import TraceEvent
from crewai.events.listeners.tracing.utils import ( from crewai.events.listeners.tracing.utils import (
get_user_id,
is_tracing_enabled_in_context, is_tracing_enabled_in_context,
should_auto_collect_first_time_traces, should_auto_collect_first_time_traces,
) )
@@ -67,7 +68,7 @@ class TraceBatchManager:
api_key=get_auth_token(), api_key=get_auth_token(),
) )
except AuthError: except AuthError:
self.plus_api = PlusAPI(api_key="") self.plus_api = PlusAPI()
self.ephemeral_trace_url = None self.ephemeral_trace_url = None
def initialize_batch( def initialize_batch(
@@ -120,7 +121,6 @@ class TraceBatchManager:
payload = { payload = {
"trace_id": self.current_batch.batch_id, "trace_id": self.current_batch.batch_id,
"execution_type": execution_metadata.get("execution_type", "crew"), "execution_type": execution_metadata.get("execution_type", "crew"),
"user_identifier": execution_metadata.get("user_context", None),
"execution_context": { "execution_context": {
"crew_fingerprint": execution_metadata.get("crew_fingerprint"), "crew_fingerprint": execution_metadata.get("crew_fingerprint"),
"crew_name": execution_metadata.get("crew_name", None), "crew_name": execution_metadata.get("crew_name", None),
@@ -140,6 +140,7 @@ class TraceBatchManager:
} }
if use_ephemeral: if use_ephemeral:
payload["ephemeral_trace_id"] = self.current_batch.batch_id payload["ephemeral_trace_id"] = self.current_batch.batch_id
payload["user_identifier"] = get_user_id()
response = ( response = (
self.plus_api.initialize_ephemeral_trace_batch(payload) self.plus_api.initialize_ephemeral_trace_batch(payload)

View File

@@ -1,4 +1,5 @@
from collections.abc import Callable from collections.abc import Callable
import contextvars
from contextvars import ContextVar, Token from contextvars import ContextVar, Token
from datetime import datetime from datetime import datetime
import getpass import getpass
@@ -18,6 +19,7 @@ from rich.console import Console
from rich.panel import Panel from rich.panel import Panel
from rich.text import Text from rich.text import Text
from crewai.utilities.lock_store import lock as store_lock
from crewai.utilities.paths import db_storage_path from crewai.utilities.paths import db_storage_path
from crewai.utilities.serialization import to_serializable from crewai.utilities.serialization import to_serializable
@@ -137,12 +139,25 @@ def _load_user_data() -> dict[str, Any]:
return {} return {}
def _save_user_data(data: dict[str, Any]) -> None: def _user_data_lock_name() -> str:
"""Return a stable lock name for the user data file."""
return f"file:{os.path.realpath(_user_data_file())}"
def update_user_data(updates: dict[str, Any]) -> None:
"""Atomically read-modify-write the user data file.
Args:
updates: Key-value pairs to merge into the existing user data.
"""
try: try:
p = _user_data_file() with store_lock(_user_data_lock_name()):
p.write_text(json.dumps(data, indent=2)) data = _load_user_data()
data.update(updates)
p = _user_data_file()
p.write_text(json.dumps(data, indent=2))
except (OSError, PermissionError) as e: except (OSError, PermissionError) as e:
logger.warning(f"Failed to save user data: {e}") logger.warning(f"Failed to update user data: {e}")
def has_user_declined_tracing() -> bool: def has_user_declined_tracing() -> bool:
@@ -357,24 +372,30 @@ def _get_generic_system_id() -> str | None:
return None return None
def get_user_id() -> str: def _generate_user_id() -> str:
"""Stable, anonymized user identifier with caching.""" """Compute an anonymized user identifier from username and machine ID."""
data = _load_user_data()
if "user_id" in data:
return cast(str, data["user_id"])
try: try:
username = getpass.getuser() username = getpass.getuser()
except Exception: except Exception:
username = "unknown" username = "unknown"
seed = f"{username}|{_get_machine_id()}" seed = f"{username}|{_get_machine_id()}"
uid = hashlib.sha256(seed.encode()).hexdigest() return hashlib.sha256(seed.encode()).hexdigest()
data["user_id"] = uid
_save_user_data(data) def get_user_id() -> str:
return uid """Stable, anonymized user identifier with caching."""
with store_lock(_user_data_lock_name()):
data = _load_user_data()
if "user_id" in data:
return cast(str, data["user_id"])
uid = _generate_user_id()
data["user_id"] = uid
p = _user_data_file()
p.write_text(json.dumps(data, indent=2))
return uid
def is_first_execution() -> bool: def is_first_execution() -> bool:
@@ -389,20 +410,23 @@ def mark_first_execution_done(user_consented: bool = False) -> None:
Args: Args:
user_consented: Whether the user consented to trace collection. user_consented: Whether the user consented to trace collection.
""" """
data = _load_user_data() with store_lock(_user_data_lock_name()):
if data.get("first_execution_done", False): data = _load_user_data()
return if data.get("first_execution_done", False):
return
data.update( uid = data.get("user_id") or _generate_user_id()
{ data.update(
"first_execution_done": True, {
"first_execution_at": datetime.now().timestamp(), "first_execution_done": True,
"user_id": get_user_id(), "first_execution_at": datetime.now().timestamp(),
"machine_id": _get_machine_id(), "user_id": uid,
"trace_consent": user_consented, "machine_id": _get_machine_id(),
} "trace_consent": user_consented,
) }
_save_user_data(data) )
p = _user_data_file()
p.write_text(json.dumps(data, indent=2))
def safe_serialize_to_dict(obj: Any, exclude: set[str] | None = None) -> dict[str, Any]: def safe_serialize_to_dict(obj: Any, exclude: set[str] | None = None) -> dict[str, Any]:
@@ -509,7 +533,8 @@ def prompt_user_for_trace_viewing(timeout_seconds: int = 20) -> bool:
# Handle all input-related errors silently # Handle all input-related errors silently
result[0] = False result[0] = False
input_thread = threading.Thread(target=get_input, daemon=True) ctx = contextvars.copy_context()
input_thread = threading.Thread(target=ctx.run, args=(get_input,), daemon=True)
input_thread.start() input_thread.start()
input_thread.join(timeout=timeout_seconds) input_thread.join(timeout=timeout_seconds)

View File

@@ -86,3 +86,11 @@ class LLMStreamChunkEvent(LLMEventBase):
tool_call: ToolCall | None = None tool_call: ToolCall | None = None
call_type: LLMCallType | None = None call_type: LLMCallType | None = None
response_id: str | None = None response_id: str | None = None
class LLMThinkingChunkEvent(LLMEventBase):
"""Event emitted when a thinking/reasoning chunk is received from a thinking model"""
type: str = "llm_thinking_chunk"
chunk: str
response_id: str | None = None

View File

@@ -43,6 +43,7 @@ def should_suppress_console_output() -> bool:
class ConsoleFormatter: class ConsoleFormatter:
tool_usage_counts: ClassVar[dict[str, int]] = {} tool_usage_counts: ClassVar[dict[str, int]] = {}
_tool_counts_lock: ClassVar[threading.Lock] = threading.Lock()
current_a2a_turn_count: int = 0 current_a2a_turn_count: int = 0
_pending_a2a_message: str | None = None _pending_a2a_message: str | None = None
@@ -445,9 +446,11 @@ To enable tracing, do any one of these:
if not self.verbose: if not self.verbose:
return return
# Update tool usage count with self._tool_counts_lock:
self.tool_usage_counts[tool_name] = self.tool_usage_counts.get(tool_name, 0) + 1 self.tool_usage_counts[tool_name] = (
iteration = self.tool_usage_counts[tool_name] self.tool_usage_counts.get(tool_name, 0) + 1
)
iteration = self.tool_usage_counts[tool_name]
content = Text() content = Text()
content.append("Tool: ", style="white") content.append("Tool: ", style="white")
@@ -474,7 +477,8 @@ To enable tracing, do any one of these:
if not self.verbose: if not self.verbose:
return return
iteration = self.tool_usage_counts.get(tool_name, 1) with self._tool_counts_lock:
iteration = self.tool_usage_counts.get(tool_name, 1)
content = Text() content = Text()
content.append("Tool Completed\n", style="green bold") content.append("Tool Completed\n", style="green bold")
@@ -500,7 +504,8 @@ To enable tracing, do any one of these:
if not self.verbose: if not self.verbose:
return return
iteration = self.tool_usage_counts.get(tool_name, 1) with self._tool_counts_lock:
iteration = self.tool_usage_counts.get(tool_name, 1)
content = Text() content = Text()
content.append("Tool Failed\n", style="red bold") content.append("Tool Failed\n", style="red bold")

View File

@@ -3,6 +3,7 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import Callable, Coroutine from collections.abc import Callable, Coroutine
from concurrent.futures import ThreadPoolExecutor, as_completed from concurrent.futures import ThreadPoolExecutor, as_completed
import contextvars
from datetime import datetime from datetime import datetime
import inspect import inspect
import json import json
@@ -302,6 +303,7 @@ class AgentExecutor(Flow[AgentReActState], CrewAgentExecutorMixin):
super().__init__( super().__init__(
suppress_flow_events=True, suppress_flow_events=True,
tracing=current_tracing if current_tracing else None, tracing=current_tracing if current_tracing else None,
max_method_calls=self.max_iter * 10,
) )
self._flow_initialized = True self._flow_initialized = True
@@ -403,7 +405,7 @@ class AgentExecutor(Flow[AgentReActState], CrewAgentExecutorMixin):
self._setup_native_tools() self._setup_native_tools()
return "initialized" return "initialized"
@listen("force_final_answer") @listen("max_iterations_exceeded")
def force_final_answer(self) -> Literal["agent_finished"]: def force_final_answer(self) -> Literal["agent_finished"]:
"""Force agent to provide final answer when max iterations exceeded.""" """Force agent to provide final answer when max iterations exceeded."""
formatted_answer = handle_max_iterations_exceeded( formatted_answer = handle_max_iterations_exceeded(
@@ -655,11 +657,11 @@ class AgentExecutor(Flow[AgentReActState], CrewAgentExecutorMixin):
return "tool_result_is_final" return "tool_result_is_final"
reasoning_prompt = self._i18n.slice("post_tool_reasoning") reasoning_prompt = self._i18n.slice("post_tool_reasoning")
reasoning_message: LLMMessage = { reasoning_message_post: LLMMessage = {
"role": "user", "role": "user",
"content": reasoning_prompt, "content": reasoning_prompt,
} }
self.state.messages.append(reasoning_message) self.state.messages.append(reasoning_message_post)
return "tool_completed" return "tool_completed"
@@ -727,7 +729,11 @@ class AgentExecutor(Flow[AgentReActState], CrewAgentExecutorMixin):
max_workers = min(8, len(runnable_tool_calls)) max_workers = min(8, len(runnable_tool_calls))
with ThreadPoolExecutor(max_workers=max_workers) as pool: with ThreadPoolExecutor(max_workers=max_workers) as pool:
future_to_idx = { future_to_idx = {
pool.submit(self._execute_single_native_tool_call, tool_call): idx pool.submit(
contextvars.copy_context().run,
self._execute_single_native_tool_call,
tool_call,
): idx
for idx, tool_call in enumerate(runnable_tool_calls) for idx, tool_call in enumerate(runnable_tool_calls)
} }
ordered_results: list[dict[str, Any] | None] = [None] * len( ordered_results: list[dict[str, Any] | None] = [None] * len(
@@ -886,9 +892,10 @@ class AgentExecutor(Flow[AgentReActState], CrewAgentExecutorMixin):
call_id, func_name, func_args = info call_id, func_name, func_args = info
# Parse arguments # Parse arguments
args_dict, parse_error = parse_tool_call_args(func_args, func_name, call_id) parsed_args, parse_error = parse_tool_call_args(func_args, func_name, call_id)
if parse_error is not None: if parse_error is not None:
return parse_error return parse_error
args_dict: dict[str, Any] = parsed_args or {}
# Get agent_key for event tracking # Get agent_key for event tracking
agent_key = getattr(self.agent, "key", "unknown") if self.agent else "unknown" agent_key = getattr(self.agent, "key", "unknown") if self.agent else "unknown"
@@ -1107,11 +1114,11 @@ class AgentExecutor(Flow[AgentReActState], CrewAgentExecutorMixin):
def check_max_iterations( def check_max_iterations(
self, self,
) -> Literal[ ) -> Literal[
"force_final_answer", "continue_reasoning", "continue_reasoning_native" "max_iterations_exceeded", "continue_reasoning", "continue_reasoning_native"
]: ]:
"""Check if max iterations reached before proceeding with reasoning.""" """Check if max iterations reached before proceeding with reasoning."""
if has_reached_max_iterations(self.state.iterations, self.max_iter): if has_reached_max_iterations(self.state.iterations, self.max_iter):
return "force_final_answer" return "max_iterations_exceeded"
if self.state.use_native_tools: if self.state.use_native_tools:
return "continue_reasoning_native" return "continue_reasoning_native"
return "continue_reasoning" return "continue_reasoning"

View File

@@ -34,6 +34,7 @@ class ConsoleProvider:
```python ```python
from crewai.flow.async_feedback import ConsoleProvider from crewai.flow.async_feedback import ConsoleProvider
@human_feedback( @human_feedback(
message="Review this:", message="Review this:",
provider=ConsoleProvider(), provider=ConsoleProvider(),
@@ -46,6 +47,7 @@ class ConsoleProvider:
```python ```python
from crewai.flow import Flow, start from crewai.flow import Flow, start
class MyFlow(Flow): class MyFlow(Flow):
@start() @start()
def gather_info(self): def gather_info(self):

View File

@@ -17,6 +17,7 @@ from collections.abc import (
ValuesView, ValuesView,
) )
from concurrent.futures import Future, ThreadPoolExecutor from concurrent.futures import Future, ThreadPoolExecutor
import contextvars
import copy import copy
import enum import enum
import inspect import inspect
@@ -497,6 +498,52 @@ class LockedListProxy(list, Generic[T]): # type: ignore[type-arg]
def __bool__(self) -> bool: def __bool__(self) -> bool:
return bool(self._list) return bool(self._list)
def index(
self, value: T, start: SupportsIndex = 0, stop: SupportsIndex | None = None
) -> int: # type: ignore[override]
if stop is None:
return self._list.index(value, start)
return self._list.index(value, start, stop)
def count(self, value: T) -> int:
return self._list.count(value)
def sort(self, *, key: Any = None, reverse: bool = False) -> None:
with self._lock:
self._list.sort(key=key, reverse=reverse)
def reverse(self) -> None:
with self._lock:
self._list.reverse()
def copy(self) -> list[T]:
return self._list.copy()
def __add__(self, other: list[T]) -> list[T]:
return self._list + other
def __radd__(self, other: list[T]) -> list[T]:
return other + self._list
def __iadd__(self, other: Iterable[T]) -> LockedListProxy[T]:
with self._lock:
self._list += list(other)
return self
def __mul__(self, n: SupportsIndex) -> list[T]:
return self._list * n
def __rmul__(self, n: SupportsIndex) -> list[T]:
return self._list * n
def __imul__(self, n: SupportsIndex) -> LockedListProxy[T]:
with self._lock:
self._list *= n
return self
def __reversed__(self) -> Iterator[T]:
return reversed(self._list)
def __eq__(self, other: object) -> bool: def __eq__(self, other: object) -> bool:
"""Compare based on the underlying list contents.""" """Compare based on the underlying list contents."""
if isinstance(other, LockedListProxy): if isinstance(other, LockedListProxy):
@@ -579,6 +626,23 @@ class LockedDictProxy(dict, Generic[T]): # type: ignore[type-arg]
def __bool__(self) -> bool: def __bool__(self) -> bool:
return bool(self._dict) return bool(self._dict)
def copy(self) -> dict[str, T]:
return self._dict.copy()
def __or__(self, other: dict[str, T]) -> dict[str, T]:
return self._dict | other
def __ror__(self, other: dict[str, T]) -> dict[str, T]:
return other | self._dict
def __ior__(self, other: dict[str, T]) -> LockedDictProxy[T]:
with self._lock:
self._dict |= other
return self
def __reversed__(self) -> Iterator[str]:
return reversed(self._dict)
def __eq__(self, other: object) -> bool: def __eq__(self, other: object) -> bool:
"""Compare based on the underlying dict contents.""" """Compare based on the underlying dict contents."""
if isinstance(other, LockedDictProxy): if isinstance(other, LockedDictProxy):
@@ -620,6 +684,10 @@ class StateProxy(Generic[T]):
if name in ("_proxy_state", "_proxy_lock"): if name in ("_proxy_state", "_proxy_lock"):
object.__setattr__(self, name, value) object.__setattr__(self, name, value)
else: else:
if isinstance(value, LockedListProxy):
value = value._list
elif isinstance(value, LockedDictProxy):
value = value._dict
with object.__getattribute__(self, "_proxy_lock"): with object.__getattribute__(self, "_proxy_lock"):
setattr(object.__getattribute__(self, "_proxy_state"), name, value) setattr(object.__getattribute__(self, "_proxy_state"), name, value)
@@ -692,6 +760,7 @@ class FlowMeta(type):
condition_type = getattr( condition_type = getattr(
attr_value, "__condition_type__", OR_CONDITION attr_value, "__condition_type__", OR_CONDITION
) )
if ( if (
hasattr(attr_value, "__trigger_condition__") hasattr(attr_value, "__trigger_condition__")
and attr_value.__trigger_condition__ is not None and attr_value.__trigger_condition__ is not None
@@ -769,6 +838,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
persistence: FlowPersistence | None = None, persistence: FlowPersistence | None = None,
tracing: bool | None = None, tracing: bool | None = None,
suppress_flow_events: bool = False, suppress_flow_events: bool = False,
max_method_calls: int = 100,
**kwargs: Any, **kwargs: Any,
) -> None: ) -> None:
"""Initialize a new Flow instance. """Initialize a new Flow instance.
@@ -777,6 +847,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
persistence: Optional persistence backend for storing flow states persistence: Optional persistence backend for storing flow states
tracing: Whether to enable tracing. True=always enable, False=always disable, None=check environment/user settings tracing: Whether to enable tracing. True=always enable, False=always disable, None=check environment/user settings
suppress_flow_events: Whether to suppress flow event emissions (internal use) suppress_flow_events: Whether to suppress flow event emissions (internal use)
max_method_calls: Maximum times a single method can be called per execution before raising RecursionError
**kwargs: Additional state values to initialize or override **kwargs: Additional state values to initialize or override
""" """
# Initialize basic instance attributes # Initialize basic instance attributes
@@ -792,6 +863,8 @@ class Flow(Generic[T], metaclass=FlowMeta):
self._completed_methods: set[FlowMethodName] = ( self._completed_methods: set[FlowMethodName] = (
set() set()
) # Track completed methods for reload ) # Track completed methods for reload
self._method_call_counts: dict[FlowMethodName, int] = {}
self._max_method_calls = max_method_calls
self._persistence: FlowPersistence | None = persistence self._persistence: FlowPersistence | None = persistence
self._is_execution_resuming: bool = False self._is_execution_resuming: bool = False
self._event_futures: list[Future[None]] = [] self._event_futures: list[Future[None]] = []
@@ -1741,8 +1814,9 @@ class Flow(Generic[T], metaclass=FlowMeta):
try: try:
asyncio.get_running_loop() asyncio.get_running_loop()
ctx = contextvars.copy_context()
with ThreadPoolExecutor(max_workers=1) as pool: with ThreadPoolExecutor(max_workers=1) as pool:
return pool.submit(asyncio.run, _run_flow()).result() return pool.submit(ctx.run, asyncio.run, _run_flow()).result()
except RuntimeError: except RuntimeError:
return asyncio.run(_run_flow()) return asyncio.run(_run_flow())
@@ -1828,6 +1902,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
self._method_outputs.clear() self._method_outputs.clear()
self._pending_and_listeners.clear() self._pending_and_listeners.clear()
self._clear_or_listeners() self._clear_or_listeners()
self._method_call_counts.clear()
else: else:
# Only enter resumption mode if there are completed methods to # Only enter resumption mode if there are completed methods to
# replay. When _completed_methods is empty (e.g. a pure # replay. When _completed_methods is empty (e.g. a pure
@@ -2165,8 +2240,6 @@ class Flow(Generic[T], metaclass=FlowMeta):
else: else:
# Run sync methods in thread pool for isolation # Run sync methods in thread pool for isolation
# This allows Agent.kickoff() to work synchronously inside Flow methods # This allows Agent.kickoff() to work synchronously inside Flow methods
import contextvars
ctx = contextvars.copy_context() ctx = contextvars.copy_context()
result = await asyncio.to_thread(ctx.run, method, *args, **kwargs) result = await asyncio.to_thread(ctx.run, method, *args, **kwargs)
finally: finally:
@@ -2569,6 +2642,16 @@ class Flow(Generic[T], metaclass=FlowMeta):
- Skips execution if method was already completed (e.g., after reload) - Skips execution if method was already completed (e.g., after reload)
- Catches and logs any exceptions during execution, preventing individual listener failures from breaking the entire flow - Catches and logs any exceptions during execution, preventing individual listener failures from breaking the entire flow
""" """
count = self._method_call_counts.get(listener_name, 0) + 1
if count > self._max_method_calls:
raise RecursionError(
f"Method '{listener_name}' has been called {self._max_method_calls} times in "
f"this flow execution, which indicates an infinite loop. "
f"This commonly happens when a @listen label matches the "
f"method's own name."
)
self._method_call_counts[listener_name] = count
if listener_name in self._completed_methods: if listener_name in self._completed_methods:
if self._is_execution_resuming: if self._is_execution_resuming:
# During resumption, skip execution but continue listeners # During resumption, skip execution but continue listeners
@@ -2633,7 +2716,9 @@ class Flow(Generic[T], metaclass=FlowMeta):
from crewai.flow.async_feedback.types import HumanFeedbackPending from crewai.flow.async_feedback.types import HumanFeedbackPending
if not isinstance(e, HumanFeedbackPending): if not isinstance(e, HumanFeedbackPending):
logger.error(f"Error executing listener {listener_name}: {e}") if not getattr(e, "_flow_listener_logged", False):
logger.error(f"Error executing listener {listener_name}: {e}")
e._flow_listener_logged = True # type: ignore[attr-defined]
raise raise
# ── User Input (self.ask) ──────────────────────────────────────── # ── User Input (self.ask) ────────────────────────────────────────
@@ -2775,8 +2860,9 @@ class Flow(Generic[T], metaclass=FlowMeta):
# Manual executor management to avoid shutdown(wait=True) # Manual executor management to avoid shutdown(wait=True)
# deadlock when the provider call outlives the timeout. # deadlock when the provider call outlives the timeout.
executor = ThreadPoolExecutor(max_workers=1) executor = ThreadPoolExecutor(max_workers=1)
ctx = contextvars.copy_context()
future = executor.submit( future = executor.submit(
provider.request_input, message, self, metadata ctx.run, provider.request_input, message, self, metadata
) )
try: try:
raw = future.result(timeout=timeout) raw = future.result(timeout=timeout)

View File

@@ -188,7 +188,7 @@ def human_feedback(
metadata: dict[str, Any] | None = None, metadata: dict[str, Any] | None = None,
provider: HumanFeedbackProvider | None = None, provider: HumanFeedbackProvider | None = None,
learn: bool = False, learn: bool = False,
learn_source: str = "hitl" learn_source: str = "hitl",
) -> Callable[[F], F]: ) -> Callable[[F], F]:
"""Decorator for Flow methods that require human feedback. """Decorator for Flow methods that require human feedback.
@@ -328,9 +328,7 @@ def human_feedback(
"""Recall past HITL lessons and use LLM to pre-review the output.""" """Recall past HITL lessons and use LLM to pre-review the output."""
try: try:
query = f"human feedback lessons for {func.__name__}: {method_output!s}" query = f"human feedback lessons for {func.__name__}: {method_output!s}"
matches = flow_instance.memory.recall( matches = flow_instance.memory.recall(query, source=learn_source)
query, source=learn_source
)
if not matches: if not matches:
return method_output return method_output
@@ -341,7 +339,10 @@ def human_feedback(
lessons=lessons, lessons=lessons,
) )
messages = [ messages = [
{"role": "system", "content": _get_hitl_prompt("hitl_pre_review_system")}, {
"role": "system",
"content": _get_hitl_prompt("hitl_pre_review_system"),
},
{"role": "user", "content": prompt}, {"role": "user", "content": prompt},
] ]
if getattr(llm_inst, "supports_function_calling", lambda: False)(): if getattr(llm_inst, "supports_function_calling", lambda: False)():
@@ -366,7 +367,10 @@ def human_feedback(
feedback=raw_feedback, feedback=raw_feedback,
) )
messages = [ messages = [
{"role": "system", "content": _get_hitl_prompt("hitl_distill_system")}, {
"role": "system",
"content": _get_hitl_prompt("hitl_distill_system"),
},
{"role": "user", "content": prompt}, {"role": "user", "content": prompt},
] ]
@@ -408,7 +412,7 @@ def human_feedback(
emit=list(emit) if emit else None, emit=list(emit) if emit else None,
default_outcome=default_outcome, default_outcome=default_outcome,
metadata=metadata or {}, metadata=metadata or {},
llm=llm if isinstance(llm, str) else None, llm=llm if isinstance(llm, str) else getattr(llm, "model", None),
) )
# Determine effective provider: # Determine effective provider:
@@ -487,7 +491,11 @@ def human_feedback(
result = _process_feedback(self, method_output, raw_feedback) result = _process_feedback(self, method_output, raw_feedback)
# Distill: extract lessons from output + feedback, store in memory # Distill: extract lessons from output + feedback, store in memory
if learn and getattr(self, "memory", None) is not None and raw_feedback.strip(): if (
learn
and getattr(self, "memory", None) is not None
and raw_feedback.strip()
):
_distill_and_store_lessons(self, method_output, raw_feedback) _distill_and_store_lessons(self, method_output, raw_feedback)
return result return result
@@ -507,7 +515,11 @@ def human_feedback(
result = _process_feedback(self, method_output, raw_feedback) result = _process_feedback(self, method_output, raw_feedback)
# Distill: extract lessons from output + feedback, store in memory # Distill: extract lessons from output + feedback, store in memory
if learn and getattr(self, "memory", None) is not None and raw_feedback.strip(): if (
learn
and getattr(self, "memory", None) is not None
and raw_feedback.strip()
):
_distill_and_store_lessons(self, method_output, raw_feedback) _distill_and_store_lessons(self, method_output, raw_feedback)
return result return result
@@ -534,7 +546,7 @@ def human_feedback(
metadata=metadata, metadata=metadata,
provider=provider, provider=provider,
learn=learn, learn=learn,
learn_source=learn_source learn_source=learn_source,
) )
wrapper.__is_flow_method__ = True wrapper.__is_flow_method__ = True

View File

@@ -1,11 +1,10 @@
""" """SQLite-based implementation of flow state persistence."""
SQLite-based implementation of flow state persistence.
"""
from __future__ import annotations from __future__ import annotations
from datetime import datetime, timezone from datetime import datetime, timezone
import json import json
import os
from pathlib import Path from pathlib import Path
import sqlite3 import sqlite3
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
@@ -13,6 +12,7 @@ from typing import TYPE_CHECKING, Any
from pydantic import BaseModel from pydantic import BaseModel
from crewai.flow.persistence.base import FlowPersistence from crewai.flow.persistence.base import FlowPersistence
from crewai.utilities.lock_store import lock as store_lock
from crewai.utilities.paths import db_storage_path from crewai.utilities.paths import db_storage_path
@@ -68,11 +68,16 @@ class SQLiteFlowPersistence(FlowPersistence):
raise ValueError("Database path must be provided") raise ValueError("Database path must be provided")
self.db_path = path # Now mypy knows this is str self.db_path = path # Now mypy knows this is str
self._lock_name = f"sqlite:{os.path.realpath(self.db_path)}"
self.init_db() self.init_db()
def init_db(self) -> None: def init_db(self) -> None:
"""Create the necessary tables if they don't exist.""" """Create the necessary tables if they don't exist."""
with sqlite3.connect(self.db_path) as conn: with (
store_lock(self._lock_name),
sqlite3.connect(self.db_path, timeout=30) as conn,
):
conn.execute("PRAGMA journal_mode=WAL")
# Main state table # Main state table
conn.execute( conn.execute(
""" """
@@ -113,6 +118,49 @@ class SQLiteFlowPersistence(FlowPersistence):
""" """
) )
def _save_state_sql(
self,
conn: sqlite3.Connection,
flow_uuid: str,
method_name: str,
state_dict: dict[str, Any],
) -> None:
"""Execute the save-state INSERT without acquiring the lock.
Args:
conn: An open SQLite connection.
flow_uuid: Unique identifier for the flow instance.
method_name: Name of the method that just completed.
state_dict: State data as a plain dict.
"""
conn.execute(
"""
INSERT INTO flow_states (
flow_uuid,
method_name,
timestamp,
state_json
) VALUES (?, ?, ?, ?)
""",
(
flow_uuid,
method_name,
datetime.now(timezone.utc).isoformat(),
json.dumps(state_dict),
),
)
@staticmethod
def _to_state_dict(state_data: dict[str, Any] | BaseModel) -> dict[str, Any]:
"""Convert state_data to a plain dict."""
if isinstance(state_data, BaseModel):
return state_data.model_dump()
if isinstance(state_data, dict):
return state_data
raise ValueError(
f"state_data must be either a Pydantic BaseModel or dict, got {type(state_data)}"
)
def save_state( def save_state(
self, self,
flow_uuid: str, flow_uuid: str,
@@ -126,33 +174,13 @@ class SQLiteFlowPersistence(FlowPersistence):
method_name: Name of the method that just completed method_name: Name of the method that just completed
state_data: Current state data (either dict or Pydantic model) state_data: Current state data (either dict or Pydantic model)
""" """
# Convert state_data to dict, handling both Pydantic and dict cases state_dict = self._to_state_dict(state_data)
if isinstance(state_data, BaseModel):
state_dict = state_data.model_dump()
elif isinstance(state_data, dict):
state_dict = state_data
else:
raise ValueError(
f"state_data must be either a Pydantic BaseModel or dict, got {type(state_data)}"
)
with sqlite3.connect(self.db_path) as conn: with (
conn.execute( store_lock(self._lock_name),
""" sqlite3.connect(self.db_path, timeout=30) as conn,
INSERT INTO flow_states ( ):
flow_uuid, self._save_state_sql(conn, flow_uuid, method_name, state_dict)
method_name,
timestamp,
state_json
) VALUES (?, ?, ?, ?)
""",
(
flow_uuid,
method_name,
datetime.now(timezone.utc).isoformat(),
json.dumps(state_dict),
),
)
def load_state(self, flow_uuid: str) -> dict[str, Any] | None: def load_state(self, flow_uuid: str) -> dict[str, Any] | None:
"""Load the most recent state for a given flow UUID. """Load the most recent state for a given flow UUID.
@@ -163,7 +191,7 @@ class SQLiteFlowPersistence(FlowPersistence):
Returns: Returns:
The most recent state as a dictionary, or None if no state exists The most recent state as a dictionary, or None if no state exists
""" """
with sqlite3.connect(self.db_path) as conn: with sqlite3.connect(self.db_path, timeout=30) as conn:
cursor = conn.execute( cursor = conn.execute(
""" """
SELECT state_json SELECT state_json
@@ -197,24 +225,14 @@ class SQLiteFlowPersistence(FlowPersistence):
context: The pending feedback context with all resume information context: The pending feedback context with all resume information
state_data: Current state data state_data: Current state data
""" """
# Import here to avoid circular imports state_dict = self._to_state_dict(state_data)
# Convert state_data to dict with (
if isinstance(state_data, BaseModel): store_lock(self._lock_name),
state_dict = state_data.model_dump() sqlite3.connect(self.db_path, timeout=30) as conn,
elif isinstance(state_data, dict): ):
state_dict = state_data self._save_state_sql(conn, flow_uuid, context.method_name, state_dict)
else:
raise ValueError(
f"state_data must be either a Pydantic BaseModel or dict, got {type(state_data)}"
)
# Also save to regular state table for consistency
self.save_state(flow_uuid, context.method_name, state_data)
# Save pending feedback context
with sqlite3.connect(self.db_path) as conn:
# Use INSERT OR REPLACE to handle re-triggering feedback on same flow
conn.execute( conn.execute(
""" """
INSERT OR REPLACE INTO pending_feedback ( INSERT OR REPLACE INTO pending_feedback (
@@ -248,7 +266,7 @@ class SQLiteFlowPersistence(FlowPersistence):
# Import here to avoid circular imports # Import here to avoid circular imports
from crewai.flow.async_feedback.types import PendingFeedbackContext from crewai.flow.async_feedback.types import PendingFeedbackContext
with sqlite3.connect(self.db_path) as conn: with sqlite3.connect(self.db_path, timeout=30) as conn:
cursor = conn.execute( cursor = conn.execute(
""" """
SELECT state_json, context_json SELECT state_json, context_json
@@ -272,7 +290,10 @@ class SQLiteFlowPersistence(FlowPersistence):
Args: Args:
flow_uuid: Unique identifier for the flow instance flow_uuid: Unique identifier for the flow instance
""" """
with sqlite3.connect(self.db_path) as conn: with (
store_lock(self._lock_name),
sqlite3.connect(self.db_path, timeout=30) as conn,
):
conn.execute( conn.execute(
""" """
DELETE FROM pending_feedback DELETE FROM pending_feedback

View File

@@ -600,7 +600,7 @@ class LiteAgent(FlowTrackable, BaseModel):
def _save_to_memory(self, output_text: str) -> None: def _save_to_memory(self, output_text: str) -> None:
"""Extract discrete memories from the run and remember each. No-op if _memory is None or read-only.""" """Extract discrete memories from the run and remember each. No-op if _memory is None or read-only."""
if self._memory is None or getattr(self._memory, "_read_only", False): if self._memory is None or self._memory.read_only:
return return
input_str = self._get_last_user_content() or "User request" input_str = self._get_last_user_content() or "User request"
try: try:

View File

@@ -26,6 +26,7 @@ from crewai.events.types.llm_events import (
LLMCallStartedEvent, LLMCallStartedEvent,
LLMCallType, LLMCallType,
LLMStreamChunkEvent, LLMStreamChunkEvent,
LLMThinkingChunkEvent,
) )
from crewai.events.types.tool_usage_events import ( from crewai.events.types.tool_usage_events import (
ToolUsageErrorEvent, ToolUsageErrorEvent,
@@ -368,9 +369,6 @@ class BaseLLM(ABC):
"""Emit LLM call started event.""" """Emit LLM call started event."""
from crewai.utilities.serialization import to_serializable from crewai.utilities.serialization import to_serializable
if not hasattr(crewai_event_bus, "emit"):
raise ValueError("crewai_event_bus does not have an emit method") from None
crewai_event_bus.emit( crewai_event_bus.emit(
self, self,
event=LLMCallStartedEvent( event=LLMCallStartedEvent(
@@ -416,9 +414,6 @@ class BaseLLM(ABC):
from_agent: Agent | None = None, from_agent: Agent | None = None,
) -> None: ) -> None:
"""Emit LLM call failed event.""" """Emit LLM call failed event."""
if not hasattr(crewai_event_bus, "emit"):
raise ValueError("crewai_event_bus does not have an emit method") from None
crewai_event_bus.emit( crewai_event_bus.emit(
self, self,
event=LLMCallFailedEvent( event=LLMCallFailedEvent(
@@ -449,9 +444,6 @@ class BaseLLM(ABC):
call_type: The type of LLM call (LLM_CALL or TOOL_CALL). call_type: The type of LLM call (LLM_CALL or TOOL_CALL).
response_id: Unique ID for a particular LLM response, chunks have same response_id. response_id: Unique ID for a particular LLM response, chunks have same response_id.
""" """
if not hasattr(crewai_event_bus, "emit"):
raise ValueError("crewai_event_bus does not have an emit method") from None
crewai_event_bus.emit( crewai_event_bus.emit(
self, self,
event=LLMStreamChunkEvent( event=LLMStreamChunkEvent(
@@ -465,6 +457,32 @@ class BaseLLM(ABC):
), ),
) )
def _emit_thinking_chunk_event(
self,
chunk: str,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_id: str | None = None,
) -> None:
"""Emit thinking/reasoning chunk event from a thinking model.
Args:
chunk: The thinking text content.
from_task: The task that initiated the call.
from_agent: The agent that initiated the call.
response_id: Unique ID for a particular LLM response.
"""
crewai_event_bus.emit(
self,
event=LLMThinkingChunkEvent(
chunk=chunk,
from_task=from_task,
from_agent=from_agent,
response_id=response_id,
call_id=get_current_call_id(),
),
)
def _handle_tool_execution( def _handle_tool_execution(
self, self,
function_name: str, function_name: str,

View File

@@ -22,7 +22,12 @@ if TYPE_CHECKING:
try: try:
from anthropic import Anthropic, AsyncAnthropic, transform_schema from anthropic import Anthropic, AsyncAnthropic, transform_schema
from anthropic.types import Message, TextBlock, ThinkingBlock, ToolUseBlock from anthropic.types import (
Message,
TextBlock,
ThinkingBlock,
ToolUseBlock,
)
from anthropic.types.beta import BetaMessage, BetaTextBlock, BetaToolUseBlock from anthropic.types.beta import BetaMessage, BetaTextBlock, BetaToolUseBlock
import httpx import httpx
except ImportError: except ImportError:
@@ -31,6 +36,11 @@ except ImportError:
) from None ) from None
TOOL_SEARCH_TOOL_TYPES: Final[tuple[str, ...]] = (
"tool_search_tool_regex_20251119",
"tool_search_tool_bm25_20251119",
)
ANTHROPIC_FILES_API_BETA: Final = "files-api-2025-04-14" ANTHROPIC_FILES_API_BETA: Final = "files-api-2025-04-14"
ANTHROPIC_STRUCTURED_OUTPUTS_BETA: Final = "structured-outputs-2025-11-13" ANTHROPIC_STRUCTURED_OUTPUTS_BETA: Final = "structured-outputs-2025-11-13"
@@ -117,6 +127,22 @@ class AnthropicThinkingConfig(BaseModel):
budget_tokens: int | None = None budget_tokens: int | None = None
class AnthropicToolSearchConfig(BaseModel):
"""Configuration for Anthropic's server-side tool search.
When enabled, tools marked with defer_loading=True are not loaded into
context immediately. Instead, Claude uses the tool search tool to
dynamically discover and load relevant tools on-demand.
Attributes:
type: The tool search variant to use.
- "regex": Claude constructs regex patterns to search tool names/descriptions.
- "bm25": Claude uses natural language queries to search tools.
"""
type: Literal["regex", "bm25"] = "bm25"
class AnthropicCompletion(BaseLLM): class AnthropicCompletion(BaseLLM):
"""Anthropic native completion implementation. """Anthropic native completion implementation.
@@ -140,6 +166,7 @@ class AnthropicCompletion(BaseLLM):
interceptor: BaseInterceptor[httpx.Request, httpx.Response] | None = None, interceptor: BaseInterceptor[httpx.Request, httpx.Response] | None = None,
thinking: AnthropicThinkingConfig | None = None, thinking: AnthropicThinkingConfig | None = None,
response_format: type[BaseModel] | None = None, response_format: type[BaseModel] | None = None,
tool_search: AnthropicToolSearchConfig | bool | None = None,
**kwargs: Any, **kwargs: Any,
): ):
"""Initialize Anthropic chat completion client. """Initialize Anthropic chat completion client.
@@ -159,6 +186,10 @@ class AnthropicCompletion(BaseLLM):
interceptor: HTTP interceptor for modifying requests/responses at transport level. interceptor: HTTP interceptor for modifying requests/responses at transport level.
response_format: Pydantic model for structured output. When provided, responses response_format: Pydantic model for structured output. When provided, responses
will be validated against this model schema. will be validated against this model schema.
tool_search: Enable Anthropic's server-side tool search. When True, uses "bm25"
variant by default. Pass an AnthropicToolSearchConfig to choose "regex" or
"bm25". When enabled, tools are automatically marked with defer_loading=True
and a tool search tool is injected into the tools list.
**kwargs: Additional parameters **kwargs: Additional parameters
""" """
super().__init__( super().__init__(
@@ -190,6 +221,13 @@ class AnthropicCompletion(BaseLLM):
self.thinking = thinking self.thinking = thinking
self.previous_thinking_blocks: list[ThinkingBlock] = [] self.previous_thinking_blocks: list[ThinkingBlock] = []
self.response_format = response_format self.response_format = response_format
# Tool search config
if tool_search is True:
self.tool_search = AnthropicToolSearchConfig()
elif isinstance(tool_search, AnthropicToolSearchConfig):
self.tool_search = tool_search
else:
self.tool_search = None
# Model-specific settings # Model-specific settings
self.is_claude_3 = "claude-3" in model.lower() self.is_claude_3 = "claude-3" in model.lower()
self.supports_tools = True self.supports_tools = True
@@ -432,10 +470,23 @@ class AnthropicCompletion(BaseLLM):
# Handle tools for Claude 3+ # Handle tools for Claude 3+
if tools and self.supports_tools: if tools and self.supports_tools:
converted_tools = self._convert_tools_for_interference(tools) converted_tools = self._convert_tools_for_interference(tools)
# When tool_search is enabled and there are 2+ regular tools,
# inject the search tool and mark regular tools with defer_loading.
# With only 1 tool there's nothing to search — skip tool search
# entirely so the normal forced tool_choice optimisation still works.
regular_tools = [
t
for t in converted_tools
if t.get("type", "") not in TOOL_SEARCH_TOOL_TYPES
]
if self.tool_search is not None and len(regular_tools) >= 2:
converted_tools = self._apply_tool_search(converted_tools)
params["tools"] = converted_tools params["tools"] = converted_tools
if available_functions and len(converted_tools) == 1: if available_functions and len(regular_tools) == 1:
tool_name = converted_tools[0].get("name") tool_name = regular_tools[0].get("name")
if tool_name and tool_name in available_functions: if tool_name and tool_name in available_functions:
params["tool_choice"] = {"type": "tool", "name": tool_name} params["tool_choice"] = {"type": "tool", "name": tool_name}
@@ -454,6 +505,12 @@ class AnthropicCompletion(BaseLLM):
anthropic_tools = [] anthropic_tools = []
for tool in tools: for tool in tools:
# Pass through tool search tool definitions unchanged
tool_type = tool.get("type", "")
if tool_type in TOOL_SEARCH_TOOL_TYPES:
anthropic_tools.append(tool)
continue
if "input_schema" in tool and "name" in tool and "description" in tool: if "input_schema" in tool and "name" in tool and "description" in tool:
anthropic_tools.append(tool) anthropic_tools.append(tool)
continue continue
@@ -466,15 +523,15 @@ class AnthropicCompletion(BaseLLM):
logging.error(f"Error converting tool to Anthropic format: {e}") logging.error(f"Error converting tool to Anthropic format: {e}")
raise e raise e
anthropic_tool = { anthropic_tool: dict[str, Any] = {
"name": name, "name": name,
"description": description, "description": description,
} }
if parameters and isinstance(parameters, dict): if parameters and isinstance(parameters, dict):
anthropic_tool["input_schema"] = parameters # type: ignore[assignment] anthropic_tool["input_schema"] = parameters
else: else:
anthropic_tool["input_schema"] = { # type: ignore[assignment] anthropic_tool["input_schema"] = {
"type": "object", "type": "object",
"properties": {}, "properties": {},
"required": [], "required": [],
@@ -484,6 +541,55 @@ class AnthropicCompletion(BaseLLM):
return anthropic_tools return anthropic_tools
def _apply_tool_search(self, tools: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""Inject tool search tool and mark regular tools with defer_loading.
When tool_search is enabled, this method:
1. Adds the appropriate tool search tool definition (regex or bm25)
2. Marks all regular tools with defer_loading=True so they are only
loaded when Claude discovers them via search
Args:
tools: Converted tool definitions in Anthropic format.
Returns:
Updated tools list with tool search tool prepended and
regular tools marked as deferred.
"""
if self.tool_search is None:
return tools
# Check if a tool search tool is already present (user passed one manually)
has_search_tool = any(
t.get("type", "") in TOOL_SEARCH_TOOL_TYPES for t in tools
)
result: list[dict[str, Any]] = []
if not has_search_tool:
# Map config type to API type identifier
type_map = {
"regex": "tool_search_tool_regex_20251119",
"bm25": "tool_search_tool_bm25_20251119",
}
tool_type = type_map[self.tool_search.type]
# Tool search tool names follow the convention: tool_search_tool_{variant}
tool_name = f"tool_search_tool_{self.tool_search.type}"
result.append({"type": tool_type, "name": tool_name})
for tool in tools:
# Don't modify tool search tools
if tool.get("type", "") in TOOL_SEARCH_TOOL_TYPES:
result.append(tool)
continue
# Mark regular tools as deferred if not already set
if "defer_loading" not in tool:
tool = {**tool, "defer_loading": True}
result.append(tool)
return result
def _extract_thinking_block( def _extract_thinking_block(
self, content_block: Any self, content_block: Any
) -> ThinkingBlock | dict[str, Any] | None: ) -> ThinkingBlock | dict[str, Any] | None:

View File

@@ -1781,6 +1781,7 @@ class BedrockCompletion(BaseLLM):
converse_messages: list[LLMMessage] = [] converse_messages: list[LLMMessage] = []
system_message: str | None = None system_message: str | None = None
pending_tool_results: list[dict[str, Any]] = []
for message in formatted_messages: for message in formatted_messages:
role = message.get("role") role = message.get("role")
@@ -1794,53 +1795,56 @@ class BedrockCompletion(BaseLLM):
system_message += f"\n\n{content}" system_message += f"\n\n{content}"
else: else:
system_message = cast(str, content) system_message = cast(str, content)
elif role == "assistant" and tool_calls:
# Convert OpenAI-style tool_calls to Bedrock toolUse format
bedrock_content = []
for tc in tool_calls:
func = tc.get("function", {})
tool_use_block = {
"toolUse": {
"toolUseId": tc.get("id", f"call_{id(tc)}"),
"name": func.get("name", ""),
"input": func.get("arguments", {})
if isinstance(func.get("arguments"), dict)
else json.loads(func.get("arguments", "{}") or "{}"),
}
}
bedrock_content.append(tool_use_block)
converse_messages.append(
{"role": "assistant", "content": bedrock_content}
)
elif role == "tool": elif role == "tool":
if not tool_call_id: if not tool_call_id:
raise ValueError("Tool message missing required tool_call_id") raise ValueError("Tool message missing required tool_call_id")
converse_messages.append( pending_tool_results.append(
{ {
"role": "user", "toolResult": {
"content": [ "toolUseId": tool_call_id,
{ "content": [{"text": str(content) if content else ""}],
"toolResult": { }
"toolUseId": tool_call_id,
"content": [
{"text": str(content) if content else ""}
],
}
}
],
} }
) )
else: else:
# Convert to Converse API format with proper content structure if pending_tool_results:
if isinstance(content, list):
# Already formatted as multimodal content blocks
converse_messages.append({"role": role, "content": content})
else:
# String content - wrap in text block
text_content = content if content else ""
converse_messages.append( converse_messages.append(
{"role": role, "content": [{"text": text_content}]} {"role": "user", "content": pending_tool_results}
) )
pending_tool_results = []
if role == "assistant" and tool_calls:
# Convert OpenAI-style tool_calls to Bedrock toolUse format
bedrock_content = []
for tc in tool_calls:
func = tc.get("function", {})
tool_use_block = {
"toolUse": {
"toolUseId": tc.get("id", f"call_{id(tc)}"),
"name": func.get("name", ""),
"input": func.get("arguments", {})
if isinstance(func.get("arguments"), dict)
else json.loads(func.get("arguments", "{}") or "{}"),
}
}
bedrock_content.append(tool_use_block)
converse_messages.append(
{"role": "assistant", "content": bedrock_content}
)
else:
# Convert to Converse API format with proper content structure
if isinstance(content, list):
# Already formatted as multimodal content blocks
converse_messages.append({"role": role, "content": content})
else:
# String content - wrap in text block
text_content = content if content else ""
converse_messages.append(
{"role": role, "content": [{"text": text_content}]}
)
if pending_tool_results:
converse_messages.append({"role": "user", "content": pending_tool_results})
# CRITICAL: Handle model-specific conversation requirements # CRITICAL: Handle model-specific conversation requirements
# Cohere and some other models require conversation to end with user message # Cohere and some other models require conversation to end with user message

View File

@@ -61,6 +61,7 @@ class GeminiCompletion(BaseLLM):
interceptor: BaseInterceptor[Any, Any] | None = None, interceptor: BaseInterceptor[Any, Any] | None = None,
use_vertexai: bool | None = None, use_vertexai: bool | None = None,
response_format: type[BaseModel] | None = None, response_format: type[BaseModel] | None = None,
thinking_config: types.ThinkingConfig | None = None,
**kwargs: Any, **kwargs: Any,
): ):
"""Initialize Google Gemini chat completion client. """Initialize Google Gemini chat completion client.
@@ -93,6 +94,10 @@ class GeminiCompletion(BaseLLM):
api_version="v1" is automatically configured. api_version="v1" is automatically configured.
response_format: Pydantic model for structured output. Used as default when response_format: Pydantic model for structured output. Used as default when
response_model is not passed to call()/acall() methods. response_model is not passed to call()/acall() methods.
thinking_config: ThinkingConfig for thinking models (gemini-2.5+, gemini-3+).
Controls thought output via include_thoughts, thinking_budget,
and thinking_level. When None, thinking models automatically
get include_thoughts=True so thought content is surfaced.
**kwargs: Additional parameters **kwargs: Additional parameters
""" """
if interceptor is not None: if interceptor is not None:
@@ -139,6 +144,14 @@ class GeminiCompletion(BaseLLM):
version_match and float(version_match.group(1)) >= 2.0 version_match and float(version_match.group(1)) >= 2.0
) )
self.thinking_config = thinking_config
if (
self.thinking_config is None
and version_match
and float(version_match.group(1)) >= 2.5
):
self.thinking_config = types.ThinkingConfig(include_thoughts=True)
@property @property
def stop(self) -> list[str]: def stop(self) -> list[str]:
"""Get stop sequences sent to the API.""" """Get stop sequences sent to the API."""
@@ -520,6 +533,9 @@ class GeminiCompletion(BaseLLM):
if self.safety_settings: if self.safety_settings:
config_params["safety_settings"] = self.safety_settings config_params["safety_settings"] = self.safety_settings
if self.thinking_config is not None:
config_params["thinking_config"] = self.thinking_config
return types.GenerateContentConfig(**config_params) return types.GenerateContentConfig(**config_params)
def _convert_tools_for_interference( # type: ignore[override] def _convert_tools_for_interference( # type: ignore[override]
@@ -618,9 +634,17 @@ class GeminiCompletion(BaseLLM):
function_response_part = types.Part.from_function_response( function_response_part = types.Part.from_function_response(
name=tool_name, response=response_data name=tool_name, response=response_data
) )
contents.append( if (
types.Content(role="user", parts=[function_response_part]) contents
) and contents[-1].role == "user"
and contents[-1].parts
and contents[-1].parts[-1].function_response is not None
):
contents[-1].parts.append(function_response_part)
else:
contents.append(
types.Content(role="user", parts=[function_response_part])
)
elif role == "assistant" and message.get("tool_calls"): elif role == "assistant" and message.get("tool_calls"):
raw_parts: list[Any] | None = message.get("raw_tool_call_parts") raw_parts: list[Any] | None = message.get("raw_tool_call_parts")
if raw_parts and all(isinstance(p, types.Part) for p in raw_parts): if raw_parts and all(isinstance(p, types.Part) for p in raw_parts):
@@ -931,15 +955,6 @@ class GeminiCompletion(BaseLLM):
if chunk.usage_metadata: if chunk.usage_metadata:
usage_data = self._extract_token_usage(chunk) usage_data = self._extract_token_usage(chunk)
if chunk.text:
full_response += chunk.text
self._emit_stream_chunk_event(
chunk=chunk.text,
from_task=from_task,
from_agent=from_agent,
response_id=response_id,
)
if chunk.candidates: if chunk.candidates:
candidate = chunk.candidates[0] candidate = chunk.candidates[0]
if candidate.content and candidate.content.parts: if candidate.content and candidate.content.parts:
@@ -976,6 +991,21 @@ class GeminiCompletion(BaseLLM):
call_type=LLMCallType.TOOL_CALL, call_type=LLMCallType.TOOL_CALL,
response_id=response_id, response_id=response_id,
) )
elif part.thought and part.text:
self._emit_thinking_chunk_event(
chunk=part.text,
from_task=from_task,
from_agent=from_agent,
response_id=response_id,
)
elif part.text:
full_response += part.text
self._emit_stream_chunk_event(
chunk=part.text,
from_task=from_task,
from_agent=from_agent,
response_id=response_id,
)
return full_response, function_calls, usage_data return full_response, function_calls, usage_data
@@ -1329,7 +1359,7 @@ class GeminiCompletion(BaseLLM):
text_parts = [ text_parts = [
part.text part.text
for part in candidate.content.parts for part in candidate.content.parts
if hasattr(part, "text") and part.text if part.text and not part.thought
] ]
return "".join(text_parts) return "".join(text_parts)

View File

@@ -11,6 +11,7 @@ into a standalone MCPToolResolver. It handles three flavours of MCP reference:
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import contextvars
import time import time
from typing import TYPE_CHECKING, Any, Final, cast from typing import TYPE_CHECKING, Any, Final, cast
from urllib.parse import urlparse from urllib.parse import urlparse
@@ -25,6 +26,7 @@ from crewai.mcp.config import (
from crewai.mcp.transports.http import HTTPTransport from crewai.mcp.transports.http import HTTPTransport
from crewai.mcp.transports.sse import SSETransport from crewai.mcp.transports.sse import SSETransport
from crewai.mcp.transports.stdio import StdioTransport from crewai.mcp.transports.stdio import StdioTransport
from crewai.utilities.string_utils import sanitize_tool_name
if TYPE_CHECKING: if TYPE_CHECKING:
@@ -74,10 +76,9 @@ class MCPToolResolver:
elif isinstance(mcp_config, str): elif isinstance(mcp_config, str):
amp_refs.append(self._parse_amp_ref(mcp_config)) amp_refs.append(self._parse_amp_ref(mcp_config))
else: else:
tools, client = self._resolve_native(mcp_config) tools, clients = self._resolve_native(mcp_config)
all_tools.extend(tools) all_tools.extend(tools)
if client: self._clients.extend(clients)
self._clients.append(client)
if amp_refs: if amp_refs:
tools, clients = self._resolve_amp(amp_refs) tools, clients = self._resolve_amp(amp_refs)
@@ -131,7 +132,7 @@ class MCPToolResolver:
all_tools: list[BaseTool] = [] all_tools: list[BaseTool] = []
all_clients: list[Any] = [] all_clients: list[Any] = []
resolved_cache: dict[str, tuple[list[BaseTool], Any | None]] = {} resolved_cache: dict[str, tuple[list[BaseTool], list[Any]]] = {}
for slug in unique_slugs: for slug in unique_slugs:
config_dict = amp_configs_map.get(slug) config_dict = amp_configs_map.get(slug)
@@ -149,10 +150,9 @@ class MCPToolResolver:
mcp_server_config = self._build_mcp_config_from_dict(config_dict) mcp_server_config = self._build_mcp_config_from_dict(config_dict)
try: try:
tools, client = self._resolve_native(mcp_server_config) tools, clients = self._resolve_native(mcp_server_config)
resolved_cache[slug] = (tools, client) resolved_cache[slug] = (tools, clients)
if client: all_clients.extend(clients)
all_clients.append(client)
except Exception as e: except Exception as e:
crewai_event_bus.emit( crewai_event_bus.emit(
self, self,
@@ -170,8 +170,9 @@ class MCPToolResolver:
slug_tools, _ = cached slug_tools, _ = cached
if specific_tool: if specific_tool:
sanitized = sanitize_tool_name(specific_tool)
all_tools.extend( all_tools.extend(
t for t in slug_tools if t.name.endswith(f"_{specific_tool}") t for t in slug_tools if t.name.endswith(f"_{sanitized}")
) )
else: else:
all_tools.extend(slug_tools) all_tools.extend(slug_tools)
@@ -198,7 +199,6 @@ class MCPToolResolver:
plus_api = PlusAPI(api_key=get_platform_integration_token()) plus_api = PlusAPI(api_key=get_platform_integration_token())
response = plus_api.get_mcp_configs(slugs) response = plus_api.get_mcp_configs(slugs)
if response.status_code == 200: if response.status_code == 200:
configs: dict[str, dict[str, Any]] = response.json().get("configs", {}) configs: dict[str, dict[str, Any]] = response.json().get("configs", {})
return configs return configs
@@ -218,6 +218,7 @@ class MCPToolResolver:
def _resolve_external(self, mcp_ref: str) -> list[BaseTool]: def _resolve_external(self, mcp_ref: str) -> list[BaseTool]:
"""Resolve an HTTPS MCP server URL into tools.""" """Resolve an HTTPS MCP server URL into tools."""
from crewai.tools.base_tool import BaseTool
from crewai.tools.mcp_tool_wrapper import MCPToolWrapper from crewai.tools.mcp_tool_wrapper import MCPToolWrapper
if "#" in mcp_ref: if "#" in mcp_ref:
@@ -227,6 +228,9 @@ class MCPToolResolver:
server_params = {"url": server_url} server_params = {"url": server_url}
server_name = self._extract_server_name(server_url) server_name = self._extract_server_name(server_url)
sanitized_specific_tool = (
sanitize_tool_name(specific_tool) if specific_tool else None
)
try: try:
tool_schemas = self._get_mcp_tool_schemas(server_params) tool_schemas = self._get_mcp_tool_schemas(server_params)
@@ -239,7 +243,7 @@ class MCPToolResolver:
tools = [] tools = []
for tool_name, schema in tool_schemas.items(): for tool_name, schema in tool_schemas.items():
if specific_tool and tool_name != specific_tool: if sanitized_specific_tool and tool_name != sanitized_specific_tool:
continue continue
try: try:
@@ -271,14 +275,16 @@ class MCPToolResolver:
) )
return [] return []
def _resolve_native( @staticmethod
self, mcp_config: MCPServerConfig def _create_transport(
) -> tuple[list[BaseTool], Any | None]: mcp_config: MCPServerConfig,
"""Resolve an ``MCPServerConfig`` into tools, returning the client for cleanup.""" ) -> tuple[StdioTransport | HTTPTransport | SSETransport, str]:
from crewai.tools.base_tool import BaseTool """Create a fresh transport instance from an MCP server config.
from crewai.tools.mcp_native_tool import MCPNativeTool
transport: StdioTransport | HTTPTransport | SSETransport Returns a ``(transport, server_name)`` tuple. Each call produces an
independent transport so that parallel tool executions never share
state.
"""
if isinstance(mcp_config, MCPServerStdio): if isinstance(mcp_config, MCPServerStdio):
transport = StdioTransport( transport = StdioTransport(
command=mcp_config.command, command=mcp_config.command,
@@ -292,38 +298,54 @@ class MCPToolResolver:
headers=mcp_config.headers, headers=mcp_config.headers,
streamable=mcp_config.streamable, streamable=mcp_config.streamable,
) )
server_name = self._extract_server_name(mcp_config.url) server_name = MCPToolResolver._extract_server_name(mcp_config.url)
elif isinstance(mcp_config, MCPServerSSE): elif isinstance(mcp_config, MCPServerSSE):
transport = SSETransport( transport = SSETransport(
url=mcp_config.url, url=mcp_config.url,
headers=mcp_config.headers, headers=mcp_config.headers,
) )
server_name = self._extract_server_name(mcp_config.url) server_name = MCPToolResolver._extract_server_name(mcp_config.url)
else: else:
raise ValueError(f"Unsupported MCP server config type: {type(mcp_config)}") raise ValueError(f"Unsupported MCP server config type: {type(mcp_config)}")
return transport, server_name
client = MCPClient( def _resolve_native(
transport=transport, self, mcp_config: MCPServerConfig
) -> tuple[list[BaseTool], list[Any]]:
"""Resolve an ``MCPServerConfig`` into tools.
Returns ``(tools, clients)`` where *clients* is always empty for
native tools (clients are now created on-demand per invocation).
A ``client_factory`` closure is passed to each ``MCPNativeTool`` so
every call -- even concurrent calls to the *same* tool -- gets its
own ``MCPClient`` + transport with no shared mutable state.
"""
from crewai.tools.base_tool import BaseTool
from crewai.tools.mcp_native_tool import MCPNativeTool
discovery_transport, server_name = self._create_transport(mcp_config)
discovery_client = MCPClient(
transport=discovery_transport,
cache_tools_list=mcp_config.cache_tools_list, cache_tools_list=mcp_config.cache_tools_list,
) )
async def _setup_client_and_list_tools() -> list[dict[str, Any]]: async def _setup_client_and_list_tools() -> list[dict[str, Any]]:
try: try:
if not client.connected: if not discovery_client.connected:
await client.connect() await discovery_client.connect()
tools_list = await client.list_tools() tools_list = await discovery_client.list_tools()
try: try:
await client.disconnect() await discovery_client.disconnect()
await asyncio.sleep(0.1) await asyncio.sleep(0.1)
except Exception as e: except Exception as e:
self._logger.log("error", f"Error during disconnect: {e}") self._logger.log("error", f"Error during disconnect: {e}")
return tools_list return tools_list
except Exception as e: except Exception as e:
if client.connected: if discovery_client.connected:
await client.disconnect() await discovery_client.disconnect()
await asyncio.sleep(0.1) await asyncio.sleep(0.1)
raise RuntimeError( raise RuntimeError(
f"Error during setup client and list tools: {e}" f"Error during setup client and list tools: {e}"
@@ -334,9 +356,10 @@ class MCPToolResolver:
asyncio.get_running_loop() asyncio.get_running_loop()
import concurrent.futures import concurrent.futures
ctx = contextvars.copy_context()
with concurrent.futures.ThreadPoolExecutor() as executor: with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit( future = executor.submit(
asyncio.run, _setup_client_and_list_tools() ctx.run, asyncio.run, _setup_client_and_list_tools()
) )
tools_list = future.result() tools_list = future.result()
except RuntimeError: except RuntimeError:
@@ -376,6 +399,13 @@ class MCPToolResolver:
filtered_tools.append(tool) filtered_tools.append(tool)
tools_list = filtered_tools tools_list = filtered_tools
def _client_factory() -> MCPClient:
transport, _ = self._create_transport(mcp_config)
return MCPClient(
transport=transport,
cache_tools_list=mcp_config.cache_tools_list,
)
tools = [] tools = []
for tool_def in tools_list: for tool_def in tools_list:
tool_name = tool_def.get("name", "") tool_name = tool_def.get("name", "")
@@ -396,7 +426,7 @@ class MCPToolResolver:
try: try:
native_tool = MCPNativeTool( native_tool = MCPNativeTool(
mcp_client=client, client_factory=_client_factory,
tool_name=tool_name, tool_name=tool_name,
tool_schema=tool_schema, tool_schema=tool_schema,
server_name=server_name, server_name=server_name,
@@ -407,10 +437,10 @@ class MCPToolResolver:
self._logger.log("error", f"Failed to create native MCP tool: {e}") self._logger.log("error", f"Failed to create native MCP tool: {e}")
continue continue
return cast(list[BaseTool], tools), client return cast(list[BaseTool], tools), []
except Exception as e: except Exception as e:
if client.connected: if discovery_client.connected:
asyncio.run(client.disconnect()) asyncio.run(discovery_client.disconnect())
raise RuntimeError(f"Failed to get native MCP tools: {e}") from e raise RuntimeError(f"Failed to get native MCP tools: {e}") from e

View File

@@ -19,6 +19,7 @@ from crewai.memory.types import (
embed_texts, embed_texts,
) )
_LAZY_IMPORTS: dict[str, tuple[str, str]] = { _LAZY_IMPORTS: dict[str, tuple[str, str]] = {
"Memory": ("crewai.memory.unified_memory", "Memory"), "Memory": ("crewai.memory.unified_memory", "Memory"),
"EncodingFlow": ("crewai.memory.encoding_flow", "EncodingFlow"), "EncodingFlow": ("crewai.memory.encoding_flow", "EncodingFlow"),

View File

@@ -308,7 +308,9 @@ def analyze_for_save(
return MemoryAnalysis.model_validate(response) return MemoryAnalysis.model_validate(response)
except Exception as e: except Exception as e:
_logger.warning( _logger.warning(
"Memory save analysis failed, using defaults: %s", e, exc_info=False, "Memory save analysis failed, using defaults: %s",
e,
exc_info=False,
) )
return _SAVE_DEFAULTS return _SAVE_DEFAULTS
@@ -366,6 +368,8 @@ def analyze_for_consolidation(
return ConsolidationPlan.model_validate(response) return ConsolidationPlan.model_validate(response)
except Exception as e: except Exception as e:
_logger.warning( _logger.warning(
"Consolidation analysis failed, defaulting to insert: %s", e, exc_info=False, "Consolidation analysis failed, defaulting to insert: %s",
e,
exc_info=False,
) )
return _CONSOLIDATION_DEFAULT return _CONSOLIDATION_DEFAULT

View File

@@ -11,7 +11,9 @@ Orchestrates the encoding side of memory in a single Flow with 5 steps:
from __future__ import annotations from __future__ import annotations
from concurrent.futures import Future, ThreadPoolExecutor from concurrent.futures import Future, ThreadPoolExecutor
import contextvars
from datetime import datetime from datetime import datetime
import logging
import math import math
from typing import Any from typing import Any
from uuid import uuid4 from uuid import uuid4
@@ -28,6 +30,8 @@ from crewai.memory.analyze import (
from crewai.memory.types import MemoryConfig, MemoryRecord, embed_texts from crewai.memory.types import MemoryConfig, MemoryRecord, embed_texts
logger = logging.getLogger(__name__)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# State models # State models
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -164,14 +168,20 @@ class EncodingFlow(Flow[EncodingState]):
def parallel_find_similar(self) -> None: def parallel_find_similar(self) -> None:
"""Search storage for similar records, concurrently for all active items.""" """Search storage for similar records, concurrently for all active items."""
items = list(self.state.items) items = list(self.state.items)
active = [(i, item) for i, item in enumerate(items) if not item.dropped and item.embedding] active = [
(i, item)
for i, item in enumerate(items)
if not item.dropped and item.embedding
]
if not active: if not active:
return return
def _search_one(item: ItemState) -> list[tuple[MemoryRecord, float]]: def _search_one(
item: ItemState,
) -> list[tuple[MemoryRecord, float]]:
scope_prefix = item.scope if item.scope and item.scope.strip("/") else None scope_prefix = item.scope if item.scope and item.scope.strip("/") else None
return self._storage.search( return self._storage.search( # type: ignore[no-any-return]
item.embedding, item.embedding,
scope_prefix=scope_prefix, scope_prefix=scope_prefix,
categories=None, categories=None,
@@ -181,14 +191,37 @@ class EncodingFlow(Flow[EncodingState]):
if len(active) == 1: if len(active) == 1:
_, item = active[0] _, item = active[0]
raw = _search_one(item) try:
raw = _search_one(item)
except Exception:
logger.warning(
"Storage search failed in parallel_find_similar, "
"treating item as new",
exc_info=True,
)
raw = []
item.similar_records = [r for r, _ in raw] item.similar_records = [r for r, _ in raw]
item.top_similarity = float(raw[0][1]) if raw else 0.0 item.top_similarity = float(raw[0][1]) if raw else 0.0
else: else:
with ThreadPoolExecutor(max_workers=min(len(active), 8)) as pool: with ThreadPoolExecutor(max_workers=min(len(active), 8)) as pool:
futures = [(i, item, pool.submit(_search_one, item)) for i, item in active] futures = [
(
i,
item,
pool.submit(contextvars.copy_context().run, _search_one, item),
)
for i, item in active
]
for _, item, future in futures: for _, item, future in futures:
raw = future.result() try:
raw = future.result()
except Exception:
logger.warning(
"Storage search failed in parallel_find_similar, "
"treating item as new",
exc_info=True,
)
raw = []
item.similar_records = [r for r, _ in raw] item.similar_records = [r for r, _ in raw]
item.top_similarity = float(raw[0][1]) if raw else 0.0 item.top_similarity = float(raw[0][1]) if raw else 0.0
@@ -250,24 +283,38 @@ class EncodingFlow(Flow[EncodingState]):
# Group B: consolidation only # Group B: consolidation only
self._apply_defaults(item) self._apply_defaults(item)
consol_futures[i] = pool.submit( consol_futures[i] = pool.submit(
contextvars.copy_context().run,
analyze_for_consolidation, analyze_for_consolidation,
item.content, list(item.similar_records), self._llm, item.content,
list(item.similar_records),
self._llm,
) )
elif not fields_provided and not has_similar: elif not fields_provided and not has_similar:
# Group C: field resolution only # Group C: field resolution only
save_futures[i] = pool.submit( save_futures[i] = pool.submit(
contextvars.copy_context().run,
analyze_for_save, analyze_for_save,
item.content, existing_scopes, existing_categories, self._llm, item.content,
existing_scopes,
existing_categories,
self._llm,
) )
else: else:
# Group D: both in parallel # Group D: both in parallel
save_futures[i] = pool.submit( save_futures[i] = pool.submit(
contextvars.copy_context().run,
analyze_for_save, analyze_for_save,
item.content, existing_scopes, existing_categories, self._llm, item.content,
existing_scopes,
existing_categories,
self._llm,
) )
consol_futures[i] = pool.submit( consol_futures[i] = pool.submit(
contextvars.copy_context().run,
analyze_for_consolidation, analyze_for_consolidation,
item.content, list(item.similar_records), self._llm, item.content,
list(item.similar_records),
self._llm,
) )
# Collect field-resolution results # Collect field-resolution results
@@ -300,8 +347,8 @@ class EncodingFlow(Flow[EncodingState]):
item.plan = ConsolidationPlan(actions=[], insert_new=True) item.plan = ConsolidationPlan(actions=[], insert_new=True)
# Collect consolidation results # Collect consolidation results
for i, future in consol_futures.items(): for i, consol_future in consol_futures.items():
items[i].plan = future.result() items[i].plan = consol_future.result()
finally: finally:
pool.shutdown(wait=False) pool.shutdown(wait=False)
@@ -339,7 +386,9 @@ class EncodingFlow(Flow[EncodingState]):
# similar_records overlap). Collect one action per record_id, first wins. # similar_records overlap). Collect one action per record_id, first wins.
# Also build a map from record_id to the original MemoryRecord for updates. # Also build a map from record_id to the original MemoryRecord for updates.
dedup_deletes: set[str] = set() # record_ids to delete dedup_deletes: set[str] = set() # record_ids to delete
dedup_updates: dict[str, tuple[int, str]] = {} # record_id -> (item_idx, new_content) dedup_updates: dict[
str, tuple[int, str]
] = {} # record_id -> (item_idx, new_content)
all_similar: dict[str, MemoryRecord] = {} # record_id -> MemoryRecord all_similar: dict[str, MemoryRecord] = {} # record_id -> MemoryRecord
for i, item in enumerate(items): for i, item in enumerate(items):
@@ -350,13 +399,24 @@ class EncodingFlow(Flow[EncodingState]):
all_similar[r.id] = r all_similar[r.id] = r
for action in item.plan.actions: for action in item.plan.actions:
rid = action.record_id rid = action.record_id
if action.action == "delete" and rid not in dedup_deletes and rid not in dedup_updates: if (
action.action == "delete"
and rid not in dedup_deletes
and rid not in dedup_updates
):
dedup_deletes.add(rid) dedup_deletes.add(rid)
elif action.action == "update" and action.new_content and rid not in dedup_deletes and rid not in dedup_updates: elif (
action.action == "update"
and action.new_content
and rid not in dedup_deletes
and rid not in dedup_updates
):
dedup_updates[rid] = (i, action.new_content) dedup_updates[rid] = (i, action.new_content)
# --- Batch re-embed all update contents in ONE call --- # --- Batch re-embed all update contents in ONE call ---
update_list = list(dedup_updates.items()) # [(record_id, (item_idx, new_content)), ...] update_list = list(
dedup_updates.items()
) # [(record_id, (item_idx, new_content)), ...]
update_embeddings: list[list[float]] = [] update_embeddings: list[list[float]] = []
if update_list: if update_list:
update_contents = [content for _, (_, content) in update_list] update_contents = [content for _, (_, content) in update_list]
@@ -377,51 +437,52 @@ class EncodingFlow(Flow[EncodingState]):
if item.dropped or item.plan is None: if item.dropped or item.plan is None:
continue continue
if item.plan.insert_new: if item.plan.insert_new:
to_insert.append((i, MemoryRecord( to_insert.append(
content=item.content, (
scope=item.resolved_scope, i,
categories=item.resolved_categories, MemoryRecord(
metadata=item.resolved_metadata, content=item.content,
importance=item.resolved_importance, scope=item.resolved_scope,
embedding=item.embedding if item.embedding else None, categories=item.resolved_categories,
source=item.resolved_source, metadata=item.resolved_metadata,
private=item.resolved_private, importance=item.resolved_importance,
))) embedding=item.embedding if item.embedding else None,
source=item.resolved_source,
# All storage mutations under one lock so no other pipeline can private=item.resolved_private,
# interleave and cause version conflicts. The lock is reentrant ),
# (RLock) so the individual storage methods re-acquire it safely.
updated_records: dict[str, MemoryRecord] = {}
with self._storage.write_lock:
if dedup_deletes:
self._storage.delete(record_ids=list(dedup_deletes))
self.state.records_deleted += len(dedup_deletes)
for rid, (_item_idx, new_content) in dedup_updates.items():
existing = all_similar.get(rid)
if existing is not None:
new_emb = update_emb_map.get(rid, [])
updated = MemoryRecord(
id=existing.id,
content=new_content,
scope=existing.scope,
categories=existing.categories,
metadata=existing.metadata,
importance=existing.importance,
created_at=existing.created_at,
last_accessed=now,
embedding=new_emb if new_emb else existing.embedding,
) )
self._storage.update(updated) )
self.state.records_updated += 1
updated_records[rid] = updated
if to_insert: updated_records: dict[str, MemoryRecord] = {}
records = [r for _, r in to_insert] if dedup_deletes:
self._storage.save(records) self._storage.delete(record_ids=list(dedup_deletes))
self.state.records_inserted += len(records) self.state.records_deleted += len(dedup_deletes)
for idx, record in to_insert:
items[idx].result_record = record for rid, (_item_idx, new_content) in dedup_updates.items():
existing = all_similar.get(rid)
if existing is not None:
new_emb = update_emb_map.get(rid, [])
updated = MemoryRecord(
id=existing.id,
content=new_content,
scope=existing.scope,
categories=existing.categories,
metadata=existing.metadata,
importance=existing.importance,
created_at=existing.created_at,
last_accessed=now,
embedding=new_emb if new_emb else existing.embedding,
)
self._storage.update(updated)
self.state.records_updated += 1
updated_records[rid] = updated
if to_insert:
records = [r for _, r in to_insert]
self._storage.save(records)
self.state.records_inserted += len(records)
for idx, record in to_insert:
items[idx].result_record = record
# Set result_record for non-insert items (after lock, using updated_records) # Set result_record for non-insert items (after lock, using updated_records)
for _i, item in enumerate(items): for _i, item in enumerate(items):

View File

@@ -3,11 +3,9 @@
from __future__ import annotations from __future__ import annotations
from datetime import datetime from datetime import datetime
from typing import TYPE_CHECKING, Any from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field, PrivateAttr, model_validator
if TYPE_CHECKING:
from crewai.memory.unified_memory import Memory
from crewai.memory.types import ( from crewai.memory.types import (
_RECALL_OVERSAMPLE_FACTOR, _RECALL_OVERSAMPLE_FACTOR,
@@ -15,22 +13,38 @@ from crewai.memory.types import (
MemoryRecord, MemoryRecord,
ScopeInfo, ScopeInfo,
) )
from crewai.memory.unified_memory import Memory
class MemoryScope: class MemoryScope(BaseModel):
"""View of Memory restricted to a root path. All operations are scoped under that path.""" """View of Memory restricted to a root path. All operations are scoped under that path."""
def __init__(self, memory: Memory, root_path: str) -> None: model_config = ConfigDict(arbitrary_types_allowed=True)
"""Initialize scope.
Args: root_path: str = Field(default="/")
memory: The underlying Memory instance.
root_path: Root path for this scope (e.g. /agent/1). _memory: Memory = PrivateAttr()
""" _root: str = PrivateAttr()
self._memory = memory
self._root = root_path.rstrip("/") or "" @model_validator(mode="wrap")
if self._root and not self._root.startswith("/"): @classmethod
self._root = "/" + self._root def _accept_memory(cls, data: Any, handler: Any) -> MemoryScope:
"""Extract memory dependency and normalize root path before validation."""
if isinstance(data, MemoryScope):
return data
memory = data.pop("memory")
instance: MemoryScope = handler(data)
instance._memory = memory
root = instance.root_path.rstrip("/") or ""
if root and not root.startswith("/"):
root = "/" + root
instance._root = root
return instance
@property
def read_only(self) -> bool:
"""Whether the underlying memory is read-only."""
return self._memory.read_only
def _scope_path(self, scope: str | None) -> str: def _scope_path(self, scope: str | None) -> str:
if not scope or scope == "/": if not scope or scope == "/":
@@ -52,7 +66,7 @@ class MemoryScope:
importance: float | None = None, importance: float | None = None,
source: str | None = None, source: str | None = None,
private: bool = False, private: bool = False,
) -> MemoryRecord: ) -> MemoryRecord | None:
"""Remember content; scope is relative to this scope's root.""" """Remember content; scope is relative to this scope's root."""
path = self._scope_path(scope) path = self._scope_path(scope)
return self._memory.remember( return self._memory.remember(
@@ -71,7 +85,7 @@ class MemoryScope:
scope: str | None = None, scope: str | None = None,
categories: list[str] | None = None, categories: list[str] | None = None,
limit: int = 10, limit: int = 10,
depth: str = "deep", depth: Literal["shallow", "deep"] = "deep",
source: str | None = None, source: str | None = None,
include_private: bool = False, include_private: bool = False,
) -> list[MemoryMatch]: ) -> list[MemoryMatch]:
@@ -138,34 +152,34 @@ class MemoryScope:
"""Return a narrower scope under this scope.""" """Return a narrower scope under this scope."""
child = path.strip("/") child = path.strip("/")
if not child: if not child:
return MemoryScope(self._memory, self._root or "/") return MemoryScope(memory=self._memory, root_path=self._root or "/")
base = self._root.rstrip("/") or "" base = self._root.rstrip("/") or ""
new_root = f"{base}/{child}" if base else f"/{child}" new_root = f"{base}/{child}" if base else f"/{child}"
return MemoryScope(self._memory, new_root) return MemoryScope(memory=self._memory, root_path=new_root)
class MemorySlice: class MemorySlice(BaseModel):
"""View over multiple scopes: recall searches all, remember is a no-op when read_only.""" """View over multiple scopes: recall searches all, remember is a no-op when read_only."""
def __init__( model_config = ConfigDict(arbitrary_types_allowed=True)
self,
memory: Memory,
scopes: list[str],
categories: list[str] | None = None,
read_only: bool = True,
) -> None:
"""Initialize slice.
Args: scopes: list[str] = Field(default_factory=list)
memory: The underlying Memory instance. categories: list[str] | None = Field(default=None)
scopes: List of scope paths to include. read_only: bool = Field(default=True)
categories: Optional category filter for recall.
read_only: If True, remember() is a silent no-op. _memory: Memory = PrivateAttr()
"""
self._memory = memory @model_validator(mode="wrap")
self._scopes = [s.rstrip("/") or "/" for s in scopes] @classmethod
self._categories = categories def _accept_memory(cls, data: Any, handler: Any) -> MemorySlice:
self._read_only = read_only """Extract memory dependency and normalize scopes before validation."""
if isinstance(data, MemorySlice):
return data
memory = data.pop("memory")
data["scopes"] = [s.rstrip("/") or "/" for s in data.get("scopes", [])]
instance: MemorySlice = handler(data)
instance._memory = memory
return instance
def remember( def remember(
self, self,
@@ -178,7 +192,7 @@ class MemorySlice:
private: bool = False, private: bool = False,
) -> MemoryRecord | None: ) -> MemoryRecord | None:
"""Remember into an explicit scope. No-op when read_only=True.""" """Remember into an explicit scope. No-op when read_only=True."""
if self._read_only: if self.read_only:
return None return None
return self._memory.remember( return self._memory.remember(
content, content,
@@ -196,14 +210,14 @@ class MemorySlice:
scope: str | None = None, scope: str | None = None,
categories: list[str] | None = None, categories: list[str] | None = None,
limit: int = 10, limit: int = 10,
depth: str = "deep", depth: Literal["shallow", "deep"] = "deep",
source: str | None = None, source: str | None = None,
include_private: bool = False, include_private: bool = False,
) -> list[MemoryMatch]: ) -> list[MemoryMatch]:
"""Recall across all slice scopes; results merged and re-ranked.""" """Recall across all slice scopes; results merged and re-ranked."""
cats = categories or self._categories cats = categories or self.categories
all_matches: list[MemoryMatch] = [] all_matches: list[MemoryMatch] = []
for sc in self._scopes: for sc in self.scopes:
matches = self._memory.recall( matches = self._memory.recall(
query, query,
scope=sc, scope=sc,
@@ -231,7 +245,7 @@ class MemorySlice:
def list_scopes(self, path: str = "/") -> list[str]: def list_scopes(self, path: str = "/") -> list[str]:
"""List scopes across all slice roots.""" """List scopes across all slice roots."""
out: list[str] = [] out: list[str] = []
for sc in self._scopes: for sc in self.scopes:
full = f"{sc.rstrip('/')}{path}" if sc != "/" else path full = f"{sc.rstrip('/')}{path}" if sc != "/" else path
out.extend(self._memory.list_scopes(full)) out.extend(self._memory.list_scopes(full))
return sorted(set(out)) return sorted(set(out))
@@ -243,15 +257,23 @@ class MemorySlice:
oldest: datetime | None = None oldest: datetime | None = None
newest: datetime | None = None newest: datetime | None = None
children: list[str] = [] children: list[str] = []
for sc in self._scopes: for sc in self.scopes:
full = f"{sc.rstrip('/')}{path}" if sc != "/" else path full = f"{sc.rstrip('/')}{path}" if sc != "/" else path
inf = self._memory.info(full) inf = self._memory.info(full)
total_records += inf.record_count total_records += inf.record_count
all_categories.update(inf.categories) all_categories.update(inf.categories)
if inf.oldest_record: if inf.oldest_record:
oldest = inf.oldest_record if oldest is None else min(oldest, inf.oldest_record) oldest = (
inf.oldest_record
if oldest is None
else min(oldest, inf.oldest_record)
)
if inf.newest_record: if inf.newest_record:
newest = inf.newest_record if newest is None else max(newest, inf.newest_record) newest = (
inf.newest_record
if newest is None
else max(newest, inf.newest_record)
)
children.extend(inf.child_scopes) children.extend(inf.child_scopes)
return ScopeInfo( return ScopeInfo(
path=path, path=path,
@@ -265,7 +287,7 @@ class MemorySlice:
def list_categories(self, path: str | None = None) -> dict[str, int]: def list_categories(self, path: str | None = None) -> dict[str, int]:
"""Categories and counts across slice scopes.""" """Categories and counts across slice scopes."""
counts: dict[str, int] = {} counts: dict[str, int] = {}
for sc in self._scopes: for sc in self.scopes:
full = (f"{sc.rstrip('/')}{path}" if sc != "/" else path) if path else sc full = (f"{sc.rstrip('/')}{path}" if sc != "/" else path) if path else sc
for k, v in self._memory.list_categories(full).items(): for k, v in self._memory.list_categories(full).items():
counts[k] = counts.get(k, 0) + v counts[k] = counts.get(k, 0) + v

View File

@@ -2,7 +2,6 @@
Implements adaptive-depth retrieval with: Implements adaptive-depth retrieval with:
- LLM query distillation into targeted sub-queries - LLM query distillation into targeted sub-queries
- Keyword-driven category filtering
- Time-based filtering from temporal hints - Time-based filtering from temporal hints
- Parallel multi-query, multi-scope search - Parallel multi-query, multi-scope search
- Confidence-based routing with iterative deepening (budget loop) - Confidence-based routing with iterative deepening (budget loop)
@@ -12,7 +11,9 @@ Implements adaptive-depth retrieval with:
from __future__ import annotations from __future__ import annotations
from concurrent.futures import ThreadPoolExecutor, as_completed from concurrent.futures import ThreadPoolExecutor, as_completed
import contextvars
from datetime import datetime from datetime import datetime
import logging
from typing import Any from typing import Any
from uuid import uuid4 from uuid import uuid4
@@ -30,6 +31,9 @@ from crewai.memory.types import (
) )
logger = logging.getLogger(__name__)
class RecallState(BaseModel): class RecallState(BaseModel):
"""State for the recall flow.""" """State for the recall flow."""
@@ -37,7 +41,6 @@ class RecallState(BaseModel):
query: str = "" query: str = ""
scope: str | None = None scope: str | None = None
categories: list[str] | None = None categories: list[str] | None = None
inferred_categories: list[str] = Field(default_factory=list)
time_cutoff: datetime | None = None time_cutoff: datetime | None = None
source: str | None = None source: str | None = None
include_private: bool = False include_private: bool = False
@@ -82,11 +85,8 @@ class RecallFlow(Flow[RecallState]):
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def _merged_categories(self) -> list[str] | None: def _merged_categories(self) -> list[str] | None:
"""Merge caller-supplied and LLM-inferred categories.""" """Return caller-supplied categories, or None if empty."""
merged = list( return self.state.categories or None
set((self.state.categories or []) + self.state.inferred_categories)
)
return merged or None
def _do_search(self) -> list[dict[str, Any]]: def _do_search(self) -> list[dict[str, Any]]:
"""Run parallel search across (embeddings x scopes) with filters. """Run parallel search across (embeddings x scopes) with filters.
@@ -108,13 +108,12 @@ class RecallFlow(Flow[RecallState]):
) )
# Post-filter by time cutoff # Post-filter by time cutoff
if self.state.time_cutoff and raw: if self.state.time_cutoff and raw:
raw = [ raw = [(r, s) for r, s in raw if r.created_at >= self.state.time_cutoff]
(r, s) for r, s in raw if r.created_at >= self.state.time_cutoff
]
# Privacy filter # Privacy filter
if not self.state.include_private and raw: if not self.state.include_private and raw:
raw = [ raw = [
(r, s) for r, s in raw (r, s)
for r, s in raw
if not r.private or r.source == self.state.source if not r.private or r.source == self.state.source
] ]
return scope, raw return scope, raw
@@ -130,38 +129,57 @@ class RecallFlow(Flow[RecallState]):
if len(tasks) <= 1: if len(tasks) <= 1:
for emb, sc in tasks: for emb, sc in tasks:
scope, results = _search_one(emb, sc) try:
scope, results = _search_one(emb, sc)
except Exception:
logger.warning(
"Storage search failed in recall flow, skipping scope",
exc_info=True,
)
continue
if results: if results:
top_composite, _ = compute_composite_score( top_composite, _ = compute_composite_score(
results[0][0], results[0][1], self._config results[0][0], results[0][1], self._config
) )
findings.append({ findings.append(
"scope": scope, {
"results": results, "scope": scope,
"top_score": top_composite, "results": results,
}) "top_score": top_composite,
}
)
else: else:
with ThreadPoolExecutor(max_workers=min(len(tasks), 4)) as pool: with ThreadPoolExecutor(max_workers=min(len(tasks), 4)) as pool:
futures = { futures = {
pool.submit(_search_one, emb, sc): (emb, sc) pool.submit(contextvars.copy_context().run, _search_one, emb, sc): (
emb,
sc,
)
for emb, sc in tasks for emb, sc in tasks
} }
for future in as_completed(futures): for future in as_completed(futures):
scope, results = future.result() try:
scope, results = future.result()
except Exception:
logger.warning(
"Storage search failed in recall flow, skipping scope",
exc_info=True,
)
continue
if results: if results:
top_composite, _ = compute_composite_score( top_composite, _ = compute_composite_score(
results[0][0], results[0][1], self._config results[0][0], results[0][1], self._config
) )
findings.append({ findings.append(
"scope": scope, {
"results": results, "scope": scope,
"top_score": top_composite, "results": results,
}) "top_score": top_composite,
}
)
self.state.chunk_findings = findings self.state.chunk_findings = findings
self.state.confidence = max( self.state.confidence = max((f["top_score"] for f in findings), default=0.0)
(f["top_score"] for f in findings), default=0.0
)
return findings return findings
# ------------------------------------------------------------------ # ------------------------------------------------------------------
@@ -212,19 +230,19 @@ class RecallFlow(Flow[RecallState]):
) )
self.state.query_analysis = analysis self.state.query_analysis = analysis
# Wire keywords -> category filter
if analysis.keywords:
self.state.inferred_categories = analysis.keywords
# Parse time_filter into a datetime cutoff # Parse time_filter into a datetime cutoff
if analysis.time_filter: if analysis.time_filter:
try: try:
self.state.time_cutoff = datetime.fromisoformat(analysis.time_filter) self.state.time_cutoff = datetime.fromisoformat(
analysis.time_filter
)
except ValueError: except ValueError:
pass pass
# Batch-embed all sub-queries in ONE call # Batch-embed all sub-queries in ONE call
queries = analysis.recall_queries if analysis.recall_queries else [self.state.query] queries = (
analysis.recall_queries if analysis.recall_queries else [self.state.query]
)
queries = queries[:3] queries = queries[:3]
embeddings = embed_texts(self._embedder, queries) embeddings = embed_texts(self._embedder, queries)
pairs: list[tuple[str, list[float]]] = [ pairs: list[tuple[str, list[float]]] = [
@@ -246,13 +264,17 @@ class RecallFlow(Flow[RecallState]):
if analysis and analysis.suggested_scopes: if analysis and analysis.suggested_scopes:
candidates = [s for s in analysis.suggested_scopes if s] candidates = [s for s in analysis.suggested_scopes if s]
else: else:
candidates = self._storage.list_scopes(scope_prefix) try:
candidates = self._storage.list_scopes(scope_prefix)
except Exception:
logger.warning(
"Storage list_scopes failed in filter_and_chunk, "
"falling back to scope prefix",
exc_info=True,
)
candidates = []
if not candidates: if not candidates:
info = self._storage.get_scope_info(scope_prefix) candidates = [scope_prefix]
if info.record_count > 0:
candidates = [scope_prefix]
else:
candidates = [scope_prefix]
self.state.candidate_scopes = candidates[:20] self.state.candidate_scopes = candidates[:20]
return self.state.candidate_scopes return self.state.candidate_scopes
@@ -305,17 +327,21 @@ class RecallFlow(Flow[RecallState]):
response = self._llm.call([{"role": "user", "content": prompt}]) response = self._llm.call([{"role": "user", "content": prompt}])
if isinstance(response, str) and "missing" in response.lower(): if isinstance(response, str) and "missing" in response.lower():
self.state.evidence_gaps.append(response[:200]) self.state.evidence_gaps.append(response[:200])
enhanced.append({ enhanced.append(
"scope": finding["scope"], {
"extraction": response, "scope": finding["scope"],
"results": finding["results"], "extraction": response,
}) "results": finding["results"],
}
)
except Exception: except Exception:
enhanced.append({ enhanced.append(
"scope": finding["scope"], {
"extraction": "", "scope": finding["scope"],
"results": finding["results"], "extraction": "",
}) "results": finding["results"],
}
)
self.state.chunk_findings = enhanced self.state.chunk_findings = enhanced
return enhanced return enhanced
@@ -327,7 +353,7 @@ class RecallFlow(Flow[RecallState]):
@router(re_search) @router(re_search)
def re_decide_depth(self) -> str: def re_decide_depth(self) -> str:
"""Re-evaluate depth after re-search. Same logic as decide_depth.""" """Re-evaluate depth after re-search. Same logic as decide_depth."""
return self.decide_depth() return self.decide_depth() # type: ignore[call-arg]
@listen("synthesize") @listen("synthesize")
def synthesize_results(self) -> list[MemoryMatch]: def synthesize_results(self) -> list[MemoryMatch]:

View File

@@ -1,5 +1,6 @@
import json import json
import logging import logging
import os
from pathlib import Path from pathlib import Path
import sqlite3 import sqlite3
from typing import Any from typing import Any
@@ -8,6 +9,7 @@ from crewai.task import Task
from crewai.utilities import Printer from crewai.utilities import Printer
from crewai.utilities.crew_json_encoder import CrewJSONEncoder from crewai.utilities.crew_json_encoder import CrewJSONEncoder
from crewai.utilities.errors import DatabaseError, DatabaseOperationError from crewai.utilities.errors import DatabaseError, DatabaseOperationError
from crewai.utilities.lock_store import lock as store_lock
from crewai.utilities.paths import db_storage_path from crewai.utilities.paths import db_storage_path
@@ -24,6 +26,7 @@ class KickoffTaskOutputsSQLiteStorage:
# Get the parent directory of the default db path and create our db file there # Get the parent directory of the default db path and create our db file there
db_path = str(Path(db_storage_path()) / "latest_kickoff_task_outputs.db") db_path = str(Path(db_storage_path()) / "latest_kickoff_task_outputs.db")
self.db_path = db_path self.db_path = db_path
self._lock_name = f"sqlite:{os.path.realpath(self.db_path)}"
self._printer: Printer = Printer() self._printer: Printer = Printer()
self._initialize_db() self._initialize_db()
@@ -38,23 +41,25 @@ class KickoffTaskOutputsSQLiteStorage:
DatabaseOperationError: If database initialization fails due to SQLite errors. DatabaseOperationError: If database initialization fails due to SQLite errors.
""" """
try: try:
with sqlite3.connect(self.db_path) as conn: with store_lock(self._lock_name):
cursor = conn.cursor() with sqlite3.connect(self.db_path, timeout=30) as conn:
cursor.execute( conn.execute("PRAGMA journal_mode=WAL")
cursor = conn.cursor()
cursor.execute(
"""
CREATE TABLE IF NOT EXISTS latest_kickoff_task_outputs (
task_id TEXT PRIMARY KEY,
expected_output TEXT,
output JSON,
task_index INTEGER,
inputs JSON,
was_replayed BOOLEAN,
timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
)
""" """
CREATE TABLE IF NOT EXISTS latest_kickoff_task_outputs (
task_id TEXT PRIMARY KEY,
expected_output TEXT,
output JSON,
task_index INTEGER,
inputs JSON,
was_replayed BOOLEAN,
timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
) )
"""
)
conn.commit() conn.commit()
except sqlite3.Error as e: except sqlite3.Error as e:
error_msg = DatabaseError.format_error(DatabaseError.INIT_ERROR, e) error_msg = DatabaseError.format_error(DatabaseError.INIT_ERROR, e)
logger.error(error_msg) logger.error(error_msg)
@@ -82,25 +87,26 @@ class KickoffTaskOutputsSQLiteStorage:
""" """
inputs = inputs or {} inputs = inputs or {}
try: try:
with sqlite3.connect(self.db_path) as conn: with store_lock(self._lock_name):
conn.execute("BEGIN TRANSACTION") with sqlite3.connect(self.db_path, timeout=30) as conn:
cursor = conn.cursor() conn.execute("BEGIN TRANSACTION")
cursor.execute( cursor = conn.cursor()
""" cursor.execute(
INSERT OR REPLACE INTO latest_kickoff_task_outputs """
(task_id, expected_output, output, task_index, inputs, was_replayed) INSERT OR REPLACE INTO latest_kickoff_task_outputs
VALUES (?, ?, ?, ?, ?, ?) (task_id, expected_output, output, task_index, inputs, was_replayed)
""", VALUES (?, ?, ?, ?, ?, ?)
( """,
str(task.id), (
task.expected_output, str(task.id),
json.dumps(output, cls=CrewJSONEncoder), task.expected_output,
task_index, json.dumps(output, cls=CrewJSONEncoder),
json.dumps(inputs, cls=CrewJSONEncoder), task_index,
was_replayed, json.dumps(inputs, cls=CrewJSONEncoder),
), was_replayed,
) ),
conn.commit() )
conn.commit()
except sqlite3.Error as e: except sqlite3.Error as e:
error_msg = DatabaseError.format_error(DatabaseError.SAVE_ERROR, e) error_msg = DatabaseError.format_error(DatabaseError.SAVE_ERROR, e)
logger.error(error_msg) logger.error(error_msg)
@@ -125,30 +131,31 @@ class KickoffTaskOutputsSQLiteStorage:
DatabaseOperationError: If updating the task output fails due to SQLite errors. DatabaseOperationError: If updating the task output fails due to SQLite errors.
""" """
try: try:
with sqlite3.connect(self.db_path) as conn: with store_lock(self._lock_name):
conn.execute("BEGIN TRANSACTION") with sqlite3.connect(self.db_path, timeout=30) as conn:
cursor = conn.cursor() conn.execute("BEGIN TRANSACTION")
cursor = conn.cursor()
fields = [] fields = []
values = [] values = []
for key, value in kwargs.items(): for key, value in kwargs.items():
fields.append(f"{key} = ?") fields.append(f"{key} = ?")
values.append( values.append(
json.dumps(value, cls=CrewJSONEncoder) json.dumps(value, cls=CrewJSONEncoder)
if isinstance(value, dict) if isinstance(value, dict)
else value else value
) )
query = f"UPDATE latest_kickoff_task_outputs SET {', '.join(fields)} WHERE task_index = ?" # nosec # noqa: S608 query = f"UPDATE latest_kickoff_task_outputs SET {', '.join(fields)} WHERE task_index = ?" # nosec # noqa: S608
values.append(task_index) values.append(task_index)
cursor.execute(query, tuple(values)) cursor.execute(query, tuple(values))
conn.commit() conn.commit()
if cursor.rowcount == 0: if cursor.rowcount == 0:
logger.warning( logger.warning(
f"No row found with task_index {task_index}. No update performed." f"No row found with task_index {task_index}. No update performed."
) )
except sqlite3.Error as e: except sqlite3.Error as e:
error_msg = DatabaseError.format_error(DatabaseError.UPDATE_ERROR, e) error_msg = DatabaseError.format_error(DatabaseError.UPDATE_ERROR, e)
logger.error(error_msg) logger.error(error_msg)
@@ -166,7 +173,7 @@ class KickoffTaskOutputsSQLiteStorage:
DatabaseOperationError: If loading task outputs fails due to SQLite errors. DatabaseOperationError: If loading task outputs fails due to SQLite errors.
""" """
try: try:
with sqlite3.connect(self.db_path) as conn: with sqlite3.connect(self.db_path, timeout=30) as conn:
cursor = conn.cursor() cursor = conn.cursor()
cursor.execute(""" cursor.execute("""
SELECT * SELECT *
@@ -205,11 +212,12 @@ class KickoffTaskOutputsSQLiteStorage:
DatabaseOperationError: If deleting task outputs fails due to SQLite errors. DatabaseOperationError: If deleting task outputs fails due to SQLite errors.
""" """
try: try:
with sqlite3.connect(self.db_path) as conn: with store_lock(self._lock_name):
conn.execute("BEGIN TRANSACTION") with sqlite3.connect(self.db_path, timeout=30) as conn:
cursor = conn.cursor() conn.execute("BEGIN TRANSACTION")
cursor.execute("DELETE FROM latest_kickoff_task_outputs") cursor = conn.cursor()
conn.commit() cursor.execute("DELETE FROM latest_kickoff_task_outputs")
conn.commit()
except sqlite3.Error as e: except sqlite3.Error as e:
error_msg = DatabaseError.format_error(DatabaseError.DELETE_ERROR, e) error_msg = DatabaseError.format_error(DatabaseError.DELETE_ERROR, e)
logger.error(error_msg) logger.error(error_msg)

View File

@@ -2,6 +2,7 @@
from __future__ import annotations from __future__ import annotations
import contextvars
from datetime import datetime from datetime import datetime
import json import json
import logging import logging
@@ -9,11 +10,12 @@ import os
from pathlib import Path from pathlib import Path
import threading import threading
import time import time
from typing import Any, ClassVar from typing import Any
import lancedb import lancedb # type: ignore[import-untyped]
from crewai.memory.types import MemoryRecord, ScopeInfo from crewai.memory.types import MemoryRecord, ScopeInfo
from crewai.utilities.lock_store import lock as store_lock
_logger = logging.getLogger(__name__) _logger = logging.getLogger(__name__)
@@ -39,15 +41,6 @@ _RETRY_BASE_DELAY = 0.2 # seconds; doubles on each retry
class LanceDBStorage: class LanceDBStorage:
"""LanceDB-backed storage for the unified memory system.""" """LanceDB-backed storage for the unified memory system."""
# Class-level registry: maps resolved database path -> shared write lock.
# When multiple Memory instances (e.g. agent + crew) independently create
# LanceDBStorage pointing at the same directory, they share one lock so
# their writes don't conflict.
# Uses RLock (reentrant) so callers can hold the lock for a batch of
# operations while the individual methods re-acquire it without deadlocking.
_path_locks: ClassVar[dict[str, threading.RLock]] = {}
_path_locks_guard: ClassVar[threading.Lock] = threading.Lock()
def __init__( def __init__(
self, self,
path: str | Path | None = None, path: str | Path | None = None,
@@ -83,13 +76,9 @@ class LanceDBStorage:
self._table_name = table_name self._table_name = table_name
self._db = lancedb.connect(str(self._path)) self._db = lancedb.connect(str(self._path))
# On macOS and Linux the default per-process open-file limit is 256.
# A LanceDB table stores one file per fragment (one fragment per save()
# call by default). With hundreds of fragments, a single full-table
# scan opens all of them simultaneously, exhausting the limit.
# Raise it proactively so scans on large tables never hit OS error 24.
try: try:
import resource import resource
soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE) soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
if soft < 4096: if soft < 4096:
resource.setrlimit(resource.RLIMIT_NOFILE, (min(hard, 4096), hard)) resource.setrlimit(resource.RLIMIT_NOFILE, (min(hard, 4096), hard))
@@ -99,68 +88,46 @@ class LanceDBStorage:
self._compact_every = compact_every self._compact_every = compact_every
self._save_count = 0 self._save_count = 0
# Get or create a shared write lock for this database path. self._lock_name = f"lancedb:{self._path.resolve()}"
resolved = str(self._path.resolve())
with LanceDBStorage._path_locks_guard:
if resolved not in LanceDBStorage._path_locks:
LanceDBStorage._path_locks[resolved] = threading.RLock()
self._write_lock = LanceDBStorage._path_locks[resolved]
# Try to open an existing table and infer dimension from its schema. # Try to open an existing table and infer dimension from its schema.
# If no table exists yet, defer creation until the first save so the # If no table exists yet, defer creation until the first save so the
# dimension can be auto-detected from the embedder's actual output. # dimension can be auto-detected from the embedder's actual output.
try: try:
self._table: lancedb.table.Table | None = self._db.open_table(self._table_name) self._table: Any = self._db.open_table(self._table_name)
self._vector_dim: int = self._infer_dim_from_table(self._table) self._vector_dim: int = self._infer_dim_from_table(self._table)
# Best-effort: create the scope index if it doesn't exist yet. with store_lock(self._lock_name):
self._ensure_scope_index() self._ensure_scope_index()
# Compact in the background if the table has accumulated many
# fragments from previous runs (each save() creates one).
self._compact_if_needed() self._compact_if_needed()
except Exception: except Exception:
_logger.debug(
"Failed to open existing LanceDB table %r", table_name, exc_info=True
)
self._table = None self._table = None
self._vector_dim = vector_dim or 0 # 0 = not yet known self._vector_dim = vector_dim or 0 # 0 = not yet known
# Explicit dim provided: create the table immediately if it doesn't exist. # Explicit dim provided: create the table immediately if it doesn't exist.
if self._table is None and vector_dim is not None: if self._table is None and vector_dim is not None:
self._vector_dim = vector_dim self._vector_dim = vector_dim
self._table = self._create_table(vector_dim) with store_lock(self._lock_name):
self._table = self._create_table(vector_dim)
@property
def write_lock(self) -> threading.RLock:
"""The shared reentrant write lock for this database path.
Callers can acquire this to hold the lock across multiple storage
operations (e.g. delete + update + save as one atomic batch).
Individual methods also acquire it internally, but since it's
reentrant (RLock), the same thread won't deadlock.
"""
return self._write_lock
@staticmethod @staticmethod
def _infer_dim_from_table(table: lancedb.table.Table) -> int: def _infer_dim_from_table(table: Any) -> int:
"""Read vector dimension from an existing table's schema.""" """Read vector dimension from an existing table's schema."""
schema = table.schema schema = table.schema
for field in schema: for field in schema:
if field.name == "vector": if field.name == "vector":
try: try:
return field.type.list_size return int(field.type.list_size)
except Exception: except Exception:
break break
return DEFAULT_VECTOR_DIM return DEFAULT_VECTOR_DIM
def _retry_write(self, op: str, *args: Any, **kwargs: Any) -> Any: def _do_write(self, op: str, *args: Any, **kwargs: Any) -> Any:
"""Execute a table operation with retry on LanceDB commit conflicts. """Execute a single table write with retry on commit conflicts.
Args: Caller must already hold ``store_lock(self._lock_name)``.
op: Method name on the table object (e.g. "add", "delete").
*args, **kwargs: Passed to the table method.
LanceDB uses optimistic concurrency: if two transactions overlap,
the second to commit fails with an ``OSError`` containing
"Commit conflict". This helper retries with exponential backoff,
refreshing the table reference before each retry so the retried
call uses the latest committed version (not a stale reference).
""" """
delay = _RETRY_BASE_DELAY delay = _RETRY_BASE_DELAY
for attempt in range(_MAX_RETRIES + 1): for attempt in range(_MAX_RETRIES + 1):
@@ -171,20 +138,24 @@ class LanceDBStorage:
raise raise
_logger.debug( _logger.debug(
"LanceDB commit conflict on %s (attempt %d/%d), retrying in %.1fs", "LanceDB commit conflict on %s (attempt %d/%d), retrying in %.1fs",
op, attempt + 1, _MAX_RETRIES, delay, op,
attempt + 1,
_MAX_RETRIES,
delay,
) )
# Refresh table to pick up the latest version before retrying.
# The next getattr(self._table, op) will use the fresh table.
try: try:
self._table = self._db.open_table(self._table_name) self._table = self._db.open_table(self._table_name)
except Exception: # noqa: S110 except Exception:
pass # table refresh is best-effort _logger.debug("Failed to re-open table during retry", exc_info=True)
time.sleep(delay) time.sleep(delay)
delay *= 2 delay *= 2
return None # unreachable, but satisfies type checker return None # unreachable, but satisfies type checker
def _create_table(self, vector_dim: int) -> lancedb.table.Table: def _create_table(self, vector_dim: int) -> Any:
"""Create a new table with the given vector dimension.""" """Create a new table with the given vector dimension.
Caller must already hold ``store_lock(self._lock_name)``.
"""
placeholder = [ placeholder = [
{ {
"id": "__schema_placeholder__", "id": "__schema_placeholder__",
@@ -200,8 +171,12 @@ class LanceDBStorage:
"vector": [0.0] * vector_dim, "vector": [0.0] * vector_dim,
} }
] ]
table = self._db.create_table(self._table_name, placeholder) try:
table.delete("id = '__schema_placeholder__'") table = self._db.create_table(self._table_name, placeholder)
except ValueError:
table = self._db.open_table(self._table_name)
else:
table.delete("id = '__schema_placeholder__'")
return table return table
def _ensure_scope_index(self) -> None: def _ensure_scope_index(self) -> None:
@@ -217,8 +192,10 @@ class LanceDBStorage:
return return
try: try:
self._table.create_scalar_index("scope", index_type="BTREE", replace=False) self._table.create_scalar_index("scope", index_type="BTREE", replace=False)
except Exception: # noqa: S110 except Exception:
pass # index already exists, table empty, or unsupported version _logger.debug(
"Scope index creation skipped (may already exist)", exc_info=True
)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Automatic background compaction # Automatic background compaction
@@ -238,8 +215,10 @@ class LanceDBStorage:
def _compact_async(self) -> None: def _compact_async(self) -> None:
"""Fire-and-forget: compact the table in a daemon background thread.""" """Fire-and-forget: compact the table in a daemon background thread."""
ctx = contextvars.copy_context()
threading.Thread( threading.Thread(
target=self._compact_safe, target=ctx.run,
args=(self._compact_safe,),
daemon=True, daemon=True,
name="lancedb-compact", name="lancedb-compact",
).start() ).start()
@@ -248,13 +227,13 @@ class LanceDBStorage:
"""Run ``table.optimize()`` in a background thread, absorbing errors.""" """Run ``table.optimize()`` in a background thread, absorbing errors."""
try: try:
if self._table is not None: if self._table is not None:
self._table.optimize() with store_lock(self._lock_name):
# Refresh the scope index so new fragments are covered. self._table.optimize()
self._ensure_scope_index() self._ensure_scope_index()
except Exception: except Exception:
_logger.debug("LanceDB background compaction failed", exc_info=True) _logger.debug("LanceDB background compaction failed", exc_info=True)
def _ensure_table(self, vector_dim: int | None = None) -> lancedb.table.Table: def _ensure_table(self, vector_dim: int | None = None) -> Any:
"""Return the table, creating it lazily if needed. """Return the table, creating it lazily if needed.
Args: Args:
@@ -280,7 +259,9 @@ class LanceDBStorage:
"last_accessed": record.last_accessed.isoformat(), "last_accessed": record.last_accessed.isoformat(),
"source": record.source or "", "source": record.source or "",
"private": record.private, "private": record.private,
"vector": record.embedding if record.embedding else [0.0] * self._vector_dim, "vector": record.embedding
if record.embedding
else [0.0] * self._vector_dim,
} }
def _row_to_record(self, row: dict[str, Any]) -> MemoryRecord: def _row_to_record(self, row: dict[str, Any]) -> MemoryRecord:
@@ -296,7 +277,9 @@ class LanceDBStorage:
id=str(row["id"]), id=str(row["id"]),
content=str(row["content"]), content=str(row["content"]),
scope=str(row["scope"]), scope=str(row["scope"]),
categories=json.loads(row["categories_str"]) if row.get("categories_str") else [], categories=json.loads(row["categories_str"])
if row.get("categories_str")
else [],
metadata=json.loads(row["metadata_str"]) if row.get("metadata_str") else {}, metadata=json.loads(row["metadata_str"]) if row.get("metadata_str") else {},
importance=float(row.get("importance", 0.5)), importance=float(row.get("importance", 0.5)),
created_at=_parse_dt(row.get("created_at")), created_at=_parse_dt(row.get("created_at")),
@@ -316,16 +299,15 @@ class LanceDBStorage:
dim = len(r.embedding) dim = len(r.embedding)
break break
is_new_table = self._table is None is_new_table = self._table is None
with self._write_lock: with store_lock(self._lock_name):
self._ensure_table(vector_dim=dim) self._ensure_table(vector_dim=dim)
rows = [self._record_to_row(r) for r in records] rows = [self._record_to_row(rec) for rec in records]
for r in rows: for row in rows:
if r["vector"] is None or len(r["vector"]) != self._vector_dim: if row["vector"] is None or len(row["vector"]) != self._vector_dim:
r["vector"] = [0.0] * self._vector_dim row["vector"] = [0.0] * self._vector_dim
self._retry_write("add", rows) self._do_write("add", rows)
# Create the scope index on the first save so it covers the initial dataset. if is_new_table:
if is_new_table: self._ensure_scope_index()
self._ensure_scope_index()
# Auto-compact every N saves so fragment files don't pile up. # Auto-compact every N saves so fragment files don't pile up.
self._save_count += 1 self._save_count += 1
if self._compact_every > 0 and self._save_count % self._compact_every == 0: if self._compact_every > 0 and self._save_count % self._compact_every == 0:
@@ -333,14 +315,14 @@ class LanceDBStorage:
def update(self, record: MemoryRecord) -> None: def update(self, record: MemoryRecord) -> None:
"""Update a record by ID. Preserves created_at, updates last_accessed.""" """Update a record by ID. Preserves created_at, updates last_accessed."""
with self._write_lock: with store_lock(self._lock_name):
self._ensure_table() self._ensure_table()
safe_id = str(record.id).replace("'", "''") safe_id = str(record.id).replace("'", "''")
self._retry_write("delete", f"id = '{safe_id}'") self._do_write("delete", f"id = '{safe_id}'")
row = self._record_to_row(record) row = self._record_to_row(record)
if row["vector"] is None or len(row["vector"]) != self._vector_dim: if row["vector"] is None or len(row["vector"]) != self._vector_dim:
row["vector"] = [0.0] * self._vector_dim row["vector"] = [0.0] * self._vector_dim
self._retry_write("add", [row]) self._do_write("add", [row])
def touch_records(self, record_ids: list[str]) -> None: def touch_records(self, record_ids: list[str]) -> None:
"""Update last_accessed to now for the given record IDs. """Update last_accessed to now for the given record IDs.
@@ -354,11 +336,11 @@ class LanceDBStorage:
""" """
if not record_ids or self._table is None: if not record_ids or self._table is None:
return return
with self._write_lock: with store_lock(self._lock_name):
now = datetime.utcnow().isoformat() now = datetime.utcnow().isoformat()
safe_ids = [str(rid).replace("'", "''") for rid in record_ids] safe_ids = [str(rid).replace("'", "''") for rid in record_ids]
ids_expr = ", ".join(f"'{rid}'" for rid in safe_ids) ids_expr = ", ".join(f"'{rid}'" for rid in safe_ids)
self._retry_write( self._do_write(
"update", "update",
where=f"id IN ({ids_expr})", where=f"id IN ({ids_expr})",
values={"last_accessed": now}, values={"last_accessed": now},
@@ -390,13 +372,17 @@ class LanceDBStorage:
prefix = scope_prefix.rstrip("/") prefix = scope_prefix.rstrip("/")
like_val = prefix + "%" like_val = prefix + "%"
query = query.where(f"scope LIKE '{like_val}'") query = query.where(f"scope LIKE '{like_val}'")
results = query.limit(limit * 3 if (categories or metadata_filter) else limit).to_list() results = query.limit(
limit * 3 if (categories or metadata_filter) else limit
).to_list()
out: list[tuple[MemoryRecord, float]] = [] out: list[tuple[MemoryRecord, float]] = []
for row in results: for row in results:
record = self._row_to_record(row) record = self._row_to_record(row)
if categories and not any(c in record.categories for c in categories): if categories and not any(c in record.categories for c in categories):
continue continue
if metadata_filter and not all(record.metadata.get(k) == v for k, v in metadata_filter.items()): if metadata_filter and not all(
record.metadata.get(k) == v for k, v in metadata_filter.items()
):
continue continue
distance = row.get("_distance", 0.0) distance = row.get("_distance", 0.0)
score = 1.0 / (1.0 + float(distance)) if distance is not None else 1.0 score = 1.0 / (1.0 + float(distance)) if distance is not None else 1.0
@@ -416,30 +402,34 @@ class LanceDBStorage:
) -> int: ) -> int:
if self._table is None: if self._table is None:
return 0 return 0
with self._write_lock: with store_lock(self._lock_name):
if record_ids and not (categories or metadata_filter): if record_ids and not (categories or metadata_filter):
before = self._table.count_rows() before = int(self._table.count_rows())
ids_expr = ", ".join(f"'{rid}'" for rid in record_ids) ids_expr = ", ".join(f"'{rid}'" for rid in record_ids)
self._retry_write("delete", f"id IN ({ids_expr})") self._do_write("delete", f"id IN ({ids_expr})")
return before - self._table.count_rows() return before - int(self._table.count_rows())
if categories or metadata_filter: if categories or metadata_filter:
rows = self._scan_rows(scope_prefix) rows = self._scan_rows(scope_prefix)
to_delete: list[str] = [] to_delete: list[str] = []
for row in rows: for row in rows:
record = self._row_to_record(row) record = self._row_to_record(row)
if categories and not any(c in record.categories for c in categories): if categories and not any(
c in record.categories for c in categories
):
continue continue
if metadata_filter and not all(record.metadata.get(k) == v for k, v in metadata_filter.items()): if metadata_filter and not all(
record.metadata.get(k) == v for k, v in metadata_filter.items()
):
continue continue
if older_than and record.created_at >= older_than: if older_than and record.created_at >= older_than:
continue continue
to_delete.append(record.id) to_delete.append(record.id)
if not to_delete: if not to_delete:
return 0 return 0
before = self._table.count_rows() before = int(self._table.count_rows())
ids_expr = ", ".join(f"'{rid}'" for rid in to_delete) ids_expr = ", ".join(f"'{rid}'" for rid in to_delete)
self._retry_write("delete", f"id IN ({ids_expr})") self._do_write("delete", f"id IN ({ids_expr})")
return before - self._table.count_rows() return before - int(self._table.count_rows())
conditions = [] conditions = []
if scope_prefix is not None and scope_prefix.strip("/"): if scope_prefix is not None and scope_prefix.strip("/"):
prefix = scope_prefix.rstrip("/") prefix = scope_prefix.rstrip("/")
@@ -449,13 +439,13 @@ class LanceDBStorage:
if older_than is not None: if older_than is not None:
conditions.append(f"created_at < '{older_than.isoformat()}'") conditions.append(f"created_at < '{older_than.isoformat()}'")
if not conditions: if not conditions:
before = self._table.count_rows() before = int(self._table.count_rows())
self._retry_write("delete", "id != ''") self._do_write("delete", "id != ''")
return before - self._table.count_rows() return before - int(self._table.count_rows())
where_expr = " AND ".join(conditions) where_expr = " AND ".join(conditions)
before = self._table.count_rows() before = int(self._table.count_rows())
self._retry_write("delete", where_expr) self._do_write("delete", where_expr)
return before - self._table.count_rows() return before - int(self._table.count_rows())
def _scan_rows( def _scan_rows(
self, self,
@@ -482,7 +472,8 @@ class LanceDBStorage:
q = q.where(f"scope LIKE '{scope_prefix.rstrip('/')}%'") q = q.where(f"scope LIKE '{scope_prefix.rstrip('/')}%'")
if columns is not None: if columns is not None:
q = q.select(columns) q = q.select(columns)
return q.limit(limit).to_list() result: list[dict[str, Any]] = q.limit(limit).to_list()
return result
def list_records( def list_records(
self, scope_prefix: str | None = None, limit: int = 200, offset: int = 0 self, scope_prefix: str | None = None, limit: int = 200, offset: int = 0
@@ -528,7 +519,7 @@ class LanceDBStorage:
for row in rows: for row in rows:
sc = str(row.get("scope", "")) sc = str(row.get("scope", ""))
if child_prefix and sc.startswith(child_prefix): if child_prefix and sc.startswith(child_prefix):
rest = sc[len(child_prefix):] rest = sc[len(child_prefix) :]
first_component = rest.split("/", 1)[0] first_component = rest.split("/", 1)[0]
if first_component: if first_component:
children.add(child_prefix + first_component) children.add(child_prefix + first_component)
@@ -539,7 +530,11 @@ class LanceDBStorage:
pass pass
created = row.get("created_at") created = row.get("created_at")
if created: if created:
dt = datetime.fromisoformat(str(created).replace("Z", "+00:00")) if isinstance(created, str) else created dt = (
datetime.fromisoformat(str(created).replace("Z", "+00:00"))
if isinstance(created, str)
else created
)
if isinstance(dt, datetime): if isinstance(dt, datetime):
if oldest is None or dt < oldest: if oldest is None or dt < oldest:
oldest = dt oldest = dt
@@ -562,7 +557,7 @@ class LanceDBStorage:
for row in rows: for row in rows:
sc = str(row.get("scope", "")) sc = str(row.get("scope", ""))
if sc.startswith(prefix) and sc != (prefix.rstrip("/") or "/"): if sc.startswith(prefix) and sc != (prefix.rstrip("/") or "/"):
rest = sc[len(prefix):] rest = sc[len(prefix) :]
first_component = rest.split("/", 1)[0] first_component = rest.split("/", 1)[0]
if first_component: if first_component:
children.add(prefix + first_component) children.add(prefix + first_component)
@@ -585,22 +580,24 @@ class LanceDBStorage:
if self._table is None: if self._table is None:
return 0 return 0
if scope_prefix is None or scope_prefix.strip("/") == "": if scope_prefix is None or scope_prefix.strip("/") == "":
return self._table.count_rows() return int(self._table.count_rows())
info = self.get_scope_info(scope_prefix) info = self.get_scope_info(scope_prefix)
return info.record_count return info.record_count
def reset(self, scope_prefix: str | None = None) -> None: def reset(self, scope_prefix: str | None = None) -> None:
if scope_prefix is None or scope_prefix.strip("/") == "": with store_lock(self._lock_name):
if self._table is not None: if scope_prefix is None or scope_prefix.strip("/") == "":
self._db.drop_table(self._table_name) if self._table is not None:
self._table = None self._db.drop_table(self._table_name)
# Dimension is preserved; table will be recreated on next save. self._table = None
return return
if self._table is None: if self._table is None:
return return
prefix = scope_prefix.rstrip("/") prefix = scope_prefix.rstrip("/")
if prefix: if prefix:
self._table.delete(f"scope >= '{prefix}' AND scope < '{prefix}/\uFFFF'") self._do_write(
"delete", f"scope >= '{prefix}' AND scope < '{prefix}/\uffff'"
)
def optimize(self) -> None: def optimize(self) -> None:
"""Compact the table synchronously and refresh the scope index. """Compact the table synchronously and refresh the scope index.
@@ -614,8 +611,9 @@ class LanceDBStorage:
""" """
if self._table is None: if self._table is None:
return return
self._table.optimize() with store_lock(self._lock_name):
self._ensure_scope_index() self._table.optimize()
self._ensure_scope_index()
async def asave(self, records: list[MemoryRecord]) -> None: async def asave(self, records: list[MemoryRecord]) -> None:
self.save(records) self.save(records)

View File

@@ -3,10 +3,13 @@
from __future__ import annotations from __future__ import annotations
from concurrent.futures import Future, ThreadPoolExecutor from concurrent.futures import Future, ThreadPoolExecutor
import contextvars
from datetime import datetime from datetime import datetime
import threading import threading
import time import time
from typing import TYPE_CHECKING, Any, Literal from typing import TYPE_CHECKING, Annotated, Any, Literal
from pydantic import BaseModel, ConfigDict, Field, PlainValidator, PrivateAttr
from crewai.events.event_bus import crewai_event_bus from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.memory_events import ( from crewai.events.types.memory_events import (
@@ -39,13 +42,18 @@ if TYPE_CHECKING:
) )
def _passthrough(v: Any) -> Any:
"""PlainValidator that accepts any value, bypassing strict union discrimination."""
return v
def _default_embedder() -> OpenAIEmbeddingFunction: def _default_embedder() -> OpenAIEmbeddingFunction:
"""Build default OpenAI embedder for memory.""" """Build default OpenAI embedder for memory."""
spec: OpenAIProviderSpec = {"provider": "openai", "config": {}} spec: OpenAIProviderSpec = {"provider": "openai", "config": {}}
return build_embedder(spec) return build_embedder(spec)
class Memory: class Memory(BaseModel):
"""Unified memory: standalone, LLM-analyzed, with intelligent recall flow. """Unified memory: standalone, LLM-analyzed, with intelligent recall flow.
Works without agent/crew. Uses LLM to infer scope, categories, importance on save. Works without agent/crew. Uses LLM to infer scope, categories, importance on save.
@@ -53,116 +61,119 @@ class Memory:
pluggable storage (LanceDB default). pluggable storage (LanceDB default).
""" """
def __init__( model_config = ConfigDict(arbitrary_types_allowed=True)
self,
llm: BaseLLM | str = "gpt-4o-mini",
storage: StorageBackend | str = "lancedb",
embedder: Any = None,
# -- Scoring weights --
# These three weights control how recall results are ranked.
# The composite score is: semantic_weight * similarity + recency_weight * decay + importance_weight * importance.
# They should sum to ~1.0 for intuitive scoring.
recency_weight: float = 0.3,
semantic_weight: float = 0.5,
importance_weight: float = 0.2,
# How quickly old memories lose relevance. The recency score halves every
# N days (exponential decay). Lower = faster forgetting; higher = longer relevance.
recency_half_life_days: int = 30,
# -- Consolidation --
# When remembering new content, if an existing record has similarity >= this
# threshold, the LLM is asked to merge/update/delete. Set to 1.0 to disable.
consolidation_threshold: float = 0.85,
# Max existing records to compare against when checking for consolidation.
consolidation_limit: int = 5,
# -- Save defaults --
# Importance assigned to new memories when no explicit value is given and
# the LLM analysis path is skipped (all fields provided by the caller).
default_importance: float = 0.5,
# -- Recall depth control --
# These thresholds govern the RecallFlow router that decides between
# returning results immediately ("synthesize") vs. doing an extra
# LLM-driven exploration round ("explore_deeper").
# confidence >= confidence_threshold_high => always synthesize
# confidence < confidence_threshold_low => explore deeper (if budget > 0)
# complex query + confidence < complex_query_threshold => explore deeper
confidence_threshold_high: float = 0.8,
confidence_threshold_low: float = 0.5,
complex_query_threshold: float = 0.7,
# How many LLM-driven exploration rounds the RecallFlow is allowed to run.
# 0 = always shallow (vector search only); higher = more thorough but slower.
exploration_budget: int = 1,
# Queries shorter than this skip LLM analysis (saving ~1-3s).
# Longer queries (full task descriptions) benefit from LLM distillation.
query_analysis_threshold: int = 200,
# When True, all write operations (remember, remember_many) are silently
# skipped. Useful for sharing a read-only view of memory across agents
# without any of them persisting new memories.
read_only: bool = False,
) -> None:
"""Initialize Memory.
Args: llm: Annotated[BaseLLM | str, PlainValidator(_passthrough)] = Field(
llm: LLM for analysis (model name or BaseLLM instance). default="gpt-4o-mini",
storage: Backend: "lancedb" or a StorageBackend instance. description="LLM for analysis (model name or BaseLLM instance).",
embedder: Embedding callable, provider config dict, or None (default OpenAI). )
recency_weight: Weight for recency in the composite relevance score. storage: Annotated[StorageBackend | str, PlainValidator(_passthrough)] = Field(
semantic_weight: Weight for semantic similarity in the composite relevance score. default="lancedb",
importance_weight: Weight for importance in the composite relevance score. description="Storage backend instance or path string.",
recency_half_life_days: Recency score halves every N days (exponential decay). )
consolidation_threshold: Similarity above which consolidation is triggered on save. embedder: Any = Field(
consolidation_limit: Max existing records to compare during consolidation. default=None,
default_importance: Default importance when not provided or inferred. description="Embedding callable, provider config dict, or None for default OpenAI.",
confidence_threshold_high: Recall confidence above which results are returned directly. )
confidence_threshold_low: Recall confidence below which deeper exploration is triggered. recency_weight: float = Field(
complex_query_threshold: For complex queries, explore deeper below this confidence. default=0.3,
exploration_budget: Number of LLM-driven exploration rounds during deep recall. description="Weight for recency in the composite relevance score.",
query_analysis_threshold: Queries shorter than this skip LLM analysis during deep recall. )
read_only: If True, remember() and remember_many() are silent no-ops. semantic_weight: float = Field(
""" default=0.5,
self._read_only = read_only description="Weight for semantic similarity in the composite relevance score.",
)
importance_weight: float = Field(
default=0.2,
description="Weight for importance in the composite relevance score.",
)
recency_half_life_days: int = Field(
default=30,
description="Recency score halves every N days (exponential decay).",
)
consolidation_threshold: float = Field(
default=0.85,
description="Similarity above which consolidation is triggered on save.",
)
consolidation_limit: int = Field(
default=5,
description="Max existing records to compare during consolidation.",
)
default_importance: float = Field(
default=0.5,
description="Default importance when not provided or inferred.",
)
confidence_threshold_high: float = Field(
default=0.8,
description="Recall confidence above which results are returned directly.",
)
confidence_threshold_low: float = Field(
default=0.5,
description="Recall confidence below which deeper exploration is triggered.",
)
complex_query_threshold: float = Field(
default=0.7,
description="For complex queries, explore deeper below this confidence.",
)
exploration_budget: int = Field(
default=1,
description="Number of LLM-driven exploration rounds during deep recall.",
)
query_analysis_threshold: int = Field(
default=200,
description="Queries shorter than this skip LLM analysis during deep recall.",
)
read_only: bool = Field(
default=False,
description="If True, remember() and remember_many() are silent no-ops.",
)
_config: MemoryConfig = PrivateAttr()
_llm_instance: BaseLLM | None = PrivateAttr(default=None)
_embedder_instance: Any = PrivateAttr(default=None)
_storage: StorageBackend = PrivateAttr()
_save_pool: ThreadPoolExecutor = PrivateAttr(
default_factory=lambda: ThreadPoolExecutor(
max_workers=1, thread_name_prefix="memory-save"
)
)
_pending_saves: list[Future[Any]] = PrivateAttr(default_factory=list)
_pending_lock: threading.Lock = PrivateAttr(default_factory=threading.Lock)
def model_post_init(self, __context: Any) -> None:
"""Initialize runtime state from field values."""
self._config = MemoryConfig( self._config = MemoryConfig(
recency_weight=recency_weight, recency_weight=self.recency_weight,
semantic_weight=semantic_weight, semantic_weight=self.semantic_weight,
importance_weight=importance_weight, importance_weight=self.importance_weight,
recency_half_life_days=recency_half_life_days, recency_half_life_days=self.recency_half_life_days,
consolidation_threshold=consolidation_threshold, consolidation_threshold=self.consolidation_threshold,
consolidation_limit=consolidation_limit, consolidation_limit=self.consolidation_limit,
default_importance=default_importance, default_importance=self.default_importance,
confidence_threshold_high=confidence_threshold_high, confidence_threshold_high=self.confidence_threshold_high,
confidence_threshold_low=confidence_threshold_low, confidence_threshold_low=self.confidence_threshold_low,
complex_query_threshold=complex_query_threshold, complex_query_threshold=self.complex_query_threshold,
exploration_budget=exploration_budget, exploration_budget=self.exploration_budget,
query_analysis_threshold=query_analysis_threshold, query_analysis_threshold=self.query_analysis_threshold,
) )
# Store raw config for lazy initialization. LLM and embedder are only self._llm_instance = None if isinstance(self.llm, str) else self.llm
# built on first access so that Memory() never fails at construction self._embedder_instance = (
# time (e.g. when auto-created by Flow without an API key set). self.embedder
self._llm_config: BaseLLM | str = llm if (self.embedder is not None and not isinstance(self.embedder, dict))
self._llm_instance: BaseLLM | None = None if isinstance(llm, str) else llm
self._embedder_config: Any = embedder
self._embedder_instance: Any = (
embedder
if (embedder is not None and not isinstance(embedder, dict))
else None else None
) )
if isinstance(storage, str): if isinstance(self.storage, str):
from crewai.memory.storage.lancedb_storage import LanceDBStorage from crewai.memory.storage.lancedb_storage import LanceDBStorage
self._storage = LanceDBStorage() if storage == "lancedb" else LanceDBStorage(path=storage) self._storage = (
LanceDBStorage()
if self.storage == "lancedb"
else LanceDBStorage(path=self.storage)
)
else: else:
self._storage = storage self._storage = self.storage
# Background save queue. max_workers=1 serializes saves to avoid
# concurrent storage mutations (two saves finding the same similar
# record and both trying to update/delete it). Within each save,
# the parallel LLM calls still run on their own thread pool.
self._save_pool = ThreadPoolExecutor(
max_workers=1, thread_name_prefix="memory-save"
)
self._pending_saves: list[Future[Any]] = []
self._pending_lock = threading.Lock()
_MEMORY_DOCS_URL = "https://docs.crewai.com/concepts/memory" _MEMORY_DOCS_URL = "https://docs.crewai.com/concepts/memory"
@@ -173,11 +184,7 @@ class Memory:
from crewai.llm import LLM from crewai.llm import LLM
try: try:
model_name = ( model_name = self.llm if isinstance(self.llm, str) else str(self.llm)
self._llm_config
if isinstance(self._llm_config, str)
else str(self._llm_config)
)
self._llm_instance = LLM(model=model_name) self._llm_instance = LLM(model=model_name)
except Exception as e: except Exception as e:
raise RuntimeError( raise RuntimeError(
@@ -197,8 +204,8 @@ class Memory:
"""Lazy embedder initialization -- only created when first needed.""" """Lazy embedder initialization -- only created when first needed."""
if self._embedder_instance is None: if self._embedder_instance is None:
try: try:
if isinstance(self._embedder_config, dict): if isinstance(self.embedder, dict):
self._embedder_instance = build_embedder(self._embedder_config) self._embedder_instance = build_embedder(self.embedder)
else: else:
self._embedder_instance = _default_embedder() self._embedder_instance = _default_embedder()
except Exception as e: except Exception as e:
@@ -223,8 +230,9 @@ class Memory:
If the pool has been shut down (e.g. after ``close()``), the save If the pool has been shut down (e.g. after ``close()``), the save
runs synchronously as a fallback so late saves still succeed. runs synchronously as a fallback so late saves still succeed.
""" """
ctx = contextvars.copy_context()
try: try:
future: Future[Any] = self._save_pool.submit(fn, *args, **kwargs) future: Future[Any] = self._save_pool.submit(ctx.run, fn, *args, **kwargs)
except RuntimeError: except RuntimeError:
# Pool shut down -- run synchronously as fallback # Pool shut down -- run synchronously as fallback
future = Future() future = Future()
@@ -356,7 +364,7 @@ class Memory:
Raises: Raises:
Exception: On save failure (events emitted). Exception: On save failure (events emitted).
""" """
if self._read_only: if self.read_only:
return None return None
_source_type = "unified_memory" _source_type = "unified_memory"
try: try:
@@ -444,7 +452,7 @@ class Memory:
Returns: Returns:
Empty list (records are not available until the background save completes). Empty list (records are not available until the background save completes).
""" """
if not contents or self._read_only: if not contents or self.read_only:
return [] return []
self._submit_save( self._submit_save(

View File

@@ -4,6 +4,7 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import Callable from collections.abc import Callable
import contextvars
from functools import wraps from functools import wraps
import inspect import inspect
from typing import TYPE_CHECKING, Any, Concatenate, ParamSpec, TypeVar, overload from typing import TYPE_CHECKING, Any, Concatenate, ParamSpec, TypeVar, overload
@@ -169,8 +170,9 @@ def _call_method(method: Callable[..., Any], *args: Any, **kwargs: Any) -> Any:
if loop and loop.is_running(): if loop and loop.is_running():
import concurrent.futures import concurrent.futures
ctx = contextvars.copy_context()
with concurrent.futures.ThreadPoolExecutor() as pool: with concurrent.futures.ThreadPoolExecutor() as pool:
return pool.submit(asyncio.run, result).result() return pool.submit(ctx.run, asyncio.run, result).result()
return asyncio.run(result) return asyncio.run(result)
return result return result

View File

@@ -4,6 +4,7 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import Callable from collections.abc import Callable
import contextvars
from functools import partial from functools import partial
import inspect import inspect
from pathlib import Path from pathlib import Path
@@ -146,8 +147,9 @@ def _resolve_result(result: Any) -> Any:
if loop and loop.is_running(): if loop and loop.is_running():
import concurrent.futures import concurrent.futures
ctx = contextvars.copy_context()
with concurrent.futures.ThreadPoolExecutor() as pool: with concurrent.futures.ThreadPoolExecutor() as pool:
return pool.submit(asyncio.run, result).result() return pool.submit(ctx.run, asyncio.run, result).result()
return asyncio.run(result) return asyncio.run(result)
return result return result

View File

@@ -1,5 +1,8 @@
"""ChromaDB client implementation.""" """ChromaDB client implementation."""
import asyncio
from collections.abc import AsyncIterator
from contextlib import AbstractContextManager, asynccontextmanager, nullcontext
import logging import logging
from typing import Any from typing import Any
@@ -29,6 +32,7 @@ from crewai.rag.core.base_client import (
BaseCollectionParams, BaseCollectionParams,
) )
from crewai.rag.types import SearchResult from crewai.rag.types import SearchResult
from crewai.utilities.lock_store import lock as store_lock
from crewai.utilities.logger_utils import suppress_logging from crewai.utilities.logger_utils import suppress_logging
@@ -52,6 +56,7 @@ class ChromaDBClient(BaseClient):
default_limit: int = 5, default_limit: int = 5,
default_score_threshold: float = 0.6, default_score_threshold: float = 0.6,
default_batch_size: int = 100, default_batch_size: int = 100,
lock_name: str = "",
) -> None: ) -> None:
"""Initialize ChromaDBClient with client and embedding function. """Initialize ChromaDBClient with client and embedding function.
@@ -61,12 +66,32 @@ class ChromaDBClient(BaseClient):
default_limit: Default number of results to return in searches. default_limit: Default number of results to return in searches.
default_score_threshold: Default minimum score for search results. default_score_threshold: Default minimum score for search results.
default_batch_size: Default batch size for adding documents. default_batch_size: Default batch size for adding documents.
lock_name: Optional lock name for cross-process synchronization.
""" """
self.client = client self.client = client
self.embedding_function = embedding_function self.embedding_function = embedding_function
self.default_limit = default_limit self.default_limit = default_limit
self.default_score_threshold = default_score_threshold self.default_score_threshold = default_score_threshold
self.default_batch_size = default_batch_size self.default_batch_size = default_batch_size
self._lock_name = lock_name
def _locked(self) -> AbstractContextManager[None]:
"""Return a cross-process lock context manager, or nullcontext if no lock name."""
return store_lock(self._lock_name) if self._lock_name else nullcontext()
@asynccontextmanager
async def _alocked(self) -> AsyncIterator[None]:
"""Async cross-process lock that acquires/releases in an executor."""
if not self._lock_name:
yield
return
lock_cm = store_lock(self._lock_name)
loop = asyncio.get_event_loop()
await loop.run_in_executor(None, lock_cm.__enter__)
try:
yield
finally:
await loop.run_in_executor(None, lock_cm.__exit__, None, None, None)
def create_collection( def create_collection(
self, **kwargs: Unpack[ChromaDBCollectionCreateParams] self, **kwargs: Unpack[ChromaDBCollectionCreateParams]
@@ -313,23 +338,24 @@ class ChromaDBClient(BaseClient):
if not documents: if not documents:
raise ValueError("Documents list cannot be empty") raise ValueError("Documents list cannot be empty")
collection = self.client.get_or_create_collection( with self._locked():
name=_sanitize_collection_name(collection_name), collection = self.client.get_or_create_collection(
embedding_function=self.embedding_function, name=_sanitize_collection_name(collection_name),
) embedding_function=self.embedding_function,
prepared = _prepare_documents_for_chromadb(documents)
for i in range(0, len(prepared.ids), batch_size):
batch_ids, batch_texts, batch_metadatas = _create_batch_slice(
prepared=prepared, start_index=i, batch_size=batch_size
) )
collection.upsert( prepared = _prepare_documents_for_chromadb(documents)
ids=batch_ids,
documents=batch_texts, for i in range(0, len(prepared.ids), batch_size):
metadatas=batch_metadatas, # type: ignore[arg-type] batch_ids, batch_texts, batch_metadatas = _create_batch_slice(
) prepared=prepared, start_index=i, batch_size=batch_size
)
collection.upsert(
ids=batch_ids,
documents=batch_texts,
metadatas=batch_metadatas, # type: ignore[arg-type]
)
async def aadd_documents(self, **kwargs: Unpack[BaseCollectionAddParams]) -> None: async def aadd_documents(self, **kwargs: Unpack[BaseCollectionAddParams]) -> None:
"""Add documents with their embeddings to a collection asynchronously. """Add documents with their embeddings to a collection asynchronously.
@@ -363,22 +389,23 @@ class ChromaDBClient(BaseClient):
if not documents: if not documents:
raise ValueError("Documents list cannot be empty") raise ValueError("Documents list cannot be empty")
collection = await self.client.get_or_create_collection( async with self._alocked():
name=_sanitize_collection_name(collection_name), collection = await self.client.get_or_create_collection(
embedding_function=self.embedding_function, name=_sanitize_collection_name(collection_name),
) embedding_function=self.embedding_function,
prepared = _prepare_documents_for_chromadb(documents)
for i in range(0, len(prepared.ids), batch_size):
batch_ids, batch_texts, batch_metadatas = _create_batch_slice(
prepared=prepared, start_index=i, batch_size=batch_size
) )
prepared = _prepare_documents_for_chromadb(documents)
await collection.upsert( for i in range(0, len(prepared.ids), batch_size):
ids=batch_ids, batch_ids, batch_texts, batch_metadatas = _create_batch_slice(
documents=batch_texts, prepared=prepared, start_index=i, batch_size=batch_size
metadatas=batch_metadatas, # type: ignore[arg-type] )
)
await collection.upsert(
ids=batch_ids,
documents=batch_texts,
metadatas=batch_metadatas, # type: ignore[arg-type]
)
def search( def search(
self, **kwargs: Unpack[ChromaDBCollectionSearchParams] self, **kwargs: Unpack[ChromaDBCollectionSearchParams]
@@ -531,7 +558,10 @@ class ChromaDBClient(BaseClient):
) )
collection_name = kwargs["collection_name"] collection_name = kwargs["collection_name"]
self.client.delete_collection(name=_sanitize_collection_name(collection_name)) with self._locked():
self.client.delete_collection(
name=_sanitize_collection_name(collection_name)
)
async def adelete_collection(self, **kwargs: Unpack[BaseCollectionParams]) -> None: async def adelete_collection(self, **kwargs: Unpack[BaseCollectionParams]) -> None:
"""Delete a collection and all its data asynchronously. """Delete a collection and all its data asynchronously.
@@ -561,9 +591,10 @@ class ChromaDBClient(BaseClient):
) )
collection_name = kwargs["collection_name"] collection_name = kwargs["collection_name"]
await self.client.delete_collection( async with self._alocked():
name=_sanitize_collection_name(collection_name) await self.client.delete_collection(
) name=_sanitize_collection_name(collection_name)
)
def reset(self) -> None: def reset(self) -> None:
"""Reset the vector database by deleting all collections and data. """Reset the vector database by deleting all collections and data.
@@ -586,7 +617,8 @@ class ChromaDBClient(BaseClient):
"Use areset() for AsyncClientAPI." "Use areset() for AsyncClientAPI."
) )
self.client.reset() with self._locked():
self.client.reset()
async def areset(self) -> None: async def areset(self) -> None:
"""Reset the vector database by deleting all collections and data asynchronously. """Reset the vector database by deleting all collections and data asynchronously.
@@ -612,4 +644,5 @@ class ChromaDBClient(BaseClient):
"Use reset() for ClientAPI." "Use reset() for ClientAPI."
) )
await self.client.reset() async with self._alocked():
await self.client.reset()

View File

@@ -1,13 +1,12 @@
"""Factory functions for creating ChromaDB clients.""" """Factory functions for creating ChromaDB clients."""
from hashlib import md5
import os import os
from chromadb import PersistentClient from chromadb import PersistentClient
import portalocker
from crewai.rag.chromadb.client import ChromaDBClient from crewai.rag.chromadb.client import ChromaDBClient
from crewai.rag.chromadb.config import ChromaDBConfig from crewai.rag.chromadb.config import ChromaDBConfig
from crewai.utilities.lock_store import lock
def create_client(config: ChromaDBConfig) -> ChromaDBClient: def create_client(config: ChromaDBConfig) -> ChromaDBClient:
@@ -25,10 +24,8 @@ def create_client(config: ChromaDBConfig) -> ChromaDBClient:
persist_dir = config.settings.persist_directory persist_dir = config.settings.persist_directory
os.makedirs(persist_dir, exist_ok=True) os.makedirs(persist_dir, exist_ok=True)
lock_id = md5(persist_dir.encode(), usedforsecurity=False).hexdigest()
lockfile = os.path.join(persist_dir, f"chromadb-{lock_id}.lock")
with portalocker.Lock(lockfile): with lock(f"chromadb:{persist_dir}"):
client = PersistentClient( client = PersistentClient(
path=persist_dir, path=persist_dir,
settings=config.settings, settings=config.settings,
@@ -42,4 +39,5 @@ def create_client(config: ChromaDBConfig) -> ChromaDBClient:
default_limit=config.limit, default_limit=config.limit,
default_score_threshold=config.score_threshold, default_score_threshold=config.score_threshold,
default_batch_size=config.batch_size, default_batch_size=config.batch_size,
lock_name=f"chromadb:{persist_dir}",
) )

View File

@@ -2,6 +2,7 @@ from __future__ import annotations
import asyncio import asyncio
from concurrent.futures import Future from concurrent.futures import Future
import contextvars
from copy import copy as shallow_copy from copy import copy as shallow_copy
import datetime import datetime
from hashlib import md5 from hashlib import md5
@@ -224,7 +225,6 @@ class Task(BaseModel):
_guardrail_retry_counts: dict[int, int] = PrivateAttr( _guardrail_retry_counts: dict[int, int] = PrivateAttr(
default_factory=dict, default_factory=dict,
) )
_execution_index: int | None = PrivateAttr(default=None)
_original_description: str | None = PrivateAttr(default=None) _original_description: str | None = PrivateAttr(default=None)
_original_expected_output: str | None = PrivateAttr(default=None) _original_expected_output: str | None = PrivateAttr(default=None)
_original_output_file: str | None = PrivateAttr(default=None) _original_output_file: str | None = PrivateAttr(default=None)
@@ -525,10 +525,11 @@ class Task(BaseModel):
) -> Future[TaskOutput]: ) -> Future[TaskOutput]:
"""Execute the task asynchronously.""" """Execute the task asynchronously."""
future: Future[TaskOutput] = Future() future: Future[TaskOutput] = Future()
ctx = contextvars.copy_context()
threading.Thread( threading.Thread(
daemon=True, daemon=True,
target=self._execute_task_async, target=ctx.run,
args=(agent, context, tools, future), args=(self._execute_task_async, agent, context, tools, future),
).start() ).start()
return future return future

View File

@@ -1,5 +1,4 @@
from crewai.telemetry.telemetry import Telemetry from crewai.telemetry.telemetry import Telemetry
__all__ = ["Telemetry"] __all__ = ["Telemetry"]

View File

@@ -173,6 +173,12 @@ class Telemetry:
self._original_handlers: dict[int, Any] = {} self._original_handlers: dict[int, Any] = {}
if threading.current_thread() is not threading.main_thread():
logger.debug(
"Skipping signal handler registration: not running in main thread"
)
return
self._register_signal_handler(signal.SIGTERM, SigTermEvent, shutdown=True) self._register_signal_handler(signal.SIGTERM, SigTermEvent, shutdown=True)
self._register_signal_handler(signal.SIGINT, SigIntEvent, shutdown=True) self._register_signal_handler(signal.SIGINT, SigIntEvent, shutdown=True)
if hasattr(signal, "SIGHUP"): if hasattr(signal, "SIGHUP"):

View File

@@ -1,7 +1,6 @@
from crewai.tools.base_tool import BaseTool, EnvVar, tool from crewai.tools.base_tool import BaseTool, EnvVar, tool
__all__ = [ __all__ = [
"BaseTool", "BaseTool",
"EnvVar", "EnvVar",

View File

@@ -1,29 +1,31 @@
"""Native MCP tool wrapper for CrewAI agents. """Native MCP tool wrapper for CrewAI agents.
This module provides a tool wrapper that reuses existing MCP client sessions This module provides a tool wrapper that creates a fresh MCP client for every
for better performance and connection management. invocation, ensuring safe parallel execution even when the same tool is called
concurrently by the executor.
""" """
import asyncio import asyncio
from collections.abc import Callable
import contextvars
from typing import Any from typing import Any
from crewai.tools import BaseTool from crewai.tools import BaseTool
class MCPNativeTool(BaseTool): class MCPNativeTool(BaseTool):
"""Native MCP tool that reuses client sessions. """Native MCP tool that creates a fresh client per invocation.
This tool wrapper is used when agents connect to MCP servers using A ``client_factory`` callable produces an independent ``MCPClient`` +
structured configurations. It reuses existing client sessions for transport for every ``_run_async`` call. This guarantees that parallel
better performance and proper connection lifecycle management. invocations -- whether of the *same* tool or *different* tools from the
same server -- never share mutable connection state (which would cause
Unlike MCPToolWrapper which connects on-demand, this tool uses anyio cancel-scope errors).
a shared MCP client instance that maintains a persistent connection.
""" """
def __init__( def __init__(
self, self,
mcp_client: Any, client_factory: Callable[[], Any],
tool_name: str, tool_name: str,
tool_schema: dict[str, Any], tool_schema: dict[str, Any],
server_name: str, server_name: str,
@@ -32,19 +34,16 @@ class MCPNativeTool(BaseTool):
"""Initialize native MCP tool. """Initialize native MCP tool.
Args: Args:
mcp_client: MCPClient instance with active session. client_factory: Zero-arg callable that returns a new MCPClient.
tool_name: Name of the tool (may be prefixed). tool_name: Name of the tool (may be prefixed).
tool_schema: Schema information for the tool. tool_schema: Schema information for the tool.
server_name: Name of the MCP server for prefixing. server_name: Name of the MCP server for prefixing.
original_tool_name: Original name of the tool on the MCP server. original_tool_name: Original name of the tool on the MCP server.
""" """
# Create tool name with server prefix to avoid conflicts
prefixed_name = f"{server_name}_{tool_name}" prefixed_name = f"{server_name}_{tool_name}"
# Handle args_schema properly - BaseTool expects a BaseModel subclass
args_schema = tool_schema.get("args_schema") args_schema = tool_schema.get("args_schema")
# Only pass args_schema if it's provided
kwargs = { kwargs = {
"name": prefixed_name, "name": prefixed_name,
"description": tool_schema.get( "description": tool_schema.get(
@@ -57,16 +56,9 @@ class MCPNativeTool(BaseTool):
super().__init__(**kwargs) super().__init__(**kwargs)
# Set instance attributes after super().__init__ self._client_factory = client_factory
self._mcp_client = mcp_client
self._original_tool_name = original_tool_name or tool_name self._original_tool_name = original_tool_name or tool_name
self._server_name = server_name self._server_name = server_name
# self._logger = logging.getLogger(__name__)
@property
def mcp_client(self) -> Any:
"""Get the MCP client instance."""
return self._mcp_client
@property @property
def original_tool_name(self) -> str: def original_tool_name(self) -> str:
@@ -93,9 +85,10 @@ class MCPNativeTool(BaseTool):
import concurrent.futures import concurrent.futures
ctx = contextvars.copy_context()
with concurrent.futures.ThreadPoolExecutor() as executor: with concurrent.futures.ThreadPoolExecutor() as executor:
coro = self._run_async(**kwargs) coro = self._run_async(**kwargs)
future = executor.submit(asyncio.run, coro) future = executor.submit(ctx.run, asyncio.run, coro)
return future.result() return future.result()
except RuntimeError: except RuntimeError:
return asyncio.run(self._run_async(**kwargs)) return asyncio.run(self._run_async(**kwargs))
@@ -108,51 +101,26 @@ class MCPNativeTool(BaseTool):
async def _run_async(self, **kwargs) -> str: async def _run_async(self, **kwargs) -> str:
"""Async implementation of tool execution. """Async implementation of tool execution.
A fresh ``MCPClient`` is created for every invocation so that
concurrent calls never share transport or session state.
Args: Args:
**kwargs: Arguments to pass to the MCP tool. **kwargs: Arguments to pass to the MCP tool.
Returns: Returns:
Result from the MCP tool execution. Result from the MCP tool execution.
""" """
# Note: Since we use asyncio.run() which creates a new event loop each time, client = self._client_factory()
# Always reconnect on-demand because asyncio.run() creates new event loops per call await client.connect()
# All MCP transport context managers (stdio, streamablehttp_client, sse_client)
# use anyio.create_task_group() which can't span different event loops
if self._mcp_client.connected:
await self._mcp_client.disconnect()
await self._mcp_client.connect()
try: try:
result = await self._mcp_client.call_tool(self.original_tool_name, kwargs) result = await client.call_tool(self.original_tool_name, kwargs)
except Exception as e:
error_str = str(e).lower()
if (
"not connected" in error_str
or "connection" in error_str
or "send" in error_str
):
await self._mcp_client.disconnect()
await self._mcp_client.connect()
# Retry the call
result = await self._mcp_client.call_tool(
self.original_tool_name, kwargs
)
else:
raise
finally: finally:
# Always disconnect after tool call to ensure clean context manager lifecycle await client.disconnect()
# This prevents "exit cancel scope in different task" errors
# All transport context managers must be exited in the same event loop they were entered
await self._mcp_client.disconnect()
# Extract result content
if isinstance(result, str): if isinstance(result, str):
return result return result
# Handle various result formats
if hasattr(result, "content") and result.content: if hasattr(result, "content") and result.content:
if isinstance(result.content, list) and len(result.content) > 0: if isinstance(result.content, list) and len(result.content) > 0:
content_item = result.content[0] content_item = result.content[0]

View File

@@ -49,7 +49,7 @@ class RecallMemoryTool(BaseTool):
all_lines: list[str] = [] all_lines: list[str] = []
seen_ids: set[str] = set() seen_ids: set[str] = set()
for query in queries: for query in queries:
matches = self.memory.recall(query) matches = self.memory.recall(query, limit=20)
for m in matches: for m in matches:
if m.record.id not in seen_ids: if m.record.id not in seen_ids:
seen_ids.add(m.record.id) seen_ids.add(m.record.id)
@@ -121,7 +121,7 @@ def create_memory_tools(memory: Any) -> list[BaseTool]:
description=i18n.tools("recall_memory"), description=i18n.tools("recall_memory"),
), ),
] ]
if not getattr(memory, "_read_only", False): if not memory.read_only:
tools.append( tools.append(
RememberTool( RememberTool(
memory=memory, memory=memory,

View File

@@ -7,7 +7,7 @@
"slices": { "slices": {
"observation": "\nObservation:", "observation": "\nObservation:",
"task": "\nCurrent Task: {input}\n\nBegin! This is VERY important to you, use the tools available and give your best Final Answer, your job depends on it!\n\nThought:", "task": "\nCurrent Task: {input}\n\nBegin! This is VERY important to you, use the tools available and give your best Final Answer, your job depends on it!\n\nThought:",
"memory": "\n\n# Useful context: \n{memory}", "memory": "\n\n# Memories from past conversations:\n{memory}\n\nIMPORTANT: The memories above are an automatic selection and may be INCOMPLETE. If the task involves counting, listing, or summing items (e.g. 'how many', 'total', 'list all'), you MUST use the Search memory tool with several different queries before answering — do NOT rely solely on the memories shown above. Enumerate each distinct item you find before giving a final count.",
"role_playing": "You are {role}. {backstory}\nYour personal goal is: {goal}", "role_playing": "You are {role}. {backstory}\nYour personal goal is: {goal}",
"tools": "\nYou ONLY have access to the following tools, and should NEVER make up tools that are not listed here:\n\n{tools}\n\nIMPORTANT: Use the following format in your response:\n\n```\nThought: you should always think about what to do\nAction: the action to take, only one name of [{tool_names}], just the name, exactly as it's written.\nAction Input: the input to the action, just a simple JSON object, enclosed in curly braces, using \" to wrap keys and values.\nObservation: the result of the action\n```\n\nOnce all necessary information is gathered, return the following format:\n\n```\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n```", "tools": "\nYou ONLY have access to the following tools, and should NEVER make up tools that are not listed here:\n\n{tools}\n\nIMPORTANT: Use the following format in your response:\n\n```\nThought: you should always think about what to do\nAction: the action to take, only one name of [{tool_names}], just the name, exactly as it's written.\nAction Input: the input to the action, just a simple JSON object, enclosed in curly braces, using \" to wrap keys and values.\nObservation: the result of the action\n```\n\nOnce all necessary information is gathered, return the following format:\n\n```\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n```",
"no_tools": "", "no_tools": "",
@@ -60,12 +60,12 @@
"description": "See image to understand its content, you can optionally ask a question about the image", "description": "See image to understand its content, you can optionally ask a question about the image",
"default_action": "Please provide a detailed description of this image, including all visual elements, context, and any notable details you can observe." "default_action": "Please provide a detailed description of this image, including all visual elements, context, and any notable details you can observe."
}, },
"recall_memory": "Search through the team's shared memory for relevant information. Pass one or more queries to search for multiple things at once. Use this when you need to find facts, decisions, preferences, or past results that may have been stored previously.", "recall_memory": "Search through the team's shared memory for relevant information. Pass one or more queries to search for multiple things at once. Use this when you need to find facts, decisions, preferences, or past results that may have been stored previously. IMPORTANT: For questions that require counting, summing, or listing items across multiple conversations (e.g. 'how many X', 'total Y', 'list all Z'), you MUST search multiple times with different phrasings to ensure you find ALL relevant items before giving a final count or total. Do not rely on a single search — items may be described differently across conversations.",
"save_to_memory": "Store one or more important facts, decisions, observations, or lessons in memory so they can be recalled later by you or other agents. Pass multiple items at once when you have several things worth remembering." "save_to_memory": "Store one or more important facts, decisions, observations, or lessons in memory so they can be recalled later by you or other agents. Pass multiple items at once when you have several things worth remembering."
}, },
"memory": { "memory": {
"query_system": "You analyze a query for searching memory.\nGiven the query and available scopes, output:\n1. keywords: Key entities or keywords that can be used to filter by category.\n2. suggested_scopes: Which available scopes are most relevant (empty for all).\n3. complexity: 'simple' or 'complex'.\n4. recall_queries: 1-3 short, targeted search phrases distilled from the query. Each should be a concise phrase optimized for semantic vector search. If the query is already short and focused, return it as-is in a single-item list. For long task descriptions, extract the distinct things worth searching for.\n5. time_filter: If the query references a time period (like 'last week', 'yesterday', 'in January'), return an ISO 8601 date string for the earliest relevant date (e.g. '2026-02-01'). Return null if no time constraint is implied.", "query_system": "You analyze a query for searching memory.\nGiven the query and available scopes, output:\n1. keywords: Key entities or keywords that can be used to filter by category.\n2. suggested_scopes: Which available scopes are most relevant (empty for all).\n3. complexity: 'simple' or 'complex'.\n4. recall_queries: 1-3 short, targeted search phrases distilled from the query. Each should be a concise phrase optimized for semantic vector search. If the query is already short and focused, return it as-is in a single-item list. For long task descriptions, extract the distinct things worth searching for.\n5. time_filter: If the query references a time period (like 'last week', 'yesterday', 'in January'), return an ISO 8601 date string for the earliest relevant date (e.g. '2026-02-01'). Return null if no time constraint is implied.",
"extract_memories_system": "You extract discrete, reusable memory statements from raw content (e.g. a task description and its result).\n\nFor the given content, output a list of memory statements. Each memory must:\n- Be one clear sentence or short statement\n- Be understandable without the original context\n- Capture a decision, fact, outcome, preference, lesson, or observation worth remembering\n- NOT be a vague summary or a restatement of the task description\n- NOT duplicate the same idea in different words\n\nIf there is nothing worth remembering (e.g. empty result, no decisions or facts), return an empty list.\nOutput a JSON object with a single key \"memories\" whose value is a list of strings.", "extract_memories_system": "You extract discrete, reusable memory statements from raw content (e.g. a task description and its result, or a conversation between a user and an assistant).\n\nFor the given content, output a list of memory statements. Each memory must:\n- Be one clear sentence or short statement\n- Be understandable without the original context\n- Capture a decision, fact, outcome, preference, lesson, or observation worth remembering\n- NOT be a vague summary or a restatement of the task description\n- NOT duplicate the same idea in different words\n\nWhen the content is a conversation, pay special attention to facts stated by the user (first-person statements). These personal facts are HIGH PRIORITY and must always be extracted:\n- What the user did, bought, made, visited, attended, or completed\n- Names of people, pets, places, brands, and specific items the user mentions\n- Quantities, durations, dates, and measurements the user states\n- Subordinate clauses and casual asides often contain important personal details (e.g. \"by the way, it took me 4 hours\" or \"my Golden Retriever Max\")\n\nPreserve exact names and numbers — never generalize (e.g. keep \"lavender gin fizz\" not just \"cocktail\", keep \"12 largemouth bass\" not just \"fish caught\", keep \"Golden Retriever\" not just \"dog\").\n\nAdditional extraction rules:\n- Presupposed facts: When the user reveals a fact indirectly in a question (e.g. \"What collar suits a Golden Retriever like Max?\" presupposes Max is a Golden Retriever), extract that fact as a separate memory.\n- Date precision: Always preserve the full date including day-of-month when stated (e.g. \"February 14th\" not just \"February\", \"March 5\" not just \"March\").\n- Life events in passing: When the user mentions a life event (birth, wedding, graduation, move, adoption) while discussing something else, extract the life event as its own memory (e.g. \"my friend David had a baby boy named Jasper\" is a birth fact, even if mentioned while planning to send congratulations).\n\nIf there is nothing worth remembering (e.g. empty result, no decisions or facts), return an empty list.\nOutput a JSON object with a single key \"memories\" whose value is a list of strings.",
"extract_memories_user": "Content:\n{content}\n\nExtract memory statements as described. Return structured output.", "extract_memories_user": "Content:\n{content}\n\nExtract memory statements as described. Return structured output.",
"query_user": "Query: {query}\n\nAvailable scopes: {available_scopes}\n{scope_desc}\n\nReturn the analysis as structured output.", "query_user": "Query: {query}\n\nAvailable scopes: {available_scopes}\n{scope_desc}\n\nReturn the analysis as structured output.",
"save_system": "You analyze content to be stored in a hierarchical memory system.\nGiven the content and the existing scopes and categories, output:\n1. suggested_scope: The best matching existing scope path, or a new path if none fit (use / for root).\n2. categories: A list of categories (reuse existing when relevant, add new ones if needed).\n3. importance: A number from 0.0 to 1.0 indicating how significant this memory is.\n4. extracted_metadata: A JSON object with any entities, dates, or topics you can extract.", "save_system": "You analyze content to be stored in a hierarchical memory system.\nGiven the content and the existing scopes and categories, output:\n1. suggested_scope: The best matching existing scope path, or a new path if none fit (use / for root).\n2. categories: A list of categories (reuse existing when relevant, add new ones if needed).\n3. importance: A number from 0.0 to 1.0 indicating how significant this memory is.\n4. extracted_metadata: A JSON object with any entities, dates, or topics you can extract.",

View File

@@ -3,6 +3,7 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import Callable, Sequence from collections.abc import Callable, Sequence
import concurrent.futures import concurrent.futures
import contextvars
import inspect import inspect
import json import json
import re import re
@@ -907,8 +908,9 @@ def summarize_messages(
chunks=chunks, llm=llm, callbacks=callbacks, i18n=i18n chunks=chunks, llm=llm, callbacks=callbacks, i18n=i18n
) )
if is_inside_event_loop(): if is_inside_event_loop():
ctx = contextvars.copy_context()
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool: with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
summarized_contents = pool.submit(asyncio.run, coro).result() summarized_contents = pool.submit(ctx.run, asyncio.run, coro).result()
else: else:
summarized_contents = asyncio.run(coro) summarized_contents = asyncio.run(coro)

Some files were not shown because too many files have changed in this diff Show More