mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-04-08 20:18:16 +00:00
9325e2f6a4
* fix: add path and URL validation to RAG tools Add validation utilities to prevent unauthorized file reads and SSRF when RAG tools accept LLM-controlled paths/URLs at runtime. Changes: - New crewai_tools.utilities.safe_path module with validate_file_path(), validate_directory_path(), and validate_url() - File paths validated against base directory (defaults to cwd). Resolves symlinks and ../ traversal. Rejects escape attempts. - URLs validated: file:// blocked entirely. HTTP/HTTPS resolves DNS and blocks private/reserved IPs (10.x, 172.16-31.x, 192.168.x, 127.x, 169.254.x, 0.0.0.0, ::1, fc00::/7). - Validation applied in RagTool.add() — catches all RAG search tools (JSON, CSV, PDF, TXT, DOCX, MDX, Directory, etc.) - Removed file:// scheme support from DataTypes.from_content() - CREWAI_TOOLS_ALLOW_UNSAFE_PATHS=true env var for backward compat - 27 tests covering traversal, symlinks, private IPs, cloud metadata, IPv6, escape hatch, and valid paths/URLs * fix: validate path/URL keyword args in RagTool.add() The original patch validated positional *args but left all keyword arguments (path=, file_path=, directory_path=, url=, website=, github_url=, youtube_url=) unvalidated, providing a trivial bypass for both path-traversal and SSRF checks. Applies validate_file_path() to path/file_path/directory_path kwargs and validate_url() to url/website/github_url/youtube_url kwargs before they reach the adapter. Adds a regression-test file covering all eight kwarg vectors plus the two existing positional-arg checks. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address CodeQL and review comments on RAG path/URL validation - Replace insecure tempfile.mktemp() with inline symlink target in test - Remove unused 'target' variable and unused tempfile import - Narrow broad except Exception: pass to only catch urlparse errors; validate_url ValueError now propagates instead of being silently swallowed - Fix ruff B904 (raise-without-from-inside-except) in safe_path.py - Fix ruff B007 (unused loop variable 'family') in safe_path.py - Use validate_directory_path in DirectorySearchTool.add() so the public utility is exercised in production code Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: fix ruff format + remaining lint issues * fix: resolve mypy type errors in RAG path/URL validation - Cast sockaddr[0] to str() to satisfy mypy (socket.getaddrinfo returns sockaddr where [0] is str but typed as str | int) - Remove now-unnecessary `type: ignore[assignment]` and `type: ignore[literal-required]` comments in rag_tool.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: unroll dynamic TypedDict key loops to satisfy mypy literal-required Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: allow tmp paths in RAG data-type tests via CREWAI_TOOLS_ALLOW_UNSAFE_PATHS TemporaryDirectory creates files under /tmp/ which is outside CWD and is correctly blocked by the new path validation. These tests exercise data-type handling, not security, so add an autouse fixture that sets CREWAI_TOOLS_ALLOW_UNSAFE_PATHS=true for the whole file. Path/URL security is covered by test_rag_tool_path_validation.py. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: allow tmp paths in search-tool and rag_tool tests via CREWAI_TOOLS_ALLOW_UNSAFE_PATHS test_search_tools.py has tests for TXTSearchTool, CSVSearchTool, MDXSearchTool, JSONSearchTool, and DirectorySearchTool that create files under /tmp/ via tempfile, which is outside CWD and correctly blocked by the new path validation. rag_tool_test.py has one test that calls tool.add() with a TemporaryDirectory path. Add the same autouse allow_tmp_paths fixture used in test_rag_tool_add_data_type.py. Security is covered separately by test_rag_tool_path_validation.py. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: update tool specifications * docs: document CodeInterpreterTool removal and RAG path/URL validation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address three review comments on path/URL validation - safe_path._is_private_or_reserved: after unwrapping IPv4-mapped IPv6 to IPv4, only check against IPv4 networks to avoid TypeError when comparing an IPv4Address against IPv6Network objects. - safe_path.validate_file_path: handle filesystem-root base_dir ('/') by not appending os.sep when the base already ends with a separator, preventing the '//'-prefix bug. - rag_tool.add: path-detection heuristic now checks for both '/' and os.sep so forward-slash paths are caught on Windows as well as Unix. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove unused _BLOCKED_NETWORKS variable after IPv4/IPv6 split * chore: update tool specifications --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
93 lines
4.1 KiB
Plaintext
93 lines
4.1 KiB
Plaintext
---
|
||
title: بحث RAG في CSV
|
||
description: أداة `CSVSearchTool` هي أداة RAG (الاسترجاع المعزز بالتوليد) قوية مصممة لعمليات البحث الدلالي داخل محتوى ملف CSV.
|
||
icon: file-csv
|
||
mode: "wide"
|
||
---
|
||
|
||
# `CSVSearchTool`
|
||
|
||
<Note>
|
||
**تجريبية**: لا نزال نعمل على تحسين الأدوات، لذا قد يحدث سلوك غير متوقع أو تغييرات في المستقبل.
|
||
</Note>
|
||
|
||
## الوصف
|
||
|
||
تُستخدم هذه الأداة لإجراء بحث RAG (الاسترجاع المعزز بالتوليد) داخل محتوى ملف CSV. تتيح للمستخدمين البحث دلالياً عن استعلامات في محتوى ملف CSV محدد. هذه الميزة مفيدة بشكل خاص لاستخراج المعلومات من مجموعات بيانات CSV الكبيرة حيث قد تكون طرق البحث التقليدية غير فعالة. جميع الأدوات التي تحتوي على "Search" في اسمها، بما في ذلك CSVSearchTool، هي أدوات RAG مصممة للبحث في مصادر بيانات مختلفة.
|
||
|
||
## التثبيت
|
||
|
||
قم بتثبيت حزمة crewai_tools
|
||
|
||
```shell
|
||
pip install 'crewai[tools]'
|
||
```
|
||
|
||
## مثال
|
||
|
||
```python Code
|
||
from crewai_tools import CSVSearchTool
|
||
|
||
# Initialize the tool with a specific CSV file.
|
||
# This setup allows the agent to only search the given CSV file.
|
||
tool = CSVSearchTool(csv='path/to/your/csvfile.csv')
|
||
|
||
# OR
|
||
|
||
# Initialize the tool without a specific CSV file.
|
||
# Agent will need to provide the CSV path at runtime.
|
||
tool = CSVSearchTool()
|
||
```
|
||
|
||
## المعاملات
|
||
|
||
يمكن استخدام المعاملات التالية لتخصيص سلوك `CSVSearchTool`:
|
||
|
||
| المعامل | النوع | الوصف |
|
||
|:---------------|:---------|:-------------------------------------------------------------------------------------------------------------------------------------|
|
||
| **csv** | `string` | _اختياري_. مسار ملف CSV المراد البحث فيه. هذا معامل إلزامي إذا تمت تهيئة الأداة بدون ملف CSV محدد؛ وإلا فهو اختياري. |
|
||
|
||
## النموذج والتضمينات المخصصة
|
||
|
||
بشكل افتراضي، تستخدم الأداة OpenAI لكل من التضمينات والتلخيص. لتخصيص النموذج، يمكنك استخدام قاموس تكوين كما يلي:
|
||
|
||
```python Code
|
||
from chromadb.config import Settings
|
||
|
||
tool = CSVSearchTool(
|
||
config={
|
||
"embedding_model": {
|
||
"provider": "openai",
|
||
"config": {
|
||
"model": "text-embedding-3-small",
|
||
# "api_key": "sk-...",
|
||
},
|
||
},
|
||
"vectordb": {
|
||
"provider": "chromadb", # or "qdrant"
|
||
"config": {
|
||
# "settings": Settings(persist_directory="/content/chroma", allow_reset=True, is_persistent=True),
|
||
# from qdrant_client.models import VectorParams, Distance
|
||
# "vectors_config": VectorParams(size=384, distance=Distance.COSINE),
|
||
}
|
||
},
|
||
}
|
||
)
|
||
```
|
||
|
||
## الأمان
|
||
|
||
### التحقق من صحة المسارات
|
||
|
||
يتم التحقق من مسارات الملفات المقدمة لهذه الأداة مقابل مجلد العمل الحالي. يتم رفض المسارات التي تحل خارج مجلد العمل وإطلاق `ValueError`.
|
||
|
||
للسماح بالمسارات خارج مجلد العمل (مثلاً في الاختبارات أو خطوط الأنابيب الموثوقة)، عيّن متغير البيئة التالي:
|
||
|
||
```shell
|
||
CREWAI_TOOLS_ALLOW_UNSAFE_PATHS=true
|
||
```
|
||
|
||
### التحقق من صحة الروابط
|
||
|
||
يتم التحقق من مدخلات الروابط: يتم حظر مخطط `file://` والطلبات التي تستهدف نطاقات IP الخاصة أو المحجوزة لمنع هجمات تزوير الطلبات من جانب الخادم (SSRF).
|