6 Commits

Author	SHA1	Message	Date
João Moura	2e95bfb4e8	fix(tools): sandbox FileWriterTool writes and fix file tool rough edges (#6692) ... * fix(tools): sandbox FileWriterTool writes and fix file tool rough edges FileReadTool confined reads to the working directory, but FileWriterTool only checked that `filename` stayed inside `directory` — and `directory` itself is an LLM-supplied schema field. An agent could therefore write anywhere the process had permission to, including ~/.ssh and site-packages, while the reader refused to read back what the writer had just written. FileWriterTool was the only filesystem tool in the package that did not go through validate_file_path; files_compressor_tool validates even its output path. Writes are now confined to base_dir (the working directory by default): the resolved directory must sit inside base_dir, and the resolved file must sit inside that directory. The pre-existing filename containment check is kept as-is and still applies even when the unsafe-paths escape hatch is on, so no existing guarantee is weakened. Both tools gain a base_dir field so a developer can widen the sandbox deliberately instead of reaching for the process-wide CREWAI_TOOLS_ALLOW_UNSAFE_PATHS kill switch. FileReadTool also stops rejecting a file_path given to its own constructor: that is developer-declared intent, and declaring one file does not expose its siblings. Also fixed: - FileReadTool scanned the whole file when reading a line window; it now stops via islice once the requested lines are collected. - FileWriterTool._run(**kwargs) made the documented positional call signature raise TypeError and turned a missing overwrite into "error accessing key". It now takes named parameters in the documented (filename, content, directory) order. - A directory naming an existing file reported "already exists and overwrite option was not passed" even with overwrite=True; it now explains the real problem. - Subdirectories inside filename are created, matching what passing directory already did. - Both tools now write and decode UTF-8 by default instead of the platform locale encoding, with an encoding field to override. The docs already claimed UTF-8 and recommended the writer to Windows users. - The writer's schema fields had no descriptions for the LLM. - Docs claimed FileReadTool parses JSON into a dict (it never has), shipped a snippet that raised TypeError, and did not mention the path sandbox. The writer README also began with a stray "Here's the rewritten README" preamble. BREAKING CHANGE: FileWriterTool no longer writes outside the working directory. Pass base_dir to authorize a different tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update tool specifications * fix(tools): make the declared FileReadTool file reachable by agents Addresses review feedback on #6692. The constructor-path exemption did not actually work the way an agent calls the tool. The description only advertises a redacted label (the basename, when the file sits outside the sandbox), but resolution required the exact absolute path, so the model's call was sandboxed and the declared file was never read. Worse, file_path was a required schema field, so the long-documented "call with no arguments to read the default file" raised a validation error instead: FileReadTool(file_path="/outside/declared.txt") .run() -> ValueError: validation failed .run(file_path="declared.txt") -> Error: File not found .run(file_path="/outside/declared.txt") -> works, but the model was never told this path file_path is now optional in the schema, so omitting it reads the default, and the declared file is addressable by the label the description shows the model as well as by its real path. Declaring one file still does not expose its siblings. The declared path is also pinned to its real path at construction, so a later chdir cannot silently repoint it at a different file — previously a relative constructor path re-resolved against the new working directory on every call. Also guards the writer's filepath resolution, which could raise ValueError out of _run for a filename containing a null byte, breaking the contract of always returning a descriptive string. The directory and read paths were already guarded. Adds docstrings to strtobool and both _run methods, corrects an Arabic tanween spelling and a kaf-as-descriptor calque in the localized read docs, and regenerates tool.specs.json for the schema change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(tools): anchor the declared read path to base_dir, not the cwd Addresses the second round of review feedback on #6692. The previous commit pinned a relative constructor file_path with os.path.realpath, which anchors to the working directory, while both format_path_for_display and validate_file_path anchor a relative path to base_dir. With the two roots disagreeing, the same relative string meant two different files — and the tool served the cwd one under a label that looks like it belongs to the sandbox: FileReadTool(file_path="data.txt", base_dir="/allowed") # cwd=/work label advertised to the model -> "data.txt" run(file_path="data.txt") -> contents of /work/data.txt That reads a file from outside base_dir, so it was a sandbox escape introduced by the exemption itself, not just a wrong-file bug. Resolution now goes through a single _resolve_against_base helper that anchors relative paths exactly the way the sandbox does, so the pinned path, the advertised label and the containment check all agree. Covered by test_relative_declared_path_anchors_to_base_dir. Also softens "always readable" to "always allowed past the containment check" in the docstring, README and docs, since bypassing containment does not guarantee the read succeeds — it can still fail on a missing file, a directory, or permissions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(tools): tell the LLM about the path sandbox in tool descriptions Addresses the low-confidence notes from the Copilot review on #6692. Both tools' descriptions were pre-sandbox wording, so the model learned about containment only by attempting a path and reading the error back. Both now state that access is confined to the tool's allowed directory and that a path resolving outside it is rejected. The wording deliberately says "the tool's allowed directory" rather than "the working directory", because the root is base_dir when one is set, and naming the absolute root would leak it into the prompt — the same reason paths are redacted in errors. Not changed: the notes also suggested advertising `encoding`. That is a constructor-only field the model cannot set, so describing it to the LLM would be misleading. Also fixes a test docstring that contradicted its own assertion — the public run() path does raise on schema validation failure, which is what the test asserts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(tools): anchor base_dir at construction so the sandbox cannot move Addresses the third review round on #6692. Both remaining findings came from the same habit: storing an unanchored string and re-resolving it later. A relative base_dir was kept verbatim and re-resolved against getcwd() on every call, while the declared file was pinned once at construction. After a chdir the sandbox root moved but the declared default did not, so one tool applied two different roots. base_dir is now resolved once — in the reader's __init__, and via a field_validator on the writer so it also applies on the model_validate path. That also covers the serialization concern. model_dump drops the private pin, and __init__ re-runs on restore, so a relative file_path was re-anchored against whatever the working directory happened to be at load time. With base_dir anchored, restore rebuilds the identical pin. The residual case is a relative file_path with no base_dir, where the sandbox root is the working directory too — so both move together and the tool stays self-consistent. Covered by test_declared_path_survives_a_serialization_round_trip and test_relative_base_dir_is_anchored_at_construction on both tools. Also corrects the writer's 'directory' description, README and docs: the default resolves inside the tool's allowed directory, which is base_dir when one is set, not always the working directory. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-07-28 02:20:26 -07:00
Rip&Tear	5d4851eac7	Fix SSRF redirect bypass in scraping fetches (#6331) ... * Validate redirects for scraping URL fetches * Prevent credential forwarding across redirects	2026-06-25 17:42:49 -07:00
Rip&Tear	d3fc0d31f8	[codex] Redact file tool paths (#6134) ... * Redact file tool paths * Fix for pull request finding 'Empty except' * Potential fix for pull request finding ---------	2026-06-12 15:50:40 +08:00
Greyson LaLonde	c5ea415cda	chore(crewai-tools): drop self-explanatory comments	2026-05-26 16:25:07 -07:00
Greyson LaLonde	868416bfe0	fix: add SSRF and path traversal protections (#5315) ... * fix: add SSRF and path traversal protections CVE-2026-2286: validate_url blocks non-http/https schemes, private IPs, loopback, link-local, reserved addresses. Applied to 11 web tools. CVE-2026-2285: validate_path confines file access to the working directory. Applied to 7 file and directory tools. * fix: drop unused assignment from validate_url call * fix: DNS rebinding protection and allow_private flag Rewrite validated URLs to use the resolved IP, preventing DNS rebinding between validation and request time. SDK-based tools use pin_ip=False since they manage their own HTTP clients. Add allow_private flag for deployments that need internal network access. * fix: unify security utilities and restore RAG chokepoint validation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor: move validation to security/ package + address review comments - Move safe_path.py to crewai_tools/security/; add safe_url.py re-export - Keep utilities/safe_path.py as a backwards-compat shim - Update all 21 import sites to use crewai_tools.security.safe_path - files_compressor_tool: validate output_path (user-controlled) - serper_scrape_website_tool: call validate_url() before building payload - brightdata_unlocker: validate_url() already called without assignment (no-op fix) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor: move validation to security/ package, keep utilities/ as compat shim - security/safe_path.py is the canonical location for all validation - utilities/safe_path.py re-exports for backward compatibility - All tool imports already point to security.safe_path - All review comments already addressed in prior commits * fix: move validation outside try/except blocks, use correct directory validator Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: use resolved paths from validation to prevent symlink TOCTOU, remove unused safe_url.py --------- Co-authored-by: Alex <alex@crewai.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-04-07 14:44:50 -03:00
alex-clawd	9325e2f6a4	fix: add path and URL validation to RAG tools (#5310) ... * fix: add path and URL validation to RAG tools Add validation utilities to prevent unauthorized file reads and SSRF when RAG tools accept LLM-controlled paths/URLs at runtime. Changes: - New crewai_tools.utilities.safe_path module with validate_file_path(), validate_directory_path(), and validate_url() - File paths validated against base directory (defaults to cwd). Resolves symlinks and ../ traversal. Rejects escape attempts. - URLs validated: file:// blocked entirely. HTTP/HTTPS resolves DNS and blocks private/reserved IPs (10.x, 172.16-31.x, 192.168.x, 127.x, 169.254.x, 0.0.0.0, ::1, fc00::/7). - Validation applied in RagTool.add() — catches all RAG search tools (JSON, CSV, PDF, TXT, DOCX, MDX, Directory, etc.) - Removed file:// scheme support from DataTypes.from_content() - CREWAI_TOOLS_ALLOW_UNSAFE_PATHS=true env var for backward compat - 27 tests covering traversal, symlinks, private IPs, cloud metadata, IPv6, escape hatch, and valid paths/URLs * fix: validate path/URL keyword args in RagTool.add() The original patch validated positional *args but left all keyword arguments (path=, file_path=, directory_path=, url=, website=, github_url=, youtube_url=) unvalidated, providing a trivial bypass for both path-traversal and SSRF checks. Applies validate_file_path() to path/file_path/directory_path kwargs and validate_url() to url/website/github_url/youtube_url kwargs before they reach the adapter. Adds a regression-test file covering all eight kwarg vectors plus the two existing positional-arg checks. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address CodeQL and review comments on RAG path/URL validation - Replace insecure tempfile.mktemp() with inline symlink target in test - Remove unused 'target' variable and unused tempfile import - Narrow broad except Exception: pass to only catch urlparse errors; validate_url ValueError now propagates instead of being silently swallowed - Fix ruff B904 (raise-without-from-inside-except) in safe_path.py - Fix ruff B007 (unused loop variable 'family') in safe_path.py - Use validate_directory_path in DirectorySearchTool.add() so the public utility is exercised in production code Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: fix ruff format + remaining lint issues * fix: resolve mypy type errors in RAG path/URL validation - Cast sockaddr[0] to str() to satisfy mypy (socket.getaddrinfo returns sockaddr where [0] is str but typed as str | int) - Remove now-unnecessary `type: ignore[assignment]` and `type: ignore[literal-required]` comments in rag_tool.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: unroll dynamic TypedDict key loops to satisfy mypy literal-required Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: allow tmp paths in RAG data-type tests via CREWAI_TOOLS_ALLOW_UNSAFE_PATHS TemporaryDirectory creates files under /tmp/ which is outside CWD and is correctly blocked by the new path validation. These tests exercise data-type handling, not security, so add an autouse fixture that sets CREWAI_TOOLS_ALLOW_UNSAFE_PATHS=true for the whole file. Path/URL security is covered by test_rag_tool_path_validation.py. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: allow tmp paths in search-tool and rag_tool tests via CREWAI_TOOLS_ALLOW_UNSAFE_PATHS test_search_tools.py has tests for TXTSearchTool, CSVSearchTool, MDXSearchTool, JSONSearchTool, and DirectorySearchTool that create files under /tmp/ via tempfile, which is outside CWD and correctly blocked by the new path validation. rag_tool_test.py has one test that calls tool.add() with a TemporaryDirectory path. Add the same autouse allow_tmp_paths fixture used in test_rag_tool_add_data_type.py. Security is covered separately by test_rag_tool_path_validation.py. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: update tool specifications * docs: document CodeInterpreterTool removal and RAG path/URL validation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address three review comments on path/URL validation - safe_path._is_private_or_reserved: after unwrapping IPv4-mapped IPv6 to IPv4, only check against IPv4 networks to avoid TypeError when comparing an IPv4Address against IPv6Network objects. - safe_path.validate_file_path: handle filesystem-root base_dir ('/') by not appending os.sep when the base already ends with a separator, preventing the '//'-prefix bug. - rag_tool.add: path-detection heuristic now checks for both '/' and os.sep so forward-slash paths are caught on Windows as well as Unix. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove unused _BLOCKED_NETWORKS variable after IPv4/IPv6 split * chore: update tool specifications --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-04-07 13:29:45 -03:00