15 Commits

Author	SHA1	Message	Date
Lucas Gomide	e9d4c57b2a	fix: run model call hooks on every path and propagate a deny (#7111) ... * fix: let a hook deny reach the caller as a deny A hook that raised `HookAborted` on `pre_model_call` never reached the code making the call: the LLM layer caught it and returned `False`, which providers translated into `ValueError("LLM call blocked by before_llm_call hook")`, dropping the reason and the source and making a policy decision indistinguishable from a provider outage. Every internal model call then absorbed that error through the `except Exception` that keeps a provider hiccup from failing a run, so memory analysis fell back to defaults and the converter and reasoning handler retried the call that was just denied. The abort now propagates out of the LLM layer while the boolean convention keeps its documented `ValueError` via `LegacyHookBlocked`, and the fail-open handlers around internal model calls re-raise it instead of degrading. * fix: dispatch model call hooks on the paths that skipped them A model call was only checked when the executor loop drove it: the `from_agent is not None` short-circuit in `base_llm` silenced the hooks for agent planning and step observation, no provider `acall` dispatched them at all, and `InternalInstructor` bypassed `llm.call` entirely. This replaces that short-circuit with an explicit `model_call_hooks_already_dispatched` window so the enclosing caller claims the dispatch, adds the pre-call dispatch to every provider's `acall`, and runs the hooks around the Instructor client call. A denial now emits a denied event instead of being logged and reported as a provider failure. * fix: report a boolean-convention deny as a deny, not an outage A `before_llm_call` hook that blocks by returning `False` reached the five native providers as a plain `ValueError`, which fell through to their generic `except Exception` and was logged and emitted as `OpenAI API call failed: ...` — the same deny raised as `HookAborted` was already labelled correctly, so the two dialects disagreed on whether a policy decision was a provider outage. The LLM layer now converts it into `LLMCallBlockedError`, still a `ValueError` so the fail-open handlers around internal model calls keep absorbing it, but its own type so a provider can report the decision it is. Since a block is raised rather than returned, the thirteen callers that turned the return flag into a raise by hand drop that line, and `_prepare_llm_call` raises the same type. * fix: keep a denied plan from letting the agent run unplanned `AgentExecutor.generate_plan` wraps `handle_agent_reasoning()` in a bare `except Exception`, so guarding the reasoning handler alone still left the deny absorbed one frame up: the executor logged "Error during planning" and the agent proceeded with no plan. It now re-raises `HookAborted` like the other planning boundaries, and the accompanying test also covers the boolean convention still degrading at a fail-open site. * fix: stop a denied knowledge query from running the task without knowledge `handle_knowledge_retrieval` and its async twin wrap the query rewrite in their own `except Exception`, so guarding `_get_knowledge_search_query` alone still let `execute_task` continue on the unaugmented prompt after a deny. Both now emit the terminal `KnowledgeSearchQueryFailedEvent` and re-raise `HookAborted`, matching the second-frame guard already added to `AgentExecutor.generate_plan`. Also documents the abort contract on `PlannerObserver.observe`. * fix: stop nine callers from re-swallowing a model call deny CodeRabbit caught the replan path re-swallowing a deny, so an AST sweep of every caller of a guarded function found the same defeat in nine places: classic and replan planning, memory recall and memory save on both `Agent` and `LiteAgent`, the base executor's save, and `LLMGuardrail.__call__`, which turned a refused call into validation feedback. Each now re-raises `HookAborted` after emitting whatever terminal event it owes, while every other failure keeps degrading as before — the knowledge guards move to that same idiom instead of duplicating their emit. * fix: pair a denied guardrail with the event it started Re-raising from `LLMGuardrail` left `process_guardrail` between its started and completed events, so a denied validation read as one still in flight rather than a policy decision. It now emits `LLMGuardrailCompletedEvent` with the deny reason before the abort leaves, matching what every other guarded site in this change already does. * fix: stop retrying a task after a hook denied its model call `Agent.execute_task` funnels every exception into `_handle_execution_error`, which re-runs the whole task up to `max_retry_limit` times, so a policy deny read as a transient blip: a crew whose first model call was denied retried and returned a normal answer. `HookAborted` now joins `_passthrough_exceptions`, the tuple already reserved for deliberate stops. The new boundary tests drive the public entry points instead of the frame that makes the call, and count model calls so a deny that gets retried fails the assertion — ten of the twelve fail against `main`. * fix: stop a denied plan step from being reported as a failed step Making model call hooks reachable on agent-bearing calls put a deny inside `StepExecutor.execute`, whose broad `except Exception` turned it into `StepResult(success=False)` and let the plan carry on; `HookAborted` now joins `ToolExecutionFailedError` in the passthrough handlers there, and `execute_todos_parallel` re-raises a deny that `return_exceptions=True` would otherwise record as one failed todo. `_emit_call_denied_event` also renders the source through the now-public `source_name`, so a hook that names itself with a callable reads as its name instead of a repr. --------- Co-authored-by: Vidit Ostwal <110953813+Vidit-Ostwal@users.noreply.github.com>	2026-08-28 12:32:09 -04:00
Lucas Gomide	3df34d9169	fix: skip interception hooks on crewai-internal flows (#7079) ... * fix: skip interception hooks on crewai-internal flows The `AgentExecutor` and the memory encoding/recall flows are `Flow` subclasses CrewAI runs for its own bookkeeping, and they were dispatching interception points as if their methods were the caller's steps — a hook saw machinery no user wrote, and a policy could deny a run over it. `Flow._skip_interception` now suppresses every point on a flow marked `is_crewai_internal`, except the execution boundary on machinery that is itself the run the caller asked for. A standalone `Agent.kickoff()` keeps its boundary and stays blockable, while the same executor bound to a crew or nested in a caller's flow stays silent, so boundaries only ever fire at the root. * docs: note that the resume match id feeds the boundary check `from_pending` seeds `_flow_match_id` from `instance.flow_id` for the usage listener's filter, and `resume_async` forces `current_flow_id` to it for the duration of the resume. `AgentExecutor._owns_execution_boundary` compares the two, so seeding the original persisted id instead would make a resumed standalone agent disown a boundary its kickoff already opened.	2026-08-25 09:54:17 -04:00
Lucas Gomide	65a4b7cede	fix: emit FlowStartedEvent when a boundary hook aborts the flow (#6953) ... * fix: emit FlowStartedEvent when a boundary hook aborts the flow A HookAborted at EXECUTION_START or INPUT propagated before `FlowStartedEvent` was emitted, so a policy deny left logs but no record of the execution. On abort, stamp the state id and open the flow scope before re-raising: the deny surfaces as a started -> failed execution while normal runs keep the existing ordering — the started event carries hook-resolved inputs and `id` rewrites keep redirecting persistence restoration. * docs: translate execution-boundary-hooks page to ar, ko, and pt-BR The English page updated on this branch had never been localized. Translate it into the three supported locales following `DOCS_TRANSLATIONS.md` and register the page in each locale's navigation in `docs/docs.json`. Untranslated link targets (the step-hooks page and the aborting-an-operation anchor) are omitted rather than pointed at English, matching the locale navigation convention.	2026-08-11 11:54:27 -07:00
Lucas Gomide	3bb87532da	fix: dispatch execution_end hook on failed crew and flow executions (#6607) ... * fix: dispatch execution_end hook on failed crew and flow executions The `execution_end` interception point only fired after a successful kickoff, so consumers never learned about failed runs. Crew kickoff paths (`kickoff`/`akickoff`) and the flow runtime (`kickoff_async`, `resume_async`) now dispatch it on the failure path too, with new additive `status` ("completed"/"failed") and `error` fields on `ExecutionEndContext`. Pairing flags guarantee exactly-once dispatch, keep the start/end pairing invariant, and the original exception propagates unchanged. * fix: track execution_end pairing per invocation for reentrant flows Reentrant kickoffs on the same Flow instance are supported (usage aggregation already accommodates them), but the instance-level pairing booleans let an inner kickoff's completion mark the outer execution as ended, skipping the outer failure's `execution_end`. The pairing state now lives in each `kickoff_async` invocation's locals, and the resume path passes a per-invocation holder into `_resume_async_body`. Crew keeps its instance flags since crew kickoffs are not reentrant on the same instance (`kickoff_for_each` copies the crew).	2026-07-21 15:25:06 -04:00
Lucas Gomide	9a49af098b	fix: sync kickoff-completed event with OUTPUT hook result (#6571) ... * fix: sync kickoff-completed event with OUTPUT hook result `CrewKickoffCompletedEvent` still carried the pre-hook `TaskOutput`, so AMP/OTEL consumers never saw `OUTPUT` mutations even though the returned `CrewOutput` was updated. Sync `final_task_output.raw` from the post-hook payload before emit, matching `FlowFinishedEvent`. * style: drop OUTPUT sync comment and rename crew output test	2026-07-16 14:37:56 -04:00
Lucas Gomide	0e5d0ecfb9	feat: add step interception points and rework execution hooks docs around @on (#6518) ... * feat: add pre_step and post_step interception points on task execution Introduces `StepContext` and the two step points in the dispatcher, and wires them around agent execution in `task.py` (sync and async paths): `pre_step` fires after `TaskStartedEvent` with the task context as payload, `post_step` fires before `TaskCompletedEvent` with the `TaskOutput`, and hook replacements are rebound in both directions. * feat: wire pre_step and post_step on flow method execution Dispatches the step points around each flow method with kind="flow_method": `pre_step` receives the dumped call params and maps returned edits back onto args/kwargs, `post_step` can rewrite the method result before it is recorded. Conformance tests cover per-method firing and output rewriting. * docs: rework execution hooks page around the @on api Replaces the standalone interception hooks catalog with a single `execution-hooks.mdx` page that teaches `@on` as the primary way to write hooks, covering the full ten-point catalog across task, flow, and LLM execution. The legacy per-point decorators stay documented in a closing section, and the `docs.json` navigation drops the removed page.	2026-07-14 14:47:18 -04:00
Lucas Gomide	a194f3867a	feat: wire execution-boundary interception points (#6517) ... * feat: wire execution-boundary interception points Adds the typed interception contexts (`crewai/hooks/contexts.py`) and wires the `execution_start`, `input`, `output`, and `execution_end` points for both crews and flows through the dispatcher. `prepare_kickoff` and `Flow.kickoff_async` fire `execution_start`/`input` so a hook can rewrite resolved inputs before the run, while `Crew._create_crew_output` and the flow tail fire `output`/`execution_end` so the final result can be observed or replaced. Closes the eight critical-path points without touching the legacy hooks. * fix: correct execution-boundary hook ordering and input aliasing Reworks the crew and flow boundary seams flagged in review. `OUTPUT` and `EXECUTION_END` now run before the completion event (`CrewKickoffCompletedEvent` and `FlowFinishedEvent`) so a `HookAborted` no longer leaves a spurious completed signal and a returned payload replacement is honored on the emitted and returned result. Boundary contexts alias `inputs` to the same object as `payload` instead of a fresh dict from `or`, so in-place edits survive read-back. Flows re-publish the resolved inputs into `flow_inputs` baggage after the `INPUT` hook so trigger-payload injection observes hook rewrites, and a resumed flow now dispatches `OUTPUT`/`EXECUTION_END` on its completion path. * chore: drop redundant seam comments from execution-boundary wiring Removes two inline comments narrating the OUTPUT/EXECUTION_END dispatch ordering in `crew.py` and the flow runtime, plus a stray sentence about enterprise adapters in the conformance-suite docstring. Comment-only cleanup, no behavior change. * fix: keep crew output typed across boundary hook dispatch `_create_crew_output` reassigned `crew_output` from the hook contexts' `payload`, which is typed `Any`, so mypy flagged `no-any-return` at the function's return. Cast the payload back to `CrewOutput` after each dispatch and split the `ExecutionEndContext` construction to satisfy `ruff format`'s line-length limit.	2026-07-14 10:43:18 -04:00
Lucas Gomide	7d21283630	feat: add generic interception-hook dispatcher (#6516) ... * feat: add generic interception-hook dispatcher Introduces `crewai/hooks/dispatch.py` as a single engine behind every interception point: a hook receives a typed context, may mutate or replace its `payload`, or raise `HookAborted(reason, source)` to stop the operation. The full `InterceptionPoint` catalog is frozen from day zero, with global and contextvar-scoped registries, an `@on` decorator, a no-op fast path, and a `HookDispatchedEvent` for telemetry. The four existing `before/after_llm_call` and `before/after_tool_call` hooks become adapters over the dispatcher, so the legacy dialect and `return False` semantics keep working unchanged while gaining the new contract. * fix: harden interception dispatcher against review findings Corrects several dispatcher edge cases surfaced in review. `_default_reducer` now reports a modification only when a `payload` is actually applied, the `agents=` filter falls back to `agent_role` for contexts without an `agent` object, and `unregister` resolves the filter wrapper stashed by `on` so a filtered hook can be removed. The tool-hook runners honor the executing agent's `verbose` flag instead of silently swallowing hook errors, and the ReAct tool path now runs `POST_TOOL_CALL` on blocked calls to match the native paths. Also adds abort-telemetry coverage and replaces the flaky absolute no-op timing budget with a relative one. * fix: honor scoped hooks on direct llm calls and register @on crew methods Direct agent-less LLM calls short-circuited on the empty global hook list, so hooks registered only for the current `scoped_hooks()` context never ran; the direct-call helpers now defer to `dispatch`, which resolves scoped hooks behind its own no-op fast path. `CrewBase` likewise only scanned the legacy `is_*_hook` markers, so `@on(InterceptionPoint.X)` methods were silently dropped — it now registers them on the dispatcher with filters applied and `self` bound. Also tightens result typing across the tool-call seams so `mypy` stays green. * refactor: scope InterceptionPoint to the points this layer wires The dispatcher only fires the model- and tool-call boundaries, so `InterceptionPoint` now lists just those four rather than the full future catalog. New points are introduced alongside the seams that dispatch them, keeping every layer free of enum members with no live consumer. The dispatcher unit tests that borrowed unused points as generic examples are remapped onto the four kept points. * test: pin per-hook fail-open at the LLM and tool seams The dispatcher swallows a hook's exception per hook rather than around the whole loop, so one buggy hook no longer silently skips every hook registered after it. These seam-level tests pin that behavior through `_setup_before_llm_call_hooks` and `run_before/after_tool_call_hooks`, and confirm an intentional `return False` block still short-circuits later hooks. * fix: run execution-scoped hooks on the agent executor model seams `_setup_before/after_llm_call_hooks` only ran the executor's snapshot hook lists, so hooks registered via `scoped_hooks()` never fired on `PRE/POST_MODEL_CALL` during normal agent execution, while the tool seams (which go through `dispatch`) merged them. The seams now append the current scope's hooks after the snapshot via `get_scoped_hooks`, matching dispatch's global-then-scoped ordering, and a scoped-only registration no longer short-circuits the seam.	2026-07-14 10:27:34 -04:00
Lucas Gomide	6452608724	fix: after_llm_call hooks no longer break native tool execution (#6531) ... * fix: don't clobber native tool-call responses in after-LLM hooks Registering any `after_llm_call` hook broke native tool execution: the executor invokes `_setup_after_llm_call_hooks` on the intermediate response that carries the model's tool calls, the non-str payload was stringified for the response-rewrite pass, and the executor then treated that string as a final answer instead of executing the tools. Structured payloads (neither `str` nor `BaseModel`) now pass through untouched, mirroring the isinstance guard `_invoke_after_llm_call_hooks` already applies on the direct-call path; hooks still fire on the follow-up textual response. Fixes #6529 * style: shorten the tool-call guard comment	2026-07-14 09:49:21 -04:00
Vinicius Brasil	9db2d44766	Add typed output schemas for CrewAI tools (#6236) ... Currently, tools have a strong input contract through `args_schema`, but no output contract. This means that anything a tool outputs is converted to string. Not only the contract is weak, but the "invisible" conversion to string can have unexpected effects when the tool returns complex objects like dicts and arrays. With this PR, a tool can _optionally_ define an output contract with `output_schema`. CrewAI validates the raw result and sends the agent JSON. ```python class ProductResult(BaseModel): sku: str name: str in_stock: bool class ProductLookupTool(BaseTool): name: str = "Product Lookup" description: str = "Look up product availability by SKU." def _run(self, sku: str) -> ProductResult: return ProductResult(sku=sku, name="USB-C dock", in_stock=True) ``` If the result does not match the schema, CrewAI warns and falls back to `str(raw_result)` instead of failing the run: ```python @tool("Product Lookup", output_schema=ProductResult) def product_lookup(sku: str) -> dict[str, object]: return {"sku": sku, "name": "USB-C dock", "in_stock": True} #=> RuntimeWarning: Failed to validate or serialize output from tool 'Bad Product Lookup' using output_schema 'ProductResult'... Falling back to str(raw_result). ``` This is additive and non-breaking. Existing tools do not need to change. Tools without `output_schema` keep the old string behavior. Invalid typed outputs warn and fall back to the old formatting path.	2026-06-19 14:33:51 -07:00
Greyson LaLonde	fd10c64148	chore(crewai): drop self-explanatory comments	2026-05-26 10:23:33 -07:00
Greyson LaLonde	98e0d1054f	fix: sanitize tool names in hook decorator filters	2026-04-08 21:02:25 +08:00
Lorenze Jay	f53bdb28ac	feat: implement before and after tool call hooks in CrewAgentExecutor… (#4287) ... * feat: implement before and after tool call hooks in CrewAgentExecutor and AgentExecutor - Added support for before and after tool call hooks in both CrewAgentExecutor and AgentExecutor classes. - Introduced ToolCallHookContext to manage context for hooks, allowing for enhanced control over tool execution. - Implemented logic to block tool execution based on before hooks and to modify results based on after hooks. - Added integration tests to validate the functionality of the new hooks, ensuring they work as expected in various scenarios. - Enhanced the overall flexibility and extensibility of tool interactions within the CrewAI framework. * Potential fix for pull request finding 'Unused local variable' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * Potential fix for pull request finding 'Unused local variable' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * test: add integration test for before hook blocking tool execution in Crew - Implemented a new test to verify that the before hook can successfully block the execution of a tool within a crew. - The test checks that the tool is not executed when the before hook returns False, ensuring proper control over tool interactions. - Enhanced the validation of hook calls to confirm that both before and after hooks are triggered appropriately, even when execution is blocked. - This addition strengthens the testing coverage for tool call hooks in the CrewAI framework. * drop unused * refactor(tests): remove OPENAI_API_KEY check from tool hook tests - Eliminated the check for the OPENAI_API_KEY environment variable in the test cases for tool hooks. - This change simplifies the test setup and allows for running tests without requiring the API key to be set, improving test accessibility and flexibility. --------- Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-01-27 14:56:50 -08:00
Lorenze Jay	c456e5c5fa	Lorenze/ensure hooks work with lite agents flows (#3981) ... * liteagent support hooks * wip llm.call hooks work - needs tests for this * fix tests * fixed more * more tool hooks test cassettes	2025-12-04 09:38:39 -08:00
Lorenze Jay	528d812263	Lorenze/feat hooks (#3902) ... * feat: implement LLM call hooks and enhance agent execution context - Introduced LLM call hooks to allow modification of messages and responses during LLM interactions. - Added support for before and after hooks in the CrewAgentExecutor, enabling dynamic adjustments to the execution flow. - Created LLMCallHookContext for comprehensive access to the executor state, facilitating in-place modifications. - Added validation for hook callables to ensure proper functionality. - Enhanced tests for LLM hooks and tool hooks to verify their behavior and error handling capabilities. - Updated LiteAgent and CrewAgentExecutor to accommodate the new crew context in their execution processes. * feat: implement LLM call hooks and enhance agent execution context - Introduced LLM call hooks to allow modification of messages and responses during LLM interactions. - Added support for before and after hooks in the CrewAgentExecutor, enabling dynamic adjustments to the execution flow. - Created LLMCallHookContext for comprehensive access to the executor state, facilitating in-place modifications. - Added validation for hook callables to ensure proper functionality. - Enhanced tests for LLM hooks and tool hooks to verify their behavior and error handling capabilities. - Updated LiteAgent and CrewAgentExecutor to accommodate the new crew context in their execution processes. * fix verbose * feat: introduce crew-scoped hook decorators and refactor hook registration - Added decorators for before and after LLM and tool calls to enhance flexibility in modifying execution behavior. - Implemented a centralized hook registration mechanism within CrewBase to automatically register crew-scoped hooks. - Removed the obsolete base.py file as its functionality has been integrated into the new decorators and registration system. - Enhanced tests for the new hook decorators to ensure proper registration and execution flow. - Updated existing hook handling to accommodate the new decorator-based approach, improving code organization and maintainability. * feat: enhance hook management with clear and unregister functions - Introduced functions to unregister specific before and after hooks for both LLM and tool calls, improving flexibility in hook management. - Added clear functions to remove all registered hooks of each type, facilitating easier state management and cleanup. - Implemented a convenience function to clear all global hooks in one call, streamlining the process for testing and execution context resets. - Enhanced tests to verify the functionality of unregistering and clearing hooks, ensuring robust behavior in various scenarios. * refactor: enhance hook type management for LLM and tool hooks - Updated hook type definitions to use generic protocols for better type safety and flexibility. - Replaced Callable type annotations with specific BeforeLLMCallHookType and AfterLLMCallHookType for clarity. - Improved the registration and retrieval functions for before and after hooks to align with the new type definitions. - Enhanced the setup functions to handle hook execution results, allowing for blocking of LLM calls based on hook logic. - Updated related tests to ensure proper functionality and type adherence across the hook management system. * feat: add execution and tool hooks documentation - Introduced new documentation for execution hooks, LLM call hooks, and tool call hooks to provide comprehensive guidance on their usage and implementation in CrewAI. - Updated existing documentation to include references to the new hooks, enhancing the learning resources available for users. - Ensured consistency across multiple languages (English, Portuguese, Korean) for the new documentation, improving accessibility for a wider audience. - Added examples and troubleshooting sections to assist users in effectively utilizing hooks for agent operations. --------- Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>	2025-11-13 10:11:50 -08:00