Lorenze/enh decouple executor from crew (#4209)

* wip restrcuturing agent executor and liteagent

* fix: handle None task in AgentExecutor to prevent errors

Added a check to ensure that if the task is None, the method returns early without attempting to access task properties. This change improves the robustness of the AgentExecutor by preventing potential errors when the task is not set.

* refactor: streamline AgentExecutor initialization by removing redundant parameters

Updated the Agent class to simplify the initialization of the AgentExecutor by removing unnecessary task and crew parameters in standalone mode. This change enhances code clarity and maintains backward compatibility by ensuring that the executor is correctly configured without redundant assignments.

* ensure executors work inside a flow due to flow in flow async structure

* refactor: enhance agent kickoff preparation by separating common logic

Updated the Agent class to introduce a new private method  that consolidates the common setup logic for both synchronous and asynchronous kickoff executions. This change improves code clarity and maintainability by reducing redundancy in the kickoff process, while ensuring that the agent can still execute effectively within both standalone and flow contexts.

* linting and tests

* fix test

* refactor: improve test for Agent kickoff parameters

Updated the test for the Agent class to ensure that the kickoff method correctly preserves parameters. The test now verifies the configuration of the agent after kickoff, enhancing clarity and maintainability. Additionally, the test for asynchronous kickoff within a flow context has been updated to reflect the Agent class instead of LiteAgent.

* refactor: update test task guardrail process output for improved validation

Refactored the test for task guardrail process output to enhance the validation of the output against the OpenAPI schema. The changes include a more structured request body and updated response handling to ensure compliance with the guardrail requirements. This update aims to improve the clarity and reliability of the test cases, ensuring that task outputs are correctly validated and feedback is appropriately provided.

* test fix cassette

* test fix cassette

* working

* working cassette

* refactor: streamline agent execution and enhance flow compatibility

Refactored the Agent class to simplify the execution method by removing the event loop check and clarifying the behavior when called from synchronous and asynchronous contexts. The changes ensure that the method operates seamlessly within flow methods, improving clarity in the documentation. Additionally, updated the AgentExecutor to set the response model to None, enhancing flexibility. New test cassettes were added to validate the functionality of agents within flow contexts, ensuring robust testing for both synchronous and asynchronous operations.

* fixed cassette

* Enhance Flow Execution Logic

- Introduced conditional execution for start methods in the Flow class.
- Unconditional start methods are prioritized during kickoff, while conditional starts are executed only if no unconditional starts are present.
- Improved handling of cyclic flows by allowing re-execution of conditional start methods triggered by routers.
- Added checks to continue execution chains for completed conditional starts.

These changes improve the flexibility and control of flow execution, ensuring that the correct methods are triggered based on the defined conditions.

* Enhance Agent and Flow Execution Logic

- Updated the Agent class to automatically detect the event loop and return a coroutine when called within a Flow, simplifying async handling for users.
- Modified Flow class to execute listeners sequentially, preventing race conditions on shared state during listener execution.
- Improved handling of coroutine results from synchronous methods, ensuring proper execution flow and state management.

These changes enhance the overall execution logic and user experience when working with agents and flows in CrewAI.

* Enhance Flow Listener Logic and Agent Imports

- Updated the Flow class to track fired OR listeners, ensuring that multi-source OR listeners only trigger once during execution. This prevents redundant executions and improves flow efficiency.
- Cleared fired OR listeners during cyclic flow resets to allow re-execution in new cycles.
- Modified the Agent class imports to include Coroutine from collections.abc, enhancing type handling for asynchronous operations.

These changes improve the control and performance of flow execution in CrewAI, ensuring more predictable behavior in complex scenarios.

* adjusted test due to new cassette

* ensure we dont finalize batch on just a liteagent finishing

* feat: cancellable parallelized flow methods

* feat: allow methods to be cancelled & run parallelized

* feat: ensure state is thread safe through proxy

* fix: check for proxy state

* fix: mimic BaseModel method

* chore: update final attr checks; test

* better description

* fix test

* chore: update test assumptions

* extra

---------

Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>

lib/crewai/tests/test_flow.py

@@ -1202,7 +1202,8 @@ def test_complex_and_or_branching():
     )
     assert execution_order.index("branch_2b") > min_branch_1_index
 
-    # Final should be last and after both 2a and 2b
+
+    # Final should be after both 2a and 2b
     assert execution_order[-1] == "final"
     assert execution_order.index("final") > execution_order.index("branch_2a")
     assert execution_order.index("final") > execution_order.index("branch_2b")
@@ -1255,10 +1256,11 @@ def test_conditional_router_paths_exclusivity():
 
 
 def test_state_consistency_across_parallel_branches():
-    """Test that state remains consistent when branches execute sequentially.
+    """Test that state remains consistent when branches execute in parallel.
 
-    Note: Branches triggered by the same parent execute sequentially, not in parallel.
-    This ensures predictable state mutations and prevents race conditions.
+    Note: Branches triggered by the same parent execute in parallel for efficiency.
+    Thread-safe state access via StateProxy ensures no race conditions.
+    We check the execution order to ensure the branches execute in parallel.
     """
     execution_order = []
 
@@ -1295,12 +1297,14 @@ def test_state_consistency_across_parallel_branches():
     flow = StateConsistencyFlow()
     flow.kickoff()
 
-    # Branches execute sequentially, so branch_a runs first, then branch_b
-    assert flow.state["branch_a_value"] == 10  # Sees initial value
-    assert flow.state["branch_b_value"] == 11  # Sees value after branch_a increment
+    assert "branch_a" in execution_order
+    assert "branch_b" in execution_order
+    assert "verify_state" in execution_order
 
-    # Final counter should reflect both increments sequentially
-    assert flow.state["counter"] == 16  # 10 + 1 + 5
+    assert flow.state["branch_a_value"] is not None
+    assert flow.state["branch_b_value"] is not None
+
+    assert flow.state["counter"] == 16
 
 
 def test_deeply_nested_conditions():