test: add test_kickoff_for_each_output_file_interpolation

Co-Authored-By: Joe Moura <joao@crewai.com>

docs/concepts/processes.mdx

@@ -10,26 +10,11 @@ icon: bars-staggered
   These processes ensure tasks are distributed and executed efficiently, in alignment with a predefined strategy.
 </Tip>
 
-## Process Types
+## Process Implementations
 
-CrewAI supports two process types that determine how tasks are assigned to agents:
-
-- **Static/Assigned Process** (formerly "Sequential"): In this process, each task must be pre-assigned to a specific agent. While the name "Sequential" was previously used, it's important to note that tasks are always executed in the order they are defined, regardless of the process type chosen. The key characteristic of this process is that it requires explicit agent assignments for each task.
-
-- **Dynamic/Unassigned Process** (formerly "Hierarchical"): In this process, you do not have to assign agents to tasks explicitly. Instead, the crew will assess available agents and automatically select the most suitable one for each task based on their roles and expertise. This requires specifying either a manager language model (`manager_llm`) or a custom manager agent (`manager_agent`) to handle the agent selection and task delegation.
-
-- **Consensual Process** (Planned): Aiming for collaborative decision-making among agents on task execution, this process type introduces a democratic approach to task management within CrewAI. It is planned for future development and is not currently implemented in the codebase.
-
-### Process Type Comparison
-
-| Aspect | Static/Assigned Process | Dynamic/Unassigned Process |
-|--------|------------------------|---------------------------|
-| Agent Assignment | Pre-assigned by developer | Automatic based on agent capabilities |
-| Task Order | Sequential (defined order) | Sequential (defined order) |
-| Manager Required | No | Yes (manager_llm or manager_agent) |
-| Use Case | Fixed workflows with known agent assignments | Dynamic workflows needing flexible assignment |
-| Configuration | Simpler setup, explicit control | Requires manager configuration |
-| Task-Agent Mapping | One-to-one, defined at creation | Determined during execution |
+- **Sequential**: Executes tasks sequentially, ensuring tasks are completed in an orderly progression.
+- **Hierarchical**: Organizes tasks in a managerial hierarchy, where tasks are delegated and executed based on a structured chain of command. A manager language model (`manager_llm`) or a custom manager agent (`manager_agent`) must be specified in the crew to enable the hierarchical process, facilitating the creation and management of tasks by the manager.
+- **Consensual Process (Planned)**: Aiming for collaborative decision-making among agents on task execution, this process type introduces a democratic approach to task management within CrewAI. It is planned for future development and is not currently implemented in the codebase.
 
 ## The Role of Processes in Teamwork
 Processes enable individual agents to operate as a cohesive unit, streamlining their efforts to achieve common objectives with efficiency and coherence.
@@ -38,149 +23,45 @@ Processes enable individual agents to operate as a cohesive unit, streamlining t
 To assign a process to a crew, specify the process type upon crew creation to set the execution strategy. For a hierarchical process, ensure to define `manager_llm` or `manager_agent` for the manager agent.
 
 ```python
-from crewai import Crew, Agent, Task
+from crewai import Crew
 from crewai.process import Process
 from langchain_openai import ChatOpenAI
 
-# Define agents with specific roles and expertise
-researcher = Agent(
-    role="Researcher",
-    goal="Conduct thorough market analysis",
-    backstory="Expert in data analysis and market research"
+# Example: Creating a crew with a sequential process
+crew = Crew(
+    agents=my_agents,
+    tasks=my_tasks,
+    process=Process.sequential
 )
 
-writer = Agent(
-    role="Writer",
-    goal="Create comprehensive reports",
-    backstory="Technical writer with expertise in market analysis"
-)
-
-# Example 1: Static/Assigned Process
-# Tasks must be explicitly assigned to agents
-research_task = Task(
-    description="Research emerging market trends",
-    agent=researcher  # Explicit agent assignment required
-)
-
-writing_task = Task(
-    description="Create market analysis report",
-    agent=writer,  # Explicit agent assignment required
-    context="Use research findings to create a detailed report"
-)
-
-# Create crew with Static/Assigned process
-static_crew = Crew(
-    agents=[researcher, writer],
-    tasks=[research_task, writing_task],
-    process=Process.sequential  # Note: Using old enum value for backward compatibility
-)
-
-# Example 2: Dynamic/Unassigned Process
-# Tasks without pre-assigned agents
-dynamic_research_task = Task(
-    description="Research emerging market trends",
-    required_skills=["market analysis", "data interpretation"]  # Help manager select agent
-)
-
-dynamic_writing_task = Task(
-    description="Create market analysis report",
-    context="Use research findings to create a detailed report",
-    required_skills=["technical writing", "data visualization"]
-)
-
-# Create crew with Dynamic/Unassigned process
-dynamic_crew = Crew(
-    agents=[researcher, writer],
-    tasks=[dynamic_research_task, dynamic_writing_task],
-    process=Process.hierarchical,  # Note: Using old enum value for backward compatibility
-    manager_llm=ChatOpenAI(model="gpt-4")  # Manager will assign tasks to suitable agents
-    # Alternative: Use custom manager agent
-    # manager_agent=custom_manager_agent
+# Example: Creating a crew with a hierarchical process
+# Ensure to provide a manager_llm or manager_agent
+crew = Crew(
+    agents=my_agents,
+    tasks=my_tasks,
+    process=Process.hierarchical,
+    manager_llm=ChatOpenAI(model="gpt-4")
+    # or
+    # manager_agent=my_manager_agent
 )
 ```
 **Note:** Ensure `my_agents` and `my_tasks` are defined prior to creating a `Crew` object, and for the hierarchical process, either `manager_llm` or `manager_agent` is also required.
 
-## Static/Assigned Process
+## Sequential Process
 
-This process type requires explicit agent assignments for each task. Tasks are executed in their defined order, with the output of one task serving as context for the next. This approach provides direct control over which agent handles each specific task.
+This method mirrors dynamic team workflows, progressing through tasks in a thoughtful and systematic manner. Task execution follows the predefined order in the task list, with the output of one task serving as context for the next.
 
 To customize task context, utilize the `context` parameter in the `Task` class to specify outputs that should be used as context for subsequent tasks.
 
-## Dynamic/Unassigned Process
+## Hierarchical Process
 
-This process type enables automatic agent selection through a manager component. You must specify either a custom manager agent or a manager language model (`manager_llm`). The manager oversees task execution by:
-- Analyzing task requirements
-- Selecting the most suitable agent based on roles and expertise
-- Delegating tasks automatically
-- Reviewing outputs and assessing task completion
+Emulates a corporate hierarchy, CrewAI allows specifying a custom manager agent or automatically creates one, requiring the specification of a manager language model (`manager_llm`). This agent oversees task execution, including planning, delegation, and validation. Tasks are not pre-assigned; the manager allocates tasks to agents based on their capabilities, reviews outputs, and assesses task completion.
 
-## Choosing the Right Process
+## Process Class: Detailed Overview
 
-When deciding between Static/Assigned and Dynamic/Unassigned processes, consider these technical factors:
-
-### Static/Assigned Process
-Consider this process when:
-- Your workflow has predefined task-agent mappings
-- You need deterministic agent assignments for auditing or compliance
-- You want to minimize runtime overhead (no manager required)
-- You have specific agents optimized for particular tasks
-- You need fine-grained control over task execution
-
-### Dynamic/Unassigned Process
-Consider this process when:
-- Your agent pool has overlapping capabilities
-- Task requirements are determined at runtime
-- You need failover capabilities between agents
-- You have a manager (LLM or agent) to handle assignment logic
-- You want to scale agent pools without modifying task definitions
-
-### Technical Considerations
-- **Performance**: Static/Assigned processes have lower overhead as they skip the agent selection phase
-- **Scalability**: Dynamic/Unassigned processes better handle changes in agent availability
-- **Maintenance**: Static assignments require updating task definitions when agent roles change
-- **Error Handling**: Dynamic processes can potentially reassign tasks on agent failures
-
-## Technical Implementation Details
-
-### Process Class
-The `Process` class is implemented as an enumeration (`Enum`), ensuring type safety and restricting process values to the defined types:
-```python
-class Process(Enum):
-    sequential = "sequential"     # Static/Assigned process
-    hierarchical = "hierarchical" # Dynamic/Unassigned process
-```
-
-### Default Configuration
-- The default process type is `Process.sequential` (Static/Assigned)
-- When using `Process.hierarchical`, a manager (either `manager_llm` or `manager_agent`) must be provided
-
-### Version Compatibility
-- Since v1.0.0: Original process types (`sequential`, `hierarchical`)
-- Current version maintains the same enum values for backward compatibility
-- Future versions will continue supporting these values while using new terminology in documentation
-
-### Error Handling
-Common error scenarios and their solutions:
-```python
-# Error: Missing manager in Dynamic/Unassigned process
-crew = Crew(
-    agents=[agent1, agent2],
-    tasks=[task1, task2],
-    process=Process.hierarchical
-    # Error: ValueError: Manager (manager_llm or manager_agent) is required for hierarchical process
-)
-
-# Error: Missing agent assignment in Static/Assigned process
-task = Task(description="Task without agent")  # Missing agent assignment
-crew = Crew(
-    agents=[agent1, agent2],
-    tasks=[task],
-    process=Process.sequential
-    # Error: ValueError: Agent assignment required for all tasks in sequential process
-)
-```
+The `Process` class is implemented as an enumeration (`Enum`), ensuring type safety and restricting process values to the defined types (`sequential`, `hierarchical`). The consensual process is planned for future inclusion, emphasizing our commitment to continuous development and innovation.
 
 ## Conclusion
 
 The structured collaboration facilitated by processes within CrewAI is crucial for enabling systematic teamwork among agents. 
-This documentation has been updated to reflect the latest features, enhancements, and the planned integration of the Consensual Process, ensuring users have access to the most current and comprehensive information.
+This documentation has been updated to reflect the latest features, enhancements, and the planned integration of the Consensual Process, ensuring users have access to the most current and comprehensive information.

docs/concepts/tasks.mdx

@@ -12,8 +12,20 @@ Tasks provide all necessary details for execution, such as a description, the ag
 
 Tasks within CrewAI can be collaborative, requiring multiple agents to work together. This is managed through the task properties and orchestrated by the Crew's process, enhancing teamwork and efficiency.
 
-### Task Execution
-Tasks are always executed in the order they are defined. For information about how tasks are assigned to agents and the different process types available (Static/Assigned vs Dynamic/Unassigned), please refer to the [Processes](/concepts/processes) section.
+### Task Execution Flow
+
+Tasks can be executed in two ways:
+- **Sequential**: Tasks are executed in the order they are defined
+- **Hierarchical**: Tasks are assigned to agents based on their roles and expertise
+
+The execution flow is defined when creating the crew:
+```python Code
+crew = Crew(
+    agents=[agent1, agent2],
+    tasks=[task1, task2],
+    process=Process.sequential  # or Process.hierarchical
+)
+```
 
 ## Task Attributes
 

tests/cassettes/test_kickoff_for_each_output_file_interpolation.yaml

@@ -0,0 +1,236 @@
+interactions:
+- request:
+    body: '{"messages": [{"role": "system", "content": "You are TestAgent. Testing
+      interpolation logic\nYour personal goal is: Validate output_file interpolation
+      with kickoff_for_each\nTo give my best complete final answer to the task use
+      the exact following format:\n\nThought: I now can give a great answer\nFinal
+      Answer: Your final answer must be the great and the most complete as possible,
+      it must be outcome described.\n\nI MUST use these formats, my job depends on
+      it!"}, {"role": "user", "content": "\nCurrent Task: Create a brief summary for
+      the ID: alpha\n\nThis is the expect criteria for your final answer: Should write
+      a file with the ID included in the path\nyou MUST return the actual complete
+      content as the final answer, not a summary.\n\nBegin! This is VERY important
+      to you, use the tools available and give your best Final Answer, your job depends
+      on it!\n\nThought:"}], "model": "gpt-4o-mini", "stop": ["\nObservation:"], "stream":
+      false}'
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate
+      connection:
+      - keep-alive
+      content-length:
+      - '948'
+      content-type:
+      - application/json
+      host:
+      - api.openai.com
+      user-agent:
+      - OpenAI/Python 1.52.1
+      x-stainless-arch:
+      - x64
+      x-stainless-async:
+      - 'false'
+      x-stainless-lang:
+      - python
+      x-stainless-os:
+      - Linux
+      x-stainless-package-version:
+      - 1.52.1
+      x-stainless-raw-response:
+      - 'true'
+      x-stainless-retry-count:
+      - '0'
+      x-stainless-runtime:
+      - CPython
+      x-stainless-runtime-version:
+      - 3.12.7
+    method: POST
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    content: "{\n  \"id\": \"chatcmpl-AjflQm9Vq8QvtY5ORMahjQVL061VL\",\n  \"object\":
+      \"chat.completion\",\n  \"created\": 1735448664,\n  \"model\": \"gpt-4o-mini-2024-07-18\",\n
+      \ \"choices\": [\n    {\n      \"index\": 0,\n      \"message\": {\n        \"role\":
+      \"assistant\",\n        \"content\": \"I now can give a great answer  \\nFinal
+      Answer: The content for the file with ID 'alpha' is as follows:\\n\\n```\\nThis
+      is the content specific to ID: alpha. This file serves as a reference for the
+      alpha ID and may include various information relevant to its usage. The ID aids
+      in categorization and organization of data.  \\n```\\n\\nThe file should be
+      created with the path including the ID, for example: `./output/alpha_file.txt`.\",\n
+      \       \"refusal\": null\n      },\n      \"logprobs\": null,\n      \"finish_reason\":
+      \"stop\"\n    }\n  ],\n  \"usage\": {\n    \"prompt_tokens\": 175,\n    \"completion_tokens\":
+      94,\n    \"total_tokens\": 269,\n    \"prompt_tokens_details\": {\n      \"cached_tokens\":
+      0,\n      \"audio_tokens\": 0\n    },\n    \"completion_tokens_details\": {\n
+      \     \"reasoning_tokens\": 0,\n      \"audio_tokens\": 0,\n      \"accepted_prediction_tokens\":
+      0,\n      \"rejected_prediction_tokens\": 0\n    }\n  },\n  \"system_fingerprint\":
+      \"fp_0aa8d3e20b\"\n}\n"
+    headers:
+      CF-Cache-Status:
+      - DYNAMIC
+      CF-RAY:
+      - 8f973fc8a8e176cd-SEA
+      Connection:
+      - keep-alive
+      Content-Encoding:
+      - gzip
+      Content-Type:
+      - application/json
+      Date:
+      - Sun, 29 Dec 2024 05:04:25 GMT
+      Server:
+      - cloudflare
+      Set-Cookie:
+      - __cf_bm=Td9pau1aHgSMMxQpKkO8mpGk5DWBDjUinUYlfJPsgcw-1735448665-1.0.1.1-wH0kiPhZtNwT_fIqYCb3obZgU2VIxoPL.QrruewXdLVLT9FeI4b6BU.sHqsBjMODx6Tvtuzz9GQRvoY1nWs6Fw;
+        path=/; expires=Sun, 29-Dec-24 05:34:25 GMT; domain=.api.openai.com; HttpOnly;
+        Secure; SameSite=None
+      - _cfuvid=P0VQx4Uj_TpVJBRs2gSXmq3srUR2DW7oj5Vmq54U47M-1735448665828-0.0.1.1-604800000;
+        path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None
+      Transfer-Encoding:
+      - chunked
+      X-Content-Type-Options:
+      - nosniff
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      openai-organization:
+      - crewai-iuxna1
+      openai-processing-ms:
+      - '1288'
+      openai-version:
+      - '2020-10-01'
+      strict-transport-security:
+      - max-age=31536000; includeSubDomains; preload
+      x-ratelimit-limit-requests:
+      - '30000'
+      x-ratelimit-limit-tokens:
+      - '150000000'
+      x-ratelimit-remaining-requests:
+      - '29999'
+      x-ratelimit-remaining-tokens:
+      - '149999786'
+      x-ratelimit-reset-requests:
+      - 2ms
+      x-ratelimit-reset-tokens:
+      - 0s
+      x-request-id:
+      - req_9dd7c0e64935fade58b5b8359fcc9c02
+    http_version: HTTP/1.1
+    status_code: 200
+- request:
+    body: '{"messages": [{"role": "system", "content": "You are TestAgent. Testing
+      interpolation logic\nYour personal goal is: Validate output_file interpolation
+      with kickoff_for_each\nTo give my best complete final answer to the task use
+      the exact following format:\n\nThought: I now can give a great answer\nFinal
+      Answer: Your final answer must be the great and the most complete as possible,
+      it must be outcome described.\n\nI MUST use these formats, my job depends on
+      it!"}, {"role": "user", "content": "\nCurrent Task: Create a brief summary for
+      the ID: beta\n\nThis is the expect criteria for your final answer: Should write
+      a file with the ID included in the path\nyou MUST return the actual complete
+      content as the final answer, not a summary.\n\nBegin! This is VERY important
+      to you, use the tools available and give your best Final Answer, your job depends
+      on it!\n\nThought:"}], "model": "gpt-4o-mini", "stop": ["\nObservation:"], "stream":
+      false}'
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate
+      connection:
+      - keep-alive
+      content-length:
+      - '947'
+      content-type:
+      - application/json
+      cookie:
+      - __cf_bm=Td9pau1aHgSMMxQpKkO8mpGk5DWBDjUinUYlfJPsgcw-1735448665-1.0.1.1-wH0kiPhZtNwT_fIqYCb3obZgU2VIxoPL.QrruewXdLVLT9FeI4b6BU.sHqsBjMODx6Tvtuzz9GQRvoY1nWs6Fw;
+        _cfuvid=P0VQx4Uj_TpVJBRs2gSXmq3srUR2DW7oj5Vmq54U47M-1735448665828-0.0.1.1-604800000
+      host:
+      - api.openai.com
+      user-agent:
+      - OpenAI/Python 1.52.1
+      x-stainless-arch:
+      - x64
+      x-stainless-async:
+      - 'false'
+      x-stainless-lang:
+      - python
+      x-stainless-os:
+      - Linux
+      x-stainless-package-version:
+      - 1.52.1
+      x-stainless-raw-response:
+      - 'true'
+      x-stainless-retry-count:
+      - '0'
+      x-stainless-runtime:
+      - CPython
+      x-stainless-runtime-version:
+      - 3.12.7
+    method: POST
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    content: "{\n  \"id\": \"chatcmpl-AjflSvzNtfBXfJSerUuWwQM47tE4N\",\n  \"object\":
+      \"chat.completion\",\n  \"created\": 1735448666,\n  \"model\": \"gpt-4o-mini-2024-07-18\",\n
+      \ \"choices\": [\n    {\n      \"index\": 0,\n      \"message\": {\n        \"role\":
+      \"assistant\",\n        \"content\": \"I now can give a great answer  \\nFinal
+      Answer: Here is the content to be written to the file for ID: beta\\n\\n```\\nThis
+      is the content specifically related to the ID: beta.\\n```\\n\\nThe file will
+      be created at the following path: `/path/to/directory/beta.txt`  \\nThank you
+      for the opportunity to complete this task!\",\n        \"refusal\": null\n      },\n
+      \     \"logprobs\": null,\n      \"finish_reason\": \"stop\"\n    }\n  ],\n
+      \ \"usage\": {\n    \"prompt_tokens\": 175,\n    \"completion_tokens\": 72,\n
+      \   \"total_tokens\": 247,\n    \"prompt_tokens_details\": {\n      \"cached_tokens\":
+      0,\n      \"audio_tokens\": 0\n    },\n    \"completion_tokens_details\": {\n
+      \     \"reasoning_tokens\": 0,\n      \"audio_tokens\": 0,\n      \"accepted_prediction_tokens\":
+      0,\n      \"rejected_prediction_tokens\": 0\n    }\n  },\n  \"system_fingerprint\":
+      \"fp_0aa8d3e20b\"\n}\n"
+    headers:
+      CF-Cache-Status:
+      - DYNAMIC
+      CF-RAY:
+      - 8f973fd1af7576cd-SEA
+      Connection:
+      - keep-alive
+      Content-Encoding:
+      - gzip
+      Content-Type:
+      - application/json
+      Date:
+      - Sun, 29 Dec 2024 05:04:27 GMT
+      Server:
+      - cloudflare
+      Transfer-Encoding:
+      - chunked
+      X-Content-Type-Options:
+      - nosniff
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      openai-organization:
+      - crewai-iuxna1
+      openai-processing-ms:
+      - '1475'
+      openai-version:
+      - '2020-10-01'
+      strict-transport-security:
+      - max-age=31536000; includeSubDomains; preload
+      x-ratelimit-limit-requests:
+      - '30000'
+      x-ratelimit-limit-tokens:
+      - '150000000'
+      x-ratelimit-remaining-requests:
+      - '29999'
+      x-ratelimit-remaining-tokens:
+      - '149999785'
+      x-ratelimit-reset-requests:
+      - 2ms
+      x-ratelimit-reset-tokens:
+      - 0s
+      x-request-id:
+      - req_dc9a842ed35e213e52c2d6fe9f11c36a
+    http_version: HTTP/1.1
+    status_code: 200
+version: 1

tests/crew_test.py

@@ -1183,6 +1183,53 @@ def test_kickoff_for_each_error_handling():
             crew.kickoff_for_each(inputs=inputs)
 
 
+@pytest.mark.vcr(filter_headers=["authorization"])
+def test_kickoff_for_each_output_file_interpolation(tmp_path):
+    """Test that output_file paths are correctly interpolated when using kickoff_for_each."""
+    # Create test directories
+    output_dir = tmp_path / "output_files"
+    output_dir.mkdir()
+
+    agent = Agent(
+        role="TestAgent",
+        goal="Validate output_file interpolation with kickoff_for_each",
+        backstory="Testing interpolation logic",
+        allow_code_execution=False,
+    )
+
+    task = Task(
+        description="Create a brief summary for the ID: {id_var}",
+        expected_output="Should write a file with the ID included in the path",
+        agent=agent,
+        output_file=str(output_dir / "summary_{id_var}.md")
+    )
+
+    crew = Crew(agents=[agent], tasks=[task], verbose=True)
+
+    inputs = [
+        {"id_var": "alpha"},
+        {"id_var": "beta"},
+    ]
+
+    # Use kickoff_for_each
+    results = crew.kickoff_for_each(inputs=inputs)
+    
+    # Verify each iteration's output_file was properly interpolated
+    for input_data, result in zip(inputs, results):
+        expected_path = output_dir / f"summary_{input_data['id_var']}.md"
+        assert expected_path.exists(), f"Expected output file {expected_path} was not created"
+        
+        # Verify the output file was created with the correct name
+        assert expected_path.exists(), (
+            f"Expected output file {expected_path} was not created"
+        )
+        
+        # Verify file contains some content
+        assert expected_path.stat().st_size > 0, (
+            f"Output file {expected_path} is empty"
+        )
+
+
 @pytest.mark.vcr(filter_headers=["authorization"])
 @pytest.mark.asyncio
 async def test_kickoff_async_basic_functionality_and_output():