Feature/kickoff consistent output (#847)

* Cleaned up task execution to now have separate paths for async and sync execution. Updating all kickoff functions to return CrewOutput. WIP. Waiting for Joao feedback on async task execution with task_output

* Consistently storing async and sync output for context

* outline tests I need to create going forward

* Major rehaul of TaskOutput and CrewOutput. Updated all tests to work with new change. Need to add in a few final tricky async tests and add a few more to verify output types on TaskOutput and CrewOutput.

* Encountering issues with callback. Need to test on main. WIP

* working on tests. WIP

* WIP. Figuring out disconnect issue.

* Cleaned up logs now that I've isolated the issue to the LLM

* more wip.

* WIP. It looks like usage metrics has always been broken for async

* Update parent crew who is managing for_each loop

* Merge in main to bugfix/kickoff-for-each-usage-metrics

* Clean up code for review

* Add new tests

* Final cleanup. Ready for review.

* Moving copy functionality from Agent to BaseAgent

* Fix renaming issue

* Fix linting errors

* use BaseAgent instead of Agent where applicable

* Fixing missing function. Working on tests.

* WIP. Needing team to review change

* Fixing issues brought about by merge

* WIP

* Implement major fixes from yesterdays group conversation. Now working on tests.

* The majority of tasks are working now. Need to fix converter class

* Fix final failing test

* Fix linting and type-checker issues

* Add more tests to fully test CrewOutput and TaskOutput changes

* Add in validation for async cannot depend on other async tasks.

* Update validators and tests

src/crewai/tasks/__init__.py

@@ -0,0 +1,4 @@
+from crewai.tasks.output_format import OutputFormat
+from crewai.tasks.task_output import TaskOutput
+
+__all__ = ["OutputFormat", "TaskOutput"]

src/crewai/tasks/output_format.py

@@ -0,0 +1,9 @@
+from enum import Enum
+
+
+class OutputFormat(str, Enum):
+    """Enum that represents the output format of a task."""
+
+    JSON = "json"
+    PYDANTIC = "pydantic"
+    RAW = "raw"

src/crewai/tasks/task_output.py

@@ -1,24 +1,78 @@
-from typing import Optional, Union
+import json
+from typing import Any, Dict, Optional
 
 from pydantic import BaseModel, Field, model_validator
 
+from crewai.tasks.output_format import OutputFormat
+
 
 class TaskOutput(BaseModel):
     """Class that represents the result of a task."""
 
     description: str = Field(description="Description of the task")
     summary: Optional[str] = Field(description="Summary of the task", default=None)
-    exported_output: Union[str, BaseModel] = Field(
-        description="Output of the task", default=None
+    raw: str = Field(
+        description="Raw output of the task", default=""
+    )  # TODO: @joao: breaking change, by renaming raw_output to raw, but now consistent with CrewOutput
+    pydantic: Optional[BaseModel] = Field(
+        description="Pydantic output of task", default=None
+    )
+    json_dict: Optional[Dict[str, Any]] = Field(
+        description="JSON dictionary of task", default=None
     )
     agent: str = Field(description="Agent that executed the task")
-    raw_output: str = Field(description="Result of the task")
+    output_format: OutputFormat = Field(
+        description="Output format of the task", default=OutputFormat.RAW
+    )
 
     @model_validator(mode="after")
     def set_summary(self):
+        """Set the summary field based on the description."""
         excerpt = " ".join(self.description.split(" ")[:10])
         self.summary = f"{excerpt}..."
         return self
 
-    def result(self):
-        return self.exported_output
+    # TODO: Joao - Adding this safety check breakes when people want to see
+    #                   The full output of a TaskOutput or CrewOutput.
+    # @property
+    # def pydantic(self) -> Optional[BaseModel]:
+    #     # Check if the final task output included a pydantic model
+    #     if self.output_format != OutputFormat.PYDANTIC:
+    #         raise ValueError(
+    #             """
+    #             Invalid output format requested.
+    #             If you would like to access the pydantic model,
+    #             please make sure to set the output_pydantic property for the task.
+    #             """
+    #         )
+
+    #     return self._pydantic
+
+    @property
+    def json(self) -> Optional[str]:
+        if self.output_format != OutputFormat.JSON:
+            raise ValueError(
+                """
+                Invalid output format requested.
+                If you would like to access the JSON output,
+                please make sure to set the output_json property for the task
+                """
+            )
+
+        return json.dumps(self.json_dict)
+
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert json_output and pydantic_output to a dictionary."""
+        output_dict = {}
+        if self.json_dict:
+            output_dict.update(self.json_dict)
+        if self.pydantic:
+            output_dict.update(self.pydantic.model_dump())
+        return output_dict
+
+    def __str__(self) -> str:
+        if self.pydantic:
+            return str(self.pydantic)
+        if self.json_dict:
+            return str(self.json_dict)
+        return self.raw