Deep Crew / Agent / Task attributes support on json (#6172)

* Enhance JSON crew project handling and validation

- Updated `create_json_crew.py` to specify input files with a brief path.
- Refactored `crew_loader.py` to improve agent and task loading logic, including the introduction of a `build_agent` function and better handling of task classes.
- Enhanced `json_loader.py` with additional validation for agent and task definitions, including support for Python references and conditional tasks.
- Added tests in `test_crew_loader.py` and `test_json_loader.py` to ensure proper loading of agents, tasks, and validation of project structures, including custom types and conditional tasks.
- Improved error handling and validation safety across the project loading process.

* Enhance JSON crew configuration options in create_json_crew.py

- Added optional fields for custom agent subclasses and advanced task options, including condition checks and output specifications.
- Improved documentation comments for better clarity on agent and task configurations.
- Updated JSON crew handling to support additional callbacks for pre- and post-execution processes.

* Enhance JSON crew template tests in test_create_crew.py

- Added assertions for new optional fields in crew and agent templates, including conditional tasks, custom converters, and input file specifications.
- Improved validation checks for manager agents and callback references to ensure proper configuration in JSON crew definitions.
- Expanded documentation references within the tests to provide clearer guidance on the expected structure and usage of crew templates.

* Fix JSON crew PR review issues

lib/cli/tests/test_create_crew.py

@@ -721,9 +721,30 @@ def test_json_create_provider_preselects_default_model(tmp_path, monkeypatch):
     assert '"guardrail_max_retries": 2' in crew_template
     assert "Docs: https://docs.crewai.com/concepts/tasks" in crew_template
     assert '"output_pydantic": null' in crew_template
+    assert '"type": "ConditionalTask"' in crew_template
+    assert '"condition": { "python": "my_project.conditions.should_run" }' in (
+        crew_template
+    )
+    assert '"output_json": { "python": "my_project.models.ReportOutput" }' in (
+        crew_template
+    )
+    assert (
+        '"converter_cls": { "python": "my_project.converters.CustomConverter" }'
+        in crew_template
+    )
     assert '"markdown": false' in crew_template
+    assert '"input_files": { "brief": "data/brief.txt" }' in crew_template
     assert "Docs: https://docs.crewai.com/concepts/crews" in crew_template
+    assert "manager_agent can reference an agents/<name>.jsonc file" in crew_template
     assert '"manager_agent": "researcher"' in crew_template
+    assert (
+        '"before_kickoff_callbacks": [{"python": '
+        '"my_project.callbacks.before_kickoff"}]'
+    ) in crew_template
+    assert (
+        '"after_kickoff_callbacks": [{"python": '
+        '"my_project.callbacks.after_kickoff"}]'
+    ) in crew_template
     assert '"output_log_file": "crew.log"' in crew_template
     assert "Crew-level LLM fields also accept object form" in crew_template
     assert '"chat_llm": {"model": "llama3", "provider": "ollama"' in (
@@ -740,7 +761,13 @@ def test_json_create_provider_preselects_default_model(tmp_path, monkeypatch):
         agent_template
     )
     assert '"role": "Senior {industry} Researcher"' in agent_template
+    assert '"type": {"python": "my_project.agents.CustomAgent"}' in agent_template
     assert "Optional agent-level guardrail" in agent_template
+    assert "Python refs must point to module-level functions/classes" in agent_template
+    assert (
+        '"step_callback": {"python": "my_project.callbacks.on_agent_step"}'
+        in agent_template
+    )
     assert '"guardrail_max_retries": 2' in agent_template
     assert "Docs: https://docs.crewai.com/concepts/agents" in agent_template
     assert '"reasoning": true' in agent_template