Enhance knowledge management in CrewAI (#2637)

* Enhance knowledge management in CrewAI

- Added `KnowledgeConfig` class to configure knowledge retrieval parameters such as `limit` and `score_threshold`.
- Updated `Agent` and `Crew` classes to utilize the new knowledge configuration for querying knowledge sources.
- Enhanced documentation to clarify the addition of knowledge sources at both agent and crew levels.
- Introduced new tips in documentation to guide users on knowledge source management and configuration.

* Refactor knowledge configuration parameters in CrewAI

- Renamed `limit` to `results_limit` in `KnowledgeConfig`, `query_knowledge`, and `query` methods for consistency and clarity.
- Updated related documentation to reflect the new parameter name, ensuring users understand the configuration options for knowledge retrieval.

* Refactor agent tests to utilize mock knowledge storage

- Updated test cases in `agent_test.py` to use `KnowledgeStorage` for mocking knowledge sources, enhancing test reliability and clarity.
- Renamed `limit` to `results_limit` in `KnowledgeConfig` for consistency with recent changes.
- Ensured that knowledge queries are properly mocked to return expected results during tests.

* Add VCR support for agent tests with query limits and score thresholds

- Introduced `@pytest.mark.vcr` decorator in `agent_test.py` for tests involving knowledge sources, ensuring consistent recording of HTTP interactions.
- Added new YAML cassette files for `test_agent_with_knowledge_sources_with_query_limit_and_score_threshold` and `test_agent_with_knowledge_sources_with_query_limit_and_score_threshold_default`, capturing the expected API responses for these tests.
- Enhanced test reliability by utilizing VCR to manage external API calls during testing.

* Update documentation to format parameter names in code style

- Changed the formatting of `results_limit` and `score_threshold` in the documentation to use code style for better clarity and emphasis.
- Ensured consistency in documentation presentation to enhance user understanding of configuration options.

* Enhance KnowledgeConfig with field descriptions

- Updated `results_limit` and `score_threshold` in `KnowledgeConfig` to use Pydantic's `Field` for improved documentation and clarity.
- Added descriptions to both parameters to provide better context for their usage in knowledge retrieval configuration.

* docstrings added

src/crewai/agent.py

@@ -114,6 +114,14 @@ class Agent(BaseAgent):
         default=None,
         description="Embedder configuration for the agent.",
     )
+    agent_knowledge_context: Optional[str] = Field(
+        default=None,
+        description="Knowledge context for the agent.",
+    )
+    crew_knowledge_context: Optional[str] = Field(
+        default=None,
+        description="Knowledge context for the crew.",
+    )
 
     @model_validator(mode="after")
     def post_init_setup(self):
@@ -234,22 +242,30 @@ class Agent(BaseAgent):
             memory = contextual_memory.build_context_for_task(task, context)
             if memory.strip() != "":
                 task_prompt += self.i18n.slice("memory").format(memory=memory)
-
+        knowledge_config = (
+            self.knowledge_config.model_dump() if self.knowledge_config else {}
+        )
         if self.knowledge:
-            agent_knowledge_snippets = self.knowledge.query([task.prompt()])
+            agent_knowledge_snippets = self.knowledge.query(
+                [task.prompt()], **knowledge_config
+            )
             if agent_knowledge_snippets:
-                agent_knowledge_context = extract_knowledge_context(
+                self.agent_knowledge_context = extract_knowledge_context(
                     agent_knowledge_snippets
                 )
-                if agent_knowledge_context:
-                    task_prompt += agent_knowledge_context
+                if self.agent_knowledge_context:
+                    task_prompt += self.agent_knowledge_context
 
         if self.crew:
-            knowledge_snippets = self.crew.query_knowledge([task.prompt()])
+            knowledge_snippets = self.crew.query_knowledge(
+                [task.prompt()], **knowledge_config
+            )
             if knowledge_snippets:
-                crew_knowledge_context = extract_knowledge_context(knowledge_snippets)
-                if crew_knowledge_context:
-                    task_prompt += crew_knowledge_context
+                self.crew_knowledge_context = extract_knowledge_context(
+                    knowledge_snippets
+                )
+                if self.crew_knowledge_context:
+                    task_prompt += self.crew_knowledge_context
 
         tools = tools or self.tools or []
         self.create_agent_executor(tools=tools, task=task)