Fix Knowledge docs Spaceflight News API dead link

This commit is contained in:
Rok Benko
2024-12-09 16:41:29 +01:00
committed by Tony Kipkemboi
parent 46be5e8097
commit 5a7a323f3a

View File

@@ -8,15 +8,13 @@ icon: book
## What is Knowledge? ## What is Knowledge?
Knowledge in CrewAI is a powerful system that allows AI agents to access and utilize external information sources during their tasks. Knowledge in CrewAI is a powerful system that allows AI agents to access and utilize external information sources during their tasks.
Think of it as giving your agents a reference library they can consult while working. Think of it as giving your agents a reference library they can consult while working.
<Info> <Info>
Key benefits of using Knowledge: Key benefits of using Knowledge: - Enhance agents with domain-specific
- Enhance agents with domain-specific information information - Support decisions with real-world data - Maintain context across
- Support decisions with real-world data conversations - Ground responses in factual information
- Maintain context across conversations
- Ground responses in factual information
</Info> </Info>
## Supported Knowledge Sources ## Supported Knowledge Sources
@@ -25,14 +23,10 @@ CrewAI supports various types of knowledge sources out of the box:
<CardGroup cols={2}> <CardGroup cols={2}>
<Card title="Text Sources" icon="text"> <Card title="Text Sources" icon="text">
- Raw strings - Raw strings - Text files (.txt) - PDF documents
- Text files (.txt)
- PDF documents
</Card> </Card>
<Card title="Structured Data" icon="table"> <Card title="Structured Data" icon="table">
- CSV files - CSV files - Excel spreadsheets - JSON documents
- Excel spreadsheets
- JSON documents
</Card> </Card>
</CardGroup> </CardGroup>
@@ -47,7 +41,7 @@ from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSourc
# Create a knowledge source # Create a knowledge source
content = "Users name is John. He is 30 years old and lives in San Francisco." content = "Users name is John. He is 30 years old and lives in San Francisco."
string_source = StringKnowledgeSource( string_source = StringKnowledgeSource(
content=content, content=content,
) )
# Create an LLM with a temperature of 0 to ensure deterministic outputs # Create an LLM with a temperature of 0 to ensure deterministic outputs
@@ -122,7 +116,6 @@ crewai reset-memories --knowledge
This is useful when you've updated your knowledge sources and want to ensure that the agents are using the most recent information. This is useful when you've updated your knowledge sources and want to ensure that the agents are using the most recent information.
## Custom Knowledge Sources ## Custom Knowledge Sources
CrewAI allows you to create custom knowledge sources for any type of data by extending the `BaseKnowledgeSource` class. Let's create a practical example that fetches and processes space news articles. CrewAI allows you to create custom knowledge sources for any type of data by extending the `BaseKnowledgeSource` class. Let's create a practical example that fetches and processes space news articles.
@@ -141,10 +134,10 @@ from pydantic import BaseModel, Field
class SpaceNewsKnowledgeSource(BaseKnowledgeSource): class SpaceNewsKnowledgeSource(BaseKnowledgeSource):
"""Knowledge source that fetches data from Space News API.""" """Knowledge source that fetches data from Space News API."""
api_endpoint: str = Field(description="API endpoint URL") api_endpoint: str = Field(description="API endpoint URL")
limit: int = Field(default=10, description="Number of articles to fetch") limit: int = Field(default=10, description="Number of articles to fetch")
def load_content(self) -> Dict[Any, str]: def load_content(self) -> Dict[Any, str]:
"""Fetch and format space news articles.""" """Fetch and format space news articles."""
try: try:
@@ -152,15 +145,15 @@ class SpaceNewsKnowledgeSource(BaseKnowledgeSource):
f"{self.api_endpoint}?limit={self.limit}" f"{self.api_endpoint}?limit={self.limit}"
) )
response.raise_for_status() response.raise_for_status()
data = response.json() data = response.json()
articles = data.get('results', []) articles = data.get('results', [])
formatted_data = self._format_articles(articles) formatted_data = self._format_articles(articles)
return {self.api_endpoint: formatted_data} return {self.api_endpoint: formatted_data}
except Exception as e: except Exception as e:
raise ValueError(f"Failed to fetch space news: {str(e)}") raise ValueError(f"Failed to fetch space news: {str(e)}")
def _format_articles(self, articles: list) -> str: def _format_articles(self, articles: list) -> str:
"""Format articles into readable text.""" """Format articles into readable text."""
formatted = "Space News Articles:\n\n" formatted = "Space News Articles:\n\n"
@@ -180,7 +173,7 @@ class SpaceNewsKnowledgeSource(BaseKnowledgeSource):
for _, text in content.items(): for _, text in content.items():
chunks = self._chunk_text(text) chunks = self._chunk_text(text)
self.chunks.extend(chunks) self.chunks.extend(chunks)
self._save_documents() self._save_documents()
# Create knowledge source # Create knowledge source
@@ -193,7 +186,7 @@ recent_news = SpaceNewsKnowledgeSource(
space_analyst = Agent( space_analyst = Agent(
role="Space News Analyst", role="Space News Analyst",
goal="Answer questions about space news accurately and comprehensively", goal="Answer questions about space news accurately and comprehensively",
backstory="""You are a space industry analyst with expertise in space exploration, backstory="""You are a space industry analyst with expertise in space exploration,
satellite technology, and space industry trends. You excel at answering questions satellite technology, and space industry trends. You excel at answering questions
about space news and providing detailed, accurate information.""", about space news and providing detailed, accurate information.""",
knowledge_sources=[recent_news], knowledge_sources=[recent_news],
@@ -220,13 +213,14 @@ result = crew.kickoff(
inputs={"user_question": "What are the latest developments in space exploration?"} inputs={"user_question": "What are the latest developments in space exploration?"}
) )
``` ```
```output Output ```output Output
# Agent: Space News Analyst # Agent: Space News Analyst
## Task: Answer this question about space news: What are the latest developments in space exploration? ## Task: Answer this question about space news: What are the latest developments in space exploration?
# Agent: Space News Analyst # Agent: Space News Analyst
## Final Answer: ## Final Answer:
The latest developments in space exploration, based on recent space news articles, include the following: The latest developments in space exploration, based on recent space news articles, include the following:
1. SpaceX has received the final regulatory approvals to proceed with the second integrated Starship/Super Heavy launch, scheduled for as soon as the morning of Nov. 17, 2023. This is a significant step in SpaceX's ambitious plans for space exploration and colonization. [Source: SpaceNews](https://spacenews.com/starship-cleared-for-nov-17-launch/) 1. SpaceX has received the final regulatory approvals to proceed with the second integrated Starship/Super Heavy launch, scheduled for as soon as the morning of Nov. 17, 2023. This is a significant step in SpaceX's ambitious plans for space exploration and colonization. [Source: SpaceNews](https://spacenews.com/starship-cleared-for-nov-17-launch/)
@@ -242,11 +236,13 @@ The latest developments in space exploration, based on recent space news article
6. The National Natural Science Foundation of China has outlined a five-year project for researchers to study the assembly of ultra-large spacecraft. This could lead to significant advancements in spacecraft technology and space exploration capabilities. [Source: SpaceNews](https://spacenews.com/china-researching-challenges-of-kilometer-scale-ultra-large-spacecraft/) 6. The National Natural Science Foundation of China has outlined a five-year project for researchers to study the assembly of ultra-large spacecraft. This could lead to significant advancements in spacecraft technology and space exploration capabilities. [Source: SpaceNews](https://spacenews.com/china-researching-challenges-of-kilometer-scale-ultra-large-spacecraft/)
7. The Center for AEroSpace Autonomy Research (CAESAR) at Stanford University is focusing on spacecraft autonomy. The center held a kickoff event on May 22, 2024, to highlight the industry, academia, and government collaboration it seeks to foster. This could lead to significant advancements in autonomous spacecraft technology. [Source: SpaceNews](https://spacenews.com/stanford-center-focuses-on-spacecraft-autonomy/) 7. The Center for AEroSpace Autonomy Research (CAESAR) at Stanford University is focusing on spacecraft autonomy. The center held a kickoff event on May 22, 2024, to highlight the industry, academia, and government collaboration it seeks to foster. This could lead to significant advancements in autonomous spacecraft technology. [Source: SpaceNews](https://spacenews.com/stanford-center-focuses-on-spacecraft-autonomy/)
``` ```
</CodeGroup> </CodeGroup>
#### Key Components Explained #### Key Components Explained
1. **Custom Knowledge Source (`SpaceNewsKnowledgeSource`)**: 1. **Custom Knowledge Source (`SpaceNewsKnowledgeSource`)**:
- Extends `BaseKnowledgeSource` for integration with CrewAI - Extends `BaseKnowledgeSource` for integration with CrewAI
- Configurable API endpoint and article limit - Configurable API endpoint and article limit
- Implements three key methods: - Implements three key methods:
@@ -255,10 +251,12 @@ The latest developments in space exploration, based on recent space news article
- `add()`: Processes and stores the content - `add()`: Processes and stores the content
2. **Agent Configuration**: 2. **Agent Configuration**:
- Specialized role as a Space News Analyst - Specialized role as a Space News Analyst
- Uses the knowledge source to access space news - Uses the knowledge source to access space news
3. **Task Setup**: 3. **Task Setup**:
- Takes a user question as input through `{user_question}` - Takes a user question as input through `{user_question}`
- Designed to provide detailed answers based on the knowledge source - Designed to provide detailed answers based on the knowledge source
@@ -267,6 +265,7 @@ The latest developments in space exploration, based on recent space news article
- Handles input/output through the kickoff method - Handles input/output through the kickoff method
This example demonstrates how to: This example demonstrates how to:
- Create a custom knowledge source that fetches real-time data - Create a custom knowledge source that fetches real-time data
- Process and format external data for AI consumption - Process and format external data for AI consumption
- Use the knowledge source to answer specific user questions - Use the knowledge source to answer specific user questions
@@ -274,13 +273,15 @@ This example demonstrates how to:
#### About the Spaceflight News API #### About the Spaceflight News API
The example uses the [Spaceflight News API](https://api.spaceflightnewsapi.net/v4/documentation), which: The example uses the [Spaceflight News API](https://api.spaceflightnewsapi.net/v4/docs/), which:
- Provides free access to space-related news articles - Provides free access to space-related news articles
- Requires no authentication - Requires no authentication
- Returns structured data about space news - Returns structured data about space news
- Supports pagination and filtering - Supports pagination and filtering
You can customize the API query by modifying the endpoint URL: You can customize the API query by modifying the endpoint URL:
```python ```python
# Fetch more articles # Fetch more articles
recent_news = SpaceNewsKnowledgeSource( recent_news = SpaceNewsKnowledgeSource(
@@ -299,14 +300,14 @@ recent_news = SpaceNewsKnowledgeSource(
<AccordionGroup> <AccordionGroup>
<Accordion title="Content Organization"> <Accordion title="Content Organization">
- Keep chunk sizes appropriate for your content type - Keep chunk sizes appropriate for your content type - Consider content
- Consider content overlap for context preservation overlap for context preservation - Organize related information into
- Organize related information into separate knowledge sources separate knowledge sources
</Accordion> </Accordion>
<Accordion title="Performance Tips"> <Accordion title="Performance Tips">
- Adjust chunk sizes based on content complexity - Adjust chunk sizes based on content complexity - Configure appropriate
- Configure appropriate embedding models embedding models - Consider using local embedding providers for faster
- Consider using local embedding providers for faster processing processing
</Accordion> </Accordion>
</AccordionGroup> </AccordionGroup>