Compare commits

..

5 Commits

Author SHA1 Message Date
Lucas Gomide
ea0a4c6bea fix: ensure Agent is able to load a list of Tools dynamically 2025-06-20 11:44:28 -03:00
Lucas Gomide
65af0173fe Merge branch 'main' into lg-agent-initialize 2025-06-19 17:21:16 -03:00
Lucas Gomide
d0e9964f2d Merge branch 'main' into lg-agent-initialize 2025-06-19 13:56:39 -03:00
Lucas Gomide
a00dd73614 Merge branch 'main' into lg-agent-initialize 2025-06-19 12:57:36 -03:00
Lucas Gomide
b63a8f738c feat: support to initialize a tool from defined Tool attributes 2025-06-17 18:31:57 -03:00
7 changed files with 10 additions and 316 deletions

View File

@@ -285,32 +285,25 @@ Watch this video tutorial for a step-by-step demonstration of deploying your cre
### 11. API Keys
When running ```crewai create crew``` command, the CLI will show you a list of available LLM providers to choose from, followed by model selection for your chosen provider.
When running ```crewai create crew``` command, the CLI will first show you the top 5 most common LLM providers and ask you to select one.
Once you've selected an LLM provider and model, you will be prompted for API keys.
Once you've selected an LLM provider, you will be prompted for API keys.
#### Available LLM Providers
#### Initial API key providers
The CLI will show you the following LLM providers to choose from:
The CLI will initially prompt for API keys for the following services:
* OpenAI
* Groq
* Anthropic
* Google Gemini
* NVIDIA NIM
* Groq
* Hugging Face
* Ollama
* Watson
* AWS Bedrock
* Azure
* Cerebras
* SambaNova
When you select a provider, the CLI will then show you available models for that provider and prompt you to enter your API key.
When you select a provider, the CLI will prompt you to enter your API key.
#### Other Options
If you select "other", you will be able to select from a list of LiteLLM supported providers.
If you select option 6, you will be able to select from a list of LiteLLM supported providers.
When you select a provider, the CLI will prompt you to enter the Key name and the API key.

View File

@@ -134,7 +134,7 @@
"tools/web-scraping/stagehandtool",
"tools/web-scraping/firecrawlcrawlwebsitetool",
"tools/web-scraping/firecrawlscrapewebsitetool",
"tools/web-scraping/oxylabsscraperstool"
"tools/web-scraping/firecrawlsearchtool"
]
},
{

View File

@@ -124,7 +124,7 @@ from crewai_tools import CrewaiEnterpriseTools
enterprise_tools = CrewaiEnterpriseTools(
actions_list=["gmail_find_email"] # only gmail_find_email tool will be available
)
gmail_tool = enterprise_tools["gmail_find_email"]
gmail_tool = enterprise_tools[0]
gmail_agent = Agent(
role="Gmail Manager",

View File

@@ -85,22 +85,6 @@ with MCPServerAdapter(server_params) as mcp_tools:
```
This general pattern shows how to integrate tools. For specific examples tailored to each transport, refer to the detailed guides below.
## Filtering Tools
```python
with MCPServerAdapter(server_params) as mcp_tools:
print(f"Available tools: {[tool.name for tool in mcp_tools]}")
my_agent = Agent(
role="MCP Tool User",
goal="Utilize tools from an MCP server.",
backstory="I can connect to MCP servers and use their tools.",
tools=mcp_tools["tool_name"], # Pass the loaded tools to your agent
reasoning=True,
verbose=True
)
# ... rest of your crew setup ...
```
## Explore MCP Integrations
<CardGroup cols={2}>

View File

@@ -56,10 +56,6 @@ These tools enable your agents to interact with the web, extract data from websi
<Card title="Stagehand Tool" icon="hand" href="/tools/web-scraping/stagehandtool">
Intelligent browser automation with natural language commands.
</Card>
<Card title="Oxylabs Scraper Tool" icon="globe" href="/tools/web-scraping/oxylabsscraperstool">
Access web data at scale with Oxylabs.
</Card>
</CardGroup>
## **Common Use Cases**
@@ -104,4 +100,4 @@ agent = Agent(
- **JavaScript-Heavy Sites**: Use `SeleniumScrapingTool` for dynamic content
- **Scale & Performance**: Use `FirecrawlScrapeWebsiteTool` for high-volume scraping
- **Cloud Infrastructure**: Use `BrowserBaseLoadTool` for scalable browser automation
- **Complex Workflows**: Use `StagehandTool` for intelligent browser interactions
- **Complex Workflows**: Use `StagehandTool` for intelligent browser interactions

View File

@@ -1,236 +0,0 @@
---
title: Oxylabs Scrapers
description: >
Oxylabs Scrapers allow to easily access the information from the respective sources. Please see the list of available sources below:
- `Amazon Product`
- `Amazon Search`
- `Google Seach`
- `Universal`
icon: globe
---
## Installation
Get the credentials by creating an Oxylabs Account [here](https://oxylabs.io).
```shell
pip install 'crewai[tools]' oxylabs
```
Check [Oxylabs Documentation](https://developers.oxylabs.io/scraping-solutions/web-scraper-api/targets) to get more information about API parameters.
# `OxylabsAmazonProductScraperTool`
### Example
```python
from crewai_tools import OxylabsAmazonProductScraperTool
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
tool = OxylabsAmazonProductScraperTool()
result = tool.run(query="AAAAABBBBCC")
print(result)
```
### Parameters
- `query` - 10-symbol ASIN code.
- `domain` - domain localization for Amazon.
- `geo_location` - the _Deliver to_ location.
- `user_agent_type` - device type and browser.
- `render` - enables JavaScript rendering when set to `html`.
- `callback_url` - URL to your callback endpoint.
- `context` - Additional advanced settings and controls for specialized requirements.
- `parse` - returns parsed data when set to true.
- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
### Advanced example
```python
from crewai_tools import OxylabsAmazonProductScraperTool
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
tool = OxylabsAmazonProductScraperTool(
config={
"domain": "com",
"parse": True,
"context": [
{
"key": "autoselect_variant",
"value": True
}
]
}
)
result = tool.run(query="AAAAABBBBCC")
print(result)
```
# `OxylabsAmazonSearchScraperTool`
### Example
```python
from crewai_tools import OxylabsAmazonSearchScraperTool
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
tool = OxylabsAmazonSearchScraperTool()
result = tool.run(query="headsets")
print(result)
```
### Parameters
- `query` - Amazon search term.
- `domain` - Domain localization for Bestbuy.
- `start_page` - starting page number.
- `pages` - number of pages to retrieve.
- `geo_location` - the _Deliver to_ location.
- `user_agent_type` - device type and browser.
- `render` - enables JavaScript rendering when set to `html`.
- `callback_url` - URL to your callback endpoint.
- `context` - Additional advanced settings and controls for specialized requirements.
- `parse` - returns parsed data when set to true.
- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
### Advanced example
```python
from crewai_tools import OxylabsAmazonSearchScraperTool
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
tool = OxylabsAmazonSearchScraperTool(
config={
"domain": 'nl',
"start_page": 2,
"pages": 2,
"parse": True,
"context": [
{'key': 'category_id', 'value': 16391693031}
],
}
)
result = tool.run(query='nirvana tshirt')
print(result)
```
# `OxylabsGoogleSearchScraperTool`
### Example
```python
from crewai_tools import OxylabsGoogleSearchScraperTool
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
tool = OxylabsGoogleSearchScraperTool()
result = tool.run(query="iPhone 16")
print(result)
```
### Parameters
- `query` - search keyword.
- `domain` - domain localization for Google.
- `start_page` - starting page number.
- `pages` - number of pages to retrieve.
- `limit` - number of results to retrieve in each page.
- `locale` - `Accept-Language` header value which changes your Google search page web interface language.
- `geo_location` - the geographical location that the result should be adapted for. Using this parameter correctly is extremely important to get the right data.
- `user_agent_type` - device type and browser.
- `render` - enables JavaScript rendering when set to `html`.
- `callback_url` - URL to your callback endpoint.
- `context` - Additional advanced settings and controls for specialized requirements.
- `parse` - returns parsed data when set to true.
- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
### Advanced example
```python
from crewai_tools import OxylabsGoogleSearchScraperTool
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
tool = OxylabsGoogleSearchScraperTool(
config={
"parse": True,
"geo_location": "Paris, France",
"user_agent_type": "tablet",
}
)
result = tool.run(query="iPhone 16")
print(result)
```
# `OxylabsUniversalScraperTool`
### Example
```python
from crewai_tools import OxylabsUniversalScraperTool
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
tool = OxylabsUniversalScraperTool()
result = tool.run(url="https://ip.oxylabs.io")
print(result)
```
### Parameters
- `url` - website url to scrape.
- `user_agent_type` - device type and browser.
- `geo_location` - sets the proxy's geolocation to retrieve data.
- `render` - enables JavaScript rendering when set to `html`.
- `callback_url` - URL to your callback endpoint.
- `context` - Additional advanced settings and controls for specialized requirements.
- `parse` - returns parsed data when set to `true`, as long as a dedicated parser exists for the submitted URL's page type.
- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
### Advanced example
```python
from crewai_tools import OxylabsUniversalScraperTool
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
tool = OxylabsUniversalScraperTool(
config={
"render": "html",
"user_agent_type": "mobile",
"context": [
{"key": "force_headers", "value": True},
{"key": "force_cookies", "value": True},
{
"key": "headers",
"value": {
"Custom-Header-Name": "custom header content",
},
},
{
"key": "cookies",
"value": [
{"key": "NID", "value": "1234567890"},
{"key": "1P JAR", "value": "0987654321"},
],
},
{"key": "http_method", "value": "get"},
{"key": "follow_redirects", "value": True},
{"key": "successful_status_codes", "value": [808, 909]},
],
}
)
result = tool.run(url="https://ip.oxylabs.io")
print(result)
```

View File

@@ -1,43 +0,0 @@
from pathlib import Path
from crewai.cli.constants import PROVIDERS
def test_cli_documentation_matches_providers():
"""Test that CLI documentation accurately reflects the available providers."""
docs_path = Path(__file__).parent.parent / "docs" / "concepts" / "cli.mdx"
with open(docs_path, 'r') as f:
docs_content = f.read()
assert "top 5" not in docs_content.lower(), "Documentation should not mention 'top 5' providers"
assert "5 most common" not in docs_content.lower(), "Documentation should not mention '5 most common' providers"
assert "list of available LLM providers" in docs_content or "following LLM providers" in docs_content, \
"Documentation should mention the availability of multiple LLM providers"
assert len(PROVIDERS) > 5, f"Expected more than 5 providers, but found {len(PROVIDERS)}"
key_providers = ["OpenAI", "Anthropic", "Gemini"]
for provider in key_providers:
assert provider in docs_content, f"Key provider {provider} should be mentioned in documentation"
def test_providers_list_matches_constants():
"""Test that the actual PROVIDERS list has the expected providers."""
expected_providers = [
"openai",
"anthropic",
"gemini",
"nvidia_nim",
"groq",
"huggingface",
"ollama",
"watson",
"bedrock",
"azure",
"cerebras",
"sambanova",
]
assert PROVIDERS == expected_providers, f"PROVIDERS list has changed. Expected {expected_providers}, got {PROVIDERS}"
assert len(PROVIDERS) == 12, f"Expected 12 providers, but found {len(PROVIDERS)}"