mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-01-09 16:18:30 +00:00
bf9e0423f2
Some checks failed
* WIP: v1 docs (#3626) (cherry picked from commit d46e20fa09bcd2f5916282f5553ddeb7183bd92c) * docs: parity for all translations * docs: full name of acronym AMP * docs: fix lingering unused code * docs: expand contextual options in docs.json * docs: add contextual action to request feature on GitHub * chore: tidy docs formatting
111 lines
2.1 KiB
Plaintext
111 lines
2.1 KiB
Plaintext
---
|
|
title: Bright Data Tools
|
|
description: Bright Data integrations for SERP search, Web Unlocker scraping, and Dataset API.
|
|
icon: spider
|
|
mode: "wide"
|
|
---
|
|
|
|
# Bright Data Tools
|
|
|
|
This set of tools integrates Bright Data services for web extraction.
|
|
|
|
## Installation
|
|
|
|
```shell
|
|
uv add crewai-tools requests aiohttp
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
- `BRIGHT_DATA_API_KEY` (required)
|
|
- `BRIGHT_DATA_ZONE` (for SERP/Web Unlocker)
|
|
|
|
Create credentials at https://brightdata.com/ (sign up, then create an API token and zone).
|
|
See their docs: https://developers.brightdata.com/
|
|
|
|
## Included Tools
|
|
|
|
- `BrightDataSearchTool`: SERP search (Google/Bing/Yandex) with geo/language/device options.
|
|
- `BrightDataWebUnlockerTool`: Scrape pages with anti-bot bypass and rendering.
|
|
- `BrightDataDatasetTool`: Run Dataset API jobs and fetch results.
|
|
|
|
## Examples
|
|
|
|
### SERP Search
|
|
|
|
```python Code
|
|
from crewai_tools import BrightDataSearchTool
|
|
|
|
tool = BrightDataSearchTool(
|
|
query="CrewAI",
|
|
country="us",
|
|
)
|
|
|
|
print(tool.run())
|
|
```
|
|
|
|
### Web Unlocker
|
|
|
|
```python Code
|
|
from crewai_tools import BrightDataWebUnlockerTool
|
|
|
|
tool = BrightDataWebUnlockerTool(
|
|
url="https://example.com",
|
|
format="markdown",
|
|
)
|
|
|
|
print(tool.run(url="https://example.com"))
|
|
```
|
|
|
|
### Dataset API
|
|
|
|
```python Code
|
|
from crewai_tools import BrightDataDatasetTool
|
|
|
|
tool = BrightDataDatasetTool(
|
|
dataset_type="ecommerce",
|
|
url="https://example.com/product",
|
|
)
|
|
|
|
print(tool.run())
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
- 401/403: verify `BRIGHT_DATA_API_KEY` and `BRIGHT_DATA_ZONE`.
|
|
- Empty/blocked content: enable rendering or try a different zone.
|
|
|
|
## Example
|
|
|
|
```python Code
|
|
from crewai import Agent, Task, Crew
|
|
from crewai_tools import BrightDataSearchTool
|
|
|
|
tool = BrightDataSearchTool(
|
|
query="CrewAI",
|
|
country="us",
|
|
)
|
|
|
|
agent = Agent(
|
|
role="Web Researcher",
|
|
goal="Search with Bright Data",
|
|
backstory="Finds reliable results",
|
|
tools=[tool],
|
|
verbose=True,
|
|
)
|
|
|
|
task = Task(
|
|
description="Search for CrewAI and summarize top results",
|
|
expected_output="Short summary with links",
|
|
agent=agent,
|
|
)
|
|
|
|
crew = Crew(
|
|
agents=[agent],
|
|
tasks=[task],
|
|
verbose=True,
|
|
)
|
|
|
|
result = crew.kickoff()
|
|
```
|