mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-04-30 23:02:50 +00:00
Private package registry (#4583)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
adding reference and explaination for package registry Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
This commit is contained in:
@@ -177,6 +177,11 @@ You need to push your crew to a GitHub repository. If you haven't created a crew
|
||||

|
||||
</Frame>
|
||||
|
||||
<Info>
|
||||
Using private Python packages? You'll need to add your registry credentials here too.
|
||||
See [Private Package Registries](/en/enterprise/guides/private-package-registry) for the required variables.
|
||||
</Info>
|
||||
|
||||
</Step>
|
||||
|
||||
<Step title="Deploy Your Crew">
|
||||
|
||||
@@ -256,6 +256,12 @@ Before deployment, ensure you have:
|
||||
1. **LLM API keys** ready (OpenAI, Anthropic, Google, etc.)
|
||||
2. **Tool API keys** if using external tools (Serper, etc.)
|
||||
|
||||
<Info>
|
||||
If your project depends on packages from a **private PyPI registry**, you'll also need to configure
|
||||
registry authentication credentials as environment variables. See the
|
||||
[Private Package Registries](/en/enterprise/guides/private-package-registry) guide for details.
|
||||
</Info>
|
||||
|
||||
<Tip>
|
||||
Test your project locally with the same environment variables before deploying
|
||||
to catch configuration issues early.
|
||||
|
||||
263
docs/en/enterprise/guides/private-package-registry.mdx
Normal file
263
docs/en/enterprise/guides/private-package-registry.mdx
Normal file
@@ -0,0 +1,263 @@
|
||||
---
|
||||
title: "Private Package Registries"
|
||||
description: "Install private Python packages from authenticated PyPI registries in CrewAI AMP"
|
||||
icon: "lock"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
<Note>
|
||||
This guide covers how to configure your CrewAI project to install Python packages
|
||||
from private PyPI registries (Azure DevOps Artifacts, GitHub Packages, GitLab, AWS CodeArtifact, etc.)
|
||||
when deploying to CrewAI AMP.
|
||||
</Note>
|
||||
|
||||
## When You Need This
|
||||
|
||||
If your project depends on internal or proprietary Python packages hosted on a private registry
|
||||
rather than the public PyPI, you'll need to:
|
||||
|
||||
1. Tell UV **where** to find the package (an index URL)
|
||||
2. Tell UV **which** packages come from that index (a source mapping)
|
||||
3. Provide **credentials** so UV can authenticate during install
|
||||
|
||||
CrewAI AMP uses [UV](https://docs.astral.sh/uv/) for dependency resolution and installation.
|
||||
UV supports authenticated private registries through `pyproject.toml` configuration combined
|
||||
with environment variables for credentials.
|
||||
|
||||
## Step 1: Configure pyproject.toml
|
||||
|
||||
Three pieces work together in your `pyproject.toml`:
|
||||
|
||||
### 1a. Declare the dependency
|
||||
|
||||
Add the private package to your `[project.dependencies]` like any other dependency:
|
||||
|
||||
```toml
|
||||
[project]
|
||||
dependencies = [
|
||||
"crewai[tools]>=0.100.1,<1.0.0",
|
||||
"my-private-package>=1.2.0",
|
||||
]
|
||||
```
|
||||
|
||||
### 1b. Define the index
|
||||
|
||||
Register your private registry as a named index under `[[tool.uv.index]]`:
|
||||
|
||||
```toml
|
||||
[[tool.uv.index]]
|
||||
name = "my-private-registry"
|
||||
url = "https://pkgs.dev.azure.com/my-org/_packaging/my-feed/pypi/simple/"
|
||||
explicit = true
|
||||
```
|
||||
|
||||
<Info>
|
||||
The `name` field is important — UV uses it to construct the environment variable names
|
||||
for authentication (see [Step 2](#step-2-set-authentication-credentials) below).
|
||||
|
||||
Setting `explicit = true` means UV won't search this index for every package — only the
|
||||
ones you explicitly map to it in `[tool.uv.sources]`. This avoids unnecessary queries
|
||||
against your private registry and protects against dependency confusion attacks.
|
||||
</Info>
|
||||
|
||||
### 1c. Map the package to the index
|
||||
|
||||
Tell UV which packages should be resolved from your private index using `[tool.uv.sources]`:
|
||||
|
||||
```toml
|
||||
[tool.uv.sources]
|
||||
my-private-package = { index = "my-private-registry" }
|
||||
```
|
||||
|
||||
### Complete example
|
||||
|
||||
```toml
|
||||
[project]
|
||||
name = "my-crew-project"
|
||||
version = "0.1.0"
|
||||
requires-python = ">=3.10,<=3.13"
|
||||
dependencies = [
|
||||
"crewai[tools]>=0.100.1,<1.0.0",
|
||||
"my-private-package>=1.2.0",
|
||||
]
|
||||
|
||||
[tool.crewai]
|
||||
type = "crew"
|
||||
|
||||
[[tool.uv.index]]
|
||||
name = "my-private-registry"
|
||||
url = "https://pkgs.dev.azure.com/my-org/_packaging/my-feed/pypi/simple/"
|
||||
explicit = true
|
||||
|
||||
[tool.uv.sources]
|
||||
my-private-package = { index = "my-private-registry" }
|
||||
```
|
||||
|
||||
After updating `pyproject.toml`, regenerate your lock file:
|
||||
|
||||
```bash
|
||||
uv lock
|
||||
```
|
||||
|
||||
<Warning>
|
||||
Always commit the updated `uv.lock` along with your `pyproject.toml` changes.
|
||||
The lock file is required for deployment — see [Prepare for Deployment](/en/enterprise/guides/prepare-for-deployment).
|
||||
</Warning>
|
||||
|
||||
## Step 2: Set Authentication Credentials
|
||||
|
||||
UV authenticates against private indexes using environment variables that follow a naming convention
|
||||
based on the index name you defined in `pyproject.toml`:
|
||||
|
||||
```
|
||||
UV_INDEX_{UPPER_NAME}_USERNAME
|
||||
UV_INDEX_{UPPER_NAME}_PASSWORD
|
||||
```
|
||||
|
||||
Where `{UPPER_NAME}` is your index name converted to **uppercase** with **hyphens replaced by underscores**.
|
||||
|
||||
For example, an index named `my-private-registry` uses:
|
||||
|
||||
| Variable | Value |
|
||||
|----------|-------|
|
||||
| `UV_INDEX_MY_PRIVATE_REGISTRY_USERNAME` | Your registry username or token name |
|
||||
| `UV_INDEX_MY_PRIVATE_REGISTRY_PASSWORD` | Your registry password or token/PAT |
|
||||
|
||||
<Warning>
|
||||
These environment variables **must** be added via the CrewAI AMP **Environment Variables** settings —
|
||||
either globally or at the deployment level. They cannot be set in `.env` files or hardcoded in your project.
|
||||
|
||||
See [Setting Environment Variables in AMP](#setting-environment-variables-in-amp) below.
|
||||
</Warning>
|
||||
|
||||
## Registry Provider Reference
|
||||
|
||||
The table below shows the index URL format and credential values for common registry providers.
|
||||
Replace placeholder values with your actual organization and feed details.
|
||||
|
||||
| Provider | Index URL | Username | Password |
|
||||
|----------|-----------|----------|----------|
|
||||
| **Azure DevOps Artifacts** | `https://pkgs.dev.azure.com/{org}/_packaging/{feed}/pypi/simple/` | Any non-empty string (e.g. `token`) | Personal Access Token (PAT) with Packaging Read scope |
|
||||
| **GitHub Packages** | `https://pypi.pkg.github.com/{owner}/simple/` | GitHub username | Personal Access Token (classic) with `read:packages` scope |
|
||||
| **GitLab Package Registry** | `https://gitlab.com/api/v4/projects/{project_id}/packages/pypi/simple/` | `__token__` | Project or Personal Access Token with `read_api` scope |
|
||||
| **AWS CodeArtifact** | Use the URL from `aws codeartifact get-repository-endpoint` | `aws` | Token from `aws codeartifact get-authorization-token` |
|
||||
| **Google Artifact Registry** | `https://{region}-python.pkg.dev/{project}/{repo}/simple/` | `_json_key_base64` | Base64-encoded service account key |
|
||||
| **JFrog Artifactory** | `https://{instance}.jfrog.io/artifactory/api/pypi/{repo}/simple/` | Username or email | API key or identity token |
|
||||
| **Self-hosted (devpi, Nexus, etc.)** | Your registry's simple API URL | Registry username | Registry password |
|
||||
|
||||
<Tip>
|
||||
For **AWS CodeArtifact**, the authorization token expires periodically.
|
||||
You'll need to refresh the `UV_INDEX_*_PASSWORD` value when it expires.
|
||||
Consider automating this in your CI/CD pipeline.
|
||||
</Tip>
|
||||
|
||||
## Setting Environment Variables in AMP
|
||||
|
||||
Private registry credentials must be configured as environment variables in CrewAI AMP.
|
||||
You have two options:
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Web Interface">
|
||||
1. Log in to [CrewAI AMP](https://app.crewai.com)
|
||||
2. Navigate to your automation
|
||||
3. Open the **Environment Variables** tab
|
||||
4. Add each variable (`UV_INDEX_*_USERNAME` and `UV_INDEX_*_PASSWORD`) with its value
|
||||
|
||||
See the [Deploy to AMP — Set Environment Variables](/en/enterprise/guides/deploy-to-amp#set-environment-variables) step for details.
|
||||
</Tab>
|
||||
<Tab title="CLI Deployment">
|
||||
Add the variables to your local `.env` file before running `crewai deploy create`.
|
||||
The CLI will securely transfer them to the platform:
|
||||
|
||||
```bash
|
||||
# .env
|
||||
OPENAI_API_KEY=sk-...
|
||||
UV_INDEX_MY_PRIVATE_REGISTRY_USERNAME=token
|
||||
UV_INDEX_MY_PRIVATE_REGISTRY_PASSWORD=your-pat-here
|
||||
```
|
||||
|
||||
```bash
|
||||
crewai deploy create
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
<Warning>
|
||||
**Never** commit credentials to your repository. Use AMP environment variables for all secrets.
|
||||
The `.env` file should be listed in `.gitignore`.
|
||||
</Warning>
|
||||
|
||||
To update credentials on an existing deployment, see [Update Your Crew — Environment Variables](/en/enterprise/guides/update-crew).
|
||||
|
||||
## How It All Fits Together
|
||||
|
||||
When CrewAI AMP builds your automation, the resolution flow works like this:
|
||||
|
||||
<Steps>
|
||||
<Step title="Build starts">
|
||||
AMP pulls your repository and reads `pyproject.toml` and `uv.lock`.
|
||||
</Step>
|
||||
<Step title="UV resolves dependencies">
|
||||
UV reads `[tool.uv.sources]` to determine which index each package should come from.
|
||||
</Step>
|
||||
<Step title="UV authenticates">
|
||||
For each private index, UV looks up `UV_INDEX_{NAME}_USERNAME` and `UV_INDEX_{NAME}_PASSWORD`
|
||||
from the environment variables you configured in AMP.
|
||||
</Step>
|
||||
<Step title="Packages install">
|
||||
UV downloads and installs all packages — both public (from PyPI) and private (from your registry).
|
||||
</Step>
|
||||
<Step title="Automation runs">
|
||||
Your crew or flow starts with all dependencies available.
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Authentication Errors During Build
|
||||
|
||||
**Symptom**: Build fails with `401 Unauthorized` or `403 Forbidden` when resolving a private package.
|
||||
|
||||
**Check**:
|
||||
- The `UV_INDEX_*` environment variable names match your index name exactly (uppercased, hyphens → underscores)
|
||||
- Credentials are set in AMP environment variables, not just in a local `.env`
|
||||
- Your token/PAT has the required read permissions for the package feed
|
||||
- The token hasn't expired (especially relevant for AWS CodeArtifact)
|
||||
|
||||
### Package Not Found
|
||||
|
||||
**Symptom**: `No matching distribution found for my-private-package`.
|
||||
|
||||
**Check**:
|
||||
- The index URL in `pyproject.toml` ends with `/simple/`
|
||||
- The `[tool.uv.sources]` entry maps the correct package name to the correct index name
|
||||
- The package is actually published to your private registry
|
||||
- Run `uv lock` locally with the same credentials to verify resolution works
|
||||
|
||||
### Lock File Conflicts
|
||||
|
||||
**Symptom**: `uv lock` fails or produces unexpected results after adding a private index.
|
||||
|
||||
**Solution**: Set the credentials locally and regenerate:
|
||||
|
||||
```bash
|
||||
export UV_INDEX_MY_PRIVATE_REGISTRY_USERNAME=token
|
||||
export UV_INDEX_MY_PRIVATE_REGISTRY_PASSWORD=your-pat
|
||||
uv lock
|
||||
```
|
||||
|
||||
Then commit the updated `uv.lock`.
|
||||
|
||||
## Related Guides
|
||||
|
||||
<CardGroup cols={3}>
|
||||
<Card title="Prepare for Deployment" icon="clipboard-check" href="/en/enterprise/guides/prepare-for-deployment">
|
||||
Verify project structure and dependencies before deploying.
|
||||
</Card>
|
||||
<Card title="Deploy to AMP" icon="rocket" href="/en/enterprise/guides/deploy-to-amp">
|
||||
Deploy your crew or flow and configure environment variables.
|
||||
</Card>
|
||||
<Card title="Update Your Crew" icon="arrows-rotate" href="/en/enterprise/guides/update-crew">
|
||||
Update environment variables and push changes to a running deployment.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
Reference in New Issue
Block a user