mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-04-30 23:02:50 +00:00
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
adding reference and explaination for package registry Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
264 lines
9.3 KiB
Plaintext
264 lines
9.3 KiB
Plaintext
---
|
|
title: "Private Package Registries"
|
|
description: "Install private Python packages from authenticated PyPI registries in CrewAI AMP"
|
|
icon: "lock"
|
|
mode: "wide"
|
|
---
|
|
|
|
<Note>
|
|
This guide covers how to configure your CrewAI project to install Python packages
|
|
from private PyPI registries (Azure DevOps Artifacts, GitHub Packages, GitLab, AWS CodeArtifact, etc.)
|
|
when deploying to CrewAI AMP.
|
|
</Note>
|
|
|
|
## When You Need This
|
|
|
|
If your project depends on internal or proprietary Python packages hosted on a private registry
|
|
rather than the public PyPI, you'll need to:
|
|
|
|
1. Tell UV **where** to find the package (an index URL)
|
|
2. Tell UV **which** packages come from that index (a source mapping)
|
|
3. Provide **credentials** so UV can authenticate during install
|
|
|
|
CrewAI AMP uses [UV](https://docs.astral.sh/uv/) for dependency resolution and installation.
|
|
UV supports authenticated private registries through `pyproject.toml` configuration combined
|
|
with environment variables for credentials.
|
|
|
|
## Step 1: Configure pyproject.toml
|
|
|
|
Three pieces work together in your `pyproject.toml`:
|
|
|
|
### 1a. Declare the dependency
|
|
|
|
Add the private package to your `[project.dependencies]` like any other dependency:
|
|
|
|
```toml
|
|
[project]
|
|
dependencies = [
|
|
"crewai[tools]>=0.100.1,<1.0.0",
|
|
"my-private-package>=1.2.0",
|
|
]
|
|
```
|
|
|
|
### 1b. Define the index
|
|
|
|
Register your private registry as a named index under `[[tool.uv.index]]`:
|
|
|
|
```toml
|
|
[[tool.uv.index]]
|
|
name = "my-private-registry"
|
|
url = "https://pkgs.dev.azure.com/my-org/_packaging/my-feed/pypi/simple/"
|
|
explicit = true
|
|
```
|
|
|
|
<Info>
|
|
The `name` field is important — UV uses it to construct the environment variable names
|
|
for authentication (see [Step 2](#step-2-set-authentication-credentials) below).
|
|
|
|
Setting `explicit = true` means UV won't search this index for every package — only the
|
|
ones you explicitly map to it in `[tool.uv.sources]`. This avoids unnecessary queries
|
|
against your private registry and protects against dependency confusion attacks.
|
|
</Info>
|
|
|
|
### 1c. Map the package to the index
|
|
|
|
Tell UV which packages should be resolved from your private index using `[tool.uv.sources]`:
|
|
|
|
```toml
|
|
[tool.uv.sources]
|
|
my-private-package = { index = "my-private-registry" }
|
|
```
|
|
|
|
### Complete example
|
|
|
|
```toml
|
|
[project]
|
|
name = "my-crew-project"
|
|
version = "0.1.0"
|
|
requires-python = ">=3.10,<=3.13"
|
|
dependencies = [
|
|
"crewai[tools]>=0.100.1,<1.0.0",
|
|
"my-private-package>=1.2.0",
|
|
]
|
|
|
|
[tool.crewai]
|
|
type = "crew"
|
|
|
|
[[tool.uv.index]]
|
|
name = "my-private-registry"
|
|
url = "https://pkgs.dev.azure.com/my-org/_packaging/my-feed/pypi/simple/"
|
|
explicit = true
|
|
|
|
[tool.uv.sources]
|
|
my-private-package = { index = "my-private-registry" }
|
|
```
|
|
|
|
After updating `pyproject.toml`, regenerate your lock file:
|
|
|
|
```bash
|
|
uv lock
|
|
```
|
|
|
|
<Warning>
|
|
Always commit the updated `uv.lock` along with your `pyproject.toml` changes.
|
|
The lock file is required for deployment — see [Prepare for Deployment](/en/enterprise/guides/prepare-for-deployment).
|
|
</Warning>
|
|
|
|
## Step 2: Set Authentication Credentials
|
|
|
|
UV authenticates against private indexes using environment variables that follow a naming convention
|
|
based on the index name you defined in `pyproject.toml`:
|
|
|
|
```
|
|
UV_INDEX_{UPPER_NAME}_USERNAME
|
|
UV_INDEX_{UPPER_NAME}_PASSWORD
|
|
```
|
|
|
|
Where `{UPPER_NAME}` is your index name converted to **uppercase** with **hyphens replaced by underscores**.
|
|
|
|
For example, an index named `my-private-registry` uses:
|
|
|
|
| Variable | Value |
|
|
|----------|-------|
|
|
| `UV_INDEX_MY_PRIVATE_REGISTRY_USERNAME` | Your registry username or token name |
|
|
| `UV_INDEX_MY_PRIVATE_REGISTRY_PASSWORD` | Your registry password or token/PAT |
|
|
|
|
<Warning>
|
|
These environment variables **must** be added via the CrewAI AMP **Environment Variables** settings —
|
|
either globally or at the deployment level. They cannot be set in `.env` files or hardcoded in your project.
|
|
|
|
See [Setting Environment Variables in AMP](#setting-environment-variables-in-amp) below.
|
|
</Warning>
|
|
|
|
## Registry Provider Reference
|
|
|
|
The table below shows the index URL format and credential values for common registry providers.
|
|
Replace placeholder values with your actual organization and feed details.
|
|
|
|
| Provider | Index URL | Username | Password |
|
|
|----------|-----------|----------|----------|
|
|
| **Azure DevOps Artifacts** | `https://pkgs.dev.azure.com/{org}/_packaging/{feed}/pypi/simple/` | Any non-empty string (e.g. `token`) | Personal Access Token (PAT) with Packaging Read scope |
|
|
| **GitHub Packages** | `https://pypi.pkg.github.com/{owner}/simple/` | GitHub username | Personal Access Token (classic) with `read:packages` scope |
|
|
| **GitLab Package Registry** | `https://gitlab.com/api/v4/projects/{project_id}/packages/pypi/simple/` | `__token__` | Project or Personal Access Token with `read_api` scope |
|
|
| **AWS CodeArtifact** | Use the URL from `aws codeartifact get-repository-endpoint` | `aws` | Token from `aws codeartifact get-authorization-token` |
|
|
| **Google Artifact Registry** | `https://{region}-python.pkg.dev/{project}/{repo}/simple/` | `_json_key_base64` | Base64-encoded service account key |
|
|
| **JFrog Artifactory** | `https://{instance}.jfrog.io/artifactory/api/pypi/{repo}/simple/` | Username or email | API key or identity token |
|
|
| **Self-hosted (devpi, Nexus, etc.)** | Your registry's simple API URL | Registry username | Registry password |
|
|
|
|
<Tip>
|
|
For **AWS CodeArtifact**, the authorization token expires periodically.
|
|
You'll need to refresh the `UV_INDEX_*_PASSWORD` value when it expires.
|
|
Consider automating this in your CI/CD pipeline.
|
|
</Tip>
|
|
|
|
## Setting Environment Variables in AMP
|
|
|
|
Private registry credentials must be configured as environment variables in CrewAI AMP.
|
|
You have two options:
|
|
|
|
<Tabs>
|
|
<Tab title="Web Interface">
|
|
1. Log in to [CrewAI AMP](https://app.crewai.com)
|
|
2. Navigate to your automation
|
|
3. Open the **Environment Variables** tab
|
|
4. Add each variable (`UV_INDEX_*_USERNAME` and `UV_INDEX_*_PASSWORD`) with its value
|
|
|
|
See the [Deploy to AMP — Set Environment Variables](/en/enterprise/guides/deploy-to-amp#set-environment-variables) step for details.
|
|
</Tab>
|
|
<Tab title="CLI Deployment">
|
|
Add the variables to your local `.env` file before running `crewai deploy create`.
|
|
The CLI will securely transfer them to the platform:
|
|
|
|
```bash
|
|
# .env
|
|
OPENAI_API_KEY=sk-...
|
|
UV_INDEX_MY_PRIVATE_REGISTRY_USERNAME=token
|
|
UV_INDEX_MY_PRIVATE_REGISTRY_PASSWORD=your-pat-here
|
|
```
|
|
|
|
```bash
|
|
crewai deploy create
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
<Warning>
|
|
**Never** commit credentials to your repository. Use AMP environment variables for all secrets.
|
|
The `.env` file should be listed in `.gitignore`.
|
|
</Warning>
|
|
|
|
To update credentials on an existing deployment, see [Update Your Crew — Environment Variables](/en/enterprise/guides/update-crew).
|
|
|
|
## How It All Fits Together
|
|
|
|
When CrewAI AMP builds your automation, the resolution flow works like this:
|
|
|
|
<Steps>
|
|
<Step title="Build starts">
|
|
AMP pulls your repository and reads `pyproject.toml` and `uv.lock`.
|
|
</Step>
|
|
<Step title="UV resolves dependencies">
|
|
UV reads `[tool.uv.sources]` to determine which index each package should come from.
|
|
</Step>
|
|
<Step title="UV authenticates">
|
|
For each private index, UV looks up `UV_INDEX_{NAME}_USERNAME` and `UV_INDEX_{NAME}_PASSWORD`
|
|
from the environment variables you configured in AMP.
|
|
</Step>
|
|
<Step title="Packages install">
|
|
UV downloads and installs all packages — both public (from PyPI) and private (from your registry).
|
|
</Step>
|
|
<Step title="Automation runs">
|
|
Your crew or flow starts with all dependencies available.
|
|
</Step>
|
|
</Steps>
|
|
|
|
## Troubleshooting
|
|
|
|
### Authentication Errors During Build
|
|
|
|
**Symptom**: Build fails with `401 Unauthorized` or `403 Forbidden` when resolving a private package.
|
|
|
|
**Check**:
|
|
- The `UV_INDEX_*` environment variable names match your index name exactly (uppercased, hyphens → underscores)
|
|
- Credentials are set in AMP environment variables, not just in a local `.env`
|
|
- Your token/PAT has the required read permissions for the package feed
|
|
- The token hasn't expired (especially relevant for AWS CodeArtifact)
|
|
|
|
### Package Not Found
|
|
|
|
**Symptom**: `No matching distribution found for my-private-package`.
|
|
|
|
**Check**:
|
|
- The index URL in `pyproject.toml` ends with `/simple/`
|
|
- The `[tool.uv.sources]` entry maps the correct package name to the correct index name
|
|
- The package is actually published to your private registry
|
|
- Run `uv lock` locally with the same credentials to verify resolution works
|
|
|
|
### Lock File Conflicts
|
|
|
|
**Symptom**: `uv lock` fails or produces unexpected results after adding a private index.
|
|
|
|
**Solution**: Set the credentials locally and regenerate:
|
|
|
|
```bash
|
|
export UV_INDEX_MY_PRIVATE_REGISTRY_USERNAME=token
|
|
export UV_INDEX_MY_PRIVATE_REGISTRY_PASSWORD=your-pat
|
|
uv lock
|
|
```
|
|
|
|
Then commit the updated `uv.lock`.
|
|
|
|
## Related Guides
|
|
|
|
<CardGroup cols={3}>
|
|
<Card title="Prepare for Deployment" icon="clipboard-check" href="/en/enterprise/guides/prepare-for-deployment">
|
|
Verify project structure and dependencies before deploying.
|
|
</Card>
|
|
<Card title="Deploy to AMP" icon="rocket" href="/en/enterprise/guides/deploy-to-amp">
|
|
Deploy your crew or flow and configure environment variables.
|
|
</Card>
|
|
<Card title="Update Your Crew" icon="arrows-rotate" href="/en/enterprise/guides/update-crew">
|
|
Update environment variables and push changes to a running deployment.
|
|
</Card>
|
|
</CardGroup>
|