mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-07-07 07:59:29 +00:00
* feat: adopt directory-based docs versioning with Edge channel Switch docs.crewai.com from navigation-only versioning (every version selector entry rendered the same docs/<lang>/* source files) to Mintlify's directory-based versioning so each version selector entry renders its own snapshot. Add an "Edge" channel under docs/edge/<lang>/* that always reflects main HEAD for unreleased work, eliminating pre-release leakage onto frozen release labels. External links to canonical /<lang>/* URLs are preserved via wildcard redirects that always land on the current default version. Layout: - docs/edge/<lang>/* rolling source (you edit here) - docs/edge/enterprise-api.*.yaml - docs/v<X.Y.Z>/<lang>/* frozen, immutable snapshots - docs/v<X.Y.Z>/enterprise-api.*.yaml - docs/images/ shared, append-only - docs/docs.json nav + redirects URLs follow the Mintlify-idiomatic shape: /edge/<lang>/<page> for Edge, /v<X.Y.Z>/<lang>/<page> for every frozen snapshot. The wildcard redirects /<lang>/:slug* -> /<default>/<lang>/:slug* keep stale links working, and every freeze rewrites them (plus all per-section/per-page redirects) so destinations always resolve to the current default without depending on a second redirect hop. Release flow integration (devtools release): - New module crewai_devtools.docs_versioning.freeze() materialises docs/v<X.Y.Z>/ from docs/edge/, rewrites openapi: refs inside the snapshot, inserts the version into every language block in docs.json, and refreshes all redirect destinations. - _update_docs_and_create_pr() in cli.py now calls that freeze during Phase 2 of devtools release. Edge changelogs are updated first (so the snapshot freeze picks them up), then the snapshot is staged alongside docs.json, branched as docs/freeze-v<X.Y.Z>, and the PR is titled [docs-freeze] docs: snapshot and changelog for v<X.Y.Z> — the title prefix the new CI guard reads. - The PR still gates tag, GitHub release, PyPI publish, and the enterprise release as before; no new PRs are added. - Pre-releases (1.X.YaN, 1.X.YbN, ...) skip the snapshot — they ride Edge — and the docs PR title omits the [docs-freeze] prefix. - docs_check (AI-generated docs scaffolding) writes to docs/edge/<lang>/* so newly-generated unreleased docs land in Edge and never accidentally touch a frozen snapshot. Migration scripts (one-shot): - scripts/docs/freeze_historical_versions.py reconstructs all 16 historical snapshots (v1.10.0 .. v1.14.7) from git tags via git archive | tar, rewriting openapi: MDX refs so each snapshot reads its own enterprise-api YAML rather than the live one. - scripts/docs/prefix_version_paths.py one-shot-migrates docs.json: rewrites every page path in 16 versioned blocks to point under docs/v<X.Y.Z>/, inserts a new Edge entry per language, tags v1.14.7 as Latest (default), prunes pages whose target file doesn't exist in the snapshot (e.g. docs/ar/ didn't exist before v1.12.0), and writes the wildcard + per-section redirects. - scripts/docs/freeze_current_edge.py is now a thin CLI wrapper around docs_versioning.freeze for manual one-off freezes (e.g. retroactively snapshotting a forgotten release). CI guards (.github/workflows/docs-snapshots.yml): - Frozen snapshots under docs/v[0-9]*/ are immutable; only PRs whose title contains [docs-freeze] (i.e. release-cut PRs generated by devtools release or the manual wrapper) may modify them. - Images under docs/images/ are append-only since snapshots share a single image directory. Deleting or renaming an image breaks every historical snapshot that still references it. Restored docs/images/crewai-otel-export.png from PR #3673; it was deleted in PR #4908 but v1.10.0 / v1.10.1 snapshots still reference it. Restoring instead of editing the snapshots preserves historical rendering fidelity and validates the new append-only rule retroactively. Tests: - lib/devtools/tests/test_docs_versioning.py covers the freeze: file copy, openapi rewrite, version insertion, default demotion, redirect upserts, per-section redirect rewriting, idempotency, and invalid inputs. Verified locally with mintlify broken-links: 0 broken links across the full site (Edge + 16 frozen versions, 4 locales). AGENTS.md (repo root) is the contributor guide for the new model; RELEASING.md is the release-cut runbook; README's Contribution section links to both. Co-authored-by: Cursor <cursoragent@cursor.com> * style: resolve linter issues --------- Co-authored-by: Cursor <cursoragent@cursor.com>
264 lines
9.3 KiB
Plaintext
264 lines
9.3 KiB
Plaintext
---
|
|
title: "Private Package Registries"
|
|
description: "Install private Python packages from authenticated PyPI registries in CrewAI AMP"
|
|
icon: "lock"
|
|
mode: "wide"
|
|
---
|
|
|
|
<Note>
|
|
This guide covers how to configure your CrewAI project to install Python packages
|
|
from private PyPI registries (Azure DevOps Artifacts, GitHub Packages, GitLab, AWS CodeArtifact, etc.)
|
|
when deploying to CrewAI AMP.
|
|
</Note>
|
|
|
|
## When You Need This
|
|
|
|
If your project depends on internal or proprietary Python packages hosted on a private registry
|
|
rather than the public PyPI, you'll need to:
|
|
|
|
1. Tell UV **where** to find the package (an index URL)
|
|
2. Tell UV **which** packages come from that index (a source mapping)
|
|
3. Provide **credentials** so UV can authenticate during install
|
|
|
|
CrewAI AMP uses [UV](https://docs.astral.sh/uv/) for dependency resolution and installation.
|
|
UV supports authenticated private registries through `pyproject.toml` configuration combined
|
|
with environment variables for credentials.
|
|
|
|
## Step 1: Configure pyproject.toml
|
|
|
|
Three pieces work together in your `pyproject.toml`:
|
|
|
|
### 1a. Declare the dependency
|
|
|
|
Add the private package to your `[project.dependencies]` like any other dependency:
|
|
|
|
```toml
|
|
[project]
|
|
dependencies = [
|
|
"crewai[tools]>=0.100.1,<1.0.0",
|
|
"my-private-package>=1.2.0",
|
|
]
|
|
```
|
|
|
|
### 1b. Define the index
|
|
|
|
Register your private registry as a named index under `[[tool.uv.index]]`:
|
|
|
|
```toml
|
|
[[tool.uv.index]]
|
|
name = "my-private-registry"
|
|
url = "https://pkgs.dev.azure.com/my-org/_packaging/my-feed/pypi/simple/"
|
|
explicit = true
|
|
```
|
|
|
|
<Info>
|
|
The `name` field is important — UV uses it to construct the environment variable names
|
|
for authentication (see [Step 2](#step-2-set-authentication-credentials) below).
|
|
|
|
Setting `explicit = true` means UV won't search this index for every package — only the
|
|
ones you explicitly map to it in `[tool.uv.sources]`. This avoids unnecessary queries
|
|
against your private registry and protects against dependency confusion attacks.
|
|
</Info>
|
|
|
|
### 1c. Map the package to the index
|
|
|
|
Tell UV which packages should be resolved from your private index using `[tool.uv.sources]`:
|
|
|
|
```toml
|
|
[tool.uv.sources]
|
|
my-private-package = { index = "my-private-registry" }
|
|
```
|
|
|
|
### Complete example
|
|
|
|
```toml
|
|
[project]
|
|
name = "my-crew-project"
|
|
version = "0.1.0"
|
|
requires-python = ">=3.10,<=3.13"
|
|
dependencies = [
|
|
"crewai[tools]>=0.100.1,<1.0.0",
|
|
"my-private-package>=1.2.0",
|
|
]
|
|
|
|
[tool.crewai]
|
|
type = "crew"
|
|
|
|
[[tool.uv.index]]
|
|
name = "my-private-registry"
|
|
url = "https://pkgs.dev.azure.com/my-org/_packaging/my-feed/pypi/simple/"
|
|
explicit = true
|
|
|
|
[tool.uv.sources]
|
|
my-private-package = { index = "my-private-registry" }
|
|
```
|
|
|
|
After updating `pyproject.toml`, regenerate your lock file:
|
|
|
|
```bash
|
|
uv lock
|
|
```
|
|
|
|
<Warning>
|
|
Always commit the updated `uv.lock` along with your `pyproject.toml` changes.
|
|
The lock file is required for deployment — see [Prepare for Deployment](/en/enterprise/guides/prepare-for-deployment).
|
|
</Warning>
|
|
|
|
## Step 2: Set Authentication Credentials
|
|
|
|
UV authenticates against private indexes using environment variables that follow a naming convention
|
|
based on the index name you defined in `pyproject.toml`:
|
|
|
|
```
|
|
UV_INDEX_{UPPER_NAME}_USERNAME
|
|
UV_INDEX_{UPPER_NAME}_PASSWORD
|
|
```
|
|
|
|
Where `{UPPER_NAME}` is your index name converted to **uppercase** with **hyphens replaced by underscores**.
|
|
|
|
For example, an index named `my-private-registry` uses:
|
|
|
|
| Variable | Value |
|
|
|----------|-------|
|
|
| `UV_INDEX_MY_PRIVATE_REGISTRY_USERNAME` | Your registry username or token name |
|
|
| `UV_INDEX_MY_PRIVATE_REGISTRY_PASSWORD` | Your registry password or token/PAT |
|
|
|
|
<Warning>
|
|
These environment variables **must** be added via the CrewAI AMP **Environment Variables** settings —
|
|
either globally or at the deployment level. They cannot be set in `.env` files or hardcoded in your project.
|
|
|
|
See [Setting Environment Variables in AMP](#setting-environment-variables-in-amp) below.
|
|
</Warning>
|
|
|
|
## Registry Provider Reference
|
|
|
|
The table below shows the index URL format and credential values for common registry providers.
|
|
Replace placeholder values with your actual organization and feed details.
|
|
|
|
| Provider | Index URL | Username | Password |
|
|
|----------|-----------|----------|----------|
|
|
| **Azure DevOps Artifacts** | `https://pkgs.dev.azure.com/{org}/_packaging/{feed}/pypi/simple/` | Any non-empty string (e.g. `token`) | Personal Access Token (PAT) with Packaging Read scope |
|
|
| **GitHub Packages** | `https://pypi.pkg.github.com/{owner}/simple/` | GitHub username | Personal Access Token (classic) with `read:packages` scope |
|
|
| **GitLab Package Registry** | `https://gitlab.com/api/v4/projects/{project_id}/packages/pypi/simple/` | `__token__` | Project or Personal Access Token with `read_api` scope |
|
|
| **AWS CodeArtifact** | Use the URL from `aws codeartifact get-repository-endpoint` | `aws` | Token from `aws codeartifact get-authorization-token` |
|
|
| **Google Artifact Registry** | `https://{region}-python.pkg.dev/{project}/{repo}/simple/` | `_json_key_base64` | Base64-encoded service account key |
|
|
| **JFrog Artifactory** | `https://{instance}.jfrog.io/artifactory/api/pypi/{repo}/simple/` | Username or email | API key or identity token |
|
|
| **Self-hosted (devpi, Nexus, etc.)** | Your registry's simple API URL | Registry username | Registry password |
|
|
|
|
<Tip>
|
|
For **AWS CodeArtifact**, the authorization token expires periodically.
|
|
You'll need to refresh the `UV_INDEX_*_PASSWORD` value when it expires.
|
|
Consider automating this in your CI/CD pipeline.
|
|
</Tip>
|
|
|
|
## Setting Environment Variables in AMP
|
|
|
|
Private registry credentials must be configured as environment variables in CrewAI AMP.
|
|
You have two options:
|
|
|
|
<Tabs>
|
|
<Tab title="Web Interface">
|
|
1. Log in to [CrewAI AMP](https://app.crewai.com)
|
|
2. Navigate to your automation
|
|
3. Open the **Environment Variables** tab
|
|
4. Add each variable (`UV_INDEX_*_USERNAME` and `UV_INDEX_*_PASSWORD`) with its value
|
|
|
|
See the [Deploy to AMP — Set Environment Variables](/en/enterprise/guides/deploy-to-amp#set-environment-variables) step for details.
|
|
</Tab>
|
|
<Tab title="CLI Deployment">
|
|
Add the variables to your local `.env` file before running `crewai deploy create`.
|
|
The CLI will securely transfer them to the platform:
|
|
|
|
```bash
|
|
# .env
|
|
OPENAI_API_KEY=sk-...
|
|
UV_INDEX_MY_PRIVATE_REGISTRY_USERNAME=token
|
|
UV_INDEX_MY_PRIVATE_REGISTRY_PASSWORD=your-pat-here
|
|
```
|
|
|
|
```bash
|
|
crewai deploy create
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
<Warning>
|
|
**Never** commit credentials to your repository. Use AMP environment variables for all secrets.
|
|
The `.env` file should be listed in `.gitignore`.
|
|
</Warning>
|
|
|
|
To update credentials on an existing deployment, see [Update Your Crew — Environment Variables](/en/enterprise/guides/update-crew).
|
|
|
|
## How It All Fits Together
|
|
|
|
When CrewAI AMP builds your automation, the resolution flow works like this:
|
|
|
|
<Steps>
|
|
<Step title="Build starts">
|
|
AMP pulls your repository and reads `pyproject.toml` and `uv.lock`.
|
|
</Step>
|
|
<Step title="UV resolves dependencies">
|
|
UV reads `[tool.uv.sources]` to determine which index each package should come from.
|
|
</Step>
|
|
<Step title="UV authenticates">
|
|
For each private index, UV looks up `UV_INDEX_{NAME}_USERNAME` and `UV_INDEX_{NAME}_PASSWORD`
|
|
from the environment variables you configured in AMP.
|
|
</Step>
|
|
<Step title="Packages install">
|
|
UV downloads and installs all packages — both public (from PyPI) and private (from your registry).
|
|
</Step>
|
|
<Step title="Automation runs">
|
|
Your crew or flow starts with all dependencies available.
|
|
</Step>
|
|
</Steps>
|
|
|
|
## Troubleshooting
|
|
|
|
### Authentication Errors During Build
|
|
|
|
**Symptom**: Build fails with `401 Unauthorized` or `403 Forbidden` when resolving a private package.
|
|
|
|
**Check**:
|
|
- The `UV_INDEX_*` environment variable names match your index name exactly (uppercased, hyphens → underscores)
|
|
- Credentials are set in AMP environment variables, not just in a local `.env`
|
|
- Your token/PAT has the required read permissions for the package feed
|
|
- The token hasn't expired (especially relevant for AWS CodeArtifact)
|
|
|
|
### Package Not Found
|
|
|
|
**Symptom**: `No matching distribution found for my-private-package`.
|
|
|
|
**Check**:
|
|
- The index URL in `pyproject.toml` ends with `/simple/`
|
|
- The `[tool.uv.sources]` entry maps the correct package name to the correct index name
|
|
- The package is actually published to your private registry
|
|
- Run `uv lock` locally with the same credentials to verify resolution works
|
|
|
|
### Lock File Conflicts
|
|
|
|
**Symptom**: `uv lock` fails or produces unexpected results after adding a private index.
|
|
|
|
**Solution**: Set the credentials locally and regenerate:
|
|
|
|
```bash
|
|
export UV_INDEX_MY_PRIVATE_REGISTRY_USERNAME=token
|
|
export UV_INDEX_MY_PRIVATE_REGISTRY_PASSWORD=your-pat
|
|
uv lock
|
|
```
|
|
|
|
Then commit the updated `uv.lock`.
|
|
|
|
## Related Guides
|
|
|
|
<CardGroup cols={3}>
|
|
<Card title="Prepare for Deployment" icon="clipboard-check" href="/en/enterprise/guides/prepare-for-deployment">
|
|
Verify project structure and dependencies before deploying.
|
|
</Card>
|
|
<Card title="Deploy to AMP" icon="rocket" href="/en/enterprise/guides/deploy-to-amp">
|
|
Deploy your crew or flow and configure environment variables.
|
|
</Card>
|
|
<Card title="Update Your Crew" icon="arrows-rotate" href="/en/enterprise/guides/update-crew">
|
|
Update environment variables and push changes to a running deployment.
|
|
</Card>
|
|
</CardGroup>
|