mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-07-02 13:48:09 +00:00
feat: adopt directory-based docs versioning with Edge channel
Switch docs.crewai.com from navigation-only versioning (every version selector entry rendered the same docs/<lang>/* source files) to Mintlify's directory-based versioning so each version selector entry renders its own snapshot. Add an "Edge" channel under docs/edge/<lang>/* that always reflects main HEAD for unreleased work, eliminating pre-release leakage onto frozen release labels. External links to canonical /<lang>/* URLs are preserved via wildcard redirects that always land on the current default version. Layout: - docs/edge/<lang>/* rolling source (you edit here) - docs/edge/enterprise-api.*.yaml - docs/v<X.Y.Z>/<lang>/* frozen, immutable snapshots - docs/v<X.Y.Z>/enterprise-api.*.yaml - docs/images/ shared, append-only - docs/docs.json nav + redirects URLs follow the Mintlify-idiomatic shape: /edge/<lang>/<page> for Edge, /v<X.Y.Z>/<lang>/<page> for every frozen snapshot. The wildcard redirects /<lang>/:slug* -> /<default>/<lang>/:slug* keep stale links working, and every freeze rewrites them (plus all per-section/per-page redirects) so destinations always resolve to the current default without depending on a second redirect hop. Release flow integration (devtools release): - New module crewai_devtools.docs_versioning.freeze() materialises docs/v<X.Y.Z>/ from docs/edge/, rewrites openapi: refs inside the snapshot, inserts the version into every language block in docs.json, and refreshes all redirect destinations. - _update_docs_and_create_pr() in cli.py now calls that freeze during Phase 2 of devtools release. Edge changelogs are updated first (so the snapshot freeze picks them up), then the snapshot is staged alongside docs.json, branched as docs/freeze-v<X.Y.Z>, and the PR is titled [docs-freeze] docs: snapshot and changelog for v<X.Y.Z> — the title prefix the new CI guard reads. - The PR still gates tag, GitHub release, PyPI publish, and the enterprise release as before; no new PRs are added. - Pre-releases (1.X.YaN, 1.X.YbN, ...) skip the snapshot — they ride Edge — and the docs PR title omits the [docs-freeze] prefix. - docs_check (AI-generated docs scaffolding) writes to docs/edge/<lang>/* so newly-generated unreleased docs land in Edge and never accidentally touch a frozen snapshot. Migration scripts (one-shot): - scripts/docs/freeze_historical_versions.py reconstructs all 16 historical snapshots (v1.10.0 .. v1.14.7) from git tags via git archive | tar, rewriting openapi: MDX refs so each snapshot reads its own enterprise-api YAML rather than the live one. - scripts/docs/prefix_version_paths.py one-shot-migrates docs.json: rewrites every page path in 16 versioned blocks to point under docs/v<X.Y.Z>/, inserts a new Edge entry per language, tags v1.14.7 as Latest (default), prunes pages whose target file doesn't exist in the snapshot (e.g. docs/ar/ didn't exist before v1.12.0), and writes the wildcard + per-section redirects. - scripts/docs/freeze_current_edge.py is now a thin CLI wrapper around docs_versioning.freeze for manual one-off freezes (e.g. retroactively snapshotting a forgotten release). CI guards (.github/workflows/docs-snapshots.yml): - Frozen snapshots under docs/v[0-9]*/ are immutable; only PRs whose title contains [docs-freeze] (i.e. release-cut PRs generated by devtools release or the manual wrapper) may modify them. - Images under docs/images/ are append-only since snapshots share a single image directory. Deleting or renaming an image breaks every historical snapshot that still references it. Restored docs/images/crewai-otel-export.png from PR #3673; it was deleted in PR #4908 but v1.10.0 / v1.10.1 snapshots still reference it. Restoring instead of editing the snapshots preserves historical rendering fidelity and validates the new append-only rule retroactively. Tests: - lib/devtools/tests/test_docs_versioning.py covers the freeze: file copy, openapi rewrite, version insertion, default demotion, redirect upserts, per-section redirect rewriting, idempotency, and invalid inputs. Verified locally with mintlify broken-links: 0 broken links across the full site (Edge + 16 frozen versions, 4 locales). AGENTS.md (repo root) is the contributor guide for the new model; RELEASING.md is the release-cut runbook; README's Contribution section links to both. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
184
scripts/docs/freeze_historical_versions.py
Normal file
184
scripts/docs/freeze_historical_versions.py
Normal file
@@ -0,0 +1,184 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Freeze historical doc versions from git tags.
|
||||
|
||||
For each release tag listed in ``HISTORICAL_TAGS`` this script extracts the
|
||||
``docs/en``, ``docs/pt-BR``, ``docs/ko``, ``docs/ar`` directories and the
|
||||
``docs/enterprise-api.*.yaml`` files at that tag and writes them under
|
||||
``docs/v<tag>/``. Files that did not yet exist at a given tag are silently
|
||||
skipped (older tags simply produce smaller snapshots).
|
||||
|
||||
Top-level ``docs/v<tag>/`` folders are the Mintlify-idiomatic layout: the
|
||||
folder name appears verbatim in the URL (``/v1.14.7/en/concepts/agents``),
|
||||
matching the official versioning examples.
|
||||
|
||||
Idempotent: if ``docs/v<tag>/`` already exists the tag is skipped unless
|
||||
``--force`` is passed.
|
||||
|
||||
Usage::
|
||||
|
||||
python scripts/docs/freeze_historical_versions.py
|
||||
python scripts/docs/freeze_historical_versions.py --tag 1.14.7
|
||||
python scripts/docs/freeze_historical_versions.py --force
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
|
||||
HISTORICAL_TAGS: list[str] = [
|
||||
"1.10.0",
|
||||
"1.10.1",
|
||||
"1.11.0",
|
||||
"1.11.1",
|
||||
"1.12.0",
|
||||
"1.12.1",
|
||||
"1.12.2",
|
||||
"1.13.0",
|
||||
"1.14.0",
|
||||
"1.14.1",
|
||||
"1.14.2",
|
||||
"1.14.3",
|
||||
"1.14.4",
|
||||
"1.14.5",
|
||||
"1.14.6",
|
||||
"1.14.7",
|
||||
]
|
||||
|
||||
SNAPSHOT_PATHS: list[str] = [
|
||||
"docs/en",
|
||||
"docs/pt-BR",
|
||||
"docs/ko",
|
||||
"docs/ar",
|
||||
"docs/enterprise-api.base.yaml",
|
||||
"docs/enterprise-api.en.yaml",
|
||||
"docs/enterprise-api.ko.yaml",
|
||||
"docs/enterprise-api.pt-BR.yaml",
|
||||
]
|
||||
|
||||
|
||||
def _repo_root() -> Path:
|
||||
out = subprocess.run(
|
||||
["git", "rev-parse", "--show-toplevel"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout.strip()
|
||||
return Path(out)
|
||||
|
||||
|
||||
def _tag_exists(tag: str) -> bool:
|
||||
rc = subprocess.run(
|
||||
["git", "rev-parse", "--verify", f"refs/tags/{tag}"],
|
||||
capture_output=True,
|
||||
).returncode
|
||||
return rc == 0
|
||||
|
||||
|
||||
def _paths_present_at_tag(tag: str, paths: list[str]) -> list[str]:
|
||||
present: list[str] = []
|
||||
for path in paths:
|
||||
rc = subprocess.run(
|
||||
["git", "cat-file", "-e", f"{tag}:{path}"],
|
||||
capture_output=True,
|
||||
).returncode
|
||||
if rc == 0:
|
||||
present.append(path)
|
||||
return present
|
||||
|
||||
|
||||
def freeze_version(tag: str, *, force: bool = False) -> None:
|
||||
root = _repo_root()
|
||||
target = root / "docs" / f"v{tag}"
|
||||
|
||||
if target.exists():
|
||||
if not force:
|
||||
print(f" skip v{tag} (already frozen at docs/v{tag}/)")
|
||||
return
|
||||
shutil.rmtree(target)
|
||||
|
||||
if not _tag_exists(tag):
|
||||
print(f" WARN tag {tag} not found, skipping", file=sys.stderr)
|
||||
return
|
||||
|
||||
paths = _paths_present_at_tag(tag, SNAPSHOT_PATHS)
|
||||
if not paths:
|
||||
print(f" WARN no snapshot paths exist at tag {tag}, skipping", file=sys.stderr)
|
||||
return
|
||||
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# git archive emits paths verbatim (e.g. docs/en/concepts/agents.mdx).
|
||||
# tar --strip-components=1 removes the leading `docs/` segment so the
|
||||
# extracted layout under `target` matches `docs/versions/v<tag>/en/...`.
|
||||
archive = subprocess.Popen(
|
||||
["git", "archive", "--format=tar", tag, *paths],
|
||||
cwd=root,
|
||||
stdout=subprocess.PIPE,
|
||||
)
|
||||
untar = subprocess.Popen(
|
||||
["tar", "-x", "--strip-components=1", "-C", str(target)],
|
||||
stdin=archive.stdout,
|
||||
)
|
||||
assert archive.stdout is not None
|
||||
archive.stdout.close()
|
||||
untar_rc = untar.wait()
|
||||
archive_rc = archive.wait()
|
||||
if archive_rc != 0 or untar_rc != 0:
|
||||
raise RuntimeError(
|
||||
f"git archive {tag} failed (archive_rc={archive_rc}, tar_rc={untar_rc})"
|
||||
)
|
||||
|
||||
_rewrite_openapi_refs(target, tag)
|
||||
|
||||
file_count = sum(1 for p in target.rglob("*") if p.is_file())
|
||||
print(f" froze v{tag} -> docs/v{tag}/ ({file_count} files)")
|
||||
|
||||
|
||||
# API Reference MDX files reference the OpenAPI spec via an absolute docs-site
|
||||
# path (e.g. ``openapi: "/enterprise-api.en.yaml GET /foo"``). When a page is
|
||||
# served from a snapshot we need that path to point at the snapshot's own copy
|
||||
# of the YAML, otherwise every frozen version would render against the latest
|
||||
# spec.
|
||||
_OPENAPI_PATTERN = re.compile(r'(openapi:\s*"\s*)/(enterprise-api\.[^"\s]+\.yaml)')
|
||||
|
||||
|
||||
def _rewrite_openapi_refs(target: Path, tag: str) -> None:
|
||||
prefix = f"v{tag}"
|
||||
for mdx in target.rglob("*.mdx"):
|
||||
text = mdx.read_text(encoding="utf-8")
|
||||
new_text, n = _OPENAPI_PATTERN.subn(rf'\1/{prefix}/\2', text)
|
||||
if n:
|
||||
mdx.write_text(new_text, encoding="utf-8")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"--tag",
|
||||
action="append",
|
||||
default=None,
|
||||
help="Limit to a specific tag (repeatable). Default: all historical tags.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--force",
|
||||
action="store_true",
|
||||
help="Overwrite existing snapshot directories.",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
tags = args.tag or HISTORICAL_TAGS
|
||||
print(f"Freezing {len(tags)} historical version(s)...")
|
||||
for tag in tags:
|
||||
freeze_version(tag, force=args.force)
|
||||
print("Done.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user