mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-07-01 05:08:12 +00:00
feat: adopt directory-based docs versioning with Edge channel (#6202)
* feat: adopt directory-based docs versioning with Edge channel Switch docs.crewai.com from navigation-only versioning (every version selector entry rendered the same docs/<lang>/* source files) to Mintlify's directory-based versioning so each version selector entry renders its own snapshot. Add an "Edge" channel under docs/edge/<lang>/* that always reflects main HEAD for unreleased work, eliminating pre-release leakage onto frozen release labels. External links to canonical /<lang>/* URLs are preserved via wildcard redirects that always land on the current default version. Layout: - docs/edge/<lang>/* rolling source (you edit here) - docs/edge/enterprise-api.*.yaml - docs/v<X.Y.Z>/<lang>/* frozen, immutable snapshots - docs/v<X.Y.Z>/enterprise-api.*.yaml - docs/images/ shared, append-only - docs/docs.json nav + redirects URLs follow the Mintlify-idiomatic shape: /edge/<lang>/<page> for Edge, /v<X.Y.Z>/<lang>/<page> for every frozen snapshot. The wildcard redirects /<lang>/:slug* -> /<default>/<lang>/:slug* keep stale links working, and every freeze rewrites them (plus all per-section/per-page redirects) so destinations always resolve to the current default without depending on a second redirect hop. Release flow integration (devtools release): - New module crewai_devtools.docs_versioning.freeze() materialises docs/v<X.Y.Z>/ from docs/edge/, rewrites openapi: refs inside the snapshot, inserts the version into every language block in docs.json, and refreshes all redirect destinations. - _update_docs_and_create_pr() in cli.py now calls that freeze during Phase 2 of devtools release. Edge changelogs are updated first (so the snapshot freeze picks them up), then the snapshot is staged alongside docs.json, branched as docs/freeze-v<X.Y.Z>, and the PR is titled [docs-freeze] docs: snapshot and changelog for v<X.Y.Z> — the title prefix the new CI guard reads. - The PR still gates tag, GitHub release, PyPI publish, and the enterprise release as before; no new PRs are added. - Pre-releases (1.X.YaN, 1.X.YbN, ...) skip the snapshot — they ride Edge — and the docs PR title omits the [docs-freeze] prefix. - docs_check (AI-generated docs scaffolding) writes to docs/edge/<lang>/* so newly-generated unreleased docs land in Edge and never accidentally touch a frozen snapshot. Migration scripts (one-shot): - scripts/docs/freeze_historical_versions.py reconstructs all 16 historical snapshots (v1.10.0 .. v1.14.7) from git tags via git archive | tar, rewriting openapi: MDX refs so each snapshot reads its own enterprise-api YAML rather than the live one. - scripts/docs/prefix_version_paths.py one-shot-migrates docs.json: rewrites every page path in 16 versioned blocks to point under docs/v<X.Y.Z>/, inserts a new Edge entry per language, tags v1.14.7 as Latest (default), prunes pages whose target file doesn't exist in the snapshot (e.g. docs/ar/ didn't exist before v1.12.0), and writes the wildcard + per-section redirects. - scripts/docs/freeze_current_edge.py is now a thin CLI wrapper around docs_versioning.freeze for manual one-off freezes (e.g. retroactively snapshotting a forgotten release). CI guards (.github/workflows/docs-snapshots.yml): - Frozen snapshots under docs/v[0-9]*/ are immutable; only PRs whose title contains [docs-freeze] (i.e. release-cut PRs generated by devtools release or the manual wrapper) may modify them. - Images under docs/images/ are append-only since snapshots share a single image directory. Deleting or renaming an image breaks every historical snapshot that still references it. Restored docs/images/crewai-otel-export.png from PR #3673; it was deleted in PR #4908 but v1.10.0 / v1.10.1 snapshots still reference it. Restoring instead of editing the snapshots preserves historical rendering fidelity and validates the new append-only rule retroactively. Tests: - lib/devtools/tests/test_docs_versioning.py covers the freeze: file copy, openapi rewrite, version insertion, default demotion, redirect upserts, per-section redirect rewriting, idempotency, and invalid inputs. Verified locally with mintlify broken-links: 0 broken links across the full site (Edge + 16 frozen versions, 4 locales). AGENTS.md (repo root) is the contributor guide for the new model; RELEASING.md is the release-cut runbook; README's Contribution section links to both. Co-authored-by: Cursor <cursoragent@cursor.com> * style: resolve linter issues --------- Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Lucas Gomide
@@ -22,6 +22,11 @@ from rich.prompt import Confirm
|
||||
import tomlkit
|
||||
|
||||
from crewai_devtools.docs_check import docs_check
|
||||
from crewai_devtools.docs_versioning import (
|
||||
InvalidVersionError,
|
||||
MissingEdgeSourcesError,
|
||||
freeze as freeze_docs,
|
||||
)
|
||||
from crewai_devtools.prompts import RELEASE_NOTES_PROMPT, TRANSLATE_RELEASE_NOTES_PROMPT
|
||||
|
||||
|
||||
@@ -390,56 +395,39 @@ def update_pyproject_dependencies(
|
||||
|
||||
|
||||
def add_docs_version(docs_json_path: Path, version: str) -> bool:
|
||||
"""Add a new version to the Mintlify docs.json versioning config.
|
||||
"""Freeze Edge into a new snapshot and register the version in docs.json.
|
||||
|
||||
Copies the current default version's tabs into a new version entry,
|
||||
sets the new version as default, and marks the previous default as
|
||||
non-default. Operates on all languages.
|
||||
Thin compatibility wrapper around :func:`crewai_devtools.docs_versioning.freeze`.
|
||||
Materialises ``docs/v<version>/`` from ``docs/edge/`` (copies files, rewrites
|
||||
``openapi:`` refs inside the snapshot), inserts a new ``vX.Y.Z`` entry into
|
||||
every language's ``versions[]`` block just after Edge, marks it
|
||||
default + ``Latest`` (demoting the prior default), and updates the wildcard
|
||||
``/<lang>/:slug*`` redirects to point at the new version.
|
||||
|
||||
Skipped (returns False) for pre-release versions like ``1.10.1b1`` since
|
||||
those don't get their own snapshot — pre-release docs stay on Edge.
|
||||
|
||||
Args:
|
||||
docs_json_path: Path to docs/docs.json.
|
||||
version: Version string (e.g., "1.10.1b1").
|
||||
version: Version string (e.g., ``"1.10.1"``). Pre-releases are skipped.
|
||||
|
||||
Returns:
|
||||
True if docs.json was updated, False otherwise.
|
||||
True if docs.json was updated, False otherwise (missing file, missing
|
||||
Edge sources, pre-release, or snapshot already up to date).
|
||||
"""
|
||||
import json
|
||||
|
||||
if not docs_json_path.exists():
|
||||
return False
|
||||
|
||||
data = json.loads(docs_json_path.read_text())
|
||||
version_label = f"v{version}"
|
||||
updated = False
|
||||
|
||||
for lang in data.get("navigation", {}).get("languages", []):
|
||||
versions = lang.get("versions", [])
|
||||
if not versions:
|
||||
continue
|
||||
|
||||
if any(v.get("version") == version_label for v in versions):
|
||||
continue
|
||||
|
||||
default_version = next(
|
||||
(v for v in versions if v.get("default")),
|
||||
versions[0],
|
||||
)
|
||||
|
||||
new_version = {
|
||||
"version": version_label,
|
||||
"default": True,
|
||||
"tabs": default_version.get("tabs", []),
|
||||
}
|
||||
|
||||
default_version.pop("default", None)
|
||||
versions.insert(0, new_version)
|
||||
updated = True
|
||||
|
||||
if not updated:
|
||||
if _is_prerelease(version):
|
||||
return False
|
||||
|
||||
docs_json_path.write_text(json.dumps(data, indent=2, ensure_ascii=False) + "\n")
|
||||
return True
|
||||
docs_root = docs_json_path.parent
|
||||
try:
|
||||
result = freeze_docs(version, docs_root)
|
||||
except (InvalidVersionError, MissingEdgeSourcesError) as e:
|
||||
console.print(f"[yellow]Warning:[/yellow] {e}")
|
||||
return False
|
||||
|
||||
return result.docsjson_entries_inserted > 0 or result.redirects_upserted > 0
|
||||
|
||||
|
||||
ChangelogLang = Literal["en", "pt-BR", "ko", "ar"]
|
||||
@@ -1104,97 +1092,127 @@ def _update_docs_and_create_pr(
|
||||
is_prerelease: bool,
|
||||
dry_run: bool,
|
||||
) -> str | None:
|
||||
"""Update changelogs and docs version switcher, create PR if needed.
|
||||
"""Update Edge changelogs, freeze a snapshot, and open the docs PR.
|
||||
|
||||
For a stable release this freezes ``docs/edge/`` into ``docs/v<version>/``
|
||||
(after the Edge changelogs have been updated so the snapshot contains the
|
||||
new entry), updates ``docs/docs.json`` to register the new version and
|
||||
canonical-URL redirects, and opens a ``[docs-freeze]`` PR. The
|
||||
``docs-snapshots`` CI guard recognises that title prefix and allows the
|
||||
snapshot directory to land.
|
||||
|
||||
For a pre-release, only the Edge changelogs are touched (pre-releases don't
|
||||
get a frozen snapshot — they ride Edge), and the PR title omits the
|
||||
``[docs-freeze]`` prefix.
|
||||
|
||||
Returns:
|
||||
The docs branch name if a PR was created, None otherwise.
|
||||
"""
|
||||
docs_json_path = cwd / "docs" / "docs.json"
|
||||
edge_root = cwd / "docs" / "edge"
|
||||
snapshot_path = cwd / "docs" / f"v{version}"
|
||||
changelog_langs: list[ChangelogLang] = ["en", "pt-BR", "ko", "ar"]
|
||||
|
||||
if not dry_run:
|
||||
docs_files_staged: list[str] = []
|
||||
|
||||
if dry_run:
|
||||
for lang in changelog_langs:
|
||||
cl_path = cwd / "docs" / lang / "changelog.mdx"
|
||||
if lang == "en":
|
||||
notes_for_lang = release_notes
|
||||
else:
|
||||
console.print(f"[dim]Translating release notes to {lang}...[/dim]")
|
||||
notes_for_lang = translate_release_notes(
|
||||
release_notes, lang, openai_client
|
||||
)
|
||||
if update_changelog(cl_path, version, notes_for_lang, lang=lang):
|
||||
console.print(f"[green]✓[/green] Updated {cl_path.relative_to(cwd)}")
|
||||
docs_files_staged.append(str(cl_path))
|
||||
else:
|
||||
console.print(
|
||||
f"[yellow]Warning:[/yellow] Changelog not found at {cl_path.relative_to(cwd)}"
|
||||
)
|
||||
|
||||
cl_path = edge_root / lang / "changelog.mdx"
|
||||
translated = " (translated)" if lang != "en" else ""
|
||||
console.print(
|
||||
f"[dim][DRY RUN][/dim] Would update "
|
||||
f"{cl_path.relative_to(cwd)}{translated}"
|
||||
)
|
||||
if not is_prerelease:
|
||||
if add_docs_version(docs_json_path, version):
|
||||
console.print(
|
||||
f"[green]✓[/green] Added v{version} to docs version switcher"
|
||||
)
|
||||
docs_files_staged.append(str(docs_json_path))
|
||||
else:
|
||||
console.print(
|
||||
f"[yellow]Warning:[/yellow] docs.json not found at {docs_json_path.relative_to(cwd)}"
|
||||
)
|
||||
|
||||
if docs_files_staged:
|
||||
docs_branch = f"docs/changelog-v{version}"
|
||||
create_or_reset_branch(docs_branch)
|
||||
for f in docs_files_staged:
|
||||
run_command(["git", "add", f])
|
||||
run_command(
|
||||
[
|
||||
"git",
|
||||
"commit",
|
||||
"-m",
|
||||
f"docs: update changelog and version for v{version}",
|
||||
]
|
||||
console.print(
|
||||
f"[dim][DRY RUN][/dim] Would freeze docs/edge -> "
|
||||
f"{snapshot_path.relative_to(cwd)} and update docs.json + redirects"
|
||||
)
|
||||
console.print("[green]✓[/green] Committed docs updates")
|
||||
|
||||
run_command(["git", "push", "-u", "origin", docs_branch])
|
||||
console.print(f"[green]✓[/green] Pushed branch {docs_branch}")
|
||||
|
||||
pr_url = run_command(
|
||||
[
|
||||
"gh",
|
||||
"pr",
|
||||
"create",
|
||||
"--base",
|
||||
"main",
|
||||
"--title",
|
||||
f"docs: update changelog and version for v{version}",
|
||||
"--body",
|
||||
"",
|
||||
]
|
||||
else:
|
||||
console.print(
|
||||
"[dim][DRY RUN][/dim] Skipping snapshot freeze (pre-release stays on Edge)"
|
||||
)
|
||||
console.print("[green]✓[/green] Created docs PR")
|
||||
console.print(f"[cyan]PR URL:[/cyan] {pr_url}")
|
||||
return docs_branch
|
||||
|
||||
prefix = "" if is_prerelease else "[docs-freeze] "
|
||||
console.print(
|
||||
f"[dim][DRY RUN][/dim] Would create branch docs/freeze-v{version}, "
|
||||
f"open PR titled '{prefix}docs: snapshot and changelog for v{version}', "
|
||||
"and wait for merge"
|
||||
)
|
||||
return None
|
||||
|
||||
docs_paths_staged: list[str] = []
|
||||
|
||||
# Step 1: update Edge changelogs first so the snapshot we freeze afterwards
|
||||
# contains the new release's entry.
|
||||
for lang in changelog_langs:
|
||||
cl_path = cwd / "docs" / lang / "changelog.mdx"
|
||||
translated = " (translated)" if lang != "en" else ""
|
||||
console.print(
|
||||
f"[dim][DRY RUN][/dim] Would update {cl_path.relative_to(cwd)}{translated}"
|
||||
)
|
||||
cl_path = edge_root / lang / "changelog.mdx"
|
||||
if lang == "en":
|
||||
notes_for_lang = release_notes
|
||||
else:
|
||||
console.print(f"[dim]Translating release notes to {lang}...[/dim]")
|
||||
notes_for_lang = translate_release_notes(release_notes, lang, openai_client)
|
||||
if update_changelog(cl_path, version, notes_for_lang, lang=lang):
|
||||
console.print(f"[green]✓[/green] Updated {cl_path.relative_to(cwd)}")
|
||||
docs_paths_staged.append(str(cl_path))
|
||||
else:
|
||||
console.print(
|
||||
f"[yellow]Warning:[/yellow] Changelog not found at "
|
||||
f"{cl_path.relative_to(cwd)}"
|
||||
)
|
||||
|
||||
# Step 2: stable releases get a frozen snapshot + docs.json updates;
|
||||
# pre-releases ride Edge so we only need the changelog edits.
|
||||
is_freeze = False
|
||||
if not is_prerelease:
|
||||
console.print(
|
||||
f"[dim][DRY RUN][/dim] Would add v{version} to docs version switcher"
|
||||
)
|
||||
else:
|
||||
console.print("[dim][DRY RUN][/dim] Skipping docs version (pre-release)")
|
||||
console.print(
|
||||
f"[dim][DRY RUN][/dim] Would create branch docs/changelog-v{version}, PR, and wait for merge"
|
||||
if add_docs_version(docs_json_path, version):
|
||||
console.print(
|
||||
f"[green]✓[/green] Froze docs/edge -> "
|
||||
f"{snapshot_path.relative_to(cwd)} and updated docs.json + redirects"
|
||||
)
|
||||
docs_paths_staged.append(str(docs_json_path))
|
||||
docs_paths_staged.append(str(snapshot_path))
|
||||
is_freeze = True
|
||||
else:
|
||||
console.print(
|
||||
f"[yellow]Warning:[/yellow] docs freeze did not modify "
|
||||
f"{docs_json_path.relative_to(cwd)} "
|
||||
"(missing file, missing Edge sources, or snapshot already current)"
|
||||
)
|
||||
|
||||
if not docs_paths_staged:
|
||||
return None
|
||||
|
||||
docs_branch = f"docs/freeze-v{version}"
|
||||
create_or_reset_branch(docs_branch)
|
||||
for path in docs_paths_staged:
|
||||
run_command(["git", "add", path])
|
||||
|
||||
# The [docs-freeze] title prefix is what the docs-snapshots CI guard reads
|
||||
# to allow writes under docs/v*/ and image deletions. Omit it for
|
||||
# pre-releases since they don't touch frozen snapshots.
|
||||
title_prefix = "[docs-freeze] " if is_freeze else ""
|
||||
pr_title = f"{title_prefix}docs: snapshot and changelog for v{version}"
|
||||
|
||||
run_command(["git", "commit", "-m", pr_title])
|
||||
console.print("[green]✓[/green] Committed docs updates")
|
||||
|
||||
run_command(["git", "push", "-u", "origin", docs_branch])
|
||||
console.print(f"[green]✓[/green] Pushed branch {docs_branch}")
|
||||
|
||||
pr_url = run_command(
|
||||
[
|
||||
"gh",
|
||||
"pr",
|
||||
"create",
|
||||
"--base",
|
||||
"main",
|
||||
"--title",
|
||||
pr_title,
|
||||
"--body",
|
||||
"",
|
||||
]
|
||||
)
|
||||
return None
|
||||
console.print("[green]✓[/green] Created docs PR")
|
||||
console.print(f"[cyan]PR URL:[/cyan] {pr_url}")
|
||||
return docs_branch
|
||||
|
||||
|
||||
def _create_tag_and_release(
|
||||
|
||||
@@ -389,13 +389,18 @@ def docs_check(base: str, write: bool, dry_run: bool) -> None:
|
||||
)
|
||||
return
|
||||
|
||||
# Newly-generated docs are by definition unreleased, so they belong in the
|
||||
# Edge channel (docs/edge/<lang>/...). Frozen snapshots under docs/v<X.Y.Z>/
|
||||
# are immutable and CI rejects writes there outside [docs-freeze] PRs.
|
||||
edge_root = docs_dir / "edge"
|
||||
|
||||
for action_item in analysis.actions:
|
||||
if action_item.action not in ("create", "update") or not action_item.file:
|
||||
continue
|
||||
|
||||
rel_path = action_item.file
|
||||
en_path = (docs_dir / "en" / rel_path).resolve()
|
||||
if not en_path.is_relative_to(docs_dir.resolve()):
|
||||
en_path = (edge_root / "en" / rel_path).resolve()
|
||||
if not en_path.is_relative_to(edge_root.resolve()):
|
||||
console.print(f" [red]✗ Skipping unsafe path: {rel_path!r}[/red]")
|
||||
continue
|
||||
console.print(f"\n[bold]Processing:[/bold] {rel_path}")
|
||||
@@ -452,10 +457,10 @@ def docs_check(base: str, write: bool, dry_run: bool) -> None:
|
||||
if not content:
|
||||
continue
|
||||
|
||||
resolved_docs = docs_dir.resolve()
|
||||
resolved_edge = edge_root.resolve()
|
||||
for lang in _TRANSLATION_LANGS:
|
||||
lang_path = (docs_dir / lang / rel_path).resolve()
|
||||
if not lang_path.is_relative_to(resolved_docs):
|
||||
lang_path = (edge_root / lang / rel_path).resolve()
|
||||
if not lang_path.is_relative_to(resolved_edge):
|
||||
continue
|
||||
|
||||
with console.status(f" [cyan]Translating to {_LANGUAGE_NAMES[lang]}..."):
|
||||
|
||||
434
lib/devtools/src/crewai_devtools/docs_versioning.py
Normal file
434
lib/devtools/src/crewai_devtools/docs_versioning.py
Normal file
@@ -0,0 +1,434 @@
|
||||
"""Freeze the current Edge docs into a per-version snapshot.
|
||||
|
||||
Used by ``devtools release`` (and the standalone
|
||||
``scripts/docs/freeze_current_edge.py`` wrapper) during the docs PR step, which
|
||||
runs *before* the release tag is created and PyPI publish is triggered. Once
|
||||
the docs PR merges, the site reflects the new release at ``/v<X.Y.Z>/...`` and
|
||||
the canonical ``/<lang>/...`` URLs (kept stable for external links) start
|
||||
redirecting to the new default version.
|
||||
|
||||
Layout assumptions (set up by ``scripts/docs/prefix_version_paths.py``):
|
||||
|
||||
- ``docs/edge/`` rolling source matching main HEAD
|
||||
- ``en/``, ``pt-BR/``, ``ko/``, ``ar/``
|
||||
- ``enterprise-api.*.yaml``
|
||||
- ``docs/v<X.Y.Z>/`` frozen, immutable snapshots
|
||||
- ``docs/docs.json`` Mintlify config: ``navigation`` + ``redirects``
|
||||
|
||||
A freeze does four things:
|
||||
|
||||
1. Copy ``docs/edge/*`` into ``docs/v<X.Y.Z>/``.
|
||||
2. Rewrite ``openapi:`` MDX refs inside the snapshot to point at the snapshot's
|
||||
own ``enterprise-api.*.yaml`` (otherwise frozen pages would render against
|
||||
whatever YAML happens to be at docs root today).
|
||||
3. Insert a new version entry into every language's ``versions[]`` block in
|
||||
``docs.json``, place it just after Edge, mark it default + Latest, and demote
|
||||
the prior default.
|
||||
4. Update wildcard redirects in ``docs.json`` so ``/<lang>/:slug*`` lands on the
|
||||
new default version.
|
||||
|
||||
Idempotent: re-running with a version that already has a snapshot directory
|
||||
*and* a docs.json entry is a no-op.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
import copy
|
||||
from dataclasses import dataclass
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
from typing import Any, Final
|
||||
|
||||
|
||||
VERSION_RE: Final[re.Pattern[str]] = re.compile(r"^\d+\.\d+\.\d+$")
|
||||
VERSION_SLUG_RE: Final[re.Pattern[str]] = re.compile(r"^v\d+\.\d+\.\d+$")
|
||||
EDGE_VERSION: Final[str] = "Edge"
|
||||
EDGE_PREFIX: Final[str] = "edge"
|
||||
LATEST_TAG: Final[str] = "Latest"
|
||||
|
||||
KNOWN_LOCALES: Final[tuple[str, ...]] = ("en", "pt-BR", "ko", "ar")
|
||||
|
||||
# Per-snapshot copies are sourced from docs/edge/<name>. The frozen layout
|
||||
# under docs/v<tag>/ omits the ``edge/`` segment so its URLs are
|
||||
# ``/v<tag>/<lang>/...``.
|
||||
SNAPSHOT_PATHS: Final[tuple[str, ...]] = (
|
||||
"en",
|
||||
"pt-BR",
|
||||
"ko",
|
||||
"ar",
|
||||
"enterprise-api.base.yaml",
|
||||
"enterprise-api.en.yaml",
|
||||
"enterprise-api.ko.yaml",
|
||||
"enterprise-api.pt-BR.yaml",
|
||||
)
|
||||
|
||||
PAGE_EXTENSIONS: Final[tuple[str, ...]] = (".mdx", ".md")
|
||||
|
||||
# Matches ``openapi: "/enterprise-api.en.yaml ..."``. The snapshot version of
|
||||
# the MDX needs the path prefixed with ``v<tag>/`` so the page reads the
|
||||
# frozen YAML rather than whichever YAML happens to live at docs root.
|
||||
_OPENAPI_PATTERN: Final[re.Pattern[str]] = re.compile(
|
||||
r'(openapi:\s*"\s*)/(enterprise-api\.[^"\s]+\.yaml)'
|
||||
)
|
||||
|
||||
|
||||
class InvalidVersionError(ValueError):
|
||||
"""Raised when a freeze is requested with a non-X.Y.Z version string."""
|
||||
|
||||
|
||||
class MissingEdgeSourcesError(RuntimeError):
|
||||
"""Raised when ``docs/edge/`` is missing or has no snapshot paths."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FreezeResult:
|
||||
"""Structured outcome of a freeze, for callers that render their own UI."""
|
||||
|
||||
version_slug: str
|
||||
snapshot_path: Path
|
||||
files_copied: int
|
||||
openapi_refs_rewritten: int
|
||||
docsjson_entries_inserted: int
|
||||
docsjson_entries_skipped: int
|
||||
redirects_upserted: int
|
||||
snapshot_already_existed: bool
|
||||
|
||||
|
||||
def freeze(version: str, docs_root: Path) -> FreezeResult:
|
||||
"""Freeze the current Edge into ``docs/v<version>/`` and update docs.json.
|
||||
|
||||
Args:
|
||||
version: Release version as ``"X.Y.Z"`` (no leading ``v``).
|
||||
docs_root: Path to the ``docs/`` directory.
|
||||
|
||||
Returns:
|
||||
``FreezeResult`` summarising what changed.
|
||||
|
||||
Raises:
|
||||
InvalidVersionError: ``version`` is not an X.Y.Z string.
|
||||
MissingEdgeSourcesError: ``docs/edge/`` is absent or empty.
|
||||
"""
|
||||
if not VERSION_RE.match(version):
|
||||
raise InvalidVersionError(f"{version!r} is not a valid X.Y.Z version string")
|
||||
|
||||
version_slug = f"v{version}"
|
||||
target = docs_root / version_slug
|
||||
docs_json = docs_root / "docs.json"
|
||||
|
||||
snapshot_already_existed = target.exists()
|
||||
if snapshot_already_existed:
|
||||
files_copied = 0
|
||||
openapi_refs_rewritten = 0
|
||||
else:
|
||||
files_copied = _copy_snapshot(docs_root, target)
|
||||
openapi_refs_rewritten = _rewrite_openapi_refs(target, version_slug)
|
||||
|
||||
inserted, skipped, redirects_upserted = _migrate_docs_json(docs_json, version_slug)
|
||||
|
||||
return FreezeResult(
|
||||
version_slug=version_slug,
|
||||
snapshot_path=target,
|
||||
files_copied=files_copied,
|
||||
openapi_refs_rewritten=openapi_refs_rewritten,
|
||||
docsjson_entries_inserted=inserted,
|
||||
docsjson_entries_skipped=skipped,
|
||||
redirects_upserted=redirects_upserted,
|
||||
snapshot_already_existed=snapshot_already_existed,
|
||||
)
|
||||
|
||||
|
||||
def _copy_snapshot(docs_root: Path, target: Path) -> int:
|
||||
"""Copy Edge sources under ``docs/edge/`` into ``target``.
|
||||
|
||||
Returns the number of files copied (recursively across directories).
|
||||
"""
|
||||
edge_root = docs_root / EDGE_PREFIX
|
||||
if not edge_root.is_dir():
|
||||
raise MissingEdgeSourcesError(
|
||||
f"Expected Edge sources under {edge_root}/. "
|
||||
"Did you forget to migrate Edge into docs/edge/?"
|
||||
)
|
||||
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
count = 0
|
||||
for name in SNAPSHOT_PATHS:
|
||||
src = edge_root / name
|
||||
if not src.exists():
|
||||
continue
|
||||
dst = target / name
|
||||
if src.is_dir():
|
||||
shutil.copytree(src, dst)
|
||||
count += sum(1 for p in dst.rglob("*") if p.is_file())
|
||||
else:
|
||||
shutil.copy2(src, dst)
|
||||
count += 1
|
||||
|
||||
if count == 0:
|
||||
raise MissingEdgeSourcesError(
|
||||
f"docs/edge/ exists but contains none of {list(SNAPSHOT_PATHS)}"
|
||||
)
|
||||
return count
|
||||
|
||||
|
||||
def _rewrite_openapi_refs(target: Path, version_slug: str) -> int:
|
||||
"""Prefix every ``openapi:`` reference in the snapshot with the version."""
|
||||
rewritten = 0
|
||||
for mdx in target.rglob("*.mdx"):
|
||||
text = mdx.read_text(encoding="utf-8")
|
||||
new_text, n = _OPENAPI_PATTERN.subn(rf"\1/{version_slug}/\2", text)
|
||||
if n:
|
||||
mdx.write_text(new_text, encoding="utf-8")
|
||||
rewritten += n
|
||||
return rewritten
|
||||
|
||||
|
||||
def _walk_pages(node: Any, transform: Callable[[str], str]) -> Any:
|
||||
"""Recursively walk a nav subtree, applying ``transform`` to every leaf."""
|
||||
if isinstance(node, str):
|
||||
return transform(node)
|
||||
if isinstance(node, list):
|
||||
return [_walk_pages(item, transform) for item in node]
|
||||
if isinstance(node, dict):
|
||||
out = dict(node)
|
||||
for key in ("pages", "tabs", "groups"):
|
||||
if key in out:
|
||||
out[key] = [_walk_pages(c, transform) for c in out[key]]
|
||||
return out
|
||||
return node
|
||||
|
||||
|
||||
def _is_version_slug(value: str) -> bool:
|
||||
return bool(VERSION_SLUG_RE.match(value))
|
||||
|
||||
|
||||
def _previous_default(versions: list[dict[str, Any]]) -> dict[str, Any] | None:
|
||||
"""Return the entry currently marked default (or the first versioned)."""
|
||||
for v in versions:
|
||||
if v.get("default") and _is_version_slug(v.get("version", "")):
|
||||
return v
|
||||
for v in versions:
|
||||
if _is_version_slug(v.get("version", "")):
|
||||
return v
|
||||
return None
|
||||
|
||||
|
||||
def _build_new_entry(
|
||||
previous: dict[str, Any], version_slug: str, locale: str, docs_root: Path
|
||||
) -> dict[str, Any] | None:
|
||||
"""Clone the previous default's nav into a new entry for ``version_slug``.
|
||||
|
||||
Page paths are rewritten from ``v<prev>/<locale>/...`` to
|
||||
``v<new>/<locale>/...``. Paths that don't resolve to a file in the
|
||||
snapshot are pruned and the now-empty groups/tabs cascade away. Returns
|
||||
``None`` if the locale has no resolvable content under the snapshot (e.g.
|
||||
a locale that wasn't present in Edge yet).
|
||||
"""
|
||||
new_entry = copy.deepcopy(previous)
|
||||
new_entry["version"] = version_slug
|
||||
new_entry["default"] = True
|
||||
new_entry["tag"] = LATEST_TAG
|
||||
|
||||
old_prefix = re.compile(rf"^{re.escape(previous['version'])}/")
|
||||
locale_prefix = f"{locale}/"
|
||||
new_prefix = f"{version_slug}/"
|
||||
|
||||
def transform(page: str) -> str:
|
||||
if page.startswith(new_prefix):
|
||||
return page
|
||||
rewritten = old_prefix.sub(new_prefix, page)
|
||||
if rewritten != page:
|
||||
return rewritten
|
||||
if page.startswith(locale_prefix):
|
||||
return f"{new_prefix}{page}"
|
||||
return page
|
||||
|
||||
rewritten = _walk_pages(new_entry, transform)
|
||||
pruned = _prune_missing_pages(rewritten, docs_root)
|
||||
# ``_prune_missing_pages`` recurses across str/list/dict, so its return
|
||||
# type is the union of those. We always call it with a dict entry, so we
|
||||
# narrow back to ``dict`` here to satisfy the typed signature.
|
||||
if not isinstance(pruned, dict) or not pruned.get("tabs"):
|
||||
return None
|
||||
return pruned
|
||||
|
||||
|
||||
def _prune_missing_pages(node: Any, docs_root: Path) -> Any:
|
||||
"""Drop pages whose target file is missing; cascade-empty groups/tabs."""
|
||||
if isinstance(node, str):
|
||||
for ext in PAGE_EXTENSIONS:
|
||||
if (docs_root / f"{node}{ext}").is_file():
|
||||
return node
|
||||
return None
|
||||
|
||||
if isinstance(node, list):
|
||||
kept = [_prune_missing_pages(item, docs_root) for item in node]
|
||||
return [k for k in kept if k is not None]
|
||||
|
||||
if isinstance(node, dict):
|
||||
out: dict[str, Any] = {}
|
||||
for key, value in node.items():
|
||||
if key in {"pages", "tabs", "groups"}:
|
||||
pruned = _prune_missing_pages(value, docs_root)
|
||||
if pruned:
|
||||
out[key] = pruned
|
||||
else:
|
||||
out[key] = value
|
||||
if "pages" in node and not out.get("pages"):
|
||||
return None
|
||||
if "groups" in node and not out.get("groups"):
|
||||
return None
|
||||
if "tabs" in node and not out.get("tabs"):
|
||||
return None
|
||||
return out
|
||||
|
||||
return node
|
||||
|
||||
|
||||
def _drop_latest_marker(entry: dict[str, Any]) -> dict[str, Any]:
|
||||
out = dict(entry)
|
||||
out.pop("default", None)
|
||||
if out.get("tag") == LATEST_TAG:
|
||||
out.pop("tag")
|
||||
return out
|
||||
|
||||
|
||||
def _update_redirects(data: dict[str, Any], version_slug: str) -> int:
|
||||
"""Make every redirect destination land on the current default version.
|
||||
|
||||
Two passes:
|
||||
|
||||
1. Upsert the wildcard ``/<locale>/:slug*`` -> ``/<version_slug>/<locale>/:slug*``
|
||||
entries so stale canonical URLs (``/en/...``, ``/ko/...``, etc.) keep
|
||||
resolving.
|
||||
2. Rewrite the destination of any pre-existing redirect that lands on a
|
||||
bare ``/<locale>/...`` or stale ``/v<old>/<locale>/...`` path so it
|
||||
lands on the current default version directly. Mintlify's link checker
|
||||
resolves each redirect independently and does not chain through them,
|
||||
so a destination that depends on a second redirect counts as broken.
|
||||
|
||||
Returns the number of redirect entries that were inserted or modified.
|
||||
"""
|
||||
redirects = data.setdefault("redirects", [])
|
||||
if not isinstance(redirects, list):
|
||||
raise RuntimeError("docs.json 'redirects' is not a list")
|
||||
|
||||
upserted = 0
|
||||
|
||||
for locale in KNOWN_LOCALES:
|
||||
source = f"/{locale}/:slug*"
|
||||
destination = f"/{version_slug}/{locale}/:slug*"
|
||||
existing = next(
|
||||
(r for r in redirects if isinstance(r, dict) and r.get("source") == source),
|
||||
None,
|
||||
)
|
||||
if existing is None:
|
||||
redirects.append(
|
||||
{"source": source, "destination": destination, "permanent": False}
|
||||
)
|
||||
upserted += 1
|
||||
elif existing.get("destination") != destination:
|
||||
existing["destination"] = destination
|
||||
existing["permanent"] = False
|
||||
upserted += 1
|
||||
|
||||
for entry in redirects:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
existing_destination = entry.get("destination")
|
||||
if not isinstance(existing_destination, str):
|
||||
continue
|
||||
new_destination = _rewrite_destination_to_version(
|
||||
existing_destination, version_slug
|
||||
)
|
||||
if new_destination != existing_destination:
|
||||
entry["destination"] = new_destination
|
||||
upserted += 1
|
||||
|
||||
return upserted
|
||||
|
||||
|
||||
def _rewrite_destination_to_version(destination: str, version_slug: str) -> str:
|
||||
"""Rewrite a redirect destination to land on ``version_slug`` directly.
|
||||
|
||||
Handles three shapes:
|
||||
|
||||
- ``/<locale>/...`` -> ``/<version_slug>/<locale>/...``
|
||||
- ``/v<X.Y.Z>/<locale>/...`` -> ``/<version_slug>/<locale>/...``
|
||||
- anything else -> unchanged
|
||||
"""
|
||||
if not destination.startswith("/"):
|
||||
return destination
|
||||
|
||||
parts = destination.lstrip("/").split("/", 2)
|
||||
if not parts:
|
||||
return destination
|
||||
|
||||
head = parts[0]
|
||||
|
||||
if head in KNOWN_LOCALES:
|
||||
return f"/{version_slug}/{destination.lstrip('/')}"
|
||||
|
||||
if VERSION_SLUG_RE.match(head) and len(parts) >= 2 and parts[1] in KNOWN_LOCALES:
|
||||
if head == version_slug:
|
||||
return destination
|
||||
rest = "/".join(parts[1:])
|
||||
return f"/{version_slug}/{rest}"
|
||||
|
||||
return destination
|
||||
|
||||
|
||||
def _migrate_docs_json(docs_json: Path, version_slug: str) -> tuple[int, int, int]:
|
||||
"""Insert a new versioned entry per language and refresh redirects."""
|
||||
data = json.loads(docs_json.read_text(encoding="utf-8"))
|
||||
docs_root = docs_json.parent
|
||||
|
||||
inserted = 0
|
||||
skipped = 0
|
||||
for block in data["navigation"]["languages"]:
|
||||
locale = block["language"]
|
||||
versions: list[dict[str, Any]] = block.get("versions", [])
|
||||
if any(v.get("version") == version_slug for v in versions):
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
previous = _previous_default(versions)
|
||||
if previous is None:
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
new_entry = _build_new_entry(previous, version_slug, locale, docs_root)
|
||||
if new_entry is None:
|
||||
# Locale has no resolvable content under the snapshot yet (e.g. a
|
||||
# locale that didn't exist in Edge). Leave the block untouched.
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
updated: list[dict[str, Any]] = []
|
||||
for v in versions:
|
||||
if v.get("default") or v.get("tag") == LATEST_TAG:
|
||||
updated.append(_drop_latest_marker(v))
|
||||
else:
|
||||
updated.append(v)
|
||||
|
||||
# Insert the new versioned entry just after Edge so the version selector
|
||||
# shows: Edge, vNEW (default/Latest), vPREV, vPREV-1, ...
|
||||
edge_idx = next(
|
||||
(i for i, v in enumerate(updated) if v.get("version") == EDGE_VERSION),
|
||||
-1,
|
||||
)
|
||||
insert_at = edge_idx + 1 if edge_idx >= 0 else 0
|
||||
updated.insert(insert_at, new_entry)
|
||||
block["versions"] = updated
|
||||
inserted += 1
|
||||
|
||||
redirects_upserted = _update_redirects(data, version_slug)
|
||||
|
||||
docs_json.write_text(
|
||||
json.dumps(data, indent=2, ensure_ascii=False) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
return inserted, skipped, redirects_upserted
|
||||
Reference in New Issue
Block a user