Compare commits

..

4 Commits

Author SHA1 Message Date
Lorenze Jay
cf6c32e79b Merge branch 'main' into lorenze/imp/issue-driven-pull-requests 2026-08-04 12:32:26 -07:00
lorenzejay
66ce0c536c docs: define issue-first contribution workflow 2026-08-04 10:48:49 -07:00
Vidit Ostwal
7accafbaf4 chore(ci): group uv patch/minor Dependabot updates (#6807)
Some checks are pending
Build uv cache / build-cache (3.10) (push) Waiting to run
Build uv cache / build-cache (3.11) (push) Waiting to run
Build uv cache / build-cache (3.12) (push) Waiting to run
Build uv cache / build-cache (3.13) (push) Waiting to run
CodeQL Advanced / Analyze (actions) (push) Waiting to run
CodeQL Advanced / Analyze (python) (push) Waiting to run
Vulnerability Scan / Detect changes (push) Waiting to run
Vulnerability Scan / pip-audit (push) Blocked by required conditions
Add a patch-minor-updates group for routine version bumps while keeping
the existing security-updates grouping. Ignore semver-major updates so
breaking upgrades stay manual.
2026-08-04 23:04:20 +05:30
Lucas Gomide
9d659a644b feat: track interception-hook dispatches in telemetry (#6805)
`HookDispatchedEvent` was already emitted from the dispatcher but never
landed in Feature Usage. Wire it through `hook_dispatched_span` so hook
adoption and abort outcomes (e.g. policy checks) show up in the same
ClickHouse aggregation as other features.
2026-08-04 13:13:24 -04:00
17 changed files with 1179 additions and 11 deletions

View File

@@ -42,6 +42,40 @@ Documentation lives in `docs/` with translations under `docs/{en,ar,ko,pt-BR}/`.
## Development Workflow
### Issue-First Contributions
CrewAI is piloting an issue-first contribution workflow. The issue is where
contributors and maintainers agree on the problem and intended solution before
implementation begins.
Issues move through the following lifecycle:
```
state:inbox -> state:design -> state:ready -> state:in-progress -> state:verification -> Done
```
Maintainers apply `state:ready` after the issue has a clear problem statement,
agreed scope and non-goals, acceptance criteria, and a verification plan. A
pull request should not begin before its issue is ready unless a maintainer has
approved an exception.
Every new pull request must reference exactly one ready issue in its description:
```
Implements #123
```
Use `Implements`, not `Closes`, `Fixes`, or `Resolves`. GitHub's closing
keywords close the issue as soon as the pull request merges, but issue-first
contributions remain open in `state:verification` until a human confirms the
result.
The issue gate currently runs in **observe mode**. It reports whether a pull
request would satisfy this policy, but it does not block or close pull requests.
Maintainers can apply `issue-gate:override` for security work, release
automation, urgent fixes, or another documented exception. Pull requests that
predate the pilot can be marked `policy:legacy`.
### Branching
Create a branch off `main` using the conventional commit type:
@@ -103,7 +137,9 @@ chore(deps): bump pydantic to 2.11
- Keep PRs focused — avoid bundling unrelated changes
- PRs over 500 lines are labeled `size/XL` automatically
- Title must follow the same conventional commit format
- Link related issues where applicable
- Reference exactly one ready issue with `Implements #<issue-number>`
- Explain how the implementation meets the issue's acceptance criteria
- Include the automated and manual verification performed
## Testing

56
.github/ISSUE_GATE.md vendored Normal file
View File

@@ -0,0 +1,56 @@
# Issue Gate Pilot
The issue gate checks whether a pull request references exactly one open issue
with the `state:ready` label. It runs on `pull_request_target`, checks out only
the trusted default branch, and never fetches or executes pull request code.
## Repository setup
Create these labels before activating the pilot:
- Issue lifecycle: `state:inbox`, `state:design`, `state:ready`,
`state:in-progress`, and `state:verification`
- Gate results: `issue-gate:passed`, `issue-gate:exempt`, and
`needs-ready-issue`
- Maintainer exceptions: `issue-gate:override` and `policy:legacy`
Configure these repository variables:
| Variable | Required | Value |
| --- | --- | --- |
| `ISSUE_GATE_MODE` | No | `observe` (default), `block`, or `close` |
| `ISSUE_GATE_CUTOFF` | Yes to activate | ISO 8601 timestamp for the first PR covered by the pilot |
| `ISSUE_GATE_READY_LABEL` | No | Defaults to `state:ready` |
Without `ISSUE_GATE_CUTOFF`, the workflow uses a future cutoff and treats every
pull request as legacy. This makes the workflow inert until maintainers choose
the activation time.
## Modes
- `observe`: Invalid pull requests receive a successful `Issue gate` status,
an explanatory comment, and the `needs-ready-issue` label.
- `block`: Invalid pull requests receive a failing status. Add `Issue gate` to
the `main` ruleset's required checks only after observation is complete.
- `close`: Invalid pull requests receive a failing status and are closed after
the comment is posted.
The script refuses to enter `block` or `close` mode without a configured cutoff.
Pull requests created by supported dependency automation, or labeled
`issue-gate:override` or `policy:legacy`, remain exempt.
## Testing the pilot
Run the `Issue Gate` workflow manually with a pull request number, or edit a
pull request description to trigger it again. Test at least these cases before
changing modes:
1. No implementation issue
2. A nonexistent, closed, or non-ready issue
3. More than one implementation issue
4. One open issue labeled `state:ready`
5. A pull request before the cutoff
6. A pull request with `issue-gate:override`
Keep the gate in `observe` mode until the sample contains at least 20 new pull
requests with no false positives.

View File

@@ -1,7 +1,7 @@
name: Bug report
description: Create a report to help us improve CrewAI
title: "[BUG]"
labels: ["bug"]
labels: ["bug", "state:inbox"]
assignees: []
body:
- type: textarea
@@ -30,6 +30,13 @@ body:
description: A clear and concise description of what you expected to happen.
validations:
required: true
- type: textarea
id: impact
attributes:
label: Impact
description: Describe who is affected and how this problem affects their work.
validations:
required: true
- type: textarea
id: screenshots-code
attributes:
@@ -106,6 +113,14 @@ body:
description: Have a solution in mind? Please suggest it here, or write "None".
validations:
required: true
- type: textarea
id: verification-plan
attributes:
label: Verification plan
description: Describe how a maintainer can confirm that a future fix resolves the problem.
placeholder: Include a minimal reproduction, expected assertions, or a manual verification procedure.
validations:
required: true
- type: textarea
id: additional-context
attributes:

View File

@@ -1,7 +1,7 @@
name: Feature request
description: Suggest a new feature for CrewAI
title: "[FEATURE]"
labels: ["feature-request"]
labels: ["feature-request", "state:inbox"]
assignees: []
body:
- type: markdown
@@ -26,8 +26,8 @@ body:
- type: textarea
id: problem
attributes:
label: Is your feature request related to a an existing bug? Please link it here.
description: A link to the bug or NA if not related to an existing bug.
label: Problem statement
description: Describe the user problem or unmet need without prescribing an implementation.
validations:
required: true
- type: textarea
@@ -44,6 +44,30 @@ body:
description: A clear and concise description of any alternative solutions or features you've considered.
validations:
required: false
- type: textarea
id: scope
attributes:
label: Proposed scope and non-goals
description: Describe what should be included and what should intentionally remain out of scope.
validations:
required: true
- type: textarea
id: acceptance-criteria
attributes:
label: Acceptance criteria
description: List the observable outcomes that would make this request complete.
placeholder: |
- [ ] A user can ...
- [ ] Existing behavior remains ...
validations:
required: true
- type: textarea
id: verification-plan
attributes:
label: Verification plan
description: Describe the automated or manual checks that should confirm the feature works.
validations:
required: true
- type: textarea
id: context
attributes:
@@ -62,4 +86,4 @@ body:
- I can test the feature once it's implemented
- No, I'm just suggesting the idea
validations:
required: true
required: true

24
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,24 @@
## Ready issue
Implements #<issue-number>
<!--
Replace the placeholder above with exactly one GitHub issue that has the
`state:ready` label. Use "Implements", not "Closes", "Fixes", or "Resolves":
the issue remains open for human verification after this PR is merged.
-->
## Summary
<!-- Explain the solution and how it follows the design agreed in the issue. -->
## Verification
<!-- List the automated and manual checks used to verify the change. -->
- [ ] Tests added or updated for the changed behavior
- [ ] Relevant tests and quality checks pass locally
## Additional context
<!-- Include screenshots, compatibility notes, follow-up work, or "None". -->

View File

@@ -1,6 +1,3 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
@@ -8,9 +5,22 @@ updates:
- package-ecosystem: uv
directory: "/"
schedule:
interval: "weekly"
interval: weekly
day: monday
open-pull-requests-limit: 10
groups:
security-updates:
applies-to: security-updates
patterns:
- "*"
patch-minor-updates:
applies-to: version-updates
patterns:
- "*"
update-types:
- patch
- minor
ignore:
- dependency-name: "*"
update-types:
- version-update:semver-major

386
.github/scripts/issue-gate.mjs vendored Normal file
View File

@@ -0,0 +1,386 @@
import { readFile } from "node:fs/promises";
import { pathToFileURL } from "node:url";
const COMMENT_MARKER = "<!-- crewai-issue-gate -->";
const GATE_LABELS = [
"issue-gate:passed",
"issue-gate:exempt",
"needs-ready-issue",
];
const EXEMPT_LABELS = new Set(["issue-gate:override", "policy:legacy"]);
const EXEMPT_ACTORS = new Set([
"dependabot[bot]",
"github-actions[bot]",
"renovate[bot]",
]);
const VALID_MODES = new Set(["observe", "block", "close"]);
class ApiError extends Error {
constructor(status, message) {
super(message);
this.name = "ApiError";
this.status = status;
}
}
/** Return unique issue references declared with `Implements #123`. */
export function parseIssueReferences(body, defaultRepository) {
const references = [];
const seen = new Set();
const pattern =
/^[\t ]*Implements[\t ]+(?:(?<repository>[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+))?#(?<number>\d+)\b/gim;
for (const match of (body ?? "").matchAll(pattern)) {
const repository = match.groups.repository ?? defaultRepository;
const number = Number.parseInt(match.groups.number, 10);
const key = `${repository.toLowerCase()}#${number}`;
if (!seen.has(key)) {
references.push({ repository, number });
seen.add(key);
}
}
return references;
}
function labelNames(item) {
return new Set((item.labels ?? []).map((label) => label.name.toLowerCase()));
}
function exemptionReason(pullRequest, cutoff) {
const labels = labelNames(pullRequest);
for (const label of EXEMPT_LABELS) {
if (labels.has(label)) {
return {
kind: label === "policy:legacy" ? "legacy" : "override",
reason: `the pull request has the \`${label}\` exemption label`,
};
}
}
const actor = pullRequest.user?.login?.toLowerCase();
if (EXEMPT_ACTORS.has(actor)) {
return {
kind: "automation",
reason: `\`${pullRequest.user.login}\` is an exempt automation account`,
};
}
if (cutoff) {
const cutoffTime = Date.parse(cutoff);
if (Number.isNaN(cutoffTime)) {
throw new Error(`ISSUE_GATE_CUTOFF is not a valid date: ${cutoff}`);
}
const createdTime = Date.parse(pullRequest.created_at);
if (Number.isNaN(createdTime)) {
throw new Error(`Pull request has an invalid creation date: ${pullRequest.created_at}`);
}
if (createdTime < cutoffTime) {
return {
kind: "legacy",
reason: `the pull request predates the pilot cutoff (${cutoff})`,
};
}
}
return null;
}
/** Evaluate a pull request without mutating GitHub state. */
export async function evaluatePullRequest({
pullRequest,
repository,
cutoff = "",
readyLabel = "state:ready",
getIssue,
}) {
const exemption = exemptionReason(pullRequest, cutoff);
if (exemption) {
return {
ok: true,
exempt: true,
exemption: exemption.kind,
reason: exemption.reason,
};
}
const references = parseIssueReferences(pullRequest.body, repository);
if (references.length === 0) {
return {
ok: false,
exempt: false,
reason: "the description does not contain `Implements #<issue-number>`",
};
}
if (references.length > 1) {
return {
ok: false,
exempt: false,
reason: "the description references more than one implementation issue",
};
}
const reference = references[0];
if (reference.repository.toLowerCase() !== repository.toLowerCase()) {
return {
ok: false,
exempt: false,
reason: `the implementation issue must belong to \`${repository}\``,
};
}
const issue = await getIssue(reference.number);
if (!issue || issue.pull_request) {
return {
ok: false,
exempt: false,
reason: `#${reference.number} is not an issue in \`${repository}\``,
};
}
if (issue.state !== "open") {
return {
ok: false,
exempt: false,
issueNumber: reference.number,
reason: `issue #${reference.number} is not open`,
};
}
if (!labelNames(issue).has(readyLabel.toLowerCase())) {
return {
ok: false,
exempt: false,
issueNumber: reference.number,
reason: `issue #${reference.number} does not have the \`${readyLabel}\` label`,
};
}
return {
ok: true,
exempt: false,
issueNumber: reference.number,
reason: `issue #${reference.number} is open and ready for implementation`,
};
}
async function githubApi(token, method, endpoint, body) {
const response = await fetch(`https://api.github.com${endpoint}`, {
method,
headers: {
Accept: "application/vnd.github+json",
Authorization: `Bearer ${token}`,
"User-Agent": "crewai-issue-gate",
"X-GitHub-Api-Version": "2022-11-28",
},
body: body === undefined ? undefined : JSON.stringify(body),
});
const responseText = await response.text();
const responseBody = responseText ? JSON.parse(responseText) : null;
if (!response.ok) {
throw new ApiError(
response.status,
`${method} ${endpoint} failed (${response.status}): ${responseBody?.message ?? responseText}`,
);
}
return responseBody;
}
function renderComment(result, mode) {
const modeExplanation =
mode === "observe"
? "The gate is in **observe mode**, so this result does not block or close the pull request."
: `The gate is in **${mode} mode**.`;
let outcome;
if (result.exempt) {
outcome = `This pull request is exempt because ${result.reason}.`;
} else if (result.ok) {
outcome = `This pull request passes: ${result.reason}.`;
} else {
outcome = `This pull request would be rejected because ${result.reason}.`;
}
return `${COMMENT_MARKER}\n### Issue gate\n\n${outcome}\n\n${modeExplanation}\n\nTo satisfy the pilot policy, describe exactly one ready issue using \`Implements #123\`. After the issue becomes ready, edit the pull request description or ask a maintainer to re-run the Issue Gate workflow.`;
}
async function updateGateComment({ token, repository, pullNumber, result, mode }) {
const [owner, repo] = repository.split("/");
const comments = await githubApi(
token,
"GET",
`/repos/${owner}/${repo}/issues/${pullNumber}/comments?per_page=100`,
);
const existing = comments.find(
(comment) => comment.user?.type === "Bot" && comment.body?.includes(COMMENT_MARKER),
);
if (!existing && result.ok) {
return;
}
const body = renderComment(result, mode);
if (existing) {
await githubApi(
token,
"PATCH",
`/repos/${owner}/${repo}/issues/comments/${existing.id}`,
{ body },
);
} else {
await githubApi(
token,
"POST",
`/repos/${owner}/${repo}/issues/${pullNumber}/comments`,
{ body },
);
}
}
async function updateGateLabel({ token, repository, pullRequest, result }) {
const [owner, repo] = repository.split("/");
const desiredLabel = result.exempt
? "issue-gate:exempt"
: result.ok
? "issue-gate:passed"
: "needs-ready-issue";
const currentLabels = labelNames(pullRequest);
for (const label of GATE_LABELS) {
if (label !== desiredLabel && currentLabels.has(label)) {
try {
await githubApi(
token,
"DELETE",
`/repos/${owner}/${repo}/issues/${pullRequest.number}/labels/${encodeURIComponent(label)}`,
);
} catch (error) {
if (!(error instanceof ApiError) || error.status !== 404) {
throw error;
}
}
}
}
if (!currentLabels.has(desiredLabel)) {
try {
await githubApi(
token,
"POST",
`/repos/${owner}/${repo}/issues/${pullRequest.number}/labels`,
{ labels: [desiredLabel] },
);
} catch (error) {
if (error instanceof ApiError && [404, 422].includes(error.status)) {
console.warn(`Could not apply missing label \`${desiredLabel}\`; create the pilot labels first.`);
return;
}
throw error;
}
}
}
async function publishStatus({ token, repository, pullRequest, result, mode }) {
const [owner, repo] = repository.split("/");
const shouldPass = mode === "observe" || result.ok;
const prefix = mode === "observe" && !result.ok ? "Observe: would fail" : result.ok ? "Pass" : "Fail";
const description = `${prefix}${result.reason}`.slice(0, 140);
await githubApi(
token,
"POST",
`/repos/${owner}/${repo}/statuses/${pullRequest.head.sha}`,
{
state: shouldPass ? "success" : "failure",
context: "Issue gate",
description,
target_url: `https://github.com/${repository}/pull/${pullRequest.number}`,
},
);
}
async function run() {
const token = process.env.GITHUB_TOKEN;
const repository = process.env.GITHUB_REPOSITORY;
const eventPath = process.env.GITHUB_EVENT_PATH;
const pullNumber = Number.parseInt(process.env.PR_NUMBER, 10);
const mode = (process.env.ISSUE_GATE_MODE || "observe").toLowerCase();
const cutoff = process.env.ISSUE_GATE_CUTOFF || "";
const readyLabel = process.env.ISSUE_GATE_READY_LABEL || "state:ready";
if (!token || !repository || !eventPath || !Number.isInteger(pullNumber)) {
throw new Error("GITHUB_TOKEN, GITHUB_REPOSITORY, GITHUB_EVENT_PATH, and PR_NUMBER are required");
}
if (!VALID_MODES.has(mode)) {
throw new Error(`ISSUE_GATE_MODE must be one of: ${[...VALID_MODES].join(", ")}`);
}
if (mode !== "observe" && !cutoff) {
throw new Error("ISSUE_GATE_CUTOFF is required before enabling block or close mode");
}
const [owner, repo] = repository.split("/");
const event = JSON.parse(await readFile(eventPath, "utf8"));
const pullRequest =
event.pull_request ??
(await githubApi(token, "GET", `/repos/${owner}/${repo}/pulls/${pullNumber}`));
const result = await evaluatePullRequest({
pullRequest,
repository,
cutoff,
readyLabel,
getIssue: async (issueNumber) => {
try {
return await githubApi(
token,
"GET",
`/repos/${owner}/${repo}/issues/${issueNumber}`,
);
} catch (error) {
if (error instanceof ApiError && error.status === 404) {
return null;
}
throw error;
}
},
});
await publishStatus({ token, repository, pullRequest, result, mode });
if (result.exemption !== "legacy") {
await updateGateLabel({ token, repository, pullRequest, result });
await updateGateComment({
token,
repository,
pullNumber: pullRequest.number,
result,
mode,
});
}
if (mode === "close" && !result.ok) {
await githubApi(
token,
"PATCH",
`/repos/${owner}/${repo}/pulls/${pullRequest.number}`,
{ state: "closed" },
);
}
console.log(JSON.stringify({ mode, pullNumber: pullRequest.number, ...result }));
if (mode !== "observe" && !result.ok) {
process.exitCode = 1;
}
}
if (import.meta.url === pathToFileURL(process.argv[1]).href) {
run().catch((error) => {
console.error(error);
process.exitCode = 1;
});
}

166
.github/scripts/issue-gate.test.mjs vendored Normal file
View File

@@ -0,0 +1,166 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
evaluatePullRequest,
parseIssueReferences,
} from "./issue-gate.mjs";
const repository = "crewAIInc/crewAI";
function pullRequest(overrides = {}) {
return {
body: "Implements #123",
created_at: "2026-08-12T00:00:00Z",
labels: [],
user: { login: "contributor" },
...overrides,
};
}
function issue(overrides = {}) {
return {
number: 123,
state: "open",
labels: [{ name: "state:ready" }],
...overrides,
};
}
async function evaluate(pullOverrides = {}, issueOverrides = {}) {
return evaluatePullRequest({
pullRequest: pullRequest(pullOverrides),
repository,
cutoff: "2026-08-11T00:00:00Z",
getIssue: async () => issue(issueOverrides),
});
}
test("parses shorthand and full repository references", () => {
assert.deepEqual(
parseIssueReferences(
"Implements #123\nImplements crewAIInc/crewAI#456",
repository,
),
[
{ repository, number: 123 },
{ repository, number: 456 },
],
);
});
test("ignores GitHub closing keywords and template placeholders", () => {
assert.deepEqual(
parseIssueReferences(
"Fixes #123\nCloses #456\nImplements #<issue-number>",
repository,
),
[],
);
});
test("deduplicates repeated references", () => {
assert.deepEqual(
parseIssueReferences("Implements #123\nimplements #123", repository),
[{ repository, number: 123 }],
);
});
test("passes an open ready issue", async () => {
const result = await evaluate();
assert.equal(result.ok, true);
assert.equal(result.exempt, false);
assert.equal(result.issueNumber, 123);
});
test("rejects a missing issue reference", async () => {
const result = await evaluate({ body: "No issue yet" });
assert.equal(result.ok, false);
assert.match(result.reason, /does not contain/);
});
test("rejects more than one implementation issue", async () => {
const result = await evaluate({
body: "Implements #123\nImplements #456",
});
assert.equal(result.ok, false);
assert.match(result.reason, /more than one/);
});
test("rejects an issue in another repository", async () => {
const result = await evaluate({
body: "Implements another/project#123",
});
assert.equal(result.ok, false);
assert.match(result.reason, /must belong/);
});
test("rejects a pull request reference masquerading as an issue", async () => {
const result = await evaluate({}, { pull_request: { url: "example" } });
assert.equal(result.ok, false);
assert.match(result.reason, /is not an issue/);
});
test("rejects a closed issue", async () => {
const result = await evaluate({}, { state: "closed" });
assert.equal(result.ok, false);
assert.match(result.reason, /is not open/);
});
test("rejects an issue without the ready label", async () => {
const result = await evaluate({}, { labels: [{ name: "state:design" }] });
assert.equal(result.ok, false);
assert.match(result.reason, /does not have/);
});
test("exempts pull requests created before the cutoff", async () => {
const result = await evaluate({
body: null,
created_at: "2026-08-10T23:59:59Z",
});
assert.equal(result.ok, true);
assert.equal(result.exempt, true);
assert.match(result.reason, /predates/);
});
test("exempts pull requests with an override label", async () => {
const result = await evaluate({
body: null,
labels: [{ name: "issue-gate:override" }],
});
assert.equal(result.ok, true);
assert.equal(result.exempt, true);
assert.match(result.reason, /override/);
});
test("exempts supported automation accounts", async () => {
const result = await evaluate({
body: null,
user: { login: "dependabot[bot]" },
});
assert.equal(result.ok, true);
assert.equal(result.exempt, true);
assert.match(result.reason, /automation account/);
});
test("fails fast for an invalid cutoff", async () => {
await assert.rejects(
evaluatePullRequest({
pullRequest: pullRequest(),
repository,
cutoff: "not-a-date",
getIssue: async () => issue(),
}),
/not a valid date/,
);
});

24
.github/workflows/issue-gate-tests.yml vendored Normal file
View File

@@ -0,0 +1,24 @@
name: Test Issue Gate
on:
pull_request:
paths:
- '.github/scripts/issue-gate.mjs'
- '.github/scripts/issue-gate.test.mjs'
- '.github/workflows/issue-gate.yml'
- '.github/workflows/issue-gate-tests.yml'
permissions:
contents: read
jobs:
test-issue-gate:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
with:
persist-credentials: false
- name: Run issue gate tests
run: node --test .github/scripts/issue-gate.test.mjs

46
.github/workflows/issue-gate.yml vendored Normal file
View File

@@ -0,0 +1,46 @@
name: Issue Gate
on:
pull_request_target:
branches: [main]
types: [opened, edited, reopened, synchronize, labeled, unlabeled]
workflow_dispatch:
inputs:
pr_number:
description: Pull request number to evaluate
required: true
type: number
permissions:
contents: read
issues: write
pull-requests: write
statuses: write
concurrency:
group: issue-gate-${{ inputs.pr_number || github.event.pull_request.number }}
cancel-in-progress: true
jobs:
issue-gate:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
# pull_request_target is privileged. Explicitly check out only the trusted
# default branch and never fetch or execute pull request code.
- name: Check out trusted gate implementation
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
with:
ref: ${{ github.event.repository.default_branch }}
persist-credentials: false
- name: Evaluate ready issue
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Until a cutoff is configured, every PR is treated as legacy. This
# keeps the newly merged workflow inert by default.
ISSUE_GATE_CUTOFF: ${{ vars.ISSUE_GATE_CUTOFF || '9999-12-31T00:00:00Z' }}
ISSUE_GATE_MODE: ${{ vars.ISSUE_GATE_MODE || 'observe' }}
ISSUE_GATE_READY_LABEL: ${{ vars.ISSUE_GATE_READY_LABEL || 'state:ready' }}
PR_NUMBER: ${{ inputs.pr_number || github.event.pull_request.number }}
run: node .github/scripts/issue-gate.mjs

View File

@@ -54,6 +54,7 @@ from crewai.events.types.flow_events import (
MethodExecutionPausedEvent,
MethodExecutionStartedEvent,
)
from crewai.events.types.hook_events import HookDispatchedEvent
from crewai.events.types.knowledge_events import (
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
@@ -875,5 +876,12 @@ class EventListener(BaseEventListener):
if has_hooks:
self._telemetry.feature_usage_span("hooks:registered")
@crewai_event_bus.on(HookDispatchedEvent)
def on_hook_dispatched(_: Any, event: HookDispatchedEvent) -> None:
self._telemetry.hook_dispatched_span(
interception_point=event.interception_point,
outcome=event.outcome,
)
event_listener = EventListener()

View File

@@ -438,6 +438,17 @@ class BaseLLM(BaseModel, ABC):
"""
return DEFAULT_SUPPORTS_STOP_WORDS
def _supports_stop_words_implementation(self) -> bool:
"""Check if stop words are configured for this LLM instance.
Native providers can override supports_stop_words() to return this value
to ensure consistent behavior based on whether stop words are actually configured.
Returns:
True if stop words are configured and can be applied
"""
return bool(self.stop_sequences)
def _apply_stop_words(self, content: str) -> str:
"""Apply stop words to truncate response content.

View File

@@ -1385,6 +1385,120 @@ class AnthropicCompletion(BaseLLM):
from_agent=from_agent,
)
# TODO: we drop this
def _handle_tool_use_conversation(
self,
initial_response: Message | BetaMessage,
tool_uses: list[_AnthropicToolUseBlock],
params: dict[str, Any],
available_functions: dict[str, Any],
from_task: Any | None = None,
from_agent: Any | None = None,
) -> str:
"""Handle the complete tool use conversation flow.
This implements the proper Anthropic tool use pattern:
1. Claude requests tool use
2. We execute the tools
3. We send tool results back to Claude
4. Claude processes results and generates final response
"""
tool_results = self._execute_tools_and_collect_results(
tool_uses, available_functions, from_task, from_agent
)
follow_up_params = params.copy()
assistant_content: list[
ThinkingBlock | ToolUseBlock | TextBlock | dict[str, Any]
] = []
for block in initial_response.content:
thinking_block = self._extract_thinking_block(block)
if thinking_block:
assistant_content.append(thinking_block)
elif _is_tool_use_block(block):
assistant_content.append(
{
"type": "tool_use",
"id": _tool_use_id(block),
"name": _tool_use_name(block),
"input": _tool_use_input(block),
}
)
elif hasattr(block, "text"):
assistant_content.append({"type": "text", "text": block.text})
assistant_message = {"role": "assistant", "content": assistant_content}
user_message = {"role": "user", "content": tool_results}
follow_up_params["messages"] = params["messages"] + [
assistant_message,
user_message,
]
try:
final_response: Message = self._get_sync_client().messages.create(
**follow_up_params
)
follow_up_usage = self._extract_anthropic_token_usage(final_response)
self._track_token_usage_internal(follow_up_usage)
final_content = ""
thinking_blocks: list[ThinkingBlock] = []
if final_response.content:
for content_block in final_response.content:
if hasattr(content_block, "text"):
final_content += content_block.text
else:
thinking_block = self._extract_thinking_block(content_block)
if thinking_block:
thinking_blocks.append(cast(ThinkingBlock, thinking_block))
if thinking_blocks:
self._previous_thinking_blocks = thinking_blocks
final_content = self._apply_stop_words(final_content)
finish_reason, final_response_id = self._extract_finish_reason_and_id(
final_response
)
self._emit_call_completed_event(
response=final_content,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=follow_up_params["messages"],
usage=follow_up_usage,
finish_reason=finish_reason,
response_id=final_response_id,
)
total_usage = {
"input_tokens": follow_up_usage.get("input_tokens", 0),
"output_tokens": follow_up_usage.get("output_tokens", 0),
"total_tokens": follow_up_usage.get("total_tokens", 0),
}
if total_usage.get("total_tokens", 0) > 0:
logging.info(f"Anthropic API tool conversation usage: {total_usage}")
return final_content
except Exception as e:
if is_context_length_exceeded(e):
logging.error(f"Context window exceeded in tool follow-up: {e}")
raise LLMContextLengthExceededError(str(e)) from e
logging.error(f"Tool follow-up conversation failed: {e}")
# Fallback to first tool result when follow-up fails
if tool_results:
return cast(str, tool_results[0]["content"])
raise e
async def _ahandle_completion(
self,
params: dict[str, Any],
@@ -1716,6 +1830,90 @@ class AnthropicCompletion(BaseLLM):
return full_response
async def _ahandle_tool_use_conversation(
self,
initial_response: Message | BetaMessage,
tool_uses: list[_AnthropicToolUseBlock],
params: dict[str, Any],
available_functions: dict[str, Any],
from_task: Any | None = None,
from_agent: Any | None = None,
) -> str:
"""Handle the complete async tool use conversation flow.
This implements the proper Anthropic tool use pattern:
1. Claude requests tool use
2. We execute the tools
3. We send tool results back to Claude
4. Claude processes results and generates final response
"""
tool_results = self._execute_tools_and_collect_results(
tool_uses, available_functions, from_task, from_agent
)
follow_up_params = params.copy()
assistant_message = {"role": "assistant", "content": initial_response.content}
user_message = {"role": "user", "content": tool_results}
follow_up_params["messages"] = params["messages"] + [
assistant_message,
user_message,
]
try:
final_response: Message = await self._get_async_client().messages.create(
**follow_up_params
)
follow_up_usage = self._extract_anthropic_token_usage(final_response)
self._track_token_usage_internal(follow_up_usage)
final_content = ""
if final_response.content:
for content_block in final_response.content:
if hasattr(content_block, "text"):
final_content += content_block.text
final_content = self._apply_stop_words(final_content)
finish_reason, final_response_id = self._extract_finish_reason_and_id(
final_response
)
self._emit_call_completed_event(
response=final_content,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=follow_up_params["messages"],
usage=follow_up_usage,
finish_reason=finish_reason,
response_id=final_response_id,
)
total_usage = {
"input_tokens": follow_up_usage.get("input_tokens", 0),
"output_tokens": follow_up_usage.get("output_tokens", 0),
"total_tokens": follow_up_usage.get("total_tokens", 0),
}
if total_usage.get("total_tokens", 0) > 0:
logging.info(f"Anthropic API tool conversation usage: {total_usage}")
return final_content
except Exception as e:
if is_context_length_exceeded(e):
logging.error(f"Context window exceeded in tool follow-up: {e}")
raise LLMContextLengthExceededError(str(e)) from e
logging.error(f"Tool follow-up conversation failed: {e}")
if tool_results:
return cast(str, tool_results[0]["content"])
raise e
def supports_function_calling(self) -> bool:
"""Check if the model supports function calling."""
return self.supports_tools

View File

@@ -2146,6 +2146,17 @@ class BedrockCompletion(BaseLLM):
)
return any(model_lower.startswith(m) for m in vision_models)
def _is_nova_model(self) -> bool:
"""Check if the model is an Amazon Nova model.
Only Nova models support S3 links for multimedia.
Returns:
True if the model is a Nova model.
"""
model_lower = self.model.lower()
return "amazon.nova-" in model_lower
def get_file_uploader(self) -> Any:
"""Get a Bedrock S3 file uploader using this LLM's AWS credentials.
@@ -2174,6 +2185,49 @@ class BedrockCompletion(BaseLLM):
except ImportError:
return None
def _get_document_format(self, content_type: str) -> str | None:
"""Map content type to Bedrock document format.
Args:
content_type: MIME type of the document.
Returns:
Bedrock format string or None if unsupported.
"""
format_map = {
"application/pdf": "pdf",
"text/csv": "csv",
"text/plain": "txt",
"text/markdown": "md",
"text/html": "html",
"application/msword": "doc",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document": "docx",
"application/vnd.ms-excel": "xls",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet": "xlsx",
}
return format_map.get(content_type)
def _get_video_format(self, content_type: str) -> str | None:
"""Map content type to Bedrock video format.
Args:
content_type: MIME type of the video.
Returns:
Bedrock format string or None if unsupported.
"""
format_map = {
"video/mp4": "mp4",
"video/quicktime": "mov",
"video/x-matroska": "mkv",
"video/webm": "webm",
"video/x-flv": "flv",
"video/mpeg": "mpeg",
"video/x-ms-wmv": "wmv",
"video/3gpp": "three_gp",
}
return format_map.get(content_type)
def format_text_content(self, text: str) -> dict[str, Any]:
"""Format text as a Bedrock content block.

View File

@@ -1148,7 +1148,8 @@ class Telemetry:
Args:
feature: Feature identifier, e.g. "planning:creation",
"mcp:connection", "a2a:delegation".
"mcp:connection", "a2a:delegation",
"hooks:pre_tool_call", "hooks:aborted".
"""
def _operation() -> None:
@@ -1160,6 +1161,21 @@ class Telemetry:
self._safe_telemetry_operation(_operation)
def hook_dispatched_span(
self,
interception_point: str,
outcome: str,
) -> None:
"""Records an interception-hook dispatch via Feature Usage.
Emits ``hooks:<point>`` on every dispatch, plus ``hooks:aborted`` when
a hook aborted the operation (e.g. a policy check). No reasons,
payloads, or other user content are recorded.
"""
self.feature_usage_span(f"hooks:{interception_point}")
if outcome == "aborted":
self.feature_usage_span("hooks:aborted")
def coding_agent_span(self) -> None:
"""Records which AI coding assistant (if any) is running this process.

View File

@@ -1576,6 +1576,30 @@ def test_anthropic_dict_tool_use_blocks_execute_available_function():
assert result == "found CrewAI"
def test_anthropic_dict_tool_use_blocks_work_in_follow_up_conversation():
from crewai.llms.providers.anthropic.completion import AnthropicCompletion
llm = AnthropicCompletion(model="claude-fable-5")
initial_response = _dict_tool_use_response()
final_response = MagicMock()
final_response.content = [types.SimpleNamespace(text="Final answer")]
final_response.usage = MagicMock(input_tokens=4, output_tokens=3)
final_response.stop_reason = "end_turn"
final_response.id = "msg_final"
mock_client = MagicMock()
mock_client.messages.create.return_value = final_response
llm._client = mock_client
result = llm._handle_tool_use_conversation(
initial_response,
initial_response.content,
params={"messages": []},
available_functions={"search_web": lambda query: f"found {query}"},
)
assert result == "Final answer"
@pytest.mark.vcr()
def test_tool_search_discovers_and_calls_tool():
"""Tool search should discover the right tool and return a tool_use block."""

View File

@@ -230,3 +230,73 @@ def test_no_signal_handler_traceback_in_non_main_thread():
mock_holder["logger"].debug.assert_any_call(
"Skipping signal handler registration: not running in main thread"
)
def test_hook_dispatched_span_counts_point_usage():
with (
patch.dict(
os.environ,
{
"CREWAI_DISABLE_TELEMETRY": "false",
"CREWAI_DISABLE_TRACKING": "false",
"OTEL_SDK_DISABLED": "false",
},
),
patch("crewai.telemetry.telemetry.TracerProvider"),
):
telemetry = Telemetry()
with patch.object(telemetry, "feature_usage_span") as feature_usage_span:
telemetry.hook_dispatched_span("pre_tool_call", "proceeded")
feature_usage_span.assert_called_once_with("hooks:pre_tool_call")
def test_hook_dispatched_span_counts_aborts():
with (
patch.dict(
os.environ,
{
"CREWAI_DISABLE_TELEMETRY": "false",
"CREWAI_DISABLE_TRACKING": "false",
"OTEL_SDK_DISABLED": "false",
},
),
patch("crewai.telemetry.telemetry.TracerProvider"),
):
telemetry = Telemetry()
with patch.object(telemetry, "feature_usage_span") as feature_usage_span:
telemetry.hook_dispatched_span("pre_tool_call", "aborted")
feature_usage_span.assert_any_call("hooks:pre_tool_call")
feature_usage_span.assert_any_call("hooks:aborted")
assert feature_usage_span.call_count == 2
def test_event_listener_tracks_hook_dispatched_events():
from crewai.events.event_bus import crewai_event_bus
from crewai.events.event_listener import event_listener
from crewai.events.types.hook_events import HookDispatchedEvent
with (
crewai_event_bus.scoped_handlers(),
patch.object(
event_listener._telemetry,
"hook_dispatched_span",
) as hook_dispatched_span,
):
event_listener.setup_listeners(crewai_event_bus)
crewai_event_bus.emit(
"test",
HookDispatchedEvent(
interception_point="pre_tool_call",
outcome="aborted",
hook_count=1,
duration_ms=1.5,
),
)
crewai_event_bus.flush()
hook_dispatched_span.assert_called_once_with(
interception_point="pre_tool_call",
outcome="aborted",
)