mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-08-04 21:41:44 +00:00
Compare commits
4 Commits
codex/llm-
...
lorenze/im
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cf6c32e79b | ||
|
|
66ce0c536c | ||
|
|
7accafbaf4 | ||
|
|
9d659a644b |
38
.github/CONTRIBUTING.md
vendored
38
.github/CONTRIBUTING.md
vendored
@@ -42,6 +42,40 @@ Documentation lives in `docs/` with translations under `docs/{en,ar,ko,pt-BR}/`.
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Issue-First Contributions
|
||||
|
||||
CrewAI is piloting an issue-first contribution workflow. The issue is where
|
||||
contributors and maintainers agree on the problem and intended solution before
|
||||
implementation begins.
|
||||
|
||||
Issues move through the following lifecycle:
|
||||
|
||||
```
|
||||
state:inbox -> state:design -> state:ready -> state:in-progress -> state:verification -> Done
|
||||
```
|
||||
|
||||
Maintainers apply `state:ready` after the issue has a clear problem statement,
|
||||
agreed scope and non-goals, acceptance criteria, and a verification plan. A
|
||||
pull request should not begin before its issue is ready unless a maintainer has
|
||||
approved an exception.
|
||||
|
||||
Every new pull request must reference exactly one ready issue in its description:
|
||||
|
||||
```
|
||||
Implements #123
|
||||
```
|
||||
|
||||
Use `Implements`, not `Closes`, `Fixes`, or `Resolves`. GitHub's closing
|
||||
keywords close the issue as soon as the pull request merges, but issue-first
|
||||
contributions remain open in `state:verification` until a human confirms the
|
||||
result.
|
||||
|
||||
The issue gate currently runs in **observe mode**. It reports whether a pull
|
||||
request would satisfy this policy, but it does not block or close pull requests.
|
||||
Maintainers can apply `issue-gate:override` for security work, release
|
||||
automation, urgent fixes, or another documented exception. Pull requests that
|
||||
predate the pilot can be marked `policy:legacy`.
|
||||
|
||||
### Branching
|
||||
|
||||
Create a branch off `main` using the conventional commit type:
|
||||
@@ -103,7 +137,9 @@ chore(deps): bump pydantic to 2.11
|
||||
- Keep PRs focused — avoid bundling unrelated changes
|
||||
- PRs over 500 lines are labeled `size/XL` automatically
|
||||
- Title must follow the same conventional commit format
|
||||
- Link related issues where applicable
|
||||
- Reference exactly one ready issue with `Implements #<issue-number>`
|
||||
- Explain how the implementation meets the issue's acceptance criteria
|
||||
- Include the automated and manual verification performed
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
56
.github/ISSUE_GATE.md
vendored
Normal file
56
.github/ISSUE_GATE.md
vendored
Normal file
@@ -0,0 +1,56 @@
|
||||
# Issue Gate Pilot
|
||||
|
||||
The issue gate checks whether a pull request references exactly one open issue
|
||||
with the `state:ready` label. It runs on `pull_request_target`, checks out only
|
||||
the trusted default branch, and never fetches or executes pull request code.
|
||||
|
||||
## Repository setup
|
||||
|
||||
Create these labels before activating the pilot:
|
||||
|
||||
- Issue lifecycle: `state:inbox`, `state:design`, `state:ready`,
|
||||
`state:in-progress`, and `state:verification`
|
||||
- Gate results: `issue-gate:passed`, `issue-gate:exempt`, and
|
||||
`needs-ready-issue`
|
||||
- Maintainer exceptions: `issue-gate:override` and `policy:legacy`
|
||||
|
||||
Configure these repository variables:
|
||||
|
||||
| Variable | Required | Value |
|
||||
| --- | --- | --- |
|
||||
| `ISSUE_GATE_MODE` | No | `observe` (default), `block`, or `close` |
|
||||
| `ISSUE_GATE_CUTOFF` | Yes to activate | ISO 8601 timestamp for the first PR covered by the pilot |
|
||||
| `ISSUE_GATE_READY_LABEL` | No | Defaults to `state:ready` |
|
||||
|
||||
Without `ISSUE_GATE_CUTOFF`, the workflow uses a future cutoff and treats every
|
||||
pull request as legacy. This makes the workflow inert until maintainers choose
|
||||
the activation time.
|
||||
|
||||
## Modes
|
||||
|
||||
- `observe`: Invalid pull requests receive a successful `Issue gate` status,
|
||||
an explanatory comment, and the `needs-ready-issue` label.
|
||||
- `block`: Invalid pull requests receive a failing status. Add `Issue gate` to
|
||||
the `main` ruleset's required checks only after observation is complete.
|
||||
- `close`: Invalid pull requests receive a failing status and are closed after
|
||||
the comment is posted.
|
||||
|
||||
The script refuses to enter `block` or `close` mode without a configured cutoff.
|
||||
Pull requests created by supported dependency automation, or labeled
|
||||
`issue-gate:override` or `policy:legacy`, remain exempt.
|
||||
|
||||
## Testing the pilot
|
||||
|
||||
Run the `Issue Gate` workflow manually with a pull request number, or edit a
|
||||
pull request description to trigger it again. Test at least these cases before
|
||||
changing modes:
|
||||
|
||||
1. No implementation issue
|
||||
2. A nonexistent, closed, or non-ready issue
|
||||
3. More than one implementation issue
|
||||
4. One open issue labeled `state:ready`
|
||||
5. A pull request before the cutoff
|
||||
6. A pull request with `issue-gate:override`
|
||||
|
||||
Keep the gate in `observe` mode until the sample contains at least 20 new pull
|
||||
requests with no false positives.
|
||||
17
.github/ISSUE_TEMPLATE/bug_report.yml
vendored
17
.github/ISSUE_TEMPLATE/bug_report.yml
vendored
@@ -1,7 +1,7 @@
|
||||
name: Bug report
|
||||
description: Create a report to help us improve CrewAI
|
||||
title: "[BUG]"
|
||||
labels: ["bug"]
|
||||
labels: ["bug", "state:inbox"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: textarea
|
||||
@@ -30,6 +30,13 @@ body:
|
||||
description: A clear and concise description of what you expected to happen.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: impact
|
||||
attributes:
|
||||
label: Impact
|
||||
description: Describe who is affected and how this problem affects their work.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: screenshots-code
|
||||
attributes:
|
||||
@@ -106,6 +113,14 @@ body:
|
||||
description: Have a solution in mind? Please suggest it here, or write "None".
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: verification-plan
|
||||
attributes:
|
||||
label: Verification plan
|
||||
description: Describe how a maintainer can confirm that a future fix resolves the problem.
|
||||
placeholder: Include a minimal reproduction, expected assertions, or a manual verification procedure.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
|
||||
32
.github/ISSUE_TEMPLATE/feature_request.yml
vendored
32
.github/ISSUE_TEMPLATE/feature_request.yml
vendored
@@ -1,7 +1,7 @@
|
||||
name: Feature request
|
||||
description: Suggest a new feature for CrewAI
|
||||
title: "[FEATURE]"
|
||||
labels: ["feature-request"]
|
||||
labels: ["feature-request", "state:inbox"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
@@ -26,8 +26,8 @@ body:
|
||||
- type: textarea
|
||||
id: problem
|
||||
attributes:
|
||||
label: Is your feature request related to a an existing bug? Please link it here.
|
||||
description: A link to the bug or NA if not related to an existing bug.
|
||||
label: Problem statement
|
||||
description: Describe the user problem or unmet need without prescribing an implementation.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
@@ -44,6 +44,30 @@ body:
|
||||
description: A clear and concise description of any alternative solutions or features you've considered.
|
||||
validations:
|
||||
required: false
|
||||
- type: textarea
|
||||
id: scope
|
||||
attributes:
|
||||
label: Proposed scope and non-goals
|
||||
description: Describe what should be included and what should intentionally remain out of scope.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: acceptance-criteria
|
||||
attributes:
|
||||
label: Acceptance criteria
|
||||
description: List the observable outcomes that would make this request complete.
|
||||
placeholder: |
|
||||
- [ ] A user can ...
|
||||
- [ ] Existing behavior remains ...
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: verification-plan
|
||||
attributes:
|
||||
label: Verification plan
|
||||
description: Describe the automated or manual checks that should confirm the feature works.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: context
|
||||
attributes:
|
||||
@@ -62,4 +86,4 @@ body:
|
||||
- I can test the feature once it's implemented
|
||||
- No, I'm just suggesting the idea
|
||||
validations:
|
||||
required: true
|
||||
required: true
|
||||
|
||||
24
.github/PULL_REQUEST_TEMPLATE.md
vendored
Normal file
24
.github/PULL_REQUEST_TEMPLATE.md
vendored
Normal file
@@ -0,0 +1,24 @@
|
||||
## Ready issue
|
||||
|
||||
Implements #<issue-number>
|
||||
|
||||
<!--
|
||||
Replace the placeholder above with exactly one GitHub issue that has the
|
||||
`state:ready` label. Use "Implements", not "Closes", "Fixes", or "Resolves":
|
||||
the issue remains open for human verification after this PR is merged.
|
||||
-->
|
||||
|
||||
## Summary
|
||||
|
||||
<!-- Explain the solution and how it follows the design agreed in the issue. -->
|
||||
|
||||
## Verification
|
||||
|
||||
<!-- List the automated and manual checks used to verify the change. -->
|
||||
|
||||
- [ ] Tests added or updated for the changed behavior
|
||||
- [ ] Relevant tests and quality checks pass locally
|
||||
|
||||
## Additional context
|
||||
|
||||
<!-- Include screenshots, compatibility notes, follow-up work, or "None". -->
|
||||
18
.github/dependabot.yml
vendored
18
.github/dependabot.yml
vendored
@@ -1,6 +1,3 @@
|
||||
# To get started with Dependabot version updates, you'll need to specify which
|
||||
# package ecosystems to update and where the package manifests are located.
|
||||
# Please see the documentation for all configuration options:
|
||||
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
|
||||
|
||||
version: 2
|
||||
@@ -8,9 +5,22 @@ updates:
|
||||
- package-ecosystem: uv
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
interval: weekly
|
||||
day: monday
|
||||
open-pull-requests-limit: 10
|
||||
groups:
|
||||
security-updates:
|
||||
applies-to: security-updates
|
||||
patterns:
|
||||
- "*"
|
||||
patch-minor-updates:
|
||||
applies-to: version-updates
|
||||
patterns:
|
||||
- "*"
|
||||
update-types:
|
||||
- patch
|
||||
- minor
|
||||
ignore:
|
||||
- dependency-name: "*"
|
||||
update-types:
|
||||
- version-update:semver-major
|
||||
|
||||
386
.github/scripts/issue-gate.mjs
vendored
Normal file
386
.github/scripts/issue-gate.mjs
vendored
Normal file
@@ -0,0 +1,386 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
const COMMENT_MARKER = "<!-- crewai-issue-gate -->";
|
||||
const GATE_LABELS = [
|
||||
"issue-gate:passed",
|
||||
"issue-gate:exempt",
|
||||
"needs-ready-issue",
|
||||
];
|
||||
const EXEMPT_LABELS = new Set(["issue-gate:override", "policy:legacy"]);
|
||||
const EXEMPT_ACTORS = new Set([
|
||||
"dependabot[bot]",
|
||||
"github-actions[bot]",
|
||||
"renovate[bot]",
|
||||
]);
|
||||
const VALID_MODES = new Set(["observe", "block", "close"]);
|
||||
|
||||
class ApiError extends Error {
|
||||
constructor(status, message) {
|
||||
super(message);
|
||||
this.name = "ApiError";
|
||||
this.status = status;
|
||||
}
|
||||
}
|
||||
|
||||
/** Return unique issue references declared with `Implements #123`. */
|
||||
export function parseIssueReferences(body, defaultRepository) {
|
||||
const references = [];
|
||||
const seen = new Set();
|
||||
const pattern =
|
||||
/^[\t ]*Implements[\t ]+(?:(?<repository>[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+))?#(?<number>\d+)\b/gim;
|
||||
|
||||
for (const match of (body ?? "").matchAll(pattern)) {
|
||||
const repository = match.groups.repository ?? defaultRepository;
|
||||
const number = Number.parseInt(match.groups.number, 10);
|
||||
const key = `${repository.toLowerCase()}#${number}`;
|
||||
|
||||
if (!seen.has(key)) {
|
||||
references.push({ repository, number });
|
||||
seen.add(key);
|
||||
}
|
||||
}
|
||||
|
||||
return references;
|
||||
}
|
||||
|
||||
function labelNames(item) {
|
||||
return new Set((item.labels ?? []).map((label) => label.name.toLowerCase()));
|
||||
}
|
||||
|
||||
function exemptionReason(pullRequest, cutoff) {
|
||||
const labels = labelNames(pullRequest);
|
||||
for (const label of EXEMPT_LABELS) {
|
||||
if (labels.has(label)) {
|
||||
return {
|
||||
kind: label === "policy:legacy" ? "legacy" : "override",
|
||||
reason: `the pull request has the \`${label}\` exemption label`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const actor = pullRequest.user?.login?.toLowerCase();
|
||||
if (EXEMPT_ACTORS.has(actor)) {
|
||||
return {
|
||||
kind: "automation",
|
||||
reason: `\`${pullRequest.user.login}\` is an exempt automation account`,
|
||||
};
|
||||
}
|
||||
|
||||
if (cutoff) {
|
||||
const cutoffTime = Date.parse(cutoff);
|
||||
if (Number.isNaN(cutoffTime)) {
|
||||
throw new Error(`ISSUE_GATE_CUTOFF is not a valid date: ${cutoff}`);
|
||||
}
|
||||
|
||||
const createdTime = Date.parse(pullRequest.created_at);
|
||||
if (Number.isNaN(createdTime)) {
|
||||
throw new Error(`Pull request has an invalid creation date: ${pullRequest.created_at}`);
|
||||
}
|
||||
|
||||
if (createdTime < cutoffTime) {
|
||||
return {
|
||||
kind: "legacy",
|
||||
reason: `the pull request predates the pilot cutoff (${cutoff})`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Evaluate a pull request without mutating GitHub state. */
|
||||
export async function evaluatePullRequest({
|
||||
pullRequest,
|
||||
repository,
|
||||
cutoff = "",
|
||||
readyLabel = "state:ready",
|
||||
getIssue,
|
||||
}) {
|
||||
const exemption = exemptionReason(pullRequest, cutoff);
|
||||
if (exemption) {
|
||||
return {
|
||||
ok: true,
|
||||
exempt: true,
|
||||
exemption: exemption.kind,
|
||||
reason: exemption.reason,
|
||||
};
|
||||
}
|
||||
|
||||
const references = parseIssueReferences(pullRequest.body, repository);
|
||||
if (references.length === 0) {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
reason: "the description does not contain `Implements #<issue-number>`",
|
||||
};
|
||||
}
|
||||
|
||||
if (references.length > 1) {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
reason: "the description references more than one implementation issue",
|
||||
};
|
||||
}
|
||||
|
||||
const reference = references[0];
|
||||
if (reference.repository.toLowerCase() !== repository.toLowerCase()) {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
reason: `the implementation issue must belong to \`${repository}\``,
|
||||
};
|
||||
}
|
||||
|
||||
const issue = await getIssue(reference.number);
|
||||
if (!issue || issue.pull_request) {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
reason: `#${reference.number} is not an issue in \`${repository}\``,
|
||||
};
|
||||
}
|
||||
|
||||
if (issue.state !== "open") {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
issueNumber: reference.number,
|
||||
reason: `issue #${reference.number} is not open`,
|
||||
};
|
||||
}
|
||||
|
||||
if (!labelNames(issue).has(readyLabel.toLowerCase())) {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
issueNumber: reference.number,
|
||||
reason: `issue #${reference.number} does not have the \`${readyLabel}\` label`,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
exempt: false,
|
||||
issueNumber: reference.number,
|
||||
reason: `issue #${reference.number} is open and ready for implementation`,
|
||||
};
|
||||
}
|
||||
|
||||
async function githubApi(token, method, endpoint, body) {
|
||||
const response = await fetch(`https://api.github.com${endpoint}`, {
|
||||
method,
|
||||
headers: {
|
||||
Accept: "application/vnd.github+json",
|
||||
Authorization: `Bearer ${token}`,
|
||||
"User-Agent": "crewai-issue-gate",
|
||||
"X-GitHub-Api-Version": "2022-11-28",
|
||||
},
|
||||
body: body === undefined ? undefined : JSON.stringify(body),
|
||||
});
|
||||
|
||||
const responseText = await response.text();
|
||||
const responseBody = responseText ? JSON.parse(responseText) : null;
|
||||
if (!response.ok) {
|
||||
throw new ApiError(
|
||||
response.status,
|
||||
`${method} ${endpoint} failed (${response.status}): ${responseBody?.message ?? responseText}`,
|
||||
);
|
||||
}
|
||||
|
||||
return responseBody;
|
||||
}
|
||||
|
||||
function renderComment(result, mode) {
|
||||
const modeExplanation =
|
||||
mode === "observe"
|
||||
? "The gate is in **observe mode**, so this result does not block or close the pull request."
|
||||
: `The gate is in **${mode} mode**.`;
|
||||
|
||||
let outcome;
|
||||
if (result.exempt) {
|
||||
outcome = `This pull request is exempt because ${result.reason}.`;
|
||||
} else if (result.ok) {
|
||||
outcome = `This pull request passes: ${result.reason}.`;
|
||||
} else {
|
||||
outcome = `This pull request would be rejected because ${result.reason}.`;
|
||||
}
|
||||
|
||||
return `${COMMENT_MARKER}\n### Issue gate\n\n${outcome}\n\n${modeExplanation}\n\nTo satisfy the pilot policy, describe exactly one ready issue using \`Implements #123\`. After the issue becomes ready, edit the pull request description or ask a maintainer to re-run the Issue Gate workflow.`;
|
||||
}
|
||||
|
||||
async function updateGateComment({ token, repository, pullNumber, result, mode }) {
|
||||
const [owner, repo] = repository.split("/");
|
||||
const comments = await githubApi(
|
||||
token,
|
||||
"GET",
|
||||
`/repos/${owner}/${repo}/issues/${pullNumber}/comments?per_page=100`,
|
||||
);
|
||||
const existing = comments.find(
|
||||
(comment) => comment.user?.type === "Bot" && comment.body?.includes(COMMENT_MARKER),
|
||||
);
|
||||
if (!existing && result.ok) {
|
||||
return;
|
||||
}
|
||||
|
||||
const body = renderComment(result, mode);
|
||||
|
||||
if (existing) {
|
||||
await githubApi(
|
||||
token,
|
||||
"PATCH",
|
||||
`/repos/${owner}/${repo}/issues/comments/${existing.id}`,
|
||||
{ body },
|
||||
);
|
||||
} else {
|
||||
await githubApi(
|
||||
token,
|
||||
"POST",
|
||||
`/repos/${owner}/${repo}/issues/${pullNumber}/comments`,
|
||||
{ body },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function updateGateLabel({ token, repository, pullRequest, result }) {
|
||||
const [owner, repo] = repository.split("/");
|
||||
const desiredLabel = result.exempt
|
||||
? "issue-gate:exempt"
|
||||
: result.ok
|
||||
? "issue-gate:passed"
|
||||
: "needs-ready-issue";
|
||||
const currentLabels = labelNames(pullRequest);
|
||||
|
||||
for (const label of GATE_LABELS) {
|
||||
if (label !== desiredLabel && currentLabels.has(label)) {
|
||||
try {
|
||||
await githubApi(
|
||||
token,
|
||||
"DELETE",
|
||||
`/repos/${owner}/${repo}/issues/${pullRequest.number}/labels/${encodeURIComponent(label)}`,
|
||||
);
|
||||
} catch (error) {
|
||||
if (!(error instanceof ApiError) || error.status !== 404) {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!currentLabels.has(desiredLabel)) {
|
||||
try {
|
||||
await githubApi(
|
||||
token,
|
||||
"POST",
|
||||
`/repos/${owner}/${repo}/issues/${pullRequest.number}/labels`,
|
||||
{ labels: [desiredLabel] },
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof ApiError && [404, 422].includes(error.status)) {
|
||||
console.warn(`Could not apply missing label \`${desiredLabel}\`; create the pilot labels first.`);
|
||||
return;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function publishStatus({ token, repository, pullRequest, result, mode }) {
|
||||
const [owner, repo] = repository.split("/");
|
||||
const shouldPass = mode === "observe" || result.ok;
|
||||
const prefix = mode === "observe" && !result.ok ? "Observe: would fail" : result.ok ? "Pass" : "Fail";
|
||||
const description = `${prefix} — ${result.reason}`.slice(0, 140);
|
||||
|
||||
await githubApi(
|
||||
token,
|
||||
"POST",
|
||||
`/repos/${owner}/${repo}/statuses/${pullRequest.head.sha}`,
|
||||
{
|
||||
state: shouldPass ? "success" : "failure",
|
||||
context: "Issue gate",
|
||||
description,
|
||||
target_url: `https://github.com/${repository}/pull/${pullRequest.number}`,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
async function run() {
|
||||
const token = process.env.GITHUB_TOKEN;
|
||||
const repository = process.env.GITHUB_REPOSITORY;
|
||||
const eventPath = process.env.GITHUB_EVENT_PATH;
|
||||
const pullNumber = Number.parseInt(process.env.PR_NUMBER, 10);
|
||||
const mode = (process.env.ISSUE_GATE_MODE || "observe").toLowerCase();
|
||||
const cutoff = process.env.ISSUE_GATE_CUTOFF || "";
|
||||
const readyLabel = process.env.ISSUE_GATE_READY_LABEL || "state:ready";
|
||||
|
||||
if (!token || !repository || !eventPath || !Number.isInteger(pullNumber)) {
|
||||
throw new Error("GITHUB_TOKEN, GITHUB_REPOSITORY, GITHUB_EVENT_PATH, and PR_NUMBER are required");
|
||||
}
|
||||
if (!VALID_MODES.has(mode)) {
|
||||
throw new Error(`ISSUE_GATE_MODE must be one of: ${[...VALID_MODES].join(", ")}`);
|
||||
}
|
||||
if (mode !== "observe" && !cutoff) {
|
||||
throw new Error("ISSUE_GATE_CUTOFF is required before enabling block or close mode");
|
||||
}
|
||||
|
||||
const [owner, repo] = repository.split("/");
|
||||
const event = JSON.parse(await readFile(eventPath, "utf8"));
|
||||
const pullRequest =
|
||||
event.pull_request ??
|
||||
(await githubApi(token, "GET", `/repos/${owner}/${repo}/pulls/${pullNumber}`));
|
||||
|
||||
const result = await evaluatePullRequest({
|
||||
pullRequest,
|
||||
repository,
|
||||
cutoff,
|
||||
readyLabel,
|
||||
getIssue: async (issueNumber) => {
|
||||
try {
|
||||
return await githubApi(
|
||||
token,
|
||||
"GET",
|
||||
`/repos/${owner}/${repo}/issues/${issueNumber}`,
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof ApiError && error.status === 404) {
|
||||
return null;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
await publishStatus({ token, repository, pullRequest, result, mode });
|
||||
if (result.exemption !== "legacy") {
|
||||
await updateGateLabel({ token, repository, pullRequest, result });
|
||||
await updateGateComment({
|
||||
token,
|
||||
repository,
|
||||
pullNumber: pullRequest.number,
|
||||
result,
|
||||
mode,
|
||||
});
|
||||
}
|
||||
|
||||
if (mode === "close" && !result.ok) {
|
||||
await githubApi(
|
||||
token,
|
||||
"PATCH",
|
||||
`/repos/${owner}/${repo}/pulls/${pullRequest.number}`,
|
||||
{ state: "closed" },
|
||||
);
|
||||
}
|
||||
|
||||
console.log(JSON.stringify({ mode, pullNumber: pullRequest.number, ...result }));
|
||||
if (mode !== "observe" && !result.ok) {
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (import.meta.url === pathToFileURL(process.argv[1]).href) {
|
||||
run().catch((error) => {
|
||||
console.error(error);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
}
|
||||
166
.github/scripts/issue-gate.test.mjs
vendored
Normal file
166
.github/scripts/issue-gate.test.mjs
vendored
Normal file
@@ -0,0 +1,166 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
|
||||
import {
|
||||
evaluatePullRequest,
|
||||
parseIssueReferences,
|
||||
} from "./issue-gate.mjs";
|
||||
|
||||
const repository = "crewAIInc/crewAI";
|
||||
|
||||
function pullRequest(overrides = {}) {
|
||||
return {
|
||||
body: "Implements #123",
|
||||
created_at: "2026-08-12T00:00:00Z",
|
||||
labels: [],
|
||||
user: { login: "contributor" },
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function issue(overrides = {}) {
|
||||
return {
|
||||
number: 123,
|
||||
state: "open",
|
||||
labels: [{ name: "state:ready" }],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
async function evaluate(pullOverrides = {}, issueOverrides = {}) {
|
||||
return evaluatePullRequest({
|
||||
pullRequest: pullRequest(pullOverrides),
|
||||
repository,
|
||||
cutoff: "2026-08-11T00:00:00Z",
|
||||
getIssue: async () => issue(issueOverrides),
|
||||
});
|
||||
}
|
||||
|
||||
test("parses shorthand and full repository references", () => {
|
||||
assert.deepEqual(
|
||||
parseIssueReferences(
|
||||
"Implements #123\nImplements crewAIInc/crewAI#456",
|
||||
repository,
|
||||
),
|
||||
[
|
||||
{ repository, number: 123 },
|
||||
{ repository, number: 456 },
|
||||
],
|
||||
);
|
||||
});
|
||||
|
||||
test("ignores GitHub closing keywords and template placeholders", () => {
|
||||
assert.deepEqual(
|
||||
parseIssueReferences(
|
||||
"Fixes #123\nCloses #456\nImplements #<issue-number>",
|
||||
repository,
|
||||
),
|
||||
[],
|
||||
);
|
||||
});
|
||||
|
||||
test("deduplicates repeated references", () => {
|
||||
assert.deepEqual(
|
||||
parseIssueReferences("Implements #123\nimplements #123", repository),
|
||||
[{ repository, number: 123 }],
|
||||
);
|
||||
});
|
||||
|
||||
test("passes an open ready issue", async () => {
|
||||
const result = await evaluate();
|
||||
|
||||
assert.equal(result.ok, true);
|
||||
assert.equal(result.exempt, false);
|
||||
assert.equal(result.issueNumber, 123);
|
||||
});
|
||||
|
||||
test("rejects a missing issue reference", async () => {
|
||||
const result = await evaluate({ body: "No issue yet" });
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /does not contain/);
|
||||
});
|
||||
|
||||
test("rejects more than one implementation issue", async () => {
|
||||
const result = await evaluate({
|
||||
body: "Implements #123\nImplements #456",
|
||||
});
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /more than one/);
|
||||
});
|
||||
|
||||
test("rejects an issue in another repository", async () => {
|
||||
const result = await evaluate({
|
||||
body: "Implements another/project#123",
|
||||
});
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /must belong/);
|
||||
});
|
||||
|
||||
test("rejects a pull request reference masquerading as an issue", async () => {
|
||||
const result = await evaluate({}, { pull_request: { url: "example" } });
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /is not an issue/);
|
||||
});
|
||||
|
||||
test("rejects a closed issue", async () => {
|
||||
const result = await evaluate({}, { state: "closed" });
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /is not open/);
|
||||
});
|
||||
|
||||
test("rejects an issue without the ready label", async () => {
|
||||
const result = await evaluate({}, { labels: [{ name: "state:design" }] });
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /does not have/);
|
||||
});
|
||||
|
||||
test("exempts pull requests created before the cutoff", async () => {
|
||||
const result = await evaluate({
|
||||
body: null,
|
||||
created_at: "2026-08-10T23:59:59Z",
|
||||
});
|
||||
|
||||
assert.equal(result.ok, true);
|
||||
assert.equal(result.exempt, true);
|
||||
assert.match(result.reason, /predates/);
|
||||
});
|
||||
|
||||
test("exempts pull requests with an override label", async () => {
|
||||
const result = await evaluate({
|
||||
body: null,
|
||||
labels: [{ name: "issue-gate:override" }],
|
||||
});
|
||||
|
||||
assert.equal(result.ok, true);
|
||||
assert.equal(result.exempt, true);
|
||||
assert.match(result.reason, /override/);
|
||||
});
|
||||
|
||||
test("exempts supported automation accounts", async () => {
|
||||
const result = await evaluate({
|
||||
body: null,
|
||||
user: { login: "dependabot[bot]" },
|
||||
});
|
||||
|
||||
assert.equal(result.ok, true);
|
||||
assert.equal(result.exempt, true);
|
||||
assert.match(result.reason, /automation account/);
|
||||
});
|
||||
|
||||
test("fails fast for an invalid cutoff", async () => {
|
||||
await assert.rejects(
|
||||
evaluatePullRequest({
|
||||
pullRequest: pullRequest(),
|
||||
repository,
|
||||
cutoff: "not-a-date",
|
||||
getIssue: async () => issue(),
|
||||
}),
|
||||
/not a valid date/,
|
||||
);
|
||||
});
|
||||
24
.github/workflows/issue-gate-tests.yml
vendored
Normal file
24
.github/workflows/issue-gate-tests.yml
vendored
Normal file
@@ -0,0 +1,24 @@
|
||||
name: Test Issue Gate
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/scripts/issue-gate.mjs'
|
||||
- '.github/scripts/issue-gate.test.mjs'
|
||||
- '.github/workflows/issue-gate.yml'
|
||||
- '.github/workflows/issue-gate-tests.yml'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test-issue-gate:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Run issue gate tests
|
||||
run: node --test .github/scripts/issue-gate.test.mjs
|
||||
46
.github/workflows/issue-gate.yml
vendored
Normal file
46
.github/workflows/issue-gate.yml
vendored
Normal file
@@ -0,0 +1,46 @@
|
||||
name: Issue Gate
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
branches: [main]
|
||||
types: [opened, edited, reopened, synchronize, labeled, unlabeled]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
pr_number:
|
||||
description: Pull request number to evaluate
|
||||
required: true
|
||||
type: number
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: write
|
||||
statuses: write
|
||||
|
||||
concurrency:
|
||||
group: issue-gate-${{ inputs.pr_number || github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
issue-gate:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
# pull_request_target is privileged. Explicitly check out only the trusted
|
||||
# default branch and never fetch or execute pull request code.
|
||||
- name: Check out trusted gate implementation
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
|
||||
with:
|
||||
ref: ${{ github.event.repository.default_branch }}
|
||||
persist-credentials: false
|
||||
|
||||
- name: Evaluate ready issue
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
# Until a cutoff is configured, every PR is treated as legacy. This
|
||||
# keeps the newly merged workflow inert by default.
|
||||
ISSUE_GATE_CUTOFF: ${{ vars.ISSUE_GATE_CUTOFF || '9999-12-31T00:00:00Z' }}
|
||||
ISSUE_GATE_MODE: ${{ vars.ISSUE_GATE_MODE || 'observe' }}
|
||||
ISSUE_GATE_READY_LABEL: ${{ vars.ISSUE_GATE_READY_LABEL || 'state:ready' }}
|
||||
PR_NUMBER: ${{ inputs.pr_number || github.event.pull_request.number }}
|
||||
run: node .github/scripts/issue-gate.mjs
|
||||
@@ -54,6 +54,7 @@ from crewai.events.types.flow_events import (
|
||||
MethodExecutionPausedEvent,
|
||||
MethodExecutionStartedEvent,
|
||||
)
|
||||
from crewai.events.types.hook_events import HookDispatchedEvent
|
||||
from crewai.events.types.knowledge_events import (
|
||||
KnowledgeQueryCompletedEvent,
|
||||
KnowledgeQueryFailedEvent,
|
||||
@@ -875,5 +876,12 @@ class EventListener(BaseEventListener):
|
||||
if has_hooks:
|
||||
self._telemetry.feature_usage_span("hooks:registered")
|
||||
|
||||
@crewai_event_bus.on(HookDispatchedEvent)
|
||||
def on_hook_dispatched(_: Any, event: HookDispatchedEvent) -> None:
|
||||
self._telemetry.hook_dispatched_span(
|
||||
interception_point=event.interception_point,
|
||||
outcome=event.outcome,
|
||||
)
|
||||
|
||||
|
||||
event_listener = EventListener()
|
||||
|
||||
@@ -438,6 +438,17 @@ class BaseLLM(BaseModel, ABC):
|
||||
"""
|
||||
return DEFAULT_SUPPORTS_STOP_WORDS
|
||||
|
||||
def _supports_stop_words_implementation(self) -> bool:
|
||||
"""Check if stop words are configured for this LLM instance.
|
||||
|
||||
Native providers can override supports_stop_words() to return this value
|
||||
to ensure consistent behavior based on whether stop words are actually configured.
|
||||
|
||||
Returns:
|
||||
True if stop words are configured and can be applied
|
||||
"""
|
||||
return bool(self.stop_sequences)
|
||||
|
||||
def _apply_stop_words(self, content: str) -> str:
|
||||
"""Apply stop words to truncate response content.
|
||||
|
||||
|
||||
@@ -1385,6 +1385,120 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
)
|
||||
|
||||
# TODO: we drop this
|
||||
def _handle_tool_use_conversation(
|
||||
self,
|
||||
initial_response: Message | BetaMessage,
|
||||
tool_uses: list[_AnthropicToolUseBlock],
|
||||
params: dict[str, Any],
|
||||
available_functions: dict[str, Any],
|
||||
from_task: Any | None = None,
|
||||
from_agent: Any | None = None,
|
||||
) -> str:
|
||||
"""Handle the complete tool use conversation flow.
|
||||
|
||||
This implements the proper Anthropic tool use pattern:
|
||||
1. Claude requests tool use
|
||||
2. We execute the tools
|
||||
3. We send tool results back to Claude
|
||||
4. Claude processes results and generates final response
|
||||
"""
|
||||
tool_results = self._execute_tools_and_collect_results(
|
||||
tool_uses, available_functions, from_task, from_agent
|
||||
)
|
||||
|
||||
follow_up_params = params.copy()
|
||||
|
||||
assistant_content: list[
|
||||
ThinkingBlock | ToolUseBlock | TextBlock | dict[str, Any]
|
||||
] = []
|
||||
for block in initial_response.content:
|
||||
thinking_block = self._extract_thinking_block(block)
|
||||
if thinking_block:
|
||||
assistant_content.append(thinking_block)
|
||||
elif _is_tool_use_block(block):
|
||||
assistant_content.append(
|
||||
{
|
||||
"type": "tool_use",
|
||||
"id": _tool_use_id(block),
|
||||
"name": _tool_use_name(block),
|
||||
"input": _tool_use_input(block),
|
||||
}
|
||||
)
|
||||
elif hasattr(block, "text"):
|
||||
assistant_content.append({"type": "text", "text": block.text})
|
||||
|
||||
assistant_message = {"role": "assistant", "content": assistant_content}
|
||||
|
||||
user_message = {"role": "user", "content": tool_results}
|
||||
|
||||
follow_up_params["messages"] = params["messages"] + [
|
||||
assistant_message,
|
||||
user_message,
|
||||
]
|
||||
|
||||
try:
|
||||
final_response: Message = self._get_sync_client().messages.create(
|
||||
**follow_up_params
|
||||
)
|
||||
|
||||
follow_up_usage = self._extract_anthropic_token_usage(final_response)
|
||||
self._track_token_usage_internal(follow_up_usage)
|
||||
|
||||
final_content = ""
|
||||
thinking_blocks: list[ThinkingBlock] = []
|
||||
|
||||
if final_response.content:
|
||||
for content_block in final_response.content:
|
||||
if hasattr(content_block, "text"):
|
||||
final_content += content_block.text
|
||||
else:
|
||||
thinking_block = self._extract_thinking_block(content_block)
|
||||
if thinking_block:
|
||||
thinking_blocks.append(cast(ThinkingBlock, thinking_block))
|
||||
|
||||
if thinking_blocks:
|
||||
self._previous_thinking_blocks = thinking_blocks
|
||||
|
||||
final_content = self._apply_stop_words(final_content)
|
||||
|
||||
finish_reason, final_response_id = self._extract_finish_reason_and_id(
|
||||
final_response
|
||||
)
|
||||
|
||||
self._emit_call_completed_event(
|
||||
response=final_content,
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
messages=follow_up_params["messages"],
|
||||
usage=follow_up_usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=final_response_id,
|
||||
)
|
||||
|
||||
total_usage = {
|
||||
"input_tokens": follow_up_usage.get("input_tokens", 0),
|
||||
"output_tokens": follow_up_usage.get("output_tokens", 0),
|
||||
"total_tokens": follow_up_usage.get("total_tokens", 0),
|
||||
}
|
||||
|
||||
if total_usage.get("total_tokens", 0) > 0:
|
||||
logging.info(f"Anthropic API tool conversation usage: {total_usage}")
|
||||
|
||||
return final_content
|
||||
|
||||
except Exception as e:
|
||||
if is_context_length_exceeded(e):
|
||||
logging.error(f"Context window exceeded in tool follow-up: {e}")
|
||||
raise LLMContextLengthExceededError(str(e)) from e
|
||||
|
||||
logging.error(f"Tool follow-up conversation failed: {e}")
|
||||
# Fallback to first tool result when follow-up fails
|
||||
if tool_results:
|
||||
return cast(str, tool_results[0]["content"])
|
||||
raise e
|
||||
|
||||
async def _ahandle_completion(
|
||||
self,
|
||||
params: dict[str, Any],
|
||||
@@ -1716,6 +1830,90 @@ class AnthropicCompletion(BaseLLM):
|
||||
|
||||
return full_response
|
||||
|
||||
async def _ahandle_tool_use_conversation(
|
||||
self,
|
||||
initial_response: Message | BetaMessage,
|
||||
tool_uses: list[_AnthropicToolUseBlock],
|
||||
params: dict[str, Any],
|
||||
available_functions: dict[str, Any],
|
||||
from_task: Any | None = None,
|
||||
from_agent: Any | None = None,
|
||||
) -> str:
|
||||
"""Handle the complete async tool use conversation flow.
|
||||
|
||||
This implements the proper Anthropic tool use pattern:
|
||||
1. Claude requests tool use
|
||||
2. We execute the tools
|
||||
3. We send tool results back to Claude
|
||||
4. Claude processes results and generates final response
|
||||
"""
|
||||
tool_results = self._execute_tools_and_collect_results(
|
||||
tool_uses, available_functions, from_task, from_agent
|
||||
)
|
||||
|
||||
follow_up_params = params.copy()
|
||||
|
||||
assistant_message = {"role": "assistant", "content": initial_response.content}
|
||||
|
||||
user_message = {"role": "user", "content": tool_results}
|
||||
|
||||
follow_up_params["messages"] = params["messages"] + [
|
||||
assistant_message,
|
||||
user_message,
|
||||
]
|
||||
|
||||
try:
|
||||
final_response: Message = await self._get_async_client().messages.create(
|
||||
**follow_up_params
|
||||
)
|
||||
|
||||
follow_up_usage = self._extract_anthropic_token_usage(final_response)
|
||||
self._track_token_usage_internal(follow_up_usage)
|
||||
|
||||
final_content = ""
|
||||
if final_response.content:
|
||||
for content_block in final_response.content:
|
||||
if hasattr(content_block, "text"):
|
||||
final_content += content_block.text
|
||||
|
||||
final_content = self._apply_stop_words(final_content)
|
||||
|
||||
finish_reason, final_response_id = self._extract_finish_reason_and_id(
|
||||
final_response
|
||||
)
|
||||
|
||||
self._emit_call_completed_event(
|
||||
response=final_content,
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
messages=follow_up_params["messages"],
|
||||
usage=follow_up_usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=final_response_id,
|
||||
)
|
||||
|
||||
total_usage = {
|
||||
"input_tokens": follow_up_usage.get("input_tokens", 0),
|
||||
"output_tokens": follow_up_usage.get("output_tokens", 0),
|
||||
"total_tokens": follow_up_usage.get("total_tokens", 0),
|
||||
}
|
||||
|
||||
if total_usage.get("total_tokens", 0) > 0:
|
||||
logging.info(f"Anthropic API tool conversation usage: {total_usage}")
|
||||
|
||||
return final_content
|
||||
|
||||
except Exception as e:
|
||||
if is_context_length_exceeded(e):
|
||||
logging.error(f"Context window exceeded in tool follow-up: {e}")
|
||||
raise LLMContextLengthExceededError(str(e)) from e
|
||||
|
||||
logging.error(f"Tool follow-up conversation failed: {e}")
|
||||
if tool_results:
|
||||
return cast(str, tool_results[0]["content"])
|
||||
raise e
|
||||
|
||||
def supports_function_calling(self) -> bool:
|
||||
"""Check if the model supports function calling."""
|
||||
return self.supports_tools
|
||||
|
||||
@@ -2146,6 +2146,17 @@ class BedrockCompletion(BaseLLM):
|
||||
)
|
||||
return any(model_lower.startswith(m) for m in vision_models)
|
||||
|
||||
def _is_nova_model(self) -> bool:
|
||||
"""Check if the model is an Amazon Nova model.
|
||||
|
||||
Only Nova models support S3 links for multimedia.
|
||||
|
||||
Returns:
|
||||
True if the model is a Nova model.
|
||||
"""
|
||||
model_lower = self.model.lower()
|
||||
return "amazon.nova-" in model_lower
|
||||
|
||||
def get_file_uploader(self) -> Any:
|
||||
"""Get a Bedrock S3 file uploader using this LLM's AWS credentials.
|
||||
|
||||
@@ -2174,6 +2185,49 @@ class BedrockCompletion(BaseLLM):
|
||||
except ImportError:
|
||||
return None
|
||||
|
||||
def _get_document_format(self, content_type: str) -> str | None:
|
||||
"""Map content type to Bedrock document format.
|
||||
|
||||
Args:
|
||||
content_type: MIME type of the document.
|
||||
|
||||
Returns:
|
||||
Bedrock format string or None if unsupported.
|
||||
"""
|
||||
format_map = {
|
||||
"application/pdf": "pdf",
|
||||
"text/csv": "csv",
|
||||
"text/plain": "txt",
|
||||
"text/markdown": "md",
|
||||
"text/html": "html",
|
||||
"application/msword": "doc",
|
||||
"application/vnd.openxmlformats-officedocument.wordprocessingml.document": "docx",
|
||||
"application/vnd.ms-excel": "xls",
|
||||
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet": "xlsx",
|
||||
}
|
||||
return format_map.get(content_type)
|
||||
|
||||
def _get_video_format(self, content_type: str) -> str | None:
|
||||
"""Map content type to Bedrock video format.
|
||||
|
||||
Args:
|
||||
content_type: MIME type of the video.
|
||||
|
||||
Returns:
|
||||
Bedrock format string or None if unsupported.
|
||||
"""
|
||||
format_map = {
|
||||
"video/mp4": "mp4",
|
||||
"video/quicktime": "mov",
|
||||
"video/x-matroska": "mkv",
|
||||
"video/webm": "webm",
|
||||
"video/x-flv": "flv",
|
||||
"video/mpeg": "mpeg",
|
||||
"video/x-ms-wmv": "wmv",
|
||||
"video/3gpp": "three_gp",
|
||||
}
|
||||
return format_map.get(content_type)
|
||||
|
||||
def format_text_content(self, text: str) -> dict[str, Any]:
|
||||
"""Format text as a Bedrock content block.
|
||||
|
||||
|
||||
@@ -1148,7 +1148,8 @@ class Telemetry:
|
||||
|
||||
Args:
|
||||
feature: Feature identifier, e.g. "planning:creation",
|
||||
"mcp:connection", "a2a:delegation".
|
||||
"mcp:connection", "a2a:delegation",
|
||||
"hooks:pre_tool_call", "hooks:aborted".
|
||||
"""
|
||||
|
||||
def _operation() -> None:
|
||||
@@ -1160,6 +1161,21 @@ class Telemetry:
|
||||
|
||||
self._safe_telemetry_operation(_operation)
|
||||
|
||||
def hook_dispatched_span(
|
||||
self,
|
||||
interception_point: str,
|
||||
outcome: str,
|
||||
) -> None:
|
||||
"""Records an interception-hook dispatch via Feature Usage.
|
||||
|
||||
Emits ``hooks:<point>`` on every dispatch, plus ``hooks:aborted`` when
|
||||
a hook aborted the operation (e.g. a policy check). No reasons,
|
||||
payloads, or other user content are recorded.
|
||||
"""
|
||||
self.feature_usage_span(f"hooks:{interception_point}")
|
||||
if outcome == "aborted":
|
||||
self.feature_usage_span("hooks:aborted")
|
||||
|
||||
def coding_agent_span(self) -> None:
|
||||
"""Records which AI coding assistant (if any) is running this process.
|
||||
|
||||
|
||||
@@ -1576,6 +1576,30 @@ def test_anthropic_dict_tool_use_blocks_execute_available_function():
|
||||
assert result == "found CrewAI"
|
||||
|
||||
|
||||
def test_anthropic_dict_tool_use_blocks_work_in_follow_up_conversation():
|
||||
from crewai.llms.providers.anthropic.completion import AnthropicCompletion
|
||||
|
||||
llm = AnthropicCompletion(model="claude-fable-5")
|
||||
initial_response = _dict_tool_use_response()
|
||||
final_response = MagicMock()
|
||||
final_response.content = [types.SimpleNamespace(text="Final answer")]
|
||||
final_response.usage = MagicMock(input_tokens=4, output_tokens=3)
|
||||
final_response.stop_reason = "end_turn"
|
||||
final_response.id = "msg_final"
|
||||
mock_client = MagicMock()
|
||||
mock_client.messages.create.return_value = final_response
|
||||
llm._client = mock_client
|
||||
|
||||
result = llm._handle_tool_use_conversation(
|
||||
initial_response,
|
||||
initial_response.content,
|
||||
params={"messages": []},
|
||||
available_functions={"search_web": lambda query: f"found {query}"},
|
||||
)
|
||||
|
||||
assert result == "Final answer"
|
||||
|
||||
|
||||
@pytest.mark.vcr()
|
||||
def test_tool_search_discovers_and_calls_tool():
|
||||
"""Tool search should discover the right tool and return a tool_use block."""
|
||||
|
||||
@@ -230,3 +230,73 @@ def test_no_signal_handler_traceback_in_non_main_thread():
|
||||
mock_holder["logger"].debug.assert_any_call(
|
||||
"Skipping signal handler registration: not running in main thread"
|
||||
)
|
||||
|
||||
|
||||
def test_hook_dispatched_span_counts_point_usage():
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"CREWAI_DISABLE_TELEMETRY": "false",
|
||||
"CREWAI_DISABLE_TRACKING": "false",
|
||||
"OTEL_SDK_DISABLED": "false",
|
||||
},
|
||||
),
|
||||
patch("crewai.telemetry.telemetry.TracerProvider"),
|
||||
):
|
||||
telemetry = Telemetry()
|
||||
with patch.object(telemetry, "feature_usage_span") as feature_usage_span:
|
||||
telemetry.hook_dispatched_span("pre_tool_call", "proceeded")
|
||||
|
||||
feature_usage_span.assert_called_once_with("hooks:pre_tool_call")
|
||||
|
||||
|
||||
def test_hook_dispatched_span_counts_aborts():
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"CREWAI_DISABLE_TELEMETRY": "false",
|
||||
"CREWAI_DISABLE_TRACKING": "false",
|
||||
"OTEL_SDK_DISABLED": "false",
|
||||
},
|
||||
),
|
||||
patch("crewai.telemetry.telemetry.TracerProvider"),
|
||||
):
|
||||
telemetry = Telemetry()
|
||||
with patch.object(telemetry, "feature_usage_span") as feature_usage_span:
|
||||
telemetry.hook_dispatched_span("pre_tool_call", "aborted")
|
||||
|
||||
feature_usage_span.assert_any_call("hooks:pre_tool_call")
|
||||
feature_usage_span.assert_any_call("hooks:aborted")
|
||||
assert feature_usage_span.call_count == 2
|
||||
|
||||
|
||||
def test_event_listener_tracks_hook_dispatched_events():
|
||||
from crewai.events.event_bus import crewai_event_bus
|
||||
from crewai.events.event_listener import event_listener
|
||||
from crewai.events.types.hook_events import HookDispatchedEvent
|
||||
|
||||
with (
|
||||
crewai_event_bus.scoped_handlers(),
|
||||
patch.object(
|
||||
event_listener._telemetry,
|
||||
"hook_dispatched_span",
|
||||
) as hook_dispatched_span,
|
||||
):
|
||||
event_listener.setup_listeners(crewai_event_bus)
|
||||
crewai_event_bus.emit(
|
||||
"test",
|
||||
HookDispatchedEvent(
|
||||
interception_point="pre_tool_call",
|
||||
outcome="aborted",
|
||||
hook_count=1,
|
||||
duration_ms=1.5,
|
||||
),
|
||||
)
|
||||
crewai_event_bus.flush()
|
||||
|
||||
hook_dispatched_span.assert_called_once_with(
|
||||
interception_point="pre_tool_call",
|
||||
outcome="aborted",
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user