mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-08-04 21:41:44 +00:00
Compare commits
3 Commits
lorenze/im
...
cursor/fix
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f0a7dcf1d3 | ||
|
|
7bb690dad0 | ||
|
|
965fdb36e4 |
38
.github/CONTRIBUTING.md
vendored
38
.github/CONTRIBUTING.md
vendored
@@ -42,40 +42,6 @@ Documentation lives in `docs/` with translations under `docs/{en,ar,ko,pt-BR}/`.
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Issue-First Contributions
|
||||
|
||||
CrewAI is piloting an issue-first contribution workflow. The issue is where
|
||||
contributors and maintainers agree on the problem and intended solution before
|
||||
implementation begins.
|
||||
|
||||
Issues move through the following lifecycle:
|
||||
|
||||
```
|
||||
state:inbox -> state:design -> state:ready -> state:in-progress -> state:verification -> Done
|
||||
```
|
||||
|
||||
Maintainers apply `state:ready` after the issue has a clear problem statement,
|
||||
agreed scope and non-goals, acceptance criteria, and a verification plan. A
|
||||
pull request should not begin before its issue is ready unless a maintainer has
|
||||
approved an exception.
|
||||
|
||||
Every new pull request must reference exactly one ready issue in its description:
|
||||
|
||||
```
|
||||
Implements #123
|
||||
```
|
||||
|
||||
Use `Implements`, not `Closes`, `Fixes`, or `Resolves`. GitHub's closing
|
||||
keywords close the issue as soon as the pull request merges, but issue-first
|
||||
contributions remain open in `state:verification` until a human confirms the
|
||||
result.
|
||||
|
||||
The issue gate currently runs in **observe mode**. It reports whether a pull
|
||||
request would satisfy this policy, but it does not block or close pull requests.
|
||||
Maintainers can apply `issue-gate:override` for security work, release
|
||||
automation, urgent fixes, or another documented exception. Pull requests that
|
||||
predate the pilot can be marked `policy:legacy`.
|
||||
|
||||
### Branching
|
||||
|
||||
Create a branch off `main` using the conventional commit type:
|
||||
@@ -137,9 +103,7 @@ chore(deps): bump pydantic to 2.11
|
||||
- Keep PRs focused — avoid bundling unrelated changes
|
||||
- PRs over 500 lines are labeled `size/XL` automatically
|
||||
- Title must follow the same conventional commit format
|
||||
- Reference exactly one ready issue with `Implements #<issue-number>`
|
||||
- Explain how the implementation meets the issue's acceptance criteria
|
||||
- Include the automated and manual verification performed
|
||||
- Link related issues where applicable
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
56
.github/ISSUE_GATE.md
vendored
56
.github/ISSUE_GATE.md
vendored
@@ -1,56 +0,0 @@
|
||||
# Issue Gate Pilot
|
||||
|
||||
The issue gate checks whether a pull request references exactly one open issue
|
||||
with the `state:ready` label. It runs on `pull_request_target`, checks out only
|
||||
the trusted default branch, and never fetches or executes pull request code.
|
||||
|
||||
## Repository setup
|
||||
|
||||
Create these labels before activating the pilot:
|
||||
|
||||
- Issue lifecycle: `state:inbox`, `state:design`, `state:ready`,
|
||||
`state:in-progress`, and `state:verification`
|
||||
- Gate results: `issue-gate:passed`, `issue-gate:exempt`, and
|
||||
`needs-ready-issue`
|
||||
- Maintainer exceptions: `issue-gate:override` and `policy:legacy`
|
||||
|
||||
Configure these repository variables:
|
||||
|
||||
| Variable | Required | Value |
|
||||
| --- | --- | --- |
|
||||
| `ISSUE_GATE_MODE` | No | `observe` (default), `block`, or `close` |
|
||||
| `ISSUE_GATE_CUTOFF` | Yes to activate | ISO 8601 timestamp for the first PR covered by the pilot |
|
||||
| `ISSUE_GATE_READY_LABEL` | No | Defaults to `state:ready` |
|
||||
|
||||
Without `ISSUE_GATE_CUTOFF`, the workflow uses a future cutoff and treats every
|
||||
pull request as legacy. This makes the workflow inert until maintainers choose
|
||||
the activation time.
|
||||
|
||||
## Modes
|
||||
|
||||
- `observe`: Invalid pull requests receive a successful `Issue gate` status,
|
||||
an explanatory comment, and the `needs-ready-issue` label.
|
||||
- `block`: Invalid pull requests receive a failing status. Add `Issue gate` to
|
||||
the `main` ruleset's required checks only after observation is complete.
|
||||
- `close`: Invalid pull requests receive a failing status and are closed after
|
||||
the comment is posted.
|
||||
|
||||
The script refuses to enter `block` or `close` mode without a configured cutoff.
|
||||
Pull requests created by supported dependency automation, or labeled
|
||||
`issue-gate:override` or `policy:legacy`, remain exempt.
|
||||
|
||||
## Testing the pilot
|
||||
|
||||
Run the `Issue Gate` workflow manually with a pull request number, or edit a
|
||||
pull request description to trigger it again. Test at least these cases before
|
||||
changing modes:
|
||||
|
||||
1. No implementation issue
|
||||
2. A nonexistent, closed, or non-ready issue
|
||||
3. More than one implementation issue
|
||||
4. One open issue labeled `state:ready`
|
||||
5. A pull request before the cutoff
|
||||
6. A pull request with `issue-gate:override`
|
||||
|
||||
Keep the gate in `observe` mode until the sample contains at least 20 new pull
|
||||
requests with no false positives.
|
||||
17
.github/ISSUE_TEMPLATE/bug_report.yml
vendored
17
.github/ISSUE_TEMPLATE/bug_report.yml
vendored
@@ -1,7 +1,7 @@
|
||||
name: Bug report
|
||||
description: Create a report to help us improve CrewAI
|
||||
title: "[BUG]"
|
||||
labels: ["bug", "state:inbox"]
|
||||
labels: ["bug"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: textarea
|
||||
@@ -30,13 +30,6 @@ body:
|
||||
description: A clear and concise description of what you expected to happen.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: impact
|
||||
attributes:
|
||||
label: Impact
|
||||
description: Describe who is affected and how this problem affects their work.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: screenshots-code
|
||||
attributes:
|
||||
@@ -113,14 +106,6 @@ body:
|
||||
description: Have a solution in mind? Please suggest it here, or write "None".
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: verification-plan
|
||||
attributes:
|
||||
label: Verification plan
|
||||
description: Describe how a maintainer can confirm that a future fix resolves the problem.
|
||||
placeholder: Include a minimal reproduction, expected assertions, or a manual verification procedure.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
|
||||
32
.github/ISSUE_TEMPLATE/feature_request.yml
vendored
32
.github/ISSUE_TEMPLATE/feature_request.yml
vendored
@@ -1,7 +1,7 @@
|
||||
name: Feature request
|
||||
description: Suggest a new feature for CrewAI
|
||||
title: "[FEATURE]"
|
||||
labels: ["feature-request", "state:inbox"]
|
||||
labels: ["feature-request"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
@@ -26,8 +26,8 @@ body:
|
||||
- type: textarea
|
||||
id: problem
|
||||
attributes:
|
||||
label: Problem statement
|
||||
description: Describe the user problem or unmet need without prescribing an implementation.
|
||||
label: Is your feature request related to a an existing bug? Please link it here.
|
||||
description: A link to the bug or NA if not related to an existing bug.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
@@ -44,30 +44,6 @@ body:
|
||||
description: A clear and concise description of any alternative solutions or features you've considered.
|
||||
validations:
|
||||
required: false
|
||||
- type: textarea
|
||||
id: scope
|
||||
attributes:
|
||||
label: Proposed scope and non-goals
|
||||
description: Describe what should be included and what should intentionally remain out of scope.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: acceptance-criteria
|
||||
attributes:
|
||||
label: Acceptance criteria
|
||||
description: List the observable outcomes that would make this request complete.
|
||||
placeholder: |
|
||||
- [ ] A user can ...
|
||||
- [ ] Existing behavior remains ...
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: verification-plan
|
||||
attributes:
|
||||
label: Verification plan
|
||||
description: Describe the automated or manual checks that should confirm the feature works.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: context
|
||||
attributes:
|
||||
@@ -86,4 +62,4 @@ body:
|
||||
- I can test the feature once it's implemented
|
||||
- No, I'm just suggesting the idea
|
||||
validations:
|
||||
required: true
|
||||
required: true
|
||||
24
.github/PULL_REQUEST_TEMPLATE.md
vendored
24
.github/PULL_REQUEST_TEMPLATE.md
vendored
@@ -1,24 +0,0 @@
|
||||
## Ready issue
|
||||
|
||||
Implements #<issue-number>
|
||||
|
||||
<!--
|
||||
Replace the placeholder above with exactly one GitHub issue that has the
|
||||
`state:ready` label. Use "Implements", not "Closes", "Fixes", or "Resolves":
|
||||
the issue remains open for human verification after this PR is merged.
|
||||
-->
|
||||
|
||||
## Summary
|
||||
|
||||
<!-- Explain the solution and how it follows the design agreed in the issue. -->
|
||||
|
||||
## Verification
|
||||
|
||||
<!-- List the automated and manual checks used to verify the change. -->
|
||||
|
||||
- [ ] Tests added or updated for the changed behavior
|
||||
- [ ] Relevant tests and quality checks pass locally
|
||||
|
||||
## Additional context
|
||||
|
||||
<!-- Include screenshots, compatibility notes, follow-up work, or "None". -->
|
||||
18
.github/dependabot.yml
vendored
18
.github/dependabot.yml
vendored
@@ -1,3 +1,6 @@
|
||||
# To get started with Dependabot version updates, you'll need to specify which
|
||||
# package ecosystems to update and where the package manifests are located.
|
||||
# Please see the documentation for all configuration options:
|
||||
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
|
||||
|
||||
version: 2
|
||||
@@ -5,22 +8,9 @@ updates:
|
||||
- package-ecosystem: uv
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: weekly
|
||||
day: monday
|
||||
open-pull-requests-limit: 10
|
||||
interval: "weekly"
|
||||
groups:
|
||||
security-updates:
|
||||
applies-to: security-updates
|
||||
patterns:
|
||||
- "*"
|
||||
patch-minor-updates:
|
||||
applies-to: version-updates
|
||||
patterns:
|
||||
- "*"
|
||||
update-types:
|
||||
- patch
|
||||
- minor
|
||||
ignore:
|
||||
- dependency-name: "*"
|
||||
update-types:
|
||||
- version-update:semver-major
|
||||
|
||||
386
.github/scripts/issue-gate.mjs
vendored
386
.github/scripts/issue-gate.mjs
vendored
@@ -1,386 +0,0 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
const COMMENT_MARKER = "<!-- crewai-issue-gate -->";
|
||||
const GATE_LABELS = [
|
||||
"issue-gate:passed",
|
||||
"issue-gate:exempt",
|
||||
"needs-ready-issue",
|
||||
];
|
||||
const EXEMPT_LABELS = new Set(["issue-gate:override", "policy:legacy"]);
|
||||
const EXEMPT_ACTORS = new Set([
|
||||
"dependabot[bot]",
|
||||
"github-actions[bot]",
|
||||
"renovate[bot]",
|
||||
]);
|
||||
const VALID_MODES = new Set(["observe", "block", "close"]);
|
||||
|
||||
class ApiError extends Error {
|
||||
constructor(status, message) {
|
||||
super(message);
|
||||
this.name = "ApiError";
|
||||
this.status = status;
|
||||
}
|
||||
}
|
||||
|
||||
/** Return unique issue references declared with `Implements #123`. */
|
||||
export function parseIssueReferences(body, defaultRepository) {
|
||||
const references = [];
|
||||
const seen = new Set();
|
||||
const pattern =
|
||||
/^[\t ]*Implements[\t ]+(?:(?<repository>[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+))?#(?<number>\d+)\b/gim;
|
||||
|
||||
for (const match of (body ?? "").matchAll(pattern)) {
|
||||
const repository = match.groups.repository ?? defaultRepository;
|
||||
const number = Number.parseInt(match.groups.number, 10);
|
||||
const key = `${repository.toLowerCase()}#${number}`;
|
||||
|
||||
if (!seen.has(key)) {
|
||||
references.push({ repository, number });
|
||||
seen.add(key);
|
||||
}
|
||||
}
|
||||
|
||||
return references;
|
||||
}
|
||||
|
||||
function labelNames(item) {
|
||||
return new Set((item.labels ?? []).map((label) => label.name.toLowerCase()));
|
||||
}
|
||||
|
||||
function exemptionReason(pullRequest, cutoff) {
|
||||
const labels = labelNames(pullRequest);
|
||||
for (const label of EXEMPT_LABELS) {
|
||||
if (labels.has(label)) {
|
||||
return {
|
||||
kind: label === "policy:legacy" ? "legacy" : "override",
|
||||
reason: `the pull request has the \`${label}\` exemption label`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const actor = pullRequest.user?.login?.toLowerCase();
|
||||
if (EXEMPT_ACTORS.has(actor)) {
|
||||
return {
|
||||
kind: "automation",
|
||||
reason: `\`${pullRequest.user.login}\` is an exempt automation account`,
|
||||
};
|
||||
}
|
||||
|
||||
if (cutoff) {
|
||||
const cutoffTime = Date.parse(cutoff);
|
||||
if (Number.isNaN(cutoffTime)) {
|
||||
throw new Error(`ISSUE_GATE_CUTOFF is not a valid date: ${cutoff}`);
|
||||
}
|
||||
|
||||
const createdTime = Date.parse(pullRequest.created_at);
|
||||
if (Number.isNaN(createdTime)) {
|
||||
throw new Error(`Pull request has an invalid creation date: ${pullRequest.created_at}`);
|
||||
}
|
||||
|
||||
if (createdTime < cutoffTime) {
|
||||
return {
|
||||
kind: "legacy",
|
||||
reason: `the pull request predates the pilot cutoff (${cutoff})`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Evaluate a pull request without mutating GitHub state. */
|
||||
export async function evaluatePullRequest({
|
||||
pullRequest,
|
||||
repository,
|
||||
cutoff = "",
|
||||
readyLabel = "state:ready",
|
||||
getIssue,
|
||||
}) {
|
||||
const exemption = exemptionReason(pullRequest, cutoff);
|
||||
if (exemption) {
|
||||
return {
|
||||
ok: true,
|
||||
exempt: true,
|
||||
exemption: exemption.kind,
|
||||
reason: exemption.reason,
|
||||
};
|
||||
}
|
||||
|
||||
const references = parseIssueReferences(pullRequest.body, repository);
|
||||
if (references.length === 0) {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
reason: "the description does not contain `Implements #<issue-number>`",
|
||||
};
|
||||
}
|
||||
|
||||
if (references.length > 1) {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
reason: "the description references more than one implementation issue",
|
||||
};
|
||||
}
|
||||
|
||||
const reference = references[0];
|
||||
if (reference.repository.toLowerCase() !== repository.toLowerCase()) {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
reason: `the implementation issue must belong to \`${repository}\``,
|
||||
};
|
||||
}
|
||||
|
||||
const issue = await getIssue(reference.number);
|
||||
if (!issue || issue.pull_request) {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
reason: `#${reference.number} is not an issue in \`${repository}\``,
|
||||
};
|
||||
}
|
||||
|
||||
if (issue.state !== "open") {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
issueNumber: reference.number,
|
||||
reason: `issue #${reference.number} is not open`,
|
||||
};
|
||||
}
|
||||
|
||||
if (!labelNames(issue).has(readyLabel.toLowerCase())) {
|
||||
return {
|
||||
ok: false,
|
||||
exempt: false,
|
||||
issueNumber: reference.number,
|
||||
reason: `issue #${reference.number} does not have the \`${readyLabel}\` label`,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
exempt: false,
|
||||
issueNumber: reference.number,
|
||||
reason: `issue #${reference.number} is open and ready for implementation`,
|
||||
};
|
||||
}
|
||||
|
||||
async function githubApi(token, method, endpoint, body) {
|
||||
const response = await fetch(`https://api.github.com${endpoint}`, {
|
||||
method,
|
||||
headers: {
|
||||
Accept: "application/vnd.github+json",
|
||||
Authorization: `Bearer ${token}`,
|
||||
"User-Agent": "crewai-issue-gate",
|
||||
"X-GitHub-Api-Version": "2022-11-28",
|
||||
},
|
||||
body: body === undefined ? undefined : JSON.stringify(body),
|
||||
});
|
||||
|
||||
const responseText = await response.text();
|
||||
const responseBody = responseText ? JSON.parse(responseText) : null;
|
||||
if (!response.ok) {
|
||||
throw new ApiError(
|
||||
response.status,
|
||||
`${method} ${endpoint} failed (${response.status}): ${responseBody?.message ?? responseText}`,
|
||||
);
|
||||
}
|
||||
|
||||
return responseBody;
|
||||
}
|
||||
|
||||
function renderComment(result, mode) {
|
||||
const modeExplanation =
|
||||
mode === "observe"
|
||||
? "The gate is in **observe mode**, so this result does not block or close the pull request."
|
||||
: `The gate is in **${mode} mode**.`;
|
||||
|
||||
let outcome;
|
||||
if (result.exempt) {
|
||||
outcome = `This pull request is exempt because ${result.reason}.`;
|
||||
} else if (result.ok) {
|
||||
outcome = `This pull request passes: ${result.reason}.`;
|
||||
} else {
|
||||
outcome = `This pull request would be rejected because ${result.reason}.`;
|
||||
}
|
||||
|
||||
return `${COMMENT_MARKER}\n### Issue gate\n\n${outcome}\n\n${modeExplanation}\n\nTo satisfy the pilot policy, describe exactly one ready issue using \`Implements #123\`. After the issue becomes ready, edit the pull request description or ask a maintainer to re-run the Issue Gate workflow.`;
|
||||
}
|
||||
|
||||
async function updateGateComment({ token, repository, pullNumber, result, mode }) {
|
||||
const [owner, repo] = repository.split("/");
|
||||
const comments = await githubApi(
|
||||
token,
|
||||
"GET",
|
||||
`/repos/${owner}/${repo}/issues/${pullNumber}/comments?per_page=100`,
|
||||
);
|
||||
const existing = comments.find(
|
||||
(comment) => comment.user?.type === "Bot" && comment.body?.includes(COMMENT_MARKER),
|
||||
);
|
||||
if (!existing && result.ok) {
|
||||
return;
|
||||
}
|
||||
|
||||
const body = renderComment(result, mode);
|
||||
|
||||
if (existing) {
|
||||
await githubApi(
|
||||
token,
|
||||
"PATCH",
|
||||
`/repos/${owner}/${repo}/issues/comments/${existing.id}`,
|
||||
{ body },
|
||||
);
|
||||
} else {
|
||||
await githubApi(
|
||||
token,
|
||||
"POST",
|
||||
`/repos/${owner}/${repo}/issues/${pullNumber}/comments`,
|
||||
{ body },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function updateGateLabel({ token, repository, pullRequest, result }) {
|
||||
const [owner, repo] = repository.split("/");
|
||||
const desiredLabel = result.exempt
|
||||
? "issue-gate:exempt"
|
||||
: result.ok
|
||||
? "issue-gate:passed"
|
||||
: "needs-ready-issue";
|
||||
const currentLabels = labelNames(pullRequest);
|
||||
|
||||
for (const label of GATE_LABELS) {
|
||||
if (label !== desiredLabel && currentLabels.has(label)) {
|
||||
try {
|
||||
await githubApi(
|
||||
token,
|
||||
"DELETE",
|
||||
`/repos/${owner}/${repo}/issues/${pullRequest.number}/labels/${encodeURIComponent(label)}`,
|
||||
);
|
||||
} catch (error) {
|
||||
if (!(error instanceof ApiError) || error.status !== 404) {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!currentLabels.has(desiredLabel)) {
|
||||
try {
|
||||
await githubApi(
|
||||
token,
|
||||
"POST",
|
||||
`/repos/${owner}/${repo}/issues/${pullRequest.number}/labels`,
|
||||
{ labels: [desiredLabel] },
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof ApiError && [404, 422].includes(error.status)) {
|
||||
console.warn(`Could not apply missing label \`${desiredLabel}\`; create the pilot labels first.`);
|
||||
return;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function publishStatus({ token, repository, pullRequest, result, mode }) {
|
||||
const [owner, repo] = repository.split("/");
|
||||
const shouldPass = mode === "observe" || result.ok;
|
||||
const prefix = mode === "observe" && !result.ok ? "Observe: would fail" : result.ok ? "Pass" : "Fail";
|
||||
const description = `${prefix} — ${result.reason}`.slice(0, 140);
|
||||
|
||||
await githubApi(
|
||||
token,
|
||||
"POST",
|
||||
`/repos/${owner}/${repo}/statuses/${pullRequest.head.sha}`,
|
||||
{
|
||||
state: shouldPass ? "success" : "failure",
|
||||
context: "Issue gate",
|
||||
description,
|
||||
target_url: `https://github.com/${repository}/pull/${pullRequest.number}`,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
async function run() {
|
||||
const token = process.env.GITHUB_TOKEN;
|
||||
const repository = process.env.GITHUB_REPOSITORY;
|
||||
const eventPath = process.env.GITHUB_EVENT_PATH;
|
||||
const pullNumber = Number.parseInt(process.env.PR_NUMBER, 10);
|
||||
const mode = (process.env.ISSUE_GATE_MODE || "observe").toLowerCase();
|
||||
const cutoff = process.env.ISSUE_GATE_CUTOFF || "";
|
||||
const readyLabel = process.env.ISSUE_GATE_READY_LABEL || "state:ready";
|
||||
|
||||
if (!token || !repository || !eventPath || !Number.isInteger(pullNumber)) {
|
||||
throw new Error("GITHUB_TOKEN, GITHUB_REPOSITORY, GITHUB_EVENT_PATH, and PR_NUMBER are required");
|
||||
}
|
||||
if (!VALID_MODES.has(mode)) {
|
||||
throw new Error(`ISSUE_GATE_MODE must be one of: ${[...VALID_MODES].join(", ")}`);
|
||||
}
|
||||
if (mode !== "observe" && !cutoff) {
|
||||
throw new Error("ISSUE_GATE_CUTOFF is required before enabling block or close mode");
|
||||
}
|
||||
|
||||
const [owner, repo] = repository.split("/");
|
||||
const event = JSON.parse(await readFile(eventPath, "utf8"));
|
||||
const pullRequest =
|
||||
event.pull_request ??
|
||||
(await githubApi(token, "GET", `/repos/${owner}/${repo}/pulls/${pullNumber}`));
|
||||
|
||||
const result = await evaluatePullRequest({
|
||||
pullRequest,
|
||||
repository,
|
||||
cutoff,
|
||||
readyLabel,
|
||||
getIssue: async (issueNumber) => {
|
||||
try {
|
||||
return await githubApi(
|
||||
token,
|
||||
"GET",
|
||||
`/repos/${owner}/${repo}/issues/${issueNumber}`,
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof ApiError && error.status === 404) {
|
||||
return null;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
await publishStatus({ token, repository, pullRequest, result, mode });
|
||||
if (result.exemption !== "legacy") {
|
||||
await updateGateLabel({ token, repository, pullRequest, result });
|
||||
await updateGateComment({
|
||||
token,
|
||||
repository,
|
||||
pullNumber: pullRequest.number,
|
||||
result,
|
||||
mode,
|
||||
});
|
||||
}
|
||||
|
||||
if (mode === "close" && !result.ok) {
|
||||
await githubApi(
|
||||
token,
|
||||
"PATCH",
|
||||
`/repos/${owner}/${repo}/pulls/${pullRequest.number}`,
|
||||
{ state: "closed" },
|
||||
);
|
||||
}
|
||||
|
||||
console.log(JSON.stringify({ mode, pullNumber: pullRequest.number, ...result }));
|
||||
if (mode !== "observe" && !result.ok) {
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (import.meta.url === pathToFileURL(process.argv[1]).href) {
|
||||
run().catch((error) => {
|
||||
console.error(error);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
}
|
||||
166
.github/scripts/issue-gate.test.mjs
vendored
166
.github/scripts/issue-gate.test.mjs
vendored
@@ -1,166 +0,0 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
|
||||
import {
|
||||
evaluatePullRequest,
|
||||
parseIssueReferences,
|
||||
} from "./issue-gate.mjs";
|
||||
|
||||
const repository = "crewAIInc/crewAI";
|
||||
|
||||
function pullRequest(overrides = {}) {
|
||||
return {
|
||||
body: "Implements #123",
|
||||
created_at: "2026-08-12T00:00:00Z",
|
||||
labels: [],
|
||||
user: { login: "contributor" },
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function issue(overrides = {}) {
|
||||
return {
|
||||
number: 123,
|
||||
state: "open",
|
||||
labels: [{ name: "state:ready" }],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
async function evaluate(pullOverrides = {}, issueOverrides = {}) {
|
||||
return evaluatePullRequest({
|
||||
pullRequest: pullRequest(pullOverrides),
|
||||
repository,
|
||||
cutoff: "2026-08-11T00:00:00Z",
|
||||
getIssue: async () => issue(issueOverrides),
|
||||
});
|
||||
}
|
||||
|
||||
test("parses shorthand and full repository references", () => {
|
||||
assert.deepEqual(
|
||||
parseIssueReferences(
|
||||
"Implements #123\nImplements crewAIInc/crewAI#456",
|
||||
repository,
|
||||
),
|
||||
[
|
||||
{ repository, number: 123 },
|
||||
{ repository, number: 456 },
|
||||
],
|
||||
);
|
||||
});
|
||||
|
||||
test("ignores GitHub closing keywords and template placeholders", () => {
|
||||
assert.deepEqual(
|
||||
parseIssueReferences(
|
||||
"Fixes #123\nCloses #456\nImplements #<issue-number>",
|
||||
repository,
|
||||
),
|
||||
[],
|
||||
);
|
||||
});
|
||||
|
||||
test("deduplicates repeated references", () => {
|
||||
assert.deepEqual(
|
||||
parseIssueReferences("Implements #123\nimplements #123", repository),
|
||||
[{ repository, number: 123 }],
|
||||
);
|
||||
});
|
||||
|
||||
test("passes an open ready issue", async () => {
|
||||
const result = await evaluate();
|
||||
|
||||
assert.equal(result.ok, true);
|
||||
assert.equal(result.exempt, false);
|
||||
assert.equal(result.issueNumber, 123);
|
||||
});
|
||||
|
||||
test("rejects a missing issue reference", async () => {
|
||||
const result = await evaluate({ body: "No issue yet" });
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /does not contain/);
|
||||
});
|
||||
|
||||
test("rejects more than one implementation issue", async () => {
|
||||
const result = await evaluate({
|
||||
body: "Implements #123\nImplements #456",
|
||||
});
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /more than one/);
|
||||
});
|
||||
|
||||
test("rejects an issue in another repository", async () => {
|
||||
const result = await evaluate({
|
||||
body: "Implements another/project#123",
|
||||
});
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /must belong/);
|
||||
});
|
||||
|
||||
test("rejects a pull request reference masquerading as an issue", async () => {
|
||||
const result = await evaluate({}, { pull_request: { url: "example" } });
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /is not an issue/);
|
||||
});
|
||||
|
||||
test("rejects a closed issue", async () => {
|
||||
const result = await evaluate({}, { state: "closed" });
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /is not open/);
|
||||
});
|
||||
|
||||
test("rejects an issue without the ready label", async () => {
|
||||
const result = await evaluate({}, { labels: [{ name: "state:design" }] });
|
||||
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /does not have/);
|
||||
});
|
||||
|
||||
test("exempts pull requests created before the cutoff", async () => {
|
||||
const result = await evaluate({
|
||||
body: null,
|
||||
created_at: "2026-08-10T23:59:59Z",
|
||||
});
|
||||
|
||||
assert.equal(result.ok, true);
|
||||
assert.equal(result.exempt, true);
|
||||
assert.match(result.reason, /predates/);
|
||||
});
|
||||
|
||||
test("exempts pull requests with an override label", async () => {
|
||||
const result = await evaluate({
|
||||
body: null,
|
||||
labels: [{ name: "issue-gate:override" }],
|
||||
});
|
||||
|
||||
assert.equal(result.ok, true);
|
||||
assert.equal(result.exempt, true);
|
||||
assert.match(result.reason, /override/);
|
||||
});
|
||||
|
||||
test("exempts supported automation accounts", async () => {
|
||||
const result = await evaluate({
|
||||
body: null,
|
||||
user: { login: "dependabot[bot]" },
|
||||
});
|
||||
|
||||
assert.equal(result.ok, true);
|
||||
assert.equal(result.exempt, true);
|
||||
assert.match(result.reason, /automation account/);
|
||||
});
|
||||
|
||||
test("fails fast for an invalid cutoff", async () => {
|
||||
await assert.rejects(
|
||||
evaluatePullRequest({
|
||||
pullRequest: pullRequest(),
|
||||
repository,
|
||||
cutoff: "not-a-date",
|
||||
getIssue: async () => issue(),
|
||||
}),
|
||||
/not a valid date/,
|
||||
);
|
||||
});
|
||||
24
.github/workflows/issue-gate-tests.yml
vendored
24
.github/workflows/issue-gate-tests.yml
vendored
@@ -1,24 +0,0 @@
|
||||
name: Test Issue Gate
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/scripts/issue-gate.mjs'
|
||||
- '.github/scripts/issue-gate.test.mjs'
|
||||
- '.github/workflows/issue-gate.yml'
|
||||
- '.github/workflows/issue-gate-tests.yml'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test-issue-gate:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Run issue gate tests
|
||||
run: node --test .github/scripts/issue-gate.test.mjs
|
||||
46
.github/workflows/issue-gate.yml
vendored
46
.github/workflows/issue-gate.yml
vendored
@@ -1,46 +0,0 @@
|
||||
name: Issue Gate
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
branches: [main]
|
||||
types: [opened, edited, reopened, synchronize, labeled, unlabeled]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
pr_number:
|
||||
description: Pull request number to evaluate
|
||||
required: true
|
||||
type: number
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: write
|
||||
statuses: write
|
||||
|
||||
concurrency:
|
||||
group: issue-gate-${{ inputs.pr_number || github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
issue-gate:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
# pull_request_target is privileged. Explicitly check out only the trusted
|
||||
# default branch and never fetch or execute pull request code.
|
||||
- name: Check out trusted gate implementation
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
|
||||
with:
|
||||
ref: ${{ github.event.repository.default_branch }}
|
||||
persist-credentials: false
|
||||
|
||||
- name: Evaluate ready issue
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
# Until a cutoff is configured, every PR is treated as legacy. This
|
||||
# keeps the newly merged workflow inert by default.
|
||||
ISSUE_GATE_CUTOFF: ${{ vars.ISSUE_GATE_CUTOFF || '9999-12-31T00:00:00Z' }}
|
||||
ISSUE_GATE_MODE: ${{ vars.ISSUE_GATE_MODE || 'observe' }}
|
||||
ISSUE_GATE_READY_LABEL: ${{ vars.ISSUE_GATE_READY_LABEL || 'state:ready' }}
|
||||
PR_NUMBER: ${{ inputs.pr_number || github.event.pull_request.number }}
|
||||
run: node .github/scripts/issue-gate.mjs
|
||||
@@ -135,7 +135,8 @@ class DataTypes:
|
||||
|
||||
if "docs" in url.netloc or ("docs" in url.path and url.scheme != "file"):
|
||||
return DataType.DOCS_SITE
|
||||
if "github.com" in url.netloc:
|
||||
hostname = (url.hostname or "").lower()
|
||||
if hostname == "github.com" or hostname.endswith(".github.com"):
|
||||
return DataType.GITHUB
|
||||
|
||||
return DataType.WEBSITE
|
||||
|
||||
32
lib/crewai-tools/tests/rag/test_data_types.py
Normal file
32
lib/crewai-tools/tests/rag/test_data_types.py
Normal file
@@ -0,0 +1,32 @@
|
||||
"""Tests for DataTypes content classification."""
|
||||
|
||||
from crewai_tools.rag.data_types import DataType, DataTypes
|
||||
|
||||
|
||||
class TestDataTypesFromContentGitHub:
|
||||
"""GitHub URL detection must use hostname matching, not substrings."""
|
||||
|
||||
def test_github_com_url(self) -> None:
|
||||
assert (
|
||||
DataTypes.from_content("https://github.com/crewai/crewai")
|
||||
== DataType.GITHUB
|
||||
)
|
||||
|
||||
def test_github_subdomain_url(self) -> None:
|
||||
assert (
|
||||
DataTypes.from_content("https://gist.github.com/user/abc")
|
||||
== DataType.GITHUB
|
||||
)
|
||||
|
||||
def test_spoofed_github_hostname_is_website(self) -> None:
|
||||
# Substring checks like `"github.com" in netloc` would misclassify this.
|
||||
assert (
|
||||
DataTypes.from_content("https://github.com.evil.example/crewai")
|
||||
== DataType.WEBSITE
|
||||
)
|
||||
|
||||
def test_github_in_path_is_not_github(self) -> None:
|
||||
assert (
|
||||
DataTypes.from_content("https://example.com/github.com/repo")
|
||||
== DataType.WEBSITE
|
||||
)
|
||||
@@ -163,8 +163,14 @@ def test_navigate_command(mock_run, stagehand_tool):
|
||||
command_type="navigate",
|
||||
)
|
||||
|
||||
# Assertions
|
||||
assert "https://example.com" in result
|
||||
# Assertions — compare the full mocked result (avoid URL substring checks)
|
||||
assert result == "Successfully navigated to https://example.com"
|
||||
mock_run.assert_called_once_with(
|
||||
stagehand_tool,
|
||||
instruction="Go to example.com",
|
||||
url="https://example.com",
|
||||
command_type="navigate",
|
||||
)
|
||||
|
||||
|
||||
@patch(
|
||||
|
||||
@@ -54,7 +54,6 @@ from crewai.events.types.flow_events import (
|
||||
MethodExecutionPausedEvent,
|
||||
MethodExecutionStartedEvent,
|
||||
)
|
||||
from crewai.events.types.hook_events import HookDispatchedEvent
|
||||
from crewai.events.types.knowledge_events import (
|
||||
KnowledgeQueryCompletedEvent,
|
||||
KnowledgeQueryFailedEvent,
|
||||
@@ -876,12 +875,5 @@ class EventListener(BaseEventListener):
|
||||
if has_hooks:
|
||||
self._telemetry.feature_usage_span("hooks:registered")
|
||||
|
||||
@crewai_event_bus.on(HookDispatchedEvent)
|
||||
def on_hook_dispatched(_: Any, event: HookDispatchedEvent) -> None:
|
||||
self._telemetry.hook_dispatched_span(
|
||||
interception_point=event.interception_point,
|
||||
outcome=event.outcome,
|
||||
)
|
||||
|
||||
|
||||
event_listener = EventListener()
|
||||
|
||||
@@ -1148,8 +1148,7 @@ class Telemetry:
|
||||
|
||||
Args:
|
||||
feature: Feature identifier, e.g. "planning:creation",
|
||||
"mcp:connection", "a2a:delegation",
|
||||
"hooks:pre_tool_call", "hooks:aborted".
|
||||
"mcp:connection", "a2a:delegation".
|
||||
"""
|
||||
|
||||
def _operation() -> None:
|
||||
@@ -1161,21 +1160,6 @@ class Telemetry:
|
||||
|
||||
self._safe_telemetry_operation(_operation)
|
||||
|
||||
def hook_dispatched_span(
|
||||
self,
|
||||
interception_point: str,
|
||||
outcome: str,
|
||||
) -> None:
|
||||
"""Records an interception-hook dispatch via Feature Usage.
|
||||
|
||||
Emits ``hooks:<point>`` on every dispatch, plus ``hooks:aborted`` when
|
||||
a hook aborted the operation (e.g. a policy check). No reasons,
|
||||
payloads, or other user content are recorded.
|
||||
"""
|
||||
self.feature_usage_span(f"hooks:{interception_point}")
|
||||
if outcome == "aborted":
|
||||
self.feature_usage_span("hooks:aborted")
|
||||
|
||||
def coding_agent_span(self) -> None:
|
||||
"""Records which AI coding assistant (if any) is running this process.
|
||||
|
||||
|
||||
@@ -38,18 +38,33 @@ def get_temperature_tool_schema() -> dict[str, Any]:
|
||||
|
||||
@pytest.fixture
|
||||
def mock_emit() -> MagicMock:
|
||||
"""Mock the event bus emit function."""
|
||||
from crewai.events.event_bus import CrewAIEventsBus
|
||||
"""Mock the singleton event bus emit used by LLM providers.
|
||||
|
||||
with patch.object(CrewAIEventsBus, "emit") as mock:
|
||||
yield mock
|
||||
Patch the singleton instance (not only the class) so a leftover
|
||||
instance-level ``emit`` from other tests cannot shadow the mock.
|
||||
"""
|
||||
from crewai.events.event_bus import CrewAIEventsBus, crewai_event_bus
|
||||
|
||||
with (
|
||||
patch.object(CrewAIEventsBus, "emit") as class_mock,
|
||||
patch.object(crewai_event_bus, "emit", new=class_mock),
|
||||
):
|
||||
yield class_mock
|
||||
|
||||
|
||||
def _event_from_emit_call(call: Any) -> Any:
|
||||
"""Return the event argument from an emit mock call."""
|
||||
event = call.kwargs.get("event")
|
||||
if event is None and len(call.args) >= 2:
|
||||
event = call.args[1]
|
||||
return event
|
||||
|
||||
|
||||
def get_tool_call_events(mock_emit: MagicMock) -> list[LLMStreamChunkEvent]:
|
||||
"""Extract tool call streaming events from mock emit calls."""
|
||||
tool_call_events = []
|
||||
for call in mock_emit.call_args_list:
|
||||
event = call[1].get("event") if len(call) > 1 else None
|
||||
event = _event_from_emit_call(call)
|
||||
if isinstance(event, LLMStreamChunkEvent) and event.call_type == LLMCallType.TOOL_CALL:
|
||||
tool_call_events.append(event)
|
||||
return tool_call_events
|
||||
@@ -59,7 +74,7 @@ def get_all_stream_events(mock_emit: MagicMock) -> list[LLMStreamChunkEvent]:
|
||||
"""Extract all streaming events from mock emit calls."""
|
||||
stream_events = []
|
||||
for call in mock_emit.call_args_list:
|
||||
event = call[1].get("event") if len(call) > 1 else None
|
||||
event = _event_from_emit_call(call)
|
||||
if isinstance(event, LLMStreamChunkEvent):
|
||||
stream_events.append(event)
|
||||
return stream_events
|
||||
|
||||
@@ -230,73 +230,3 @@ def test_no_signal_handler_traceback_in_non_main_thread():
|
||||
mock_holder["logger"].debug.assert_any_call(
|
||||
"Skipping signal handler registration: not running in main thread"
|
||||
)
|
||||
|
||||
|
||||
def test_hook_dispatched_span_counts_point_usage():
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"CREWAI_DISABLE_TELEMETRY": "false",
|
||||
"CREWAI_DISABLE_TRACKING": "false",
|
||||
"OTEL_SDK_DISABLED": "false",
|
||||
},
|
||||
),
|
||||
patch("crewai.telemetry.telemetry.TracerProvider"),
|
||||
):
|
||||
telemetry = Telemetry()
|
||||
with patch.object(telemetry, "feature_usage_span") as feature_usage_span:
|
||||
telemetry.hook_dispatched_span("pre_tool_call", "proceeded")
|
||||
|
||||
feature_usage_span.assert_called_once_with("hooks:pre_tool_call")
|
||||
|
||||
|
||||
def test_hook_dispatched_span_counts_aborts():
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"CREWAI_DISABLE_TELEMETRY": "false",
|
||||
"CREWAI_DISABLE_TRACKING": "false",
|
||||
"OTEL_SDK_DISABLED": "false",
|
||||
},
|
||||
),
|
||||
patch("crewai.telemetry.telemetry.TracerProvider"),
|
||||
):
|
||||
telemetry = Telemetry()
|
||||
with patch.object(telemetry, "feature_usage_span") as feature_usage_span:
|
||||
telemetry.hook_dispatched_span("pre_tool_call", "aborted")
|
||||
|
||||
feature_usage_span.assert_any_call("hooks:pre_tool_call")
|
||||
feature_usage_span.assert_any_call("hooks:aborted")
|
||||
assert feature_usage_span.call_count == 2
|
||||
|
||||
|
||||
def test_event_listener_tracks_hook_dispatched_events():
|
||||
from crewai.events.event_bus import crewai_event_bus
|
||||
from crewai.events.event_listener import event_listener
|
||||
from crewai.events.types.hook_events import HookDispatchedEvent
|
||||
|
||||
with (
|
||||
crewai_event_bus.scoped_handlers(),
|
||||
patch.object(
|
||||
event_listener._telemetry,
|
||||
"hook_dispatched_span",
|
||||
) as hook_dispatched_span,
|
||||
):
|
||||
event_listener.setup_listeners(crewai_event_bus)
|
||||
crewai_event_bus.emit(
|
||||
"test",
|
||||
HookDispatchedEvent(
|
||||
interception_point="pre_tool_call",
|
||||
outcome="aborted",
|
||||
hook_count=1,
|
||||
duration_ms=1.5,
|
||||
),
|
||||
)
|
||||
crewai_event_bus.flush()
|
||||
|
||||
hook_dispatched_span.assert_called_once_with(
|
||||
interception_point="pre_tool_call",
|
||||
outcome="aborted",
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user