Merge branch 'lorenze/imp/parallel-tool-calling' of github.com:crewAIInc/crewAI into lorenze/imp/parallel-tool-calling

This commit is contained in:
lorenzejay
2026-02-19 10:54:12 -08:00
6 changed files with 414 additions and 266 deletions

View File

@@ -38,22 +38,21 @@ CrewAI Enterprise provides a comprehensive Human-in-the-Loop (HITL) management s
Configure human review checkpoints within your Flows using the `@human_feedback` decorator. When execution reaches a review point, the system pauses, notifies the assignee via email, and waits for a response. Configure human review checkpoints within your Flows using the `@human_feedback` decorator. When execution reaches a review point, the system pauses, notifies the assignee via email, and waits for a response.
```python ```python
from crewai.flow.flow import Flow, start, listen from crewai.flow.flow import Flow, start, listen, or_
from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult
class ContentApprovalFlow(Flow): class ContentApprovalFlow(Flow):
@start() @start()
def generate_content(self): def generate_content(self):
# AI generates content
return "Generated marketing copy for Q1 campaign..." return "Generated marketing copy for Q1 campaign..."
@listen(generate_content)
@human_feedback( @human_feedback(
message="Please review this content for brand compliance:", message="Please review this content for brand compliance:",
emit=["approved", "rejected", "needs_revision"], emit=["approved", "rejected", "needs_revision"],
) )
def review_content(self, content): @listen(or_("generate_content", "needs_revision"))
return content def review_content(self):
return "Marketing copy for review..."
@listen("approved") @listen("approved")
def publish_content(self, result: HumanFeedbackResult): def publish_content(self, result: HumanFeedbackResult):
@@ -62,10 +61,6 @@ class ContentApprovalFlow(Flow):
@listen("rejected") @listen("rejected")
def archive_content(self, result: HumanFeedbackResult): def archive_content(self, result: HumanFeedbackResult):
print(f"Content rejected. Reason: {result.feedback}") print(f"Content rejected. Reason: {result.feedback}")
@listen("needs_revision")
def revise_content(self, result: HumanFeedbackResult):
print(f"Revision requested: {result.feedback}")
``` ```
For complete implementation details, see the [Human Feedback in Flows](/en/learn/human-feedback-in-flows) guide. For complete implementation details, see the [Human Feedback in Flows](/en/learn/human-feedback-in-flows) guide.

View File

@@ -98,33 +98,43 @@ def handle_feedback(self, result):
When you specify `emit`, the decorator becomes a router. The human's free-form feedback is interpreted by an LLM and collapsed into one of the specified outcomes: When you specify `emit`, the decorator becomes a router. The human's free-form feedback is interpreted by an LLM and collapsed into one of the specified outcomes:
```python Code ```python Code
@start() from crewai.flow.flow import Flow, start, listen, or_
@human_feedback( from crewai.flow.human_feedback import human_feedback
message="Do you approve this content for publication?",
emit=["approved", "rejected", "needs_revision"],
llm="gpt-4o-mini",
default_outcome="needs_revision",
)
def review_content(self):
return "Draft blog post content here..."
@listen("approved") class ReviewFlow(Flow):
def publish(self, result): @start()
print(f"Publishing! User said: {result.feedback}") def generate_content(self):
return "Draft blog post content here..."
@listen("rejected") @human_feedback(
def discard(self, result): message="Do you approve this content for publication?",
print(f"Discarding. Reason: {result.feedback}") emit=["approved", "rejected", "needs_revision"],
llm="gpt-4o-mini",
default_outcome="needs_revision",
)
@listen(or_("generate_content", "needs_revision"))
def review_content(self):
return "Draft blog post content here..."
@listen("needs_revision") @listen("approved")
def revise(self, result): def publish(self, result):
print(f"Revising based on: {result.feedback}") print(f"Publishing! User said: {result.feedback}")
@listen("rejected")
def discard(self, result):
print(f"Discarding. Reason: {result.feedback}")
``` ```
When the human says something like "needs more detail", the LLM collapses that to `"needs_revision"`, which triggers `review_content` again via `or_()` — creating a revision loop. The loop continues until the outcome is `"approved"` or `"rejected"`.
<Tip> <Tip>
The LLM uses structured outputs (function calling) when available to guarantee the response is one of your specified outcomes. This makes routing reliable and predictable. The LLM uses structured outputs (function calling) when available to guarantee the response is one of your specified outcomes. This makes routing reliable and predictable.
</Tip> </Tip>
<Warning>
A `@start()` method only runs once at the beginning of the flow. If you need a revision loop, separate the start method from the review method and use `@listen(or_("trigger", "revision_outcome"))` on the review method to enable the self-loop.
</Warning>
## HumanFeedbackResult ## HumanFeedbackResult
The `HumanFeedbackResult` dataclass contains all information about a human feedback interaction: The `HumanFeedbackResult` dataclass contains all information about a human feedback interaction:
@@ -188,127 +198,183 @@ Each `HumanFeedbackResult` is appended to `human_feedback_history`, so multiple
## Complete Example: Content Approval Workflow ## Complete Example: Content Approval Workflow
Here's a full example implementing a content review and approval workflow: Here's a full example implementing a content review and approval workflow with a revision loop:
<CodeGroup> <CodeGroup>
```python Code ```python Code
from crewai.flow.flow import Flow, start, listen from crewai.flow.flow import Flow, start, listen, or_
from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult
from pydantic import BaseModel from pydantic import BaseModel
class ContentState(BaseModel): class ContentState(BaseModel):
topic: str = ""
draft: str = "" draft: str = ""
final_content: str = ""
revision_count: int = 0 revision_count: int = 0
status: str = "pending"
class ContentApprovalFlow(Flow[ContentState]): class ContentApprovalFlow(Flow[ContentState]):
"""A flow that generates content and gets human approval.""" """A flow that generates content and loops until the human approves."""
@start() @start()
def get_topic(self): def generate_draft(self):
self.state.topic = input("What topic should I write about? ") self.state.draft = "# AI Safety\n\nThis is a draft about AI Safety..."
return self.state.topic
@listen(get_topic)
def generate_draft(self, topic):
# In real use, this would call an LLM
self.state.draft = f"# {topic}\n\nThis is a draft about {topic}..."
return self.state.draft return self.state.draft
@listen(generate_draft)
@human_feedback( @human_feedback(
message="Please review this draft. Reply 'approved', 'rejected', or provide revision feedback:", message="Please review this draft. Approve, reject, or describe what needs changing:",
emit=["approved", "rejected", "needs_revision"], emit=["approved", "rejected", "needs_revision"],
llm="gpt-4o-mini", llm="gpt-4o-mini",
default_outcome="needs_revision", default_outcome="needs_revision",
) )
def review_draft(self, draft): @listen(or_("generate_draft", "needs_revision"))
return draft def review_draft(self):
self.state.revision_count += 1
return f"{self.state.draft} (v{self.state.revision_count})"
@listen("approved") @listen("approved")
def publish_content(self, result: HumanFeedbackResult): def publish_content(self, result: HumanFeedbackResult):
self.state.final_content = result.output self.state.status = "published"
print("\n✅ Content approved and published!") print(f"Content approved and published! Reviewer said: {result.feedback}")
print(f"Reviewer comment: {result.feedback}")
return "published" return "published"
@listen("rejected") @listen("rejected")
def handle_rejection(self, result: HumanFeedbackResult): def handle_rejection(self, result: HumanFeedbackResult):
print("\n❌ Content rejected") self.state.status = "rejected"
print(f"Reason: {result.feedback}") print(f"Content rejected. Reason: {result.feedback}")
return "rejected" return "rejected"
@listen("needs_revision")
def revise_content(self, result: HumanFeedbackResult):
self.state.revision_count += 1
print(f"\n📝 Revision #{self.state.revision_count} requested")
print(f"Feedback: {result.feedback}")
# In a real flow, you might loop back to generate_draft
# For this example, we just acknowledge
return "revision_requested"
# Run the flow
flow = ContentApprovalFlow() flow = ContentApprovalFlow()
result = flow.kickoff() result = flow.kickoff()
print(f"\nFlow completed. Revisions requested: {flow.state.revision_count}") print(f"\nFlow completed. Status: {flow.state.status}, Reviews: {flow.state.revision_count}")
``` ```
```text Output ```text Output
What topic should I write about? AI Safety ==================================================
OUTPUT FOR REVIEW:
==================================================
# AI Safety
This is a draft about AI Safety... (v1)
==================================================
Please review this draft. Approve, reject, or describe what needs changing:
(Press Enter to skip, or type your feedback)
Your feedback: Needs more detail on alignment research
================================================== ==================================================
OUTPUT FOR REVIEW: OUTPUT FOR REVIEW:
================================================== ==================================================
# AI Safety # AI Safety
This is a draft about AI Safety... This is a draft about AI Safety... (v2)
================================================== ==================================================
Please review this draft. Reply 'approved', 'rejected', or provide revision feedback: Please review this draft. Approve, reject, or describe what needs changing:
(Press Enter to skip, or type your feedback) (Press Enter to skip, or type your feedback)
Your feedback: Looks good, approved! Your feedback: Looks good, approved!
Content approved and published! Content approved and published! Reviewer said: Looks good, approved!
Reviewer comment: Looks good, approved!
Flow completed. Revisions requested: 0 Flow completed. Status: published, Reviews: 2
``` ```
</CodeGroup> </CodeGroup>
The key pattern is `@listen(or_("generate_draft", "needs_revision"))` — the review method listens to both the initial trigger and its own revision outcome, creating a self-loop that repeats until the human approves or rejects.
## Combining with Other Decorators ## Combining with Other Decorators
The `@human_feedback` decorator works with other flow decorators. Place it as the innermost decorator (closest to the function): The `@human_feedback` decorator works with `@start()`, `@listen()`, and `or_()`. Both decorator orderings work — the framework propagates attributes in both directions — but the recommended patterns are:
```python Code ```python Code
# Correct: @human_feedback is innermost (closest to the function) # One-shot review at the start of a flow (no self-loop)
@start() @start()
@human_feedback(message="Review this:") @human_feedback(message="Review this:", emit=["approved", "rejected"], llm="gpt-4o-mini")
def my_start_method(self): def my_start_method(self):
return "content" return "content"
# Linear review on a listener (no self-loop)
@listen(other_method) @listen(other_method)
@human_feedback(message="Review this too:") @human_feedback(message="Review this too:", emit=["good", "bad"], llm="gpt-4o-mini")
def my_listener(self, data): def my_listener(self, data):
return f"processed: {data}" return f"processed: {data}"
# Self-loop: review that can loop back for revisions
@human_feedback(message="Approve or revise?", emit=["approved", "revise"], llm="gpt-4o-mini")
@listen(or_("upstream_method", "revise"))
def review_with_loop(self):
return "content for review"
``` ```
<Tip> ### Self-loop pattern
Place `@human_feedback` as the innermost decorator (last/closest to the function) so it wraps the method directly and can capture the return value before passing to the flow system.
</Tip> To create a revision loop, the review method must listen to **both** an upstream trigger and its own revision outcome using `or_()`:
```python Code
@start()
def generate(self):
return "initial draft"
@human_feedback(
message="Approve or request changes?",
emit=["revise", "approved"],
llm="gpt-4o-mini",
default_outcome="approved",
)
@listen(or_("generate", "revise"))
def review(self):
return "content"
@listen("approved")
def publish(self):
return "published"
```
When the outcome is `"revise"`, the flow routes back to `review` (because it listens to `"revise"` via `or_()`). When the outcome is `"approved"`, the flow continues to `publish`. This works because the flow engine exempts routers from the "fire once" rule, allowing them to re-execute on each loop iteration.
### Chained routers
A listener triggered by one router's outcome can itself be a router:
```python Code
@start()
def generate(self):
return "draft content"
@human_feedback(message="First review:", emit=["approved", "rejected"], llm="gpt-4o-mini")
@listen("generate")
def first_review(self):
return "draft content"
@human_feedback(message="Final review:", emit=["publish", "hold"], llm="gpt-4o-mini")
@listen("approved")
def final_review(self, prev):
return "final content"
@listen("publish")
def on_publish(self, prev):
return "published"
@listen("hold")
def on_hold(self, prev):
return "held for later"
```
### Limitations
- **`@start()` methods run once**: A `@start()` method cannot self-loop. If you need a revision cycle, use a separate `@start()` method as the entry point and put the `@human_feedback` on a `@listen()` method.
- **No `@start()` + `@listen()` on the same method**: This is a Flow framework constraint. A method is either a start point or a listener, not both.
## Best Practices ## Best Practices
### 1. Write Clear Request Messages ### 1. Write Clear Request Messages
The `request` parameter is what the human sees. Make it actionable: The `message` parameter is what the human sees. Make it actionable:
```python Code ```python Code
# ✅ Good - clear and actionable # ✅ Good - clear and actionable
@@ -516,9 +582,9 @@ class ContentPipeline(Flow):
@start() @start()
@human_feedback( @human_feedback(
message="Approve this content for publication?", message="Approve this content for publication?",
emit=["approved", "rejected", "needs_revision"], emit=["approved", "rejected"],
llm="gpt-4o-mini", llm="gpt-4o-mini",
default_outcome="needs_revision", default_outcome="rejected",
provider=SlackNotificationProvider("#content-reviews"), provider=SlackNotificationProvider("#content-reviews"),
) )
def generate_content(self): def generate_content(self):
@@ -534,11 +600,6 @@ class ContentPipeline(Flow):
print(f"Archived. Reason: {result.feedback}") print(f"Archived. Reason: {result.feedback}")
return {"status": "archived"} return {"status": "archived"}
@listen("needs_revision")
def queue_revision(self, result):
print(f"Queued for revision: {result.feedback}")
return {"status": "revision_needed"}
# Starting the flow (will pause and wait for Slack response) # Starting the flow (will pause and wait for Slack response)
def start_content_pipeline(): def start_content_pipeline():
@@ -594,22 +655,22 @@ Over time, the human sees progressively better pre-reviewed output because each
```python Code ```python Code
class ArticleReviewFlow(Flow): class ArticleReviewFlow(Flow):
@start() @start()
def generate_article(self):
return self.crew.kickoff(inputs={"topic": "AI Safety"}).raw
@human_feedback( @human_feedback(
message="Review this article draft:", message="Review this article draft:",
emit=["approved", "needs_revision"], emit=["approved", "needs_revision"],
llm="gpt-4o-mini", llm="gpt-4o-mini",
learn=True, # enable HITL learning learn=True, # enable HITL learning
) )
def generate_article(self): @listen(or_("generate_article", "needs_revision"))
return self.crew.kickoff(inputs={"topic": "AI Safety"}).raw def review_article(self):
return self.last_human_feedback.output if self.last_human_feedback else "article draft"
@listen("approved") @listen("approved")
def publish(self): def publish(self):
print(f"Publishing: {self.last_human_feedback.output}") print(f"Publishing: {self.last_human_feedback.output}")
@listen("needs_revision")
def revise(self):
print("Revising based on feedback...")
``` ```
**First run**: The human sees the raw output and says "Always include citations for factual claims." The lesson is distilled and stored in memory. **First run**: The human sees the raw output and says "Always include citations for factual claims." The lesson is distilled and stored in memory.

View File

@@ -38,22 +38,21 @@ CrewAI Enterprise는 AI 워크플로우를 협업적인 인간-AI 프로세스
`@human_feedback` 데코레이터를 사용하여 Flow 내에 인간 검토 체크포인트를 구성합니다. 실행이 검토 포인트에 도달하면 시스템이 일시 중지되고, 담당자에게 이메일로 알리며, 응답을 기다립니다. `@human_feedback` 데코레이터를 사용하여 Flow 내에 인간 검토 체크포인트를 구성합니다. 실행이 검토 포인트에 도달하면 시스템이 일시 중지되고, 담당자에게 이메일로 알리며, 응답을 기다립니다.
```python ```python
from crewai.flow.flow import Flow, start, listen from crewai.flow.flow import Flow, start, listen, or_
from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult
class ContentApprovalFlow(Flow): class ContentApprovalFlow(Flow):
@start() @start()
def generate_content(self): def generate_content(self):
# AI가 콘텐츠 생성
return "Q1 캠페인용 마케팅 카피 생성..." return "Q1 캠페인용 마케팅 카피 생성..."
@listen(generate_content)
@human_feedback( @human_feedback(
message="브랜드 준수를 위해 이 콘텐츠를 검토해 주세요:", message="브랜드 준수를 위해 이 콘텐츠를 검토해 주세요:",
emit=["approved", "rejected", "needs_revision"], emit=["approved", "rejected", "needs_revision"],
) )
def review_content(self, content): @listen(or_("generate_content", "needs_revision"))
return content def review_content(self):
return "검토용 마케팅 카피..."
@listen("approved") @listen("approved")
def publish_content(self, result: HumanFeedbackResult): def publish_content(self, result: HumanFeedbackResult):
@@ -62,10 +61,6 @@ class ContentApprovalFlow(Flow):
@listen("rejected") @listen("rejected")
def archive_content(self, result: HumanFeedbackResult): def archive_content(self, result: HumanFeedbackResult):
print(f"콘텐츠 거부됨. 사유: {result.feedback}") print(f"콘텐츠 거부됨. 사유: {result.feedback}")
@listen("needs_revision")
def revise_content(self, result: HumanFeedbackResult):
print(f"수정 요청: {result.feedback}")
``` ```
완전한 구현 세부 사항은 [Flow에서 인간 피드백](/ko/learn/human-feedback-in-flows) 가이드를 참조하세요. 완전한 구현 세부 사항은 [Flow에서 인간 피드백](/ko/learn/human-feedback-in-flows) 가이드를 참조하세요.

View File

@@ -98,33 +98,43 @@ def handle_feedback(self, result):
`emit`을 지정하면, 데코레이터는 라우터가 됩니다. 인간의 자유 형식 피드백이 LLM에 의해 해석되어 지정된 outcome 중 하나로 매핑됩니다: `emit`을 지정하면, 데코레이터는 라우터가 됩니다. 인간의 자유 형식 피드백이 LLM에 의해 해석되어 지정된 outcome 중 하나로 매핑됩니다:
```python Code ```python Code
@start() from crewai.flow.flow import Flow, start, listen, or_
@human_feedback( from crewai.flow.human_feedback import human_feedback
message="이 콘텐츠의 출판을 승인하시겠습니까?",
emit=["approved", "rejected", "needs_revision"],
llm="gpt-4o-mini",
default_outcome="needs_revision",
)
def review_content(self):
return "블로그 게시물 초안 내용..."
@listen("approved") class ReviewFlow(Flow):
def publish(self, result): @start()
print(f"출판 중! 사용자 의견: {result.feedback}") def generate_content(self):
return "블로그 게시물 초안 내용..."
@listen("rejected") @human_feedback(
def discard(self, result): message="이 콘텐츠의 출판을 승인하시겠습니까?",
print(f"폐기됨. 이유: {result.feedback}") emit=["approved", "rejected", "needs_revision"],
llm="gpt-4o-mini",
default_outcome="needs_revision",
)
@listen(or_("generate_content", "needs_revision"))
def review_content(self):
return "블로그 게시물 초안 내용..."
@listen("needs_revision") @listen("approved")
def revise(self, result): def publish(self, result):
print(f"다음을 기반으로 수정 중: {result.feedback}") print(f"출판 중! 사용자 의견: {result.feedback}")
@listen("rejected")
def discard(self, result):
print(f"폐기됨. 이유: {result.feedback}")
``` ```
사용자가 "더 자세한 내용이 필요합니다"와 같이 말하면, LLM이 이를 `"needs_revision"`으로 매핑하고, `or_()`를 통해 `review_content`가 다시 트리거됩니다 — 수정 루프가 생성됩니다. outcome이 `"approved"` 또는 `"rejected"`가 될 때까지 루프가 계속됩니다.
<Tip> <Tip>
LLM은 가능한 경우 구조화된 출력(function calling)을 사용하여 응답이 지정된 outcome 중 하나임을 보장합니다. 이로 인해 라우팅이 신뢰할 수 있고 예측 가능해집니다. LLM은 가능한 경우 구조화된 출력(function calling)을 사용하여 응답이 지정된 outcome 중 하나임을 보장합니다. 이로 인해 라우팅이 신뢰할 수 있고 예측 가능해집니다.
</Tip> </Tip>
<Warning>
`@start()` 메서드는 flow 시작 시 한 번만 실행됩니다. 수정 루프가 필요한 경우, start 메서드를 review 메서드와 분리하고 review 메서드에 `@listen(or_("trigger", "revision_outcome"))`를 사용하여 self-loop을 활성화하세요.
</Warning>
## HumanFeedbackResult ## HumanFeedbackResult
`HumanFeedbackResult` 데이터클래스는 인간 피드백 상호작용에 대한 모든 정보를 포함합니다: `HumanFeedbackResult` 데이터클래스는 인간 피드백 상호작용에 대한 모든 정보를 포함합니다:
@@ -193,116 +203,162 @@ def summarize(self):
<CodeGroup> <CodeGroup>
```python Code ```python Code
from crewai.flow.flow import Flow, start, listen from crewai.flow.flow import Flow, start, listen, or_
from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult
from pydantic import BaseModel from pydantic import BaseModel
class ContentState(BaseModel): class ContentState(BaseModel):
topic: str = ""
draft: str = "" draft: str = ""
final_content: str = ""
revision_count: int = 0 revision_count: int = 0
status: str = "pending"
class ContentApprovalFlow(Flow[ContentState]): class ContentApprovalFlow(Flow[ContentState]):
"""콘텐츠를 생성하고 인간의 승인을 받는 Flow입니다.""" """콘텐츠를 생성하고 승인될 때까지 반복하는 Flow."""
@start() @start()
def get_topic(self): def generate_draft(self):
self.state.topic = input("어떤 주제에 대해 글을 쓸까요? ") self.state.draft = "# AI 안전\n\nAI 안전에 대한 초안..."
return self.state.topic
@listen(get_topic)
def generate_draft(self, topic):
# 실제 사용에서는 LLM을 호출합니다
self.state.draft = f"# {topic}\n\n{topic}에 대한 초안입니다..."
return self.state.draft return self.state.draft
@listen(generate_draft)
@human_feedback( @human_feedback(
message="이 초안을 검토해 주세요. 'approved', 'rejected'로 답하거나 수정 피드백을 제공해 주세요:", message="이 초안을 검토해 주세요. 승인, 거부 또는 변경이 필요한 사항을 설명해 주세요:",
emit=["approved", "rejected", "needs_revision"], emit=["approved", "rejected", "needs_revision"],
llm="gpt-4o-mini", llm="gpt-4o-mini",
default_outcome="needs_revision", default_outcome="needs_revision",
) )
def review_draft(self, draft): @listen(or_("generate_draft", "needs_revision"))
return draft def review_draft(self):
self.state.revision_count += 1
return f"{self.state.draft} (v{self.state.revision_count})"
@listen("approved") @listen("approved")
def publish_content(self, result: HumanFeedbackResult): def publish_content(self, result: HumanFeedbackResult):
self.state.final_content = result.output self.state.status = "published"
print("\n✅ 콘텐츠 승인되어 출판되었습니다!") print(f"콘텐츠 승인 및 게시! 리뷰어 의견: {result.feedback}")
print(f"검토자 코멘트: {result.feedback}")
return "published" return "published"
@listen("rejected") @listen("rejected")
def handle_rejection(self, result: HumanFeedbackResult): def handle_rejection(self, result: HumanFeedbackResult):
print("\n❌ 콘텐츠가 거부되었습니다") self.state.status = "rejected"
print(f"이유: {result.feedback}") print(f"콘텐츠 거부됨. 이유: {result.feedback}")
return "rejected" return "rejected"
@listen("needs_revision")
def revise_content(self, result: HumanFeedbackResult):
self.state.revision_count += 1
print(f"\n📝 수정 #{self.state.revision_count} 요청됨")
print(f"피드백: {result.feedback}")
# 실제 Flow에서는 generate_draft로 돌아갈 수 있습니다
# 이 예제에서는 단순히 확인합니다
return "revision_requested"
# Flow 실행
flow = ContentApprovalFlow() flow = ContentApprovalFlow()
result = flow.kickoff() result = flow.kickoff()
print(f"\nFlow 완료. 요청된 수정: {flow.state.revision_count}") print(f"\nFlow 완료. 상태: {flow.state.status}, 검토 횟수: {flow.state.revision_count}")
``` ```
```text Output ```text Output
어떤 주제에 대해 글을 쓸까요? AI 안전 ==================================================
OUTPUT FOR REVIEW:
==================================================
# AI 안전
AI 안전에 대한 초안... (v1)
==================================================
이 초안을 검토해 주세요. 승인, 거부 또는 변경이 필요한 사항을 설명해 주세요:
(Press Enter to skip, or type your feedback)
Your feedback: 더 자세한 내용이 필요합니다
================================================== ==================================================
OUTPUT FOR REVIEW: OUTPUT FOR REVIEW:
================================================== ==================================================
# AI 안전 # AI 안전
AI 안전에 대한 초안입니다... AI 안전에 대한 초안... (v2)
================================================== ==================================================
이 초안을 검토해 주세요. 'approved', 'rejected'로 답하거나 수정 피드백을 제공해 주세요: 이 초안을 검토해 주세요. 승인, 거부 또는 변경이 필요한 사항을 설명해 주세요:
(Press Enter to skip, or type your feedback) (Press Enter to skip, or type your feedback)
Your feedback: 좋아 보입니다, 승인! Your feedback: 좋아 보입니다, 승인!
콘텐츠 승인되어 출판되었습니다! 콘텐츠 승인 및 게시! 리뷰어 의견: 좋아 보입니다, 승인!
검토자 코멘트: 좋아 보입니다, 승인!
Flow 완료. 요청된 수정: 0 Flow 완료. 상태: published, 검토 횟수: 2
``` ```
</CodeGroup> </CodeGroup>
## 다른 데코레이터와 결합하기 ## 다른 데코레이터와 결합하기
`@human_feedback` 데코레이터는 다른 Flow 데코레이터와 함께 작동합니다. 가장 안쪽 데코레이터(함수에 가장 가까운)로 배치하세요: `@human_feedback` 데코레이터는 `@start()`, `@listen()`, `or_()`와 함께 작동합니다. 데코레이터 순서는 두 가지 모두 동작합니다—프레임워크가 양방향으로 속성을 전파합니다—하지만 권장 패턴은 다음과 같습니다:
```python Code ```python Code
# 올바름: @human_feedback이 가장 안쪽(함수에 가장 가까움) # Flow 시작 시 일회성 검토 (self-loop 없음)
@start() @start()
@human_feedback(message="이것을 검토해 주세요:") @human_feedback(message="이것을 검토해 주세요:", emit=["approved", "rejected"], llm="gpt-4o-mini")
def my_start_method(self): def my_start_method(self):
return "content" return "content"
# 리스너에서 선형 검토 (self-loop 없음)
@listen(other_method) @listen(other_method)
@human_feedback(message="이것도 검토해 주세요:") @human_feedback(message="이것도 검토해 주세요:", emit=["good", "bad"], llm="gpt-4o-mini")
def my_listener(self, data): def my_listener(self, data):
return f"processed: {data}" return f"processed: {data}"
# Self-loop: 수정을 위해 반복할 수 있는 검토
@human_feedback(message="승인 또는 수정 요청?", emit=["approved", "revise"], llm="gpt-4o-mini")
@listen(or_("upstream_method", "revise"))
def review_with_loop(self):
return "content for review"
``` ```
<Tip> ### Self-loop 패턴
`@human_feedback`를 가장 안쪽 데코레이터(마지막/함수에 가장 가까움)로 배치하여 메서드를 직접 래핑하고 Flow 시스템에 전달하기 전에 반환 값을 캡처할 수 있도록 하세요.
</Tip> 수정 루프를 만들려면 `or_()`를 사용하여 검토 메서드가 **상위 트리거**와 **자체 수정 outcome**을 모두 리스닝해야 합니다:
```python Code
@start()
def generate(self):
return "initial draft"
@human_feedback(
message="승인하시겠습니까, 아니면 변경을 요청하시겠습니까?",
emit=["revise", "approved"],
llm="gpt-4o-mini",
default_outcome="approved",
)
@listen(or_("generate", "revise"))
def review(self):
return "content"
@listen("approved")
def publish(self):
return "published"
```
outcome이 `"revise"`이면 flow가 `review`로 다시 라우팅됩니다 (`or_()`를 통해 `"revise"`를 리스닝하기 때문). outcome이 `"approved"`이면 flow가 `publish`로 계속됩니다. flow 엔진이 라우터를 "한 번만 실행" 규칙에서 제외하여 각 루프 반복마다 재실행할 수 있기 때문에 이 패턴이 동작합니다.
### 체인된 라우터
한 라우터의 outcome으로 트리거된 리스너가 그 자체로 라우터가 될 수 있습니다:
```python Code
@start()
@human_feedback(message="첫 번째 검토:", emit=["approved", "rejected"], llm="gpt-4o-mini")
def draft(self):
return "draft content"
@listen("approved")
@human_feedback(message="최종 검토:", emit=["publish", "revise"], llm="gpt-4o-mini")
def final_review(self, prev):
return "final content"
@listen("publish")
def on_publish(self, prev):
return "published"
```
### 제한 사항
- **`@start()` 메서드는 한 번만 실행**: `@start()` 메서드는 self-loop할 수 없습니다. 수정 주기가 필요하면 별도의 `@start()` 메서드를 진입점으로 사용하고 `@listen()` 메서드에 `@human_feedback`를 배치하세요.
- **동일 메서드에 `@start()` + `@listen()` 불가**: 이는 Flow 프레임워크 제약입니다. 메서드는 시작점이거나 리스너여야 하며, 둘 다일 수 없습니다.
## 모범 사례 ## 모범 사례
@@ -516,9 +572,9 @@ class ContentPipeline(Flow):
@start() @start()
@human_feedback( @human_feedback(
message="이 콘텐츠의 출판을 승인하시겠습니까?", message="이 콘텐츠의 출판을 승인하시겠습니까?",
emit=["approved", "rejected", "needs_revision"], emit=["approved", "rejected"],
llm="gpt-4o-mini", llm="gpt-4o-mini",
default_outcome="needs_revision", default_outcome="rejected",
provider=SlackNotificationProvider("#content-reviews"), provider=SlackNotificationProvider("#content-reviews"),
) )
def generate_content(self): def generate_content(self):
@@ -534,11 +590,6 @@ class ContentPipeline(Flow):
print(f"보관됨. 이유: {result.feedback}") print(f"보관됨. 이유: {result.feedback}")
return {"status": "archived"} return {"status": "archived"}
@listen("needs_revision")
def queue_revision(self, result):
print(f"수정 대기열에 추가됨: {result.feedback}")
return {"status": "revision_needed"}
# Flow 시작 (Slack 응답을 기다리며 일시 중지) # Flow 시작 (Slack 응답을 기다리며 일시 중지)
def start_content_pipeline(): def start_content_pipeline():
@@ -594,22 +645,22 @@ async def on_slack_feedback_async(flow_id: str, slack_message: str):
```python Code ```python Code
class ArticleReviewFlow(Flow): class ArticleReviewFlow(Flow):
@start() @start()
@human_feedback(
message="Review this article draft:",
emit=["approved", "needs_revision"],
llm="gpt-4o-mini",
learn=True, # HITL 학습 활성화
)
def generate_article(self): def generate_article(self):
return self.crew.kickoff(inputs={"topic": "AI Safety"}).raw return self.crew.kickoff(inputs={"topic": "AI Safety"}).raw
@human_feedback(
message="이 글 초안을 검토해 주세요:",
emit=["approved", "needs_revision"],
llm="gpt-4o-mini",
learn=True,
)
@listen(or_("generate_article", "needs_revision"))
def review_article(self):
return self.last_human_feedback.output if self.last_human_feedback else "article draft"
@listen("approved") @listen("approved")
def publish(self): def publish(self):
print(f"Publishing: {self.last_human_feedback.output}") print(f"Publishing: {self.last_human_feedback.output}")
@listen("needs_revision")
def revise(self):
print("Revising based on feedback...")
``` ```
**첫 번째 실행**: 인간이 원시 출력을 보고 "사실에 대한 주장에는 항상 인용을 포함하세요."라고 말합니다. 교훈이 추출되어 메모리에 저장됩니다. **첫 번째 실행**: 인간이 원시 출력을 보고 "사실에 대한 주장에는 항상 인용을 포함하세요."라고 말합니다. 교훈이 추출되어 메모리에 저장됩니다.

View File

@@ -38,22 +38,21 @@ O CrewAI Enterprise oferece um sistema abrangente de gerenciamento Human-in-the-
Configure checkpoints de revisão humana em seus Flows usando o decorador `@human_feedback`. Quando a execução atinge um ponto de revisão, o sistema pausa, notifica o responsável via email e aguarda uma resposta. Configure checkpoints de revisão humana em seus Flows usando o decorador `@human_feedback`. Quando a execução atinge um ponto de revisão, o sistema pausa, notifica o responsável via email e aguarda uma resposta.
```python ```python
from crewai.flow.flow import Flow, start, listen from crewai.flow.flow import Flow, start, listen, or_
from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult
class ContentApprovalFlow(Flow): class ContentApprovalFlow(Flow):
@start() @start()
def generate_content(self): def generate_content(self):
# IA gera conteúdo
return "Texto de marketing gerado para campanha Q1..." return "Texto de marketing gerado para campanha Q1..."
@listen(generate_content)
@human_feedback( @human_feedback(
message="Por favor, revise este conteúdo para conformidade com a marca:", message="Por favor, revise este conteúdo para conformidade com a marca:",
emit=["approved", "rejected", "needs_revision"], emit=["approved", "rejected", "needs_revision"],
) )
def review_content(self, content): @listen(or_("generate_content", "needs_revision"))
return content def review_content(self):
return "Texto de marketing para revisão..."
@listen("approved") @listen("approved")
def publish_content(self, result: HumanFeedbackResult): def publish_content(self, result: HumanFeedbackResult):
@@ -62,10 +61,6 @@ class ContentApprovalFlow(Flow):
@listen("rejected") @listen("rejected")
def archive_content(self, result: HumanFeedbackResult): def archive_content(self, result: HumanFeedbackResult):
print(f"Conteúdo rejeitado. Motivo: {result.feedback}") print(f"Conteúdo rejeitado. Motivo: {result.feedback}")
@listen("needs_revision")
def revise_content(self, result: HumanFeedbackResult):
print(f"Revisão solicitada: {result.feedback}")
``` ```
Para detalhes completos de implementação, consulte o guia [Feedback Humano em Flows](/pt-BR/learn/human-feedback-in-flows). Para detalhes completos de implementação, consulte o guia [Feedback Humano em Flows](/pt-BR/learn/human-feedback-in-flows).

View File

@@ -98,33 +98,43 @@ def handle_feedback(self, result):
Quando você especifica `emit`, o decorador se torna um roteador. O feedback livre do humano é interpretado por um LLM e mapeado para um dos outcomes especificados: Quando você especifica `emit`, o decorador se torna um roteador. O feedback livre do humano é interpretado por um LLM e mapeado para um dos outcomes especificados:
```python Code ```python Code
@start() from crewai.flow.flow import Flow, start, listen, or_
@human_feedback( from crewai.flow.human_feedback import human_feedback
message="Você aprova este conteúdo para publicação?",
emit=["approved", "rejected", "needs_revision"],
llm="gpt-4o-mini",
default_outcome="needs_revision",
)
def review_content(self):
return "Rascunho do post do blog aqui..."
@listen("approved") class ReviewFlow(Flow):
def publish(self, result): @start()
print(f"Publicando! Usuário disse: {result.feedback}") def generate_content(self):
return "Rascunho do post do blog aqui..."
@listen("rejected") @human_feedback(
def discard(self, result): message="Você aprova este conteúdo para publicação?",
print(f"Descartando. Motivo: {result.feedback}") emit=["approved", "rejected", "needs_revision"],
llm="gpt-4o-mini",
default_outcome="needs_revision",
)
@listen(or_("generate_content", "needs_revision"))
def review_content(self):
return "Rascunho do post do blog aqui..."
@listen("needs_revision") @listen("approved")
def revise(self, result): def publish(self, result):
print(f"Revisando baseado em: {result.feedback}") print(f"Publicando! Usuário disse: {result.feedback}")
@listen("rejected")
def discard(self, result):
print(f"Descartando. Motivo: {result.feedback}")
``` ```
Quando o humano diz algo como "precisa de mais detalhes", o LLM mapeia para `"needs_revision"`, que dispara `review_content` novamente via `or_()` — criando um loop de revisão. O loop continua até que o outcome seja `"approved"` ou `"rejected"`.
<Tip> <Tip>
O LLM usa saídas estruturadas (function calling) quando disponível para garantir que a resposta seja um dos seus outcomes especificados. Isso torna o roteamento confiável e previsível. O LLM usa saídas estruturadas (function calling) quando disponível para garantir que a resposta seja um dos seus outcomes especificados. Isso torna o roteamento confiável e previsível.
</Tip> </Tip>
<Warning>
Um método `@start()` só executa uma vez no início do flow. Se você precisa de um loop de revisão, separe o método start do método de revisão e use `@listen(or_("trigger", "revision_outcome"))` no método de revisão para habilitar o self-loop.
</Warning>
## HumanFeedbackResult ## HumanFeedbackResult
O dataclass `HumanFeedbackResult` contém todas as informações sobre uma interação de feedback humano: O dataclass `HumanFeedbackResult` contém todas as informações sobre uma interação de feedback humano:
@@ -193,116 +203,162 @@ Aqui está um exemplo completo implementando um fluxo de revisão e aprovação
<CodeGroup> <CodeGroup>
```python Code ```python Code
from crewai.flow.flow import Flow, start, listen from crewai.flow.flow import Flow, start, listen, or_
from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult from crewai.flow.human_feedback import human_feedback, HumanFeedbackResult
from pydantic import BaseModel from pydantic import BaseModel
class ContentState(BaseModel): class ContentState(BaseModel):
topic: str = ""
draft: str = "" draft: str = ""
final_content: str = ""
revision_count: int = 0 revision_count: int = 0
status: str = "pending"
class ContentApprovalFlow(Flow[ContentState]): class ContentApprovalFlow(Flow[ContentState]):
"""Um flow que gera conteúdo e obtém aprovação humana.""" """Um flow que gera conteúdo e faz loop até o humano aprovar."""
@start() @start()
def get_topic(self): def generate_draft(self):
self.state.topic = input("Sobre qual tópico devo escrever? ") self.state.draft = "# IA Segura\n\nEste é um rascunho sobre IA Segura..."
return self.state.topic
@listen(get_topic)
def generate_draft(self, topic):
# Em uso real, isso chamaria um LLM
self.state.draft = f"# {topic}\n\nEste é um rascunho sobre {topic}..."
return self.state.draft return self.state.draft
@listen(generate_draft)
@human_feedback( @human_feedback(
message="Por favor, revise este rascunho. Responda 'approved', 'rejected', ou forneça feedback de revisão:", message="Por favor, revise este rascunho. Aprove, rejeite ou descreva o que precisa mudar:",
emit=["approved", "rejected", "needs_revision"], emit=["approved", "rejected", "needs_revision"],
llm="gpt-4o-mini", llm="gpt-4o-mini",
default_outcome="needs_revision", default_outcome="needs_revision",
) )
def review_draft(self, draft): @listen(or_("generate_draft", "needs_revision"))
return draft def review_draft(self):
self.state.revision_count += 1
return f"{self.state.draft} (v{self.state.revision_count})"
@listen("approved") @listen("approved")
def publish_content(self, result: HumanFeedbackResult): def publish_content(self, result: HumanFeedbackResult):
self.state.final_content = result.output self.state.status = "published"
print("\n✅ Conteúdo aprovado e publicado!") print(f"Conteúdo aprovado e publicado! Revisor disse: {result.feedback}")
print(f"Comentário do revisor: {result.feedback}")
return "published" return "published"
@listen("rejected") @listen("rejected")
def handle_rejection(self, result: HumanFeedbackResult): def handle_rejection(self, result: HumanFeedbackResult):
print("\n❌ Conteúdo rejeitado") self.state.status = "rejected"
print(f"Motivo: {result.feedback}") print(f"Conteúdo rejeitado. Motivo: {result.feedback}")
return "rejected" return "rejected"
@listen("needs_revision")
def revise_content(self, result: HumanFeedbackResult):
self.state.revision_count += 1
print(f"\n📝 Revisão #{self.state.revision_count} solicitada")
print(f"Feedback: {result.feedback}")
# Em um flow real, você pode voltar para generate_draft
# Para este exemplo, apenas reconhecemos
return "revision_requested"
# Executar o flow
flow = ContentApprovalFlow() flow = ContentApprovalFlow()
result = flow.kickoff() result = flow.kickoff()
print(f"\nFlow concluído. Revisões solicitadas: {flow.state.revision_count}") print(f"\nFlow finalizado. Status: {flow.state.status}, Revisões: {flow.state.revision_count}")
``` ```
```text Output ```text Output
Sobre qual tópico devo escrever? Segurança em IA ==================================================
OUTPUT FOR REVIEW:
==================================================
# IA Segura
Este é um rascunho sobre IA Segura... (v1)
==================================================
Por favor, revise este rascunho. Aprove, rejeite ou descreva o que precisa mudar:
(Press Enter to skip, or type your feedback)
Your feedback: Preciso de mais detalhes sobre segurança em IA.
================================================== ==================================================
OUTPUT FOR REVIEW: OUTPUT FOR REVIEW:
================================================== ==================================================
# Segurança em IA # IA Segura
Este é um rascunho sobre Segurança em IA... Este é um rascunho sobre IA Segura... (v2)
================================================== ==================================================
Por favor, revise este rascunho. Responda 'approved', 'rejected', ou forneça feedback de revisão: Por favor, revise este rascunho. Aprove, rejeite ou descreva o que precisa mudar:
(Press Enter to skip, or type your feedback) (Press Enter to skip, or type your feedback)
Your feedback: Parece bom, aprovado! Your feedback: Parece bom, aprovado!
Conteúdo aprovado e publicado! Conteúdo aprovado e publicado! Revisor disse: Parece bom, aprovado!
Comentário do revisor: Parece bom, aprovado!
Flow concluído. Revisões solicitadas: 0 Flow finalizado. Status: published, Revisões: 2
``` ```
</CodeGroup> </CodeGroup>
## Combinando com Outros Decoradores ## Combinando com Outros Decoradores
O decorador `@human_feedback` funciona com outros decoradores de flow. Coloque-o como o decorador mais interno (mais próximo da função): O decorador `@human_feedback` funciona com `@start()`, `@listen()` e `or_()`. Ambas as ordens de decoradores funcionam — o framework propaga atributos em ambas as direções — mas os padrões recomendados são:
```python Code ```python Code
# Correto: @human_feedback é o mais interno (mais próximo da função) # Revisão única no início do flow (sem self-loop)
@start() @start()
@human_feedback(message="Revise isto:") @human_feedback(message="Revise isto:", emit=["approved", "rejected"], llm="gpt-4o-mini")
def my_start_method(self): def my_start_method(self):
return "content" return "content"
# Revisão linear em um listener (sem self-loop)
@listen(other_method) @listen(other_method)
@human_feedback(message="Revise isto também:") @human_feedback(message="Revise isto também:", emit=["good", "bad"], llm="gpt-4o-mini")
def my_listener(self, data): def my_listener(self, data):
return f"processed: {data}" return f"processed: {data}"
# Self-loop: revisão que pode voltar para revisões
@human_feedback(message="Aprovar ou revisar?", emit=["approved", "revise"], llm="gpt-4o-mini")
@listen(or_("upstream_method", "revise"))
def review_with_loop(self):
return "content for review"
``` ```
<Tip> ### Padrão de self-loop
Coloque `@human_feedback` como o decorador mais interno (último/mais próximo da função) para que ele envolva o método diretamente e possa capturar o valor de retorno antes de passar para o sistema de flow.
</Tip> Para criar um loop de revisão, o método de revisão deve escutar **ambos** um gatilho upstream e seu próprio outcome de revisão usando `or_()`:
```python Code
@start()
def generate(self):
return "initial draft"
@human_feedback(
message="Aprovar ou solicitar alterações?",
emit=["revise", "approved"],
llm="gpt-4o-mini",
default_outcome="approved",
)
@listen(or_("generate", "revise"))
def review(self):
return "content"
@listen("approved")
def publish(self):
return "published"
```
Quando o outcome é `"revise"`, o flow roteia de volta para `review` (porque ele escuta `"revise"` via `or_()`). Quando o outcome é `"approved"`, o flow continua para `publish`. Isso funciona porque o engine de flow isenta roteadores da regra "fire once", permitindo que eles re-executem em cada iteração do loop.
### Roteadores encadeados
Um listener disparado pelo outcome de um roteador pode ser ele mesmo um roteador:
```python Code
@start()
@human_feedback(message="Primeira revisão:", emit=["approved", "rejected"], llm="gpt-4o-mini")
def draft(self):
return "draft content"
@listen("approved")
@human_feedback(message="Revisão final:", emit=["publish", "revise"], llm="gpt-4o-mini")
def final_review(self, prev):
return "final content"
@listen("publish")
def on_publish(self, prev):
return "published"
```
### Limitações
- **Métodos `@start()` executam uma vez**: Um método `@start()` não pode fazer self-loop. Se você precisa de um ciclo de revisão, use um método `@start()` separado como ponto de entrada e coloque o `@human_feedback` em um método `@listen()`.
- **Sem `@start()` + `@listen()` no mesmo método**: Esta é uma restrição do framework de Flow. Um método é ou um ponto de início ou um listener, não ambos.
## Melhores Práticas ## Melhores Práticas
@@ -516,9 +572,9 @@ class ContentPipeline(Flow):
@start() @start()
@human_feedback( @human_feedback(
message="Aprova este conteúdo para publicação?", message="Aprova este conteúdo para publicação?",
emit=["approved", "rejected", "needs_revision"], emit=["approved", "rejected"],
llm="gpt-4o-mini", llm="gpt-4o-mini",
default_outcome="needs_revision", default_outcome="rejected",
provider=SlackNotificationProvider("#content-reviews"), provider=SlackNotificationProvider("#content-reviews"),
) )
def generate_content(self): def generate_content(self):
@@ -534,11 +590,6 @@ class ContentPipeline(Flow):
print(f"Arquivado. Motivo: {result.feedback}") print(f"Arquivado. Motivo: {result.feedback}")
return {"status": "archived"} return {"status": "archived"}
@listen("needs_revision")
def queue_revision(self, result):
print(f"Na fila para revisão: {result.feedback}")
return {"status": "revision_needed"}
# Iniciando o flow (vai pausar e aguardar resposta do Slack) # Iniciando o flow (vai pausar e aguardar resposta do Slack)
def start_content_pipeline(): def start_content_pipeline():
@@ -594,22 +645,22 @@ Com o tempo, o humano vê saídas pré-revisadas progressivamente melhores porqu
```python Code ```python Code
class ArticleReviewFlow(Flow): class ArticleReviewFlow(Flow):
@start() @start()
def generate_article(self):
return self.crew.kickoff(inputs={"topic": "AI Safety"}).raw
@human_feedback( @human_feedback(
message="Review this article draft:", message="Revise este rascunho do artigo:",
emit=["approved", "needs_revision"], emit=["approved", "needs_revision"],
llm="gpt-4o-mini", llm="gpt-4o-mini",
learn=True, # enable HITL learning learn=True, # enable HITL learning
) )
def generate_article(self): @listen(or_("generate_article", "needs_revision"))
return self.crew.kickoff(inputs={"topic": "AI Safety"}).raw def review_article(self):
return self.last_human_feedback.output if self.last_human_feedback else "article draft"
@listen("approved") @listen("approved")
def publish(self): def publish(self):
print(f"Publishing: {self.last_human_feedback.output}") print(f"Publishing: {self.last_human_feedback.output}")
@listen("needs_revision")
def revise(self):
print("Revising based on feedback...")
``` ```
**Primeira execução**: O humano vê a saída bruta e diz "Sempre inclua citações para afirmações factuais." A lição é destilada e armazenada na memória. **Primeira execução**: O humano vê a saída bruta e diz "Sempre inclua citações para afirmações factuais." A lição é destilada e armazenada na memória.