Lucas Gomide 08fa3797ca Introducing Agent evaluation (#3130) ...

* feat: add exchanged messages in LLMCallCompletedEvent

* feat: add GoalAlignment metric for Agent evaluation

* feat: add SemanticQuality metric for Agent evaluation

* feat: add Tool Metrics for Agent evaluation

* feat: add Reasoning Metrics for Agent evaluation, still in progress

* feat: add AgentEvaluator class

This class will evaluate Agent' results and report to user

* fix: do not evaluate Agent by default

This is a experimental feature we still need refine it further

* test: add Agent eval tests

* fix: render all feedback per iteration

* style: resolve linter issues

* style: fix mypy issues

* fix: allow messages be empty on LLMCallCompletedEvent

2025-07-11 13:18:03 -04:00

..

agents

feat: add capability to track LLM calls by task and agent (#3087)

2025-07-01 09:30:16 -04:00

cli

fix: use production workos environment id (#3129)

2025-07-09 17:09:01 -04:00

crews

Brandon/cre 19 workflows (#1347)

2024-09-27 12:11:17 -03:00

evaluation

Introducing Agent evaluation (#3130)

2025-07-11 13:18:03 -04:00

flow

fix: fix pydantic support to 2.7.x (#3016)

2025-06-16 16:20:10 -04:00

knowledge

Support Python 3.13 (#2844)

2025-06-02 18:12:24 -04:00

llms

feat: add capability to track LLM calls by task and agent (#3087)

2025-07-01 09:30:16 -04:00

memory

Introduce MemoryEvents to monitor their usage (#3098)

2025-07-01 22:50:39 -04:00

project

feat: enhance CrewBase MCP tools support to allow selecting multiple tools per agent (#3065)

2025-06-25 14:59:55 -04:00

security

adding fingerprints (#2332)

2025-03-14 03:00:30 -03:00

tasks

LiteAgent w/ Guardrail (#2982)

2025-06-10 13:32:32 -04:00

telemetry

Add Scarf analytics tracking (#3086)

2025-06-30 17:48:45 -04:00

tools

LiteAgent w/ Guardrail (#2982)

2025-06-10 13:32:32 -04:00

translations

improviong reasoning prompt

2025-05-25 15:24:59 -07:00

types

Brandon/eng 266 conversation crew v1 (#1843)

2025-01-06 16:12:43 -05:00

utilities

Introducing Agent evaluation (#3130)

2025-07-11 13:18:03 -04:00

__init__.py

chore: update crewAI and dependencies to version 0.141.0 and 0.51.0 (#3128)

2025-07-09 10:37:06 -07:00

agent.py

Introduce MemoryEvents to monitor their usage (#3098)

2025-07-01 22:50:39 -04:00

crew.py

Introducing Agent evaluation (#3130)

2025-07-11 13:18:03 -04:00

lite_agent.py

Introducing Agent evaluation (#3130)

2025-07-11 13:18:03 -04:00

llm.py

Introducing Agent evaluation (#3130)

2025-07-11 13:18:03 -04:00

process.py

Hierarchical process (#206)

2024-02-02 13:56:35 -03:00

task.py

feat: add console logging for LLM guardrail events (#3105)

2025-07-02 16:19:22 -04:00