This commit fixes the race condition described in issue #4168 where
token tracking was inaccurate when multiple async tasks from the same
agent ran concurrently.
The fix introduces:
1. Per-agent locks to serialize async task execution for accurate token
tracking when multiple async tasks from the same agent run concurrently
2. Token capture callback that captures both tokens_before and tokens_after
inside the thread (after acquiring the lock), not when the task is queued
3. Updated _process_async_tasks to handle the new return type from
execute_async which now returns (TaskOutput, tokens_before, tokens_after)
This ensures that token deltas are accurately attributed to each task
even when multiple async tasks from the same agent overlap in execution.
Tests added:
- test_async_task_token_tracking_uses_per_agent_lock
- test_async_task_token_callback_captures_tokens_inside_thread
- test_async_task_per_agent_lock_serializes_execution
Co-Authored-By: João <joao@crewai.com>
5dc87c04af