From 27083f41319a7d71d4c65d541c558d79ad2a30a1 Mon Sep 17 00:00:00 2001 From: Ran Shemtov Date: Wed, 12 Aug 2026 19:01:16 +0200 Subject: [PATCH] docs: add Frontend guides (CopilotKit + AG-UI) (#6686) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs: add Frontend guides section (CopilotKit + AG-UI) Add a Frontend sub-group under Guides documenting how to build user interfaces for CrewAI Crews and Flows with CopilotKit over the AG-UI protocol. Pages: overview, generative UI, tool-based generative UI, agentic generative UI, human-in-the-loop, shared state, frontend actions, predictive state updates, and channels. * docs: mirror Frontend guides into v1.15.5 (Latest) Also register the Frontend sub-group and pages under the default v1.15.5 version so the section is visible without switching to Edge. * docs(frontend): address audit — correct APIs and claims - Use useRenderTool for display-only tool rendering (was useFrontendTool) - Correct state 'auto-streams' claims: snapshot at step boundaries, document copilotkit_emit_state for mid-step progress - Fix setState usage to spread full state (replace, not merge) - Add tool description to the frontend-action example - Rewrite Channels with the real @copilotkit/channels createBot API (Slack + Discord adapters); drop unsupported platform claims - Note self-hosted vs managed CopilotKit paths and pin package versions * docs(frontend): remove versions callout from overview * docs(frontend): drop package-generation framing from emit_state note * docs(frontend): add generative UI spectrum (A2UI, reasoning) + Conversational Flows Rewrite generative-ui as the controlled/declarative/open-ended spectrum; add A2UI (declarative), Reasoning (controlled), and a Conversational Flows page; add a backend-tools section to tool-based; note the three execution shapes in the overview. * docs(frontend): address review — edge-only, attribute access, safe defaults Remove the docs/v1.15.5 mirror (versioned snapshots are cut from edge by the release tooling; the docs-snapshots CI guard rejects manual docs/v* writes). Use attribute access on the LiteLLM message in shared-state, guard setState against undefined agent/recipe, and use Field(default_factory=list) for the agent-state list. --- docs/docs.json | 18 ++ docs/edge/en/guides/frontend/a2ui.mdx | 127 ++++++++++ .../guides/frontend/agentic-generative-ui.mdx | 208 +++++++++++++++ docs/edge/en/guides/frontend/channels.mdx | 125 +++++++++ .../guides/frontend/conversational-flows.mdx | 106 ++++++++ .../en/guides/frontend/frontend-actions.mdx | 121 +++++++++ .../edge/en/guides/frontend/generative-ui.mdx | 107 ++++++++ .../en/guides/frontend/human-in-the-loop.mdx | 191 ++++++++++++++ docs/edge/en/guides/frontend/overview.mdx | 238 ++++++++++++++++++ .../frontend/predictive-state-updates.mdx | 142 +++++++++++ docs/edge/en/guides/frontend/reasoning.mdx | 68 +++++ docs/edge/en/guides/frontend/shared-state.mdx | 210 ++++++++++++++++ .../frontend/tool-based-generative-ui.mdx | 235 +++++++++++++++++ 13 files changed, 1896 insertions(+) create mode 100644 docs/edge/en/guides/frontend/a2ui.mdx create mode 100644 docs/edge/en/guides/frontend/agentic-generative-ui.mdx create mode 100644 docs/edge/en/guides/frontend/channels.mdx create mode 100644 docs/edge/en/guides/frontend/conversational-flows.mdx create mode 100644 docs/edge/en/guides/frontend/frontend-actions.mdx create mode 100644 docs/edge/en/guides/frontend/generative-ui.mdx create mode 100644 docs/edge/en/guides/frontend/human-in-the-loop.mdx create mode 100644 docs/edge/en/guides/frontend/overview.mdx create mode 100644 docs/edge/en/guides/frontend/predictive-state-updates.mdx create mode 100644 docs/edge/en/guides/frontend/reasoning.mdx create mode 100644 docs/edge/en/guides/frontend/shared-state.mdx create mode 100644 docs/edge/en/guides/frontend/tool-based-generative-ui.mdx diff --git a/docs/docs.json b/docs/docs.json index a7b8f02011..b4495f5582 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -118,6 +118,24 @@ "edge/en/guides/flows/inputs-id-deprecation" ] }, + { + "group": "Frontend", + "icon": "browser", + "pages": [ + "edge/en/guides/frontend/overview", + "edge/en/guides/frontend/conversational-flows", + "edge/en/guides/frontend/generative-ui", + "edge/en/guides/frontend/tool-based-generative-ui", + "edge/en/guides/frontend/agentic-generative-ui", + "edge/en/guides/frontend/reasoning", + "edge/en/guides/frontend/a2ui", + "edge/en/guides/frontend/human-in-the-loop", + "edge/en/guides/frontend/shared-state", + "edge/en/guides/frontend/frontend-actions", + "edge/en/guides/frontend/predictive-state-updates", + "edge/en/guides/frontend/channels" + ] + }, { "group": "Tools", "icon": "wrench", diff --git a/docs/edge/en/guides/frontend/a2ui.mdx b/docs/edge/en/guides/frontend/a2ui.mdx new file mode 100644 index 0000000000..9d3b9fdae1 --- /dev/null +++ b/docs/edge/en/guides/frontend/a2ui.mdx @@ -0,0 +1,127 @@ +--- +title: A2UI +description: The declarative tier of generative UI — the agent assembles a surface from a catalog of components you own. +icon: table-cells +mode: "wide" +--- + +## The agent assembles the UI + +[Tool-based rendering](/en/guides/frontend/tool-based-generative-ui) maps one tool to one component: the agent picks a component, you draw it. A2UI is the **declarative** tier of the [generative-UI spectrum](/en/guides/frontend/generative-ui#declarative) — instead of picking a single component, the agent **assembles a surface** by combining building blocks from a catalog you define. + +You still own the components. The agent can only use what is in your catalog, so it can never render something you did not ship. What the agent decides is the **layout and the data** — how those building blocks come together into a panel, and what goes in them. + + +A2UI works with [Flows](/en/concepts/flows). Both modes below — dynamic and fixed-schema — run as Flows served over AG-UI, exactly like the rest of this section. + + +## The catalog (same for every mode) + +The frontend wiring is identical no matter which backend mode you use: you register a **catalog** on the `` provider with the `a2ui` prop. + +```tsx +import { CopilotKit } from "@copilotkit/react-core"; +import { catalog } from "@/a2ui-catalog"; + + + {/* ... */} + +``` + +The catalog is your set of React components keyed by a catalog id — a `FlightCard`, a `HotelCard`, a `Chart`, whatever your app needs. The agent references catalog ids; CopilotKit paints your components with the data the agent supplies. + + +Authoring the catalog itself — the id schema, prop mapping, and composition rules — is deeper than this page covers. See the [CopilotKit A2UI docs](https://docs.copilotkit.ai) for the full authoring reference. Here we focus on the two backend modes and when to reach for each. + + +## Two backend modes + +A2UI backends come in two shapes. In **dynamic** mode the agent designs the surface; in **fixed-schema** mode you pre-author the layout and the agent only fills in data. + +| Mode | Who designs the layout | Backend | Predictability | +| --- | --- | --- | --- | +| **[Dynamic](#dynamic)** | The agent, from the conversation | No A2UI tool — auto-injected | Novel layouts, LLM layout step | +| **[Fixed-schema](#fixed-schema)** | You, up front | Backend tools return an envelope | Deterministic, no layout step | + +### Dynamic + +The Flow wires **no** A2UI tool. Enable A2UI on the runtime for this agent and it gains a `generate_a2ui` tool automatically. A sub-agent designs a surface from the conversation against your catalog, streams it to the frontend progressively, and self-heals invalid output through a validate-then-retry recovery pass. You write a normal agentic-chat Flow; the tool is injected for you. + + + + + +Same as above — pass your catalog through the `a2ui` prop: + +```tsx + + {/* ... */} + +``` + + + + + +Your backend is a plain agentic-chat Flow. You do not define an A2UI tool — the runtime injects `generate_a2ui` when A2UI is enabled for the agent, and the sub-agent invents the layout from the conversation. + + + + + +When a turn calls for UI, the agent assembles a surface from your catalog, streams the components in as it designs them, and repairs any invalid output before it reaches the screen. Your registered components render in the layout the agent chose. + + + + + +### Fixed-schema + +When you already know the layout and only the data changes per call, pre-author the surface and let the agent fill it. The Flow wires backend tools (for example `search_flights`, `search_hotels`). Each tool returns an **A2UI operations envelope** as its result — `createSurface` -> `updateComponents` -> `updateDataModel` — which the frontend paints. There is no sub-agent, no generation, and no recovery pass: the layout JSON is authored by you, and only the data varies. + +Install the toolkit that provides the envelope helpers: + +```bash +pip install ag-ui-a2ui-toolkit +``` + +Build the envelope with the toolkit helpers and emit it as the tool result: + +```python +from ag_ui_a2ui_toolkit import ( + A2UI_OPERATIONS_KEY, + create_surface, + update_components, + update_data_model, +) +from ag_ui_crewai.sdk import copilotkit_emit_tool_result, copilotkit_stream +``` + +The tool assembles the `createSurface` -> `updateComponents` -> `updateDataModel` operations into an envelope keyed by `A2UI_OPERATIONS_KEY`, then hands it back with `copilotkit_emit_tool_result(...)`. Because the layout is fixed, the same tool always produces the same shape — only the values differ from call to call. + +## When to use which + + + + The layout is not known ahead of time and you want the agent to compose novel surfaces from your primitives. You gain flexibility and pay for an LLM layout step. + + + The layout is known and only the data varies. More predictable and deterministic — no generation, no recovery, no LLM in the layout path. + + + +Both modes share the same frontend: one catalog, registered once on the provider. Start with fixed-schema when your surfaces are stable, and reach for dynamic when you want the agent to design layouts you did not anticipate. + +## Related + + + + The full spectrum — A2UI is its declarative tier. + + + Map one tool to one component (controlled). + + + Render live agent state (controlled). + + diff --git a/docs/edge/en/guides/frontend/agentic-generative-ui.mdx b/docs/edge/en/guides/frontend/agentic-generative-ui.mdx new file mode 100644 index 0000000000..952cb98a83 --- /dev/null +++ b/docs/edge/en/guides/frontend/agentic-generative-ui.mdx @@ -0,0 +1,208 @@ +--- +title: Agentic Generative UI +description: Render your CrewAI Flow's live state as UI that updates as the agent works through multi-step tasks. +icon: list-check +mode: "wide" +--- + +## Render the agent's live state + +Some work does not fit into a single tool call. A research task, a multi-step plan, a long-running job: the interesting thing to show the user is not one result, but *progress*. Agentic generative UI renders the agent's **state** and re-renders it every time that state changes. + +The pattern has two halves: + +1. Your Flow writes progress into its own state as it works. +2. Your frontend reads that state with `useAgent` and paints it, re-rendering as the state streams in. + +The Flow's state reaches the frontend over AG-UI without you wiring up any transport. A state snapshot is emitted automatically at each step (method) boundary of the Flow, and you can push intermediate updates during a long-running step by calling `copilotkit_emit_state` explicitly. You subclass the state to add your own fields, update them in the Flow, and read them in React. + + +State-driven rendering requires a **Flow** with custom state (`Flow[AgentState]`). Crews are chat-oriented and do not expose custom state this way, so with a Crew use [tool rendering](/en/guides/frontend/tool-based-generative-ui) instead. + + +## Build a live task planner + +This example builds a planner that breaks a request into about ten steps and streams them to the UI as a checklist. It assumes you already have a CrewAI server and a CopilotKit frontend wired up. If you do not, start with the [Frontend Overview](/en/guides/frontend/overview). + + + + + +Subclass `CopilotKitState` to declare the state your UI needs. `CopilotKitState` already carries the conversation (`messages`); you add whatever else you want to render, here a list of task steps. + +```python +from typing import List, Literal +from pydantic import BaseModel, Field +from ag_ui_crewai.sdk import CopilotKitState + + +class TaskStep(BaseModel): + description: str + status: Literal["enabled", "disabled"] + + +class AgentState(CopilotKitState): + steps: List[TaskStep] = Field(default_factory=list) +``` + +Everything on `AgentState` is included in the state snapshot the frontend receives. A snapshot is emitted automatically at each step boundary, so writing to `self.state` is enough for the UI to pick it up between steps. To update the UI *during* a long step, emit explicitly (shown below). + + + + + +Type your Flow with the custom state (`Flow[AgentState]`) and let the model fill it in. Here the LLM calls a `generate_task_steps` tool; the streamed tool call lands in the conversation and the steps become visible in state. + +```python +from crewai.flow.flow import Flow, start +from litellm import acompletion +from ag_ui_crewai.sdk import copilotkit_stream + +GENERATE_TASK_STEPS_TOOL = { + "type": "function", + "function": { + "name": "generate_task_steps", + "description": "Break a task into about 10 short imperative steps.", + "parameters": { + "type": "object", + "properties": { + "steps": { + "type": "array", + "items": { + "type": "object", + "properties": { + "description": {"type": "string"}, + "status": {"type": "string", "enum": ["enabled"]}, + }, + "required": ["description", "status"], + }, + }, + }, + "required": ["steps"], + }, + }, +} + + +class TaskPlannerFlow(Flow[AgentState]): + @start() + async def chat(self): + response = await copilotkit_stream( + await acompletion( + model="openai/gpt-4o", + messages=[ + {"role": "system", "content": "Plan the task the user asks for."}, + *self.state.messages, + ], + tools=[GENERATE_TASK_STEPS_TOOL], + parallel_tool_calls=False, + stream=True, + ) + ) + message = response.choices[0].message + self.state.messages.append(message) +``` + +Wrapping the LLM call in `copilotkit_stream` streams the assistant's tokens and tool call to the frontend as they are produced. The `steps` you write to `self.state` are sent in the state snapshot emitted at the end of this step. + + + + + +The automatic snapshot fires at step boundaries. If a single step does substantial work and you want the checklist to fill in *as it happens*, emit intermediate state yourself with `copilotkit_emit_state`. Each call pushes the current state to the frontend immediately. + +```python +from ag_ui_crewai.sdk import copilotkit_emit_state + +class TaskPlannerFlow(Flow[AgentState]): + @start() + async def execute(self): + for step in self.state.steps: + step.status = "disabled" # mark done as you go + await copilotkit_emit_state(self.state) # push update now + await do_work(step) +``` + +Import `copilotkit_emit_state` from `ag_ui_crewai.sdk`. It requires the CopilotKit SDK (`pip install "copilotkit[crewai]"`). Reach for it only when a step is long enough that waiting for its boundary snapshot would feel unresponsive. + + + + + +Register the Flow exactly as any other, on its own path: + +```python +# server.py +from fastapi import FastAPI +from ag_ui_crewai.endpoint import add_crewai_flow_fastapi_endpoint +from my_agents.task_planner import TaskPlannerFlow + +app = FastAPI(title="CrewAI Agent Server") + +add_crewai_flow_fastapi_endpoint( + app=app, + flow=TaskPlannerFlow(), + path="/task_planner", +) +``` + +See the [Frontend Overview](/en/guides/frontend/overview) for the full server, runtime, and provider setup, and remember to register the agent (here `task_planner`) in your CopilotKit runtime route. + + + + + +On the frontend, `useAgent` gives you the agent's live state. Subscribe to state changes so your component re-renders every time the Flow writes an update. + +```tsx +"use client"; +import { useAgent, UseAgentUpdate } from "@copilotkit/react-core/v2"; + +function TaskPlan() { + const { agent } = useAgent({ + agentId: "task_planner", + updates: [UseAgentUpdate.OnStateChanged], + }); + + const steps = agent?.state?.steps ?? []; + + return ( +
    + {steps.map((s, i) => ( +
  • {s.description}
    • + ))} +
+ ); +} +``` + +`useAgent` returns `{ agent }`. A few things to know: + +- `agent.state` is the live Flow state. Its shape matches the fields you added to `AgentState`, so `agent.state.steps` is your list of task steps. +- `agent.isRunning` tells you when the agent is actively working, useful for showing a spinner or disabling input. +- `updates: [UseAgentUpdate.OnStateChanged]` re-renders the component whenever state changes, so the checklist fills in as the Flow streams its steps. + + + + + +## Where this goes next + +Reading state is the foundation. Two guides build directly on it: + +- [Shared State](/en/guides/frontend/shared-state) adds the other direction: editing the agent's state from the UI and having the Flow pick up the change. +- [Predictive State](/en/guides/frontend/predictive-state-updates) streams a tool's in-progress arguments into state so the UI reflects work before it is committed. + +## Related + + + + Sync agent state and app UI in both directions. + + + Stream in-progress tool arguments into state. + + + Map agent tool calls to components. + + diff --git a/docs/edge/en/guides/frontend/channels.mdx b/docs/edge/en/guides/frontend/channels.mdx new file mode 100644 index 0000000000..b881ef6b1a --- /dev/null +++ b/docs/edge/en/guides/frontend/channels.mdx @@ -0,0 +1,125 @@ +--- +title: Channels +description: Run the same CrewAI agent as a chat bot on Slack and Discord with the CopilotKit Channels SDK. +icon: slack +mode: "wide" +--- + +## Meet your users where they already are + +The CrewAI agent you built in the [Overview](/en/guides/frontend/overview) does not have to live behind a web app. The same Crew or Flow can run as a bot inside a messaging platform. No rebuild, no second copy of your agent logic: the agent stays exposed over the [AG-UI protocol](https://docs.ag-ui.com), and a bot process drives it. + +CopilotKit's [Channels SDK](https://docs.copilotkit.ai/reference/channels) provides that bot process. It ships a platform-agnostic engine plus per-platform adapters. + +## How it fits together + +Nothing about your agent server changes. It keeps serving your Crew or Flow over AG-UI exactly as in the Overview. What you add is a separate **bot process**: it connects to a platform adapter, listens for messages, and runs your agent when it is messaged. The reply streams back into the channel. + +``` +Slack / Discord ──► Channels bot process ──► CrewAI server (AG-UI) ──► Crew / Flow +``` + +Your agent server can keep serving the web frontend from the Overview at the same time. The web app and the bot are just two clients of one AG-UI endpoint. + +## Slack + + + + + +```bash +npm install @copilotkit/channels @copilotkit/channels-slack @ag-ui/crewai +``` + + + + + +Create an app in the Slack API dashboard for your workspace, enable Socket Mode, and grant it the message and event scopes it needs to read and post in channels. Then expose its tokens to the bot process: + +```bash +export SLACK_BOT_TOKEN=xoxb-... # bot user token +export SLACK_APP_TOKEN=xapp-... # app-level token (Socket Mode) +``` + + + + + +`createBot` wires a Slack adapter to your agent. The `agent` factory returns a `CrewAIAgent` pointed at the AG-UI path your server exposes (the same URL you registered in the runtime in the Overview). + +```ts +// bot.ts +import { createBot } from "@copilotkit/channels"; +import { slack, defaultSlackTools, defaultSlackContext } from "@copilotkit/channels-slack"; +import { CrewAIAgent } from "@ag-ui/crewai"; + +const bot = createBot({ + adapters: [ + slack({ + botToken: process.env.SLACK_BOT_TOKEN!, // xoxb-… + appToken: process.env.SLACK_APP_TOKEN!, // xapp-… (Socket Mode) + }), + ], + agent: (threadId) => new CrewAIAgent({ url: "http://localhost:8000/recipe" }), + tools: [...defaultSlackTools], + context: [...defaultSlackContext], +}); + +bot.start(); +``` + + + + + +Start the bot process alongside your agent server: + +```bash +uvicorn server:app --port 8000 # terminal 1 — CrewAI agent server +node bot.ts # terminal 2 — Slack bot +``` + +Message the bot in Slack and it runs your Crew or Flow, streaming the reply back into the thread. + + + + + + +Slack app scopes, Socket Mode setup, and the full adapter options are maintained by CopilotKit. Follow the [Slack channel reference](https://docs.copilotkit.ai/reference/channels/slack) together with Slack's own app setup guide for the authoritative steps. + + +## Discord + +Discord uses the same `createBot` engine with the Discord adapter from `@copilotkit/channels-discord`: + +```ts +import { createBot } from "@copilotkit/channels"; +import { discord } from "@copilotkit/channels-discord"; +import { CrewAIAgent } from "@ag-ui/crewai"; + +const bot = createBot({ + adapters: [discord({ token: process.env.DISCORD_BOT_TOKEN! })], + agent: (threadId) => new CrewAIAgent({ url: "http://localhost:8000/recipe" }), +}); + +bot.start(); +``` + +See the [Discord channel reference](https://docs.copilotkit.ai/reference/channels/discord) for the exact adapter options and bot setup. + +## Platform support + +Slack and Discord have official Channels adapters (`@copilotkit/channels-slack`, `@copilotkit/channels-discord`). Microsoft Teams is available through CopilotKit's managed offering (currently waitlisted). Check the [Channels reference](https://docs.copilotkit.ai/reference/channels) for the current list before promising a platform. + +## Related + + + + Serve your Crew or Flow over AG-UI — the foundation every channel builds on. + + + Pause the agent to collect user approval or input mid-run. + + diff --git a/docs/edge/en/guides/frontend/conversational-flows.mdx b/docs/edge/en/guides/frontend/conversational-flows.mdx new file mode 100644 index 0000000000..c5e629ccd7 --- /dev/null +++ b/docs/edge/en/guides/frontend/conversational-flows.mdx @@ -0,0 +1,106 @@ +--- +title: Conversational Flows +description: Serve native, session-aware CrewAI Flows over AG-UI with managed conversation state and full frontend parity. +icon: comments +mode: "wide" +--- + +## Three execution shapes, one bridge + +Behind the AG-UI bridge, a CrewAI backend can take one of three shapes. Knowing which one you are serving decides how you author the backend, not how you build the frontend. + +| Shape | What it is | How it is entered | +| --- | --- | --- | +| **Regular Flows** | Author-controlled `@start`/`@listen`/`@router` graphs. The default used throughout these guides. | `kickoff` / `astream` | +| **Conversational Flows** | Native, session-aware, turn-based Flows with managed conversation state. | `stream_turn(message, session_id=...)` | +| **Crews** | Closed autonomous task/agent loops. Basic chat only, a separate compatibility path. | Not the focus here. | + +Conversational Flows are a newer CrewAI capability, and an important thing to be clear about up front: **they are Flows, not Crews.** They now run at full regular-Flow feature parity. This page introduces them and shows how they fit the rest of the frontend guides. + + +Reach for a Conversational Flow when you want native multi-turn conversation with CrewAI managing session state and history for you, rather than wiring turn and state handling into a regular Flow yourself. If you are new here, start with the [Frontend Overview](/en/guides/frontend/overview) for the base server, runtime, and provider setup. + + +## Register a Conversational Flow + +You register a Conversational Flow through the same endpoint helper as any other Flow, with one extra argument: `conversational=True`. + +```python +# server.py +from ag_ui_crewai.endpoint import add_crewai_flow_fastapi_endpoint + +add_crewai_flow_fastapi_endpoint( + app, + flow, + "/conversation", + conversational=True, +) +``` + +Two requirements must hold for this to work: + +- The Flow instance declares `conversational = True`. +- The Flow exposes CrewAI's public, callable `stream_turn(message, session_id=...)`. + +Detection is capability-based, not version-gated: the bridge checks that the Flow actually offers turn-based conversation, rather than keying off a version number. + + +If those requirements are not met, the request fails loudly with a `RUN_ERROR` (code `AGUI_CREWAI_CONVERSATIONAL_FLOW_UNSUPPORTED`). It never silently falls back to regular kickoff semantics, so you always know exactly which path you are on. + + +Authoring the Flow itself, including how you implement `stream_turn`, belongs to CrewAI's Conversational Flows documentation. This page stays at the registration and integration boundary. + +## Session and state + +Conversational Flows manage session state and history for you across turns. You do not re-thread history manually. + +- The AG-UI `threadId` **is** the CrewAI conversation `session_id`. The same thread is the same conversation. +- Before each turn the bridge hydrates the Flow's state and conversation history, then calls `stream_turn`. CrewAI restores the stored session state, and a per-request overlay reapplies the incoming AG-UI state and history so the browser's latest edits win over stale storage. + +The result: from the backend author's side, each turn arrives already carrying the conversation's state, and CrewAI persists what you write for the next turn. + +## Frontend parity + +This is the point to hold onto: **Conversational Flows run through the same event pipeline as regular Flows, so the frontend code is identical.** + +There is no Conversational-Flow-specific frontend API. Every feature in these guides works exactly the same way with a Conversational Flow as it does with a regular Flow, using the same hooks and components: + + + + Map agent tool calls to your React components. + + + Render the Flow's live state as it works. + + + Keep agent state and app UI in two-way sync. + + + Pause the agent for user approval or input mid-turn. + + + Stream in-progress tool arguments into state. + + + Show the model's thinking in the chat. + + + Render agent-authored UI from a component catalog. + + + +The only difference is on the backend: how you author the Flow (turn-based `stream_turn` with managed session state) and the `conversational=True` registration. Once the endpoint is up, everything you already know about building the frontend applies unchanged. + +## Related + + + + Wire a Crew or Flow to a Next.js frontend end to end. + + + Render tool calls and agent state as custom components. + + + Gate agent actions behind user approval. + + diff --git a/docs/edge/en/guides/frontend/frontend-actions.mdx b/docs/edge/en/guides/frontend/frontend-actions.mdx new file mode 100644 index 0000000000..4a41d118df --- /dev/null +++ b/docs/edge/en/guides/frontend/frontend-actions.mdx @@ -0,0 +1,121 @@ +--- +title: Frontend Actions +description: Let your CrewAI agent call functions that run in the user's browser, from switching themes to navigating your app. +icon: bolt +mode: "wide" +--- + +## Let the agent act on the app + +A frontend action is a tool the agent calls that runs code in the browser instead of on the server. The model decides to invoke it; your handler switches the theme, navigates, highlights an element, or updates your app data; and the result flows back to the agent. + +It uses the same hook as tool-based generative UI, `useFrontendTool`. The difference is what you give it: a `handler` that runs code, instead of (or alongside) a `render` that draws UI. + + +Frontend actions work with both Crews and Flows. Any agent that binds `copilotkit.actions` into its LLM call can invoke them. + + +## Build a frontend action + +The example below lets the agent switch the app into dark mode on request. + + + + + +Call `useFrontendTool` with a `handler`. The handler runs in the browser when the agent invokes the tool, and the string it returns is fed back to the agent. + +```tsx +"use client"; +import { useFrontendTool } from "@copilotkit/react-core/v2"; +import { z } from "zod"; + +useFrontendTool({ + agentId: "assistant", + name: "set_theme", + description: "Switch the app between light and dark mode.", + parameters: z.object({ + theme: z.enum(["light", "dark"]), + }), + followUp: false, + handler: async ({ theme }) => { + document.documentElement.dataset.theme = theme; // runs in the browser + return `Theme set to ${theme}.`; + }, +}); +``` + +The arguments: + +- **`name`** — the tool name the model calls (`set_theme`). +- **`description`** — a short explanation of what the tool does. The model reads it to decide *when* to call the tool, so make it specific. Omitting it leaves the model guessing from the name alone. +- **`parameters`** — a [zod](https://zod.dev) schema describing the arguments the model must supply. CopilotKit turns this into the tool's JSON schema and validates the incoming call. +- **`handler(args)`** — runs in the browser with the parsed arguments. Do your side effect here (set the theme, navigate, update state). The string you return is handed back to the agent as the tool result. +- **`followUp: false`** — stops the agent from taking another turn after the action runs. Leave it out (or set `true`) when you want the agent to respond after acting. + + + + + +The agent can only call a tool it has been given. In your Flow, pass the frontend-registered tools into the LLM `tools` list with `*self.state.copilotkit.actions`. + +```python +from crewai.flow.flow import Flow, start +from litellm import acompletion +from ag_ui_crewai.sdk import copilotkit_stream, CopilotKitState + +class AssistantFlow(Flow[CopilotKitState]): + @start() + async def chat(self): + response = await copilotkit_stream( + await acompletion( + model="openai/gpt-4o", + messages=[ + {"role": "system", "content": "Help the user. Use the tools available to control the app."}, + *self.state.messages, + ], + tools=[*self.state.copilotkit.actions], # tools the frontend registered + parallel_tool_calls=False, + stream=True, + ) + ) + message = response.choices[0].message + self.state.messages.append(message) +``` + +`self.state.copilotkit.actions` holds the tool definitions for every frontend action registered with `useFrontendTool`. Spreading them into the LLM `tools` list is what makes the agent able to invoke browser-side actions. `copilotkit_stream` streams the response, including the tool call, back to the frontend, where CopilotKit runs the matching handler. + + + + + +Expose the Flow over AG-UI with `add_crewai_flow_fastapi_endpoint(...)` and register it in the CopilotKit runtime, exactly as in the [Frontend Overview](/en/guides/frontend/overview). Once both are running, asking the assistant to "switch to dark mode" triggers `set_theme`, and the page flips. + + + + + +## Actions vs. generative UI + +`useFrontendTool` covers both ends of a spectrum, and you pick per tool: + +| You provide | What it does | +| --- | --- | +| **`handler`** | Runs code in the browser (a frontend action) | +| **`render`** | Draws UI for the tool call (generative UI) | + +You can supply either one, or both. A `handler` with a `render` alongside it performs the action and draws UI while it runs. For render-only tools that just display the result of an agent action, see [Tool-Based Generative UI](/en/guides/frontend/tool-based-generative-ui). + +## Related + + + + Map agent tool calls to React components. + + + Gate agent actions behind user approval. + + + Keep agent state and your app UI in two-way sync. + + diff --git a/docs/edge/en/guides/frontend/generative-ui.mdx b/docs/edge/en/guides/frontend/generative-ui.mdx new file mode 100644 index 0000000000..b8e2da30b9 --- /dev/null +++ b/docs/edge/en/guides/frontend/generative-ui.mdx @@ -0,0 +1,107 @@ +--- +title: Generative UI +description: Render your CrewAI agent's work as live React components, across the full spectrum from author-controlled to agent-invented UI. +icon: wand-magic-sparkles +mode: "wide" +--- + +## Beyond the chat bubble + +Generative UI means the agent's work shows up as real interface, not just text. When your Crew or Flow calls a tool, updates its state, or reasons about a problem, you decide what the user sees: a progress checklist, a recipe card, a chart, a whole assembled panel. + +CopilotKit renders generative UI along a **spectrum**, from fully author-controlled (you decide every pixel) to agent-invented (the agent assembles the surface): + +| Tier | Who decides the UI | CrewAI mechanism | +| --- | --- | --- | +| **[Controlled](#controlled)** | You — a fixed set of components the agent picks from | `useRenderTool`, `useAgent`, reasoning | +| **[Declarative](#declarative)** | The agent — assembles a surface from *your* component catalog | [A2UI](/en/guides/frontend/a2ui) | +| **[Open-ended](#open-ended)** | An external tool/server invents the surface | MCP tools | + +The tiers compose freely; a single app usually mixes them. + +## Controlled + +You own the components. The agent chooses which to show and with what data. This is the most predictable tier and where most apps start. + +### Tool rendering + +The agent calls a tool on the backend. You register a matching component on the frontend with `useRenderTool`, and CopilotKit renders it, streaming the arguments in as they arrive. + +```tsx +"use client"; +import { useRenderTool } from "@copilotkit/react-core/v2"; +import { z } from "zod"; + +useRenderTool({ + name: "generate_recipe", + parameters: z.object({ + title: z.string(), + ingredients: z.array(z.string()), + }), + render: ({ args }) => , +}); +``` + + +`useRenderTool` renders a tool call. When a tool also needs to *run* code in the browser, use [`useFrontendTool`](/en/guides/frontend/frontend-actions) (a `handler`, with optional `render`). + + +See [Tool-Based Generative UI](/en/guides/frontend/tool-based-generative-ui) for the full walkthrough, including progressive rendering as arguments stream, and [Backend Tool Rendering](/en/guides/frontend/tool-based-generative-ui#backend-tools) for tools your Crew or Flow executes server-side. + +### State rendering + +Instead of reacting to a single tool call, render the agent's **state** as it changes. This is the right pattern for multi-step work: read the agent's working state with `useAgent` and paint it however you like. + +```tsx +"use client"; +import { useAgent } from "@copilotkit/react-core/v2"; + +function TaskProgress() { + const { agent } = useAgent({ agentId: "task_runner" }); + const steps = agent?.state?.steps ?? []; + return ; +} +``` + +See [Agentic Generative UI](/en/guides/frontend/agentic-generative-ui) for streaming state from a Flow, and [Shared State](/en/guides/frontend/shared-state) for editing that state from the UI. + +### Reasoning + +When the model reasons before answering, that thinking renders in the chat automatically. No component to write. See [Reasoning](/en/guides/frontend/reasoning). + +## Declarative + +The agent goes beyond picking a component: it **assembles a surface** by combining building blocks from a catalog *you* define. You still own the components (the agent can only use what is in your catalog), but the layout is the agent's. + +This is [A2UI](/en/guides/frontend/a2ui). You register a catalog on the provider: + +```tsx + + {/* ... */} + +``` + +The agent then builds surfaces from that catalog — either dynamically (it designs the layout from the conversation) or from a fixed schema your backend fills with data. See [A2UI](/en/guides/frontend/a2ui) for both modes and error recovery. + +## Open-ended + +At the far end, the surface is invented outside your app entirely. For CrewAI this comes through **MCP**: tools served by an MCP server the agent connects to render as tool calls in the chat, the same way backend tools do. This is the least constrained and the least predictable tier. + +MCP tool calls surface as standard tool-call UI — render them with `useRenderTool` like any other tool. Full agent-invented "MCP App" surfaces are an emerging capability; see the [CopilotKit docs](https://docs.copilotkit.ai) for the current state. + +## Related + + + + Map agent tool calls to components (controlled). + + + Render live agent state (controlled). + + + Let the agent assemble surfaces from your catalog (declarative). + + + Render the agent's thinking. + + diff --git a/docs/edge/en/guides/frontend/human-in-the-loop.mdx b/docs/edge/en/guides/frontend/human-in-the-loop.mdx new file mode 100644 index 0000000000..61f9c6749b --- /dev/null +++ b/docs/edge/en/guides/frontend/human-in-the-loop.mdx @@ -0,0 +1,191 @@ +--- +title: Human-in-the-Loop +description: Pause your CrewAI agent mid-run to collect a user decision, then resume the agent with their answer. +icon: user-check +mode: "wide" +--- + +## Put the user in the loop + +Some steps should not happen without a human saying yes. Human-in-the-loop pauses the agent mid-run, renders an interactive component in the frontend, and waits. The user makes a choice; the agent resumes with that choice and continues. + +The mechanism is a tool the frontend registers. When the model calls it, the run halts at that tool call until the user responds. Nothing happens automatically: the agent stays parked until `respond()` hands control back. + +In the example below, the agent proposes a list of task steps. The user enables or disables each step and confirms. The agent then continues, respecting exactly what the user approved. + + +This pattern works with Flows. It relies on the Flow's chat loop re-entering after `respond()`: the returned value comes back as a tool result, and the agent's next turn acts on it. + + +## Build it + + + + + +In your Flow, add the frontend-registered actions to the model's tool list with `*self.state.copilotkit.actions`. Those actions are the tools your frontend registered (via `useHumanInTheLoop`). Binding them lets the model call them; the run pauses at that tool call until the user responds. + +```python +# human_in_the_loop_flow.py +from crewai.flow.flow import Flow, start, router, listen +from litellm import acompletion +from ag_ui_crewai.sdk import copilotkit_stream, CopilotKitState + + +class HumanInTheLoopFlow(Flow[CopilotKitState]): + @start() + @listen("route_follow_up") + async def start_flow(self): + pass + + @router(start_flow) + async def chat(self): + system_prompt = ( + "You perform tasks for the user. When asked to do a task, call the " + "tool the frontend provides so the user can approve or adjust the steps " + "before you continue." + ) + + response = await copilotkit_stream( + await acompletion( + model="openai/gpt-4o", + messages=[ + {"role": "system", "content": system_prompt}, + *self.state.messages, + ], + tools=[*self.state.copilotkit.actions], # tools registered by the frontend + parallel_tool_calls=False, + stream=True, + ) + ) + + message = response.choices[0].message + self.state.messages.append(message) + return "route_end" + + @listen("route_end") + async def end(self): + pass +``` + +`CopilotKitState` carries the frontend-registered actions on `self.state.copilotkit.actions`. When the model calls one, the run pauses there. After the user responds, the returned value lands in `self.state.messages` as the tool result, and the Flow loops back through `chat` so the model can act on the decision. + + + + + +Expose the Flow from your FastAPI server with `add_crewai_flow_fastapi_endpoint`, the same way as every other agent. See [Frontend Overview](/en/guides/frontend/overview) for the full server, runtime, and provider setup. + +```python +# server.py +from fastapi import FastAPI +from ag_ui_crewai.endpoint import add_crewai_flow_fastapi_endpoint +from my_agents.human_in_the_loop_flow import HumanInTheLoopFlow + +app = FastAPI(title="CrewAI Agent Server") + +add_crewai_flow_fastapi_endpoint( + app=app, + flow=HumanInTheLoopFlow(), + path="/human_in_the_loop", +) +``` + + + + + +`useHumanInTheLoop` registers the tool the agent pauses on and gives you a `render` function to draw the interactive UI. When the agent calls the tool, your component appears; when the user acts, you call `respond()` to resume the agent. + +```tsx +"use client"; +import { useHumanInTheLoop } from "@copilotkit/react-core/v2"; +import { z } from "zod"; + +useHumanInTheLoop({ + agentId: "human_in_the_loop", + name: "generate_task_steps", + parameters: z.object({ + steps: z.array( + z.object({ + description: z.string(), + status: z.enum(["enabled", "disabled", "executing"]), + }) + ), + }), + render: ({ args, respond, status }) => ( + respond?.(chosen)} + /> + ), +}); +``` + +The `render` function receives: + +- **`args`** — the tool arguments the model produced (here, the proposed `steps`). These stream in as the model generates them. +- **`status`** — the tool call's lifecycle. While it is `"executing"`, the agent is paused and waiting on the human. +- **`respond(value)`** — resumes the agent with the user's decision. The agent's next turn sees the returned value and acts on it. + + + + + +Your component reads `args.steps`, lets the user toggle each one, and calls `respond()` with the final selection. That value is what the agent continues with. + +```tsx +function StepReview({ steps, waiting, onConfirm }) { + const [choices, setChoices] = useState(steps); + + const toggle = (i) => + setChoices((prev) => + prev.map((s, idx) => + idx === i + ? { ...s, status: s.status === "enabled" ? "disabled" : "enabled" } + : s + ) + ); + + return ( +
+ {choices.map((step, i) => ( + + ))} + +
+ ); +} +``` + +Once the user clicks Confirm, `respond()` fires, the run resumes, and the Flow's `chat` step runs again with the user's choices in the message history. + + + + + +## Related + + + + Let the agent call functions that run in the browser. + + + Keep agent state and your app UI in two-way sync. + + + Render live agent state as custom components. + + diff --git a/docs/edge/en/guides/frontend/overview.mdx b/docs/edge/en/guides/frontend/overview.mdx new file mode 100644 index 0000000000..326c917216 --- /dev/null +++ b/docs/edge/en/guides/frontend/overview.mdx @@ -0,0 +1,238 @@ +--- +title: Frontend Overview +description: Build interactive user interfaces for your CrewAI agents with CopilotKit and the AG-UI protocol. +icon: browser +mode: "wide" +--- + +## Give your agents a user interface + +CrewAI runs your agents. [CopilotKit](https://copilotkit.ai) gives them a frontend. Together they let you build applications where users chat with a Crew or Flow, watch it work in real time, approve its decisions, and see its output rendered as live UI instead of walls of text. + +The two connect through the [AG-UI protocol](https://docs.ag-ui.com). The `ag-ui-crewai` package exposes any Crew or Flow as an AG-UI endpoint. CopilotKit's React hooks and components consume that endpoint. This unlocks experiences that go well beyond a chat box: + + + + Render agent tool calls and state as your own React components. + + + Pause the agent to collect user approval or input mid-run. + + + Keep agent state and your app UI in two-way sync. + + + Run the same agent as a Slack, Discord, or Teams bot. + + + +This guide gets a Crew or Flow talking to a Next.js frontend end to end. The rest of the section builds on the app you set up here. + +## Architecture + +There are three pieces: + +1. **CrewAI agent server** — a Python process that serves your Crew or Flow over AG-UI (FastAPI + `ag-ui-crewai`). +2. **CopilotKit runtime** — a Next.js route that registers your agent and proxies requests to it. +3. **React frontend** — the `` provider plus chat and generative-UI components. + +``` +React app ──► CopilotKit runtime (/api/copilotkit) ──► CrewAI server (AG-UI) ──► Crew / Flow +``` + + +This guide covers the **self-hosted** path: you run the CrewAI agent server yourself with `ag-ui-crewai`, and it works locally with no managed service. CopilotKit also offers a **managed** path (CopilotKit Cloud / Enterprise Intelligence) with hosted threads and an inspector — see the [CopilotKit CrewAI quickstart](https://docs.copilotkit.ai/crewai-crews/quickstart) if you want that instead. The frontend code in this section is the same either way; only how the agent is hosted and registered differs. + + + +CrewAI runs behind AG-UI in three shapes: regular **Flows** (used throughout these guides), **[Conversational Flows](/en/guides/frontend/conversational-flows)** (native, session-aware, turn-based, at full feature parity), and **Crews** (basic chat). The frontend in this section is identical across them — only the backend authoring and registration differ. + + +## Integration guide + + + + + +Install the integration package into your CrewAI project: + +```bash +pip install ag-ui-crewai +``` + +Expose your agent from a FastAPI app. Flows use `add_crewai_flow_fastapi_endpoint`; Crews use `add_crewai_crew_fastapi_endpoint`. You can register as many as you want, each on its own path. + + + +```python Flow +# server.py +from fastapi import FastAPI +from ag_ui_crewai.endpoint import add_crewai_flow_fastapi_endpoint +from my_agents.recipe_flow import RecipeFlow + +app = FastAPI(title="CrewAI Agent Server") + +add_crewai_flow_fastapi_endpoint( + app=app, + flow=RecipeFlow(), + path="/recipe", +) +``` + +```python Crew +# server.py +from fastapi import FastAPI +from ag_ui_crewai.endpoint import add_crewai_crew_fastapi_endpoint +from my_agents.research_crew import ResearchCrew + +app = FastAPI(title="CrewAI Agent Server") + +add_crewai_crew_fastapi_endpoint( + app=app, + crew=ResearchCrew().crew(), + path="/research", +) +``` + + + +Run it: + +```bash +uvicorn server:app --port 8000 +``` + + +Set the environment variables for your LLM provider (for example `OPENAI_API_KEY`) before starting the server. + + + + + + +If you do not have a frontend yet, scaffold one: + +```bash +npx create-next-app@latest my-app +cd my-app +``` + +Install CopilotKit and the CrewAI AG-UI client: + +```bash +npm install @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime @ag-ui/crewai +``` + + + + + +Create a route that registers your CrewAI agent(s) with the CopilotKit runtime. Each agent points at a path on your Python server via `CrewAIAgent`. + +```ts +// app/api/copilotkit/route.ts +import { + CopilotRuntime, + InMemoryAgentRunner, + createCopilotEndpoint, +} from "@copilotkit/runtime/v2"; +import { CrewAIAgent } from "@ag-ui/crewai"; +import { handle } from "hono/vercel"; + +const runtime = new CopilotRuntime({ + agents: { + recipe: new CrewAIAgent({ url: "http://localhost:8000/recipe" }), + }, + runner: new InMemoryAgentRunner(), +}); + +const app = createCopilotEndpoint({ + runtime, + basePath: "/api/copilotkit", +}); + +const handler = handle(app); +export const GET = handler; +export const POST = handler; +``` + + + + + +Point `` at the runtime route and name the agent you registered. + +```tsx +// app/page.tsx +"use client"; +import { CopilotKit } from "@copilotkit/react-core"; +import { CopilotSidebar } from "@copilotkit/react-core/v2"; +import "@copilotkit/react-core/v2/styles.css"; + +export default function Page() { + return ( + + + + + ); +} +``` + + + + + +Start both processes and open the app. Chatting in the sidebar now runs your Crew or Flow. + +```bash +uvicorn server:app --port 8000 # terminal 1 +npm run dev # terminal 2 +``` + + + + + +## Chat UI options + +CopilotKit ships three interchangeable chat surfaces. Swap the component; the wiring is identical. + + + +```tsx Sidebar +import { CopilotSidebar } from "@copilotkit/react-core/v2"; + + +``` + +```tsx Popup +import { CopilotPopup } from "@copilotkit/react-core/v2"; + + +``` + +```tsx Inline +import { CopilotChat } from "@copilotkit/react-core/v2"; + + +``` + + + +## Where to go next + + + + Render tool calls and agent state as custom components. + + + Let the agent call functions that run in the browser. + + + Gate agent actions behind user approval. + + + Stream in-progress state to the UI as the agent works. + + diff --git a/docs/edge/en/guides/frontend/predictive-state-updates.mdx b/docs/edge/en/guides/frontend/predictive-state-updates.mdx new file mode 100644 index 0000000000..ed9204311b --- /dev/null +++ b/docs/edge/en/guides/frontend/predictive-state-updates.mdx @@ -0,0 +1,142 @@ +--- +title: Predictive State Updates +description: Stream an in-progress tool call's arguments into agent state so the UI updates optimistically while the agent is still generating. +icon: gauge-high +mode: "wide" +--- + +## Show the work as it happens + +Normally a tool call is atomic from the UI's point of view: the agent decides what to write, and your interface only sees the result once the call finishes. For a tool that produces a large document that means a long pause followed by everything snapping into place at once. + +Predictive state updates remove the wait. You project a streaming tool argument onto a field of the agent's state, so as the model generates the argument token by token, that state field fills in live. A document the agent is writing appears in the editor as it is typed, not after. + + +Predictive state relies on a Flow with custom state (`Flow[AgentState]`). It projects a streaming tool argument onto a state field, so there is no equivalent for a bare Crew. + + +## How it compares to Shared State + +Both patterns read the agent's state from the frontend, but they solve different problems: + +| Pattern | What it does | +| --- | --- | +| **Predictive state** | One-way. Streams an in-progress tool argument into a state field so the UI updates *during* generation, before the call completes. | +| **[Shared State](/en/guides/frontend/shared-state)** | Two-way. The UI reads *and writes* the agent's committed state, keeping app and agent in sync across turns. | + +Reach for predictive state when you want an optimistic, in-flight preview of what the agent is producing. Reach for [Shared State](/en/guides/frontend/shared-state) when the user needs to edit that state back. + +## Walkthrough + +This assumes you already have a Crew or Flow served over AG-UI and a CopilotKit frontend wired up. If not, start with the [Frontend Overview](/en/guides/frontend/overview). + + + + + +Predictive state projects a tool argument onto a state field, so your Flow needs a typed state field to receive it. Add the field you want to stream into to your `CopilotKitState` subclass. + +```python +from typing import Optional +from crewai.flow.flow import Flow, start, router, listen +from litellm import acompletion +from ag_ui_crewai.sdk import copilotkit_stream, copilotkit_predict_state, CopilotKitState + +WRITE_DOCUMENT_TOOL = { + "type": "function", + "function": { + "name": "write_document", + "description": "Write the full document in markdown.", + "parameters": { + "type": "object", + "properties": { + "document": {"type": "string", "description": "The document to write"}, + }, + }, + }, +} + +class AgentState(CopilotKitState): + document: Optional[str] = None + +class DocumentFlow(Flow[AgentState]): + @start() + @listen("route_follow_up") + async def start_flow(self): + pass +``` + + + + + +Call `copilotkit_predict_state` **before** you start streaming the completion. It tells the runtime to project the named tool argument onto the named state field: as the `write_document` call streams its `document` argument, the `document` state field updates live. + +```python + @router(start_flow) + async def chat(self): + # Map the `document` state field to the `document` argument of write_document. + # As the tool call streams, the state field updates live. + await copilotkit_predict_state({ + "document": {"tool_name": "write_document", "tool_argument": "document"}, + }) + + response = await copilotkit_stream( + await acompletion( + model="openai/gpt-4o", + messages=[ + {"role": "system", "content": "Write and edit the document with write_document."}, + *self.state.messages, + ], + tools=[*self.state.copilotkit.actions, WRITE_DOCUMENT_TOOL], + parallel_tool_calls=False, + stream=True, + ) + ) + message = response.choices[0].message + self.state.messages.append(message) +``` + +The key is `copilotkit_predict_state({ "": {"tool_name": ..., "tool_argument": ...} })`. Without it, the frontend would only see `document` once the tool call completed. With it, the partial argument streams onto the field while the agent is still generating. + +Serve the Flow with `add_crewai_flow_fastapi_endpoint(...)` as shown in the [Frontend Overview](/en/guides/frontend/overview). + + + + + +On the frontend, read the field with `useAgent` and subscribe to state changes. Because the backend is projecting the streaming argument onto `document`, this component re-renders as the agent types. + +```tsx +"use client"; +import { useAgent, UseAgentUpdate } from "@copilotkit/react-core/v2"; + +function DocumentView() { + const { agent } = useAgent({ + agentId: "document", + updates: [UseAgentUpdate.OnStateChanged], + }); + const document = (agent?.state as { document?: string })?.document ?? ""; + return
{document}
; // updates as the agent types +} +``` + +The `document` field fills in progressively as the agent generates the `write_document` call, so the editor updates in real time rather than snapping in at the end. + + + + + +## Related + + + + Read and write the agent's state two-way. + + + Render live agent state as it changes. + + + Map agent tool calls to components. + + diff --git a/docs/edge/en/guides/frontend/reasoning.mdx b/docs/edge/en/guides/frontend/reasoning.mdx new file mode 100644 index 0000000000..104c37e944 --- /dev/null +++ b/docs/edge/en/guides/frontend/reasoning.mdx @@ -0,0 +1,68 @@ +--- +title: Reasoning +description: Show the model's thinking in the chat automatically, with no component to build. +icon: brain +mode: "wide" +--- + +## Thinking, rendered for free + +When a reasoning-capable model thinks before it answers, CopilotKit renders that thinking right in the chat. This is the simplest generative-UI pattern in the whole section: there is nothing to build. No hook, no component, no props. Use a reasoning-capable model, keep the streaming wrapper your Flows already have, and the chat surface from the [Overview](/en/guides/frontend/overview) does the rest. + +## Use a reasoning-capable model + +Reasoning is surfaced automatically by `copilotkit_stream`, which every Flow example already wraps the model call in. The bridge reads the model's reasoning deltas and emits them to the frontend. It is provider-agnostic and works over both of CrewAI's streaming transports, so the only thing you change is the model. + +```python +# recipe_flow.py +from crewai.flow.flow import Flow, start +from ag_ui_crewai.sdk import copilotkit_stream, CopilotKitState +from litellm import acompletion + + +class RecipeFlow(Flow[CopilotKitState]): + @start() + async def chat(self): + response = await copilotkit_stream( + acompletion( + # any reasoning-capable model, e.g. deepseek-reasoner + model="deepseek/deepseek-reasoner", + messages=self.state.messages, + stream=True, + ) + ) + message = response.choices[0].message + self.state.messages.append(message) +``` + +Models that emit reasoning over the standard channel include DeepSeek `deepseek-reasoner`, Anthropic extended thinking (Claude), and Gemini thinking, among others. Swap the `model` for one of these and its thinking starts streaming through. + +This works the same for both Crews and Flows, since both run their model calls through `copilotkit_stream`. + +## Render it + +There is no frontend step. The `CopilotChat`, `CopilotSidebar`, or `CopilotPopup` surface you already mounted shows the reasoning as it streams, above the answer it produced. + +```tsx +import { CopilotChat } from "@copilotkit/react-core/v2"; + + +``` + + +There is no `useReasoning` hook and no reasoning component to write. Reasoning is not something you wire up on the frontend; it renders automatically as long as the model emits it. + + +## Related + + + + The full spectrum, from author-controlled to agent-invented UI. + + + Render live agent state as the Flow works. + + + Set up the chat surface and runtime. + + diff --git a/docs/edge/en/guides/frontend/shared-state.mdx b/docs/edge/en/guides/frontend/shared-state.mdx new file mode 100644 index 0000000000..c42fac2cf4 --- /dev/null +++ b/docs/edge/en/guides/frontend/shared-state.mdx @@ -0,0 +1,210 @@ +--- +title: Shared State +description: Keep your CrewAI agent's state and your app's UI in two-way sync, so edits on either side flow to the other. +icon: arrows-rotate +mode: "wide" +--- + +## One state, both directions + +Shared state is a single state object that the agent and the UI both read and write. The agent updates it as it works and your React components render it live. When the user edits that same state in the UI, the change flows back so the agent sees it on its next turn. + +The classic example is a recipe: the agent drafts it, the user tweaks an ingredient or an instruction, and the agent picks up from the edited version. Neither side owns the state; they share it. + + +Shared state relies on a Flow with custom state. Define an `AgentState` that subclasses `CopilotKitState` and type your Flow as `Flow[AgentState]`. Crews do not carry custom state, so this pattern is Flow-only. + + +## How it works + + + + + +Subclass `CopilotKitState` so the agent keeps CopilotKit's message plumbing, then add your own fields. Here the shared field is `recipe`. + +```python +# recipe_flow.py +import json +from typing import List, Optional +from pydantic import BaseModel, Field +from crewai.flow.flow import Flow, start, router, listen +from litellm import acompletion +from ag_ui_crewai.sdk import copilotkit_stream, CopilotKitState + + +class Ingredient(BaseModel): + name: str + amount: str + + +class Recipe(BaseModel): + title: str + ingredients: List[Ingredient] = Field(default_factory=list) + instructions: List[str] = Field(default_factory=list) + + +class AgentState(CopilotKitState): + recipe: Optional[Recipe] = None +``` + + + + + +The agent reads the current state by dumping it into the system prompt, and writes it back by assigning to `self.state.recipe`. A `generate_recipe` tool lets the model return the updated recipe as structured arguments. + +```python +GENERATE_RECIPE_TOOL = { + "type": "function", + "function": { + "name": "generate_recipe", + "description": "Generate or modify the recipe.", + "parameters": { + "type": "object", + "properties": {"recipe": {"type": "object"}}, + "required": ["recipe"], + }, + }, +} + + +class SharedStateFlow(Flow[AgentState]): + @start() + @listen("route_follow_up") + async def start_flow(self): + pass + + @router(start_flow) + async def chat(self): + # The current shared state is visible to the model. + system_prompt = f"""You help the user build a recipe. + Current recipe: {self.state.model_dump_json(indent=2)} + Modify it by calling generate_recipe.""" + + response = await copilotkit_stream( + await acompletion( + model="openai/gpt-4o", + messages=[ + {"role": "system", "content": system_prompt}, + *self.state.messages, + ], + tools=[*self.state.copilotkit.actions, GENERATE_RECIPE_TOOL], + parallel_tool_calls=False, + stream=True, + ) + ) + message = response.choices[0].message + self.state.messages.append(message) + + if message.tool_calls: + call = message.tool_calls[0] + if call.function.name == "generate_recipe": + args = json.loads(call.function.arguments) + self.state.recipe = Recipe(**args["recipe"]) # write to shared state + self.state.messages.append({ + "role": "tool", + "content": "Recipe updated.", + "tool_call_id": call.id, + }) + return "route_follow_up" + return "route_end" + + @listen("route_end") + async def end(self): + pass +``` + +Two things make this shared rather than one-way: dumping `self.state` into the prompt means the agent always works from the latest recipe (including edits the user made in the UI), and assigning `self.state.recipe` puts the new value into the state snapshot sent to connected clients at the end of the step. For updates during a long step, emit explicitly with `copilotkit_emit_state` (see [Agentic Generative UI](/en/guides/frontend/agentic-generative-ui)). + + + + + +Expose the Flow from your FastAPI app with `add_crewai_flow_fastapi_endpoint`, then register it in the CopilotKit runtime. See the [Frontend Overview](/en/guides/frontend/overview) for the full server and runtime setup. + +```python +# server.py +from fastapi import FastAPI +from ag_ui_crewai.endpoint import add_crewai_flow_fastapi_endpoint +from recipe_flow import SharedStateFlow + +app = FastAPI(title="CrewAI Agent Server") + +add_crewai_flow_fastapi_endpoint( + app=app, + flow=SharedStateFlow(), + path="/shared_state", +) +``` + + + + + +`useAgent` gives you both directions in one hook. Read the shared state off `agent.state`, and write it back with `agent.setState(...)`. Subscribe to `OnStateChanged` so your component re-renders whenever the agent updates the state. + +```tsx +"use client"; +import { useAgent, UseAgentUpdate } from "@copilotkit/react-core/v2"; + +function RecipeEditor() { + const { agent } = useAgent({ + agentId: "shared_state", + updates: [UseAgentUpdate.OnStateChanged], + }); + + const state = agent?.state as { recipe?: Recipe } | undefined; + const isLoading = agent?.isRunning; + + const recipe = state?.recipe; + + // setState replaces the whole state object, so spread the current + // state and override only the field you changed. Passing just + // `{ recipe }` would drop messages and other runtime fields. + const updateRecipe = (patch: Partial) => + agent?.setState({ ...(agent.state ?? {}), recipe: { ...(recipe ?? {}), ...patch } }); + + return ( +
+ updateRecipe({ title: e.target.value })} + /> + {/* render inputs for ingredients and instructions the same way */} +
+ ); +} +``` + +`agent.state` reads the shared state, `agent.setState(...)` writes it back so the agent sees the change on its next turn, and `agent.isRunning` reflects whether the agent is currently working. + + +`setState` **replaces** the entire state object rather than merging. Always spread the current state (`{ ...agent.state, ... }`) and override only the fields you are changing, or you will drop the conversation and other runtime fields the agent depends on. + + + + + + +## The two-way loop + +Putting the pieces together, a single recipe object is kept in sync in both directions: + +- **Agent edits, UI updates.** The Flow assigns `self.state.recipe`, the new value ships in the step's state snapshot, and `OnStateChanged` re-renders your inputs. +- **User edits, agent sees it.** A change in the UI calls `agent.setState(...)`, and because the Flow dumps `self.state` into its prompt, the agent works from the edited recipe on its next turn. + +## Related + + + + Render live agent state as it changes. + + + Stream in-progress state to the UI as the agent works. + + + Pause the agent to collect user approval or input mid-run. + + diff --git a/docs/edge/en/guides/frontend/tool-based-generative-ui.mdx b/docs/edge/en/guides/frontend/tool-based-generative-ui.mdx new file mode 100644 index 0000000000..2ef4e63d2b --- /dev/null +++ b/docs/edge/en/guides/frontend/tool-based-generative-ui.mdx @@ -0,0 +1,235 @@ +--- +title: Tool-Based Generative UI +description: Map a CrewAI agent's tool calls to React components and stream the arguments in as they arrive. +icon: puzzle-piece +mode: "wide" +--- + +## Render tool calls as components + +When your Crew or Flow calls a tool, you rarely want the raw arguments dumped into the chat. Tool-based generative UI maps each tool the agent calls to a React component you own. The agent decides *when* to call the tool; you decide what the user sees. + +Because CopilotKit streams the tool call to the frontend as the model generates it, the arguments fill in progressively. Your component can paint the moment the first field arrives and update as the rest stream in. + +This guide builds a haiku generator: the agent calls a `generate_haiku` tool, and the frontend renders each haiku as a card. It assumes you already have a Crew or Flow talking to a Next.js app. If not, start with the [Frontend Overview](/en/guides/frontend/overview) for the full server, runtime, and provider setup. + + +Tool rendering works with both Crews and Flows. The example below uses a Flow, but the frontend wiring is identical either way. + + +## Walkthrough + + + + + +Declare the tool with a JSON schema and pass it to the model. The `copilotkit_stream` wrapper together with `stream=True` is what streams the tool call to the frontend as it is generated, one argument chunk at a time. + +```python +# haiku_flow.py +from crewai.flow.flow import Flow, start +from litellm import acompletion +from ag_ui_crewai.sdk import copilotkit_stream, CopilotKitState + +GENERATE_HAIKU_TOOL = { + "type": "function", + "function": { + "name": "generate_haiku", + "description": "Generate a haiku in Japanese and its English translation", + "parameters": { + "type": "object", + "properties": { + "japanese": { + "type": "array", + "items": {"type": "string"}, + "description": "Three lines in Japanese", + }, + "english": { + "type": "array", + "items": {"type": "string"}, + "description": "Three lines in English", + }, + }, + "required": ["japanese", "english"], + }, + }, +} + + +class HaikuFlow(Flow[CopilotKitState]): + @start() + async def chat(self): + system_prompt = "You help the user write haikus. Use the generate_haiku tool." + + response = await copilotkit_stream( + await acompletion( + model="openai/gpt-4o", + messages=[ + {"role": "system", "content": system_prompt}, + *self.state.messages, + ], + tools=[GENERATE_HAIKU_TOOL], + parallel_tool_calls=False, + stream=True, + ) + ) + + message = response.choices[0].message + self.state.messages.append(message) + + if message.tool_calls: + self.state.messages.append({ + "tool_call_id": message.tool_calls[0].id, + "role": "tool", + "content": "Haiku generated.", + }) +``` + +The tool has no Python implementation. It exists only so the model emits a structured call the frontend can render. After the call, append a short tool result so the conversation stays well-formed for the next turn. + + + + + +Expose the Flow from your FastAPI app on its own path: + +```python +# server.py +from fastapi import FastAPI +from ag_ui_crewai.endpoint import add_crewai_flow_fastapi_endpoint +from haiku_flow import HaikuFlow + +app = FastAPI(title="CrewAI Agent Server") + +add_crewai_flow_fastapi_endpoint( + app=app, + flow=HaikuFlow(), + path="/haiku", +) +``` + +Register the agent with the CopilotKit runtime and point `` at it exactly as shown in the [Frontend Overview](/en/guides/frontend/overview). The rest of this guide assumes the agent is registered under the id `haiku`. + + + + + +On the frontend, call `useRenderTool` with the same `name` the backend declared. `useRenderTool` is the hook for *rendering* a tool call: it takes a `render` function and nothing to execute, because this tool is pure display. + + +Use `useRenderTool` when the tool only draws UI. If the tool also needs to *run* something in the browser, use [`useFrontendTool`](/en/guides/frontend/frontend-actions) instead, which pairs a `handler` with an optional `render`. + + +```tsx +"use client"; +import { useRenderTool } from "@copilotkit/react-core/v2"; +import { z } from "zod"; + +useRenderTool({ + name: "generate_haiku", + parameters: z.object({ + japanese: z.array(z.string()), + english: z.array(z.string()), + }), + render: ({ args, status }) => { + if (!args.japanese) return <>; // still streaming + return ; + }, +}); +``` + +The tool is scoped to the active agent by the `` provider, so no `agentId` is needed here. A few things to note: + +- **`name` must match the backend tool name** exactly (`generate_haiku`). That match is how CopilotKit routes the call to this component. +- **`render` receives `{ args, status }`.** `args` fills in progressively as the model streams the call; early on it may be empty or partial. `status` moves through `"inProgress"` / `"executing"` to `"complete"` if you want to show a loading state while arguments stream. +- **Guard against partial args.** Return an empty fragment until the fields you need exist. Here we wait for `args.japanese` before rendering the card. + + + + + +The `render` function delegates to an ordinary React component. Nothing about it is CopilotKit-specific: it takes props and returns markup. + +```tsx +function HaikuCard({ + japanese, + english, +}: { + japanese: string[]; + english: string[]; +}) { + return ( +
+ {japanese.map((line, i) => ( +
+ {line} + {english?.[i]} +
+ ))} +
+ ); +} +``` + +Because `english` streams in alongside `japanese`, use optional access (`english?.[i]`) so the card renders cleanly while the translation is still arriving. + + + + + +Start both processes and ask the assistant for a haiku. The card renders as the arguments stream in, filling out line by line. + +```bash +uvicorn server:app --port 8000 # terminal 1 +npm run dev # terminal 2 +``` + + + + + +## How progressive rendering works + +The model does not emit the tool call all at once. It streams tokens, and CopilotKit re-invokes your `render` function every time a new chunk of arguments arrives: + +1. The call begins. `args` is empty, so your guard returns an empty fragment. +2. `args.japanese` fills in line by line. The card appears and grows. +3. `args.english` fills in. Translations slot into place. +4. The call completes. `args` holds the final, fully-validated object. + +This is why the partial-args guard matters: `render` runs against incomplete data by design. Read only the fields you have, and let the rest paint as they arrive. + +## Backend tools + +The `generate_haiku` tool above has no Python implementation — it exists only so the model emits a structured call the frontend renders. But a **real tool your Crew or Flow runs server-side** renders the same way. + +When an Agent or Crew executes a tool during its run, the bridge surfaces that tool call along with its **result**. Register a `useRenderTool` for the tool's name and read `result` in the render: + +```tsx +useRenderTool({ + name: "get_weather", + parameters: z.object({ location: z.string() }), + render: ({ args, result, status }) => { + if (status !== "complete") return ; + return ; + }, +}); +``` + + +A backend tool must return a **JSON string**, not a Python dict. The bridge stringifies tool output, so a raw dict arrives as a Python repr the browser cannot `JSON.parse`. Return `json.dumps(...)` from the tool. + + +## Related + + + + Render live agent state as it changes across a multi-step run. + + + Pause the agent to collect user approval or input mid-run. + + + Let the agent call functions that run in the browser. + +