Agent

This document details the core orchestration engine of the agent. The system is built on LangGraph, utilizing a dual-LLM architecture that separates the reasoning/tool-calling process from the final user-facing response generation.

Agent loop

1. State Graph and Routing (agent.py)

The agent’s workflow is modeled as a state machine (StateGraph) handling the MessagesState. It relies on two distinct LLM clients (ROUTER_LLM and FINAL_LLM) to orchestrate the workflow safely and efficiently.

Key Nodes

  • Router Node: Analyzes the conversation history and decides the next step. It is bound to the domain-specific tools (e.g., fetching logs or files) as well as two explicit control tools: ready_to_answer and declare_out_of_scope.

  • Tools Node: A prebuilt LangGraph node that executes the requested tool and appends the result to the state.

  • Handle Tool Error Node: A self-correction mechanism that catches malformed JSON or tool call syntax errors, instructing the Router LLM to fix the syntax and retry.

  • Generate Final Response Node: Intercepts the final state, removes the routing metadata, injects critical directives (e.g., missing context handling or out-of-scope refusals), and triggers the FINAL_LLM to formulate the final answer.

Circuit Breakers

The graph includes safety mechanisms to prevent infinite tool loops. If a tool returns a [MISSING_CONTEXT] error, the router is bypassed, and a critical directive is immediately passed to the final generator to alert the user.

2. Execution and Streaming (execute_agent.py)

The execution module is responsible for instantiating the LangGraph agent and streaming responses back to the client asynchronously.

Execution Modes

  • Production Mode (execute_agent_prod): Optimized for end-users. It streams strictly the "messages" event, yielding only the final strings generated by the FINAL_LLM for the UI.

  • Debug Mode (execute_agent_debug): Verbose execution used for internal testing. It tracks both "messages" (for intermediate router thoughts and tool calls) and "updates" (for state transitions between graph nodes).

Observability and State Management

  • Checkpointer: Uses AsyncPostgresSaver to persist conversation state across turns.

  • Tracing: Seamlessly integrates with Langfuse and LangSmith via callbacks, tagging sessions based on the execution environment (prod vs debug) and mapping them to the user’s chat_id.