LLM
The LLM type initializes a Large Language Model (LLM).
ENV TYPEYou use an LLM in conjunction with Env. The Env type is used to represent the environment in which an LLM operates. It allows the LLM to interact with inputs and outputs, such as directories, containers, and custom modules.
API reference
A conversation with a large language model (LLM): queue prompts, expose tools, and step the model until it completes its turn.
contextWindow- The model's total context window in tokens, or null if unknown (e.g. a local or uncatalogued model).
hasPending- Report whether anything is queued to send to the model: an unsent prompt or unevaluated tool results. When true, another step will do work; when false, the turn is complete.
id- A unique identifier for this LLM.
lastReply- The text of the model's most recent reply.
model- The model the conversation is running against, after resolving any configured default.
portableID- A portable, self-contained ID for the conversation that node() can resolve in any session. Unlike id, which may return an engine-local runtime handle valid only within the current session, this returns the recipe form suitable for persisting and later restoring the conversation.
provider- The provider serving the model, e.g. "anthropic", "openai", "google", or "local".
tools- Render documentation for the tools currently exposed to the model.
transcript- The message history rendered as a plain-text transcript, suitable for feeding back to an LLM (e.g. for summarization).
bindResult- returns the type of the current state
env- return the LLM's current environment
fork- Fork the conversation, so that otherwise-identical follow-ups evaluate independently instead of deduplicating to a single cached result.
loop- Send the queued prompt and step the model against the available tools, until it ends its turn: a reply with no tool calls and nothing left queued.
messages- The full message history, as structured messages.
replay- Re-emit telemetry spans for the full message history, so a loaded conversation displays in the TUI.
step- Advance the conversation by a single step: send the queued prompt or tool results to the model, evaluate any tool calls it makes, and queue their results. Use loop to step until the model ends its turn.
sync- Force evaluation of the conversation's pending operations (prompts, steps, loops) in the engine.
tokenUsage- The cumulative token usage, summed across every API call in the conversation.
withBlockedFunction- Return a new LLM with the specified function no longer exposed as a tool
withEnv- allow the LLM to interact with an environment via MCP
withMCPServer- Add an external MCP server to the LLM
withModel- Change the model for the rest of the conversation. The message history is preserved; the new model takes effect on the next step.
withObject- Track an object so the LLM can reference it in subsequent tool calls.
withoutDefaultSystemPrompt- Disable the default system prompt
withoutMessageHistory- Clear the message history, keeping only the system prompts.
withoutSystemPrompts- Clear the user-added system prompts, keeping only the default system prompt.
withPrompt- Queue a user prompt, to be sent to the model on the next step or loop.
withPromptFile- Queue a file's contents as a user prompt, like withPrompt.
withResponse- Append an assistant response to the message history without calling the model, e.g. to reconstruct a conversation from another source.
withStaticTools- Use a static set of tools for method calls, e.g. for MCP clients that do not support dynamic tool registration
withSystemPrompt- Add a system prompt, instructing the model across the whole conversation.
withToolResult- Append the result of a tool call to the message history.
contextWindow: Int
The model's total context window in tokens, or null if unknown (e.g. a local or uncatalogued model).
hasPending: Boolean!
Report whether anything is queued to send to the model: an unsent prompt or unevaluated tool results. When true, another step will do work; when false, the turn is complete.
id: ID!
A unique identifier for this LLM.
lastReply: String!
The text of the model's most recent reply.
model: String!
The model the conversation is running against, after resolving any configured default.
portableID: ID!
A portable, self-contained ID for the conversation that node() can resolve in any session. Unlike id, which may return an engine-local runtime handle valid only within the current session, this returns the recipe form suitable for persisting and later restoring the conversation.
provider: String!
The provider serving the model, e.g. "anthropic", "openai", "google", or "local".
tools: String!
Render documentation for the tools currently exposed to the model.
transcript: String!
The message history rendered as a plain-text transcript, suitable for feeding back to an LLM (e.g. for summarization).
bindResult(name: String!): Binding
returns the type of the current state
name: String!
fork(label: String!): LLM!
Fork the conversation, so that otherwise-identical follow-ups evaluate independently instead of deduplicating to a single cached result.
label: String!A label distinguishing this fork from its siblings, e.g. "attempt-2" when retrying a flaky evaluation.
loop(maxSteps: Int, maxTokens: Int): LLM!
Send the queued prompt and step the model against the available tools, until it ends its turn: a reply with no tool calls and nothing left queued.
maxSteps: IntCap the number of steps. The loop fails if the cap is reached before the model ends its turn.
maxTokens: IntCap the model's output tokens on each step. Defaults to the model's maximum.
messages: [LLMMessage!]!
The full message history, as structured messages.
replay: LLM!
Re-emit telemetry spans for the full message history, so a loaded conversation displays in the TUI.
step(maxTokens: Int): LLM!
Advance the conversation by a single step: send the queued prompt or tool results to the model, evaluate any tool calls it makes, and queue their results. Use loop to step until the model ends its turn.
maxTokens: IntCap the model's output tokens for this step. Defaults to the model's maximum.
sync: LLM!
Force evaluation of the conversation's pending operations (prompts, steps, loops) in the engine.
tokenUsage: LLMTokenUsage!
The cumulative token usage, summed across every API call in the conversation.
withBlockedFunction(typeName: String!, function: String!): LLM!
Return a new LLM with the specified function no longer exposed as a tool
typeName: String!The type name whose function will be blocked
function: String!The function to block
Will be converted to lowerCamelCase if necessary.
withMCPServer(name: String!, service: Service!): LLM!
Add an external MCP server to the LLM
name: String!The name of the MCP server
service: Service!The MCP service to run and communicate with over stdio
withModel(model: String!, provider: String): LLM!
Change the model for the rest of the conversation. The message history is preserved; the new model takes effect on the next step.
model: String!The model to use, e.g. "claude-sonnet-4-5" or "gpt-5.4".
provider: StringThe provider serving the model, e.g. "openai". Overrides the provider otherwise inferred from the model name — useful when the name matches no known pattern (e.g. a fine-tune), or matches the wrong one.
withObject(tag: String!, object: ID!): LLM!
Track an object so the LLM can reference it in subsequent tool calls.
tag: String!Arbitrary string tag for the object, typically in TypeName#Number format
object: ID!The object to track, as a generic ID
withoutDefaultSystemPrompt: LLM!
Disable the default system prompt
withoutMessageHistory: LLM!
Clear the message history, keeping only the system prompts.
withoutSystemPrompts: LLM!
Clear the user-added system prompts, keeping only the default system prompt.
withPrompt(prompt: String!): LLM!
Queue a user prompt, to be sent to the model on the next step or loop.
prompt: String!The prompt to send
withPromptFile(file: File!): LLM!
Queue a file's contents as a user prompt, like withPrompt.
file: File!The file to read the prompt from
withResponse(content: [LLMContentBlockInput!]!,inputTokens: Int = 0,outputTokens: Int = 0,cachedTokenReads: Int = 0,cachedTokenWrites: Int = 0,totalTokens: Int = 0): LLM!
Append an assistant response to the message history without calling the model, e.g. to reconstruct a conversation from another source.
content: [LLMContentBlockInput!]!The response content
inputTokens: Int = 0Uncached input tokens sent
outputTokens: Int = 0Tokens received from the model, including text and tool calls
cachedTokenReads: Int = 0Cached input tokens read
cachedTokenWrites: Int = 0Cached input tokens written
totalTokens: Int = 0Total tokens consumed by this response
withStaticTools: LLM!
Use a static set of tools for method calls, e.g. for MCP clients that do not support dynamic tool registration
withSystemPrompt(prompt: String!): LLM!
Add a system prompt, instructing the model across the whole conversation.
prompt: String!The system prompt to send
withToolResult(callId: String!,content: String!,errored: Boolean!): LLM!
Append the result of a tool call to the message history.
callId: String!The ID of the tool call this result responds to
content: String!The content returned by the tool
errored: Boolean!Whether the tool call resulted in an error