Introduction

In AI Internals - RAG the model read evidence you packed into the prompt. It did not open a ticket, charge a card, or query a live database. Those are actions, and actions need a different contract than snippets.

Tools are named functions the model can request, with typed arguments your code executes. An agent is usually just a loop: call the model, run the tools it asked for, append the results, call again until it stops or you stop it.

The message loop from Prompts & Context still holds. Tool calls and tool results are extra roles in the same list. Most production bugs come from treating "agent" as a product feature instead of a control loop with budgets, allowlists, and failure paths.

Tools are side effects with a schema

Without tools, the completion is only tokens. With tools, the model can emit a structured request like "call get_order with {order_id: \"…\"}" and your runtime turns that into an HTTP call, a SQL query, or a webhook.

The model never executes the tool. Your process does. That split matters for security and for debugging: if a refund fires, it was your handler, not the GPU.

A tool definition is a name, a short description the model uses to decide when to call it, and a JSON Schema for arguments. Keep descriptions concrete ("Fetch an order by public ID") and keep schemas tight (enums, required fields, max lengths). Vague tools become lottery tickets for wrong calls.

How tool calling fits the chat API

Providers differ in field names (tools vs legacy functions, tool vs function roles), but the shape is stable:

1. You send messages plus a tools array of schemas.
2. The model returns either normal assistant content, or one or more tool_calls (id, name, arguments JSON).
3. Your app validates arguments, runs the handlers, and appends messages with role: "tool" (and the matching tool_call_id).
4. You call the model again with the extended history so it can answer from the results — or request another tool.

Parallel tool calls in one turn are common: look up a user and an order at once. Your runtime should tolerate that and not assume a single call per response.

tool_choice (or equivalent) is the control knob:

• auto — model decides whether to call a tool
• required / any — force at least one call
• named tool — force that specific tool
• none — forbid tools for this turn (useful after you already have results)

Treat argument JSON as untrusted input. Parse it, validate against your schema again in code, reject bad shapes before they hit production systems. The model can invent fields that were never in the schema.

A minimal tool round-trip

Below is one user question, one tool, and the second model call after you inject the result. Real SDKs hide some of the message plumbing; the wire shape is what you need to reason about failures.

package main

import (
  "bytes"
  "encoding/json"
  "fmt"
  "io"
  "net/http"
  "os"
)

func postChat(body map[string]any) map[string]any {
  raw, _ := json.Marshal(body)
  req, _ := http.NewRequest(
    "POST",
    "https://api.openai.com/v1/chat/completions",
    bytes.NewReader(raw),
  )
  req.Header.Set("Authorization", "Bearer "+os.Getenv("OPENAI_API_KEY"))
  req.Header.Set("Content-Type", "application/json")
  res, err := http.DefaultClient.Do(req)
  if err != nil {
    panic(err)
  }
  defer res.Body.Close()
  data, _ := io.ReadAll(res.Body)
  var out map[string]any
  _ = json.Unmarshal(data, &out)
  return out
}

func getOrder(orderID string) string {
  // Stand-in for your DB / billing API.
  return fmt.Sprintf(`{"order_id":%q,"status":"shipped","total_cents":4999}`, orderID)
}

func main() {
  tools := []map[string]any{
    {
      "type": "function",
      "function": map[string]any{
        "name":        "get_order",
        "description": "Fetch an order by public ID.",
        "parameters": map[string]any{
          "type": "object",
          "properties": map[string]any{
            "order_id": map[string]any{"type": "string"},
          },
          "required": []string{"order_id"},
        },
      },
    },
  }

  messages := []map[string]any{
    {
      "role":    "system",
      "content": "Use get_order when the user asks about a specific order. Do not invent order fields.",
    },
    {
      "role":    "user",
      "content": "What is the status of order ord_123?",
    },
  }

  first := postChat(map[string]any{
    "model":    "gpt-4.1-mini",
    "messages": messages,
    "tools":    tools,
  })

  // In real code: walk choices[0].message.tool_calls.
  // Here we show the follow-up shape after you ran the tool.
  callID := "call_1"
  orderID := "ord_123"
  result := getOrder(orderID)

  messages = append(messages,
    map[string]any{
      "role": "assistant",
      "tool_calls": []map[string]any{
        {
          "id":   callID,
          "type": "function",
          "function": map[string]any{
            "name":      "get_order",
            "arguments": fmt.Sprintf(`{"order_id":%q}`, orderID),
          },
        },
      },
    },
    map[string]any{
      "role":         "tool",
      "tool_call_id": callID,
      "content":      result,
    },
  )

  second := postChat(map[string]any{
    "model":       "gpt-4.1-mini",
    "messages":    messages,
    "tools":       tools,
    "tool_choice": "none",
  })
  fmt.Println(second)
}

Notice tool_choice: "none" on the second call: you already have the data; you want prose (or structured output), not another speculative call. That pattern scales better than hoping the model stops by itself.

What people mean by "agent"

Marketing uses "agent" for almost any tool-using chat bot. Internally, an agent is a policy over the loop above:

The interesting design choices are not "use LangGraph or not." They are:

• Which tools exist — every tool is attack surface and cognitive load for the model.
• Step budget — hard cap on LLM rounds (often 3–8 for product flows).
• Spend budget — max tokens or dollars per session.
• Stop rules — success schema filled, user confirmation required, or human handoff.
• Memory — what from prior steps stays in messages vs what you summarize or drop.

A single forced tool call (tool_choice set to get_order) is not an agent. A planner that can search, write files, and open PRs in a loop is. Most product features need the first shape, not the second.

When not to agent

Reach for a fixed pipeline before a free loop:

• Known sequence. If every refund is "verify order → check policy → create credit," encode that in code. Use the model to fill slots or classify edge cases, not to rediscover the workflow each time.
• RAG-only answers. Reading docs does not need tools if you already retrieve and ground in RAG. Calling search_docs from an agent just moves retrieval into a slower, less controllable path unless the query truly must be rewritten mid-loop.
• High-stakes writes. Money movement, permission changes, deletes: require explicit confirmation UI or a human step. Do not let an open loop call charge_card because the model "thought it was done."
• Latency budgets. Each extra LLM round is another prefill. Users feel three round-trips; demos hide them behind streaming theater.
• Eval vacuum. If you cannot score whether the agent finished correctly, a loop will amplify flaky prompts into flaky production.

Agents earn their keep when the next action depends on intermediate results you cannot branch on statically — ambiguous intents, multi-system lookups with unknown order, or research tasks where the stop condition is "enough evidence," not "step 3 of 3."

Failure modes that show up in the loop

• Hallucinated tool names or arguments. Validate every call against your registry. Unknown name → tool error message back to the model, or hard fail — never invent a handler.
• Infinite or long loops. Same tool with the same args twice in a row is a smell; detect and break. Cap steps in code, not in the system prompt alone.
• Poisoned tool output. A scraped page or DB row that says "ignore previous instructions" is still tokens. Prefer structured tool payloads over raw HTML; strip or summarize hostile content.
• Partial success. Two parallel tools: one fails, one returns. The model may answer as if both worked. Surface errors explicitly in the tool message ({"error":"not_found"}) and teach the system prompt to report them.
• Over-trusting the planner. The model asks for delete_user because the user said "remove them." Authorization belongs in your handler (session, tenant, RBAC), not in the tool description.
• Context bloat. Dumping full tool payloads every step blows the window. Keep raw results in your store; put compact summaries or IDs into messages when possible.
• Silent side effects. Logging "assistant said refunded" without an idempotency key on the refund API is how you double-charge. Tool handlers need the same discipline as any other write path.

Debug habit: log each step's tool names, validated args, latency, and whether you truncated results. The final user-visible sentence is the least useful artifact when the loop misbehaves.

Comparison: tool shapes

Start at the top of that table and move down only when a fixed path fails on real traffic. Adding tools is cheap; adding an unbounded loop is a product decision.

Conclusion

Tools turn the chat API into a request for side effects your code must execute safely. Agents are that request wrapped in a loop with budgets and stop rules. The same message contract still applies: schemas in, tool results back in as tokens, completion out.

Next up is Orchestration — composing prompts, retrieval, and tool steps into explicit pipelines (graph-style or otherwise) without pretending every path needs an autonomous agent.