Introduction

In AI Internals - Tools & Agents, we treated an agent as a control loop with tool calls, budgets, and stop rules. That loop is useful, but not every product flow should be a planner deciding the next move on the fly.

Orchestration is the part where you make the path explicit: classify intent, retrieve evidence, call specific tools, and synthesize an answer with known boundaries. You still use the same message contract from Prompts & Context, but now your app owns the sequence.

When people say "our agent is flaky," they are often missing orchestration, not model quality.

What orchestration actually means

At runtime, orchestration is policy + wiring:

• Policy: which step runs first, what can branch, and what must never be optional.
• Wiring: how each step reads inputs and produces outputs for the next step.

Instead of one open loop, you build a bounded pipeline with checkpoints.

This shape gives you two practical wins:

1. You can test each stage independently.
2. You can enforce guardrails before side effects happen.

Building blocks of a pipeline

Most production assistants use a small set of reusable stages:

1. Intent classification: "Q&A", "account action", "write operation", "unknown".
2. Retrieval: fetch supporting docs or records only when needed.
3. Tool execution: call allowlisted handlers with validated args.
4. Synthesis: produce user-facing text or structured output.
5. Post-processing: redact secrets, normalize tone, attach citations.

The model can appear in several of those stages, but it should not decide the existence of the stages themselves.

For example, if intent=refund_request, your code can require:

• policy check tool
• user confirmation state
• idempotency key on write tool

No prompt trick should bypass that.

A practical orchestration skeleton

Below is a minimal pipeline skeleton with fixed stages and one bounded loop for recoverable errors. The point is not framework syntax; the point is explicit control.

type State struct {
  UserInput     string
  Intent        string
  Evidence      []string
  ToolResults   map[string]string
  FinalAnswer   string
  Attempt       int
}

func RunPipeline(s *State) error {
  s.Intent = classifyIntent(s.UserInput)

  if s.Intent == "unknown" {
    s.FinalAnswer = "Can you clarify what you want to do?"
    return nil
  }

  if needsRetrieval(s.Intent) {
    s.Evidence = retrieveEvidence(s.UserInput)
  }

  if needsTools(s.Intent) {
    // Tool args must be schema-validated before execution.
    s.ToolResults = runAllowlistedTools(s.Intent, s.UserInput)
  }

  for s.Attempt = 1; s.Attempt <= 2; s.Attempt++ {
    s.FinalAnswer = synthesizeAnswer(s)
    if passesQualityGate(s.FinalAnswer) {
      return nil
    }
  }

  return fmt.Errorf("quality gate failed")
}

Notice the loop is bounded (<= 2) and only around synthesis. Retrieval and write-side tool execution stay deterministic.

Branches, fallbacks, and stop conditions

Real flows branch. The key is to branch in code on explicit signals:

• classifier label
• tool result status
• confidence threshold
• policy result

Avoid "if the model feels uncertain." Instead, make uncertainty a field you validate.

A strong fallback beats a long retry chain. Two retries plus a clean handoff usually feels better than six rounds of vague assistant text.

Orchestration vs autonomous agent loops

You can think of this as a control-surface choice:

Start from explicit orchestration and only add agent behavior where static branching fails on real traffic.

Observability that makes orchestration debuggable

Log per-stage artifacts, not just final answers:

• request_id, user_id, intent
• retrieved chunk IDs and scores
• tool names, validated args hash, latency, status
• model name, input/output tokens, cost estimate
• fallback reason

This lets you answer "why did this response happen?" without replaying blind from chat text alone.

Conclusion

Orchestration is how you turn model capability into a reliable product path: explicit stages, guarded branches, bounded retries, and measurable outcomes. Agents still matter, but they should sit inside a controlled pipeline, not replace it.

Next up is Harness — what happens when this pipeline runs inside a session that lasts hours: a durable transcript, compaction, interruption and resume, and the sandbox tools execute in.