Introduction

In AI Internals - How LLMs Work we stayed inside the model stack: tokens in, next-token prediction, sampling out.

This part moves one layer up. Prompts are not magic text snippets. They are a context-construction strategy: which instructions the model sees first, which facts get injected, and what output shape is legal for your app to consume.

When this layer is sloppy, you get random behavior and blame the model. When this layer is disciplined, the same model becomes much easier to control.

Prompting is context engineering

A chat request is a list of role-tagged messages. The serving stack flattens those messages with a chat template, tokenizes them, then runs prefill and decode.

So "prompt engineering" is mostly answering three concrete questions:

1. Which instructions are stable across requests?
2. Which request-specific facts should be injected?
3. How do we force an output format that downstream code can trust?

The model does not see your database, your hidden state, or your business rules unless you explicitly serialize them into that context. If a rule matters, it must be in tokens.

Roles and instruction precedence

Most chat APIs expose system, user, and assistant roles (plus tool/function messages when needed). In practice:

• system is where stable behavior belongs: voice, constraints, response contract, safety boundaries.
• user carries the live task.
• assistant in history is prior model output and can accidentally steer future turns if you keep too much of it.

Think in terms of precedence, not vibes:

1. Put hard constraints in system.
2. Put task specifics in user.
3. Keep history only when it adds necessary state.

If you spread core rules across ten user turns, they are easier to dilute, contradict, or truncate when the context window gets tight.

A practical message contract

You do not need giant prompts. You need a stable contract with clear sections and explicit output rules.

This shape works well for control tasks:

System:
- Role: what the model is acting as
- Constraints: non-negotiable rules
- Output contract: exact schema or format

User:
- Objective: what to do now
- Input data: the raw payload
- Success criteria: what "good" means for this request

Here is the same call in Go, Python, and TypeScript returning strict JSON:

package main

import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
)

func main() {
body, _ := json.Marshal(map[string]any{
  "model": "gpt-4.1-mini",
  "response_format": map[string]any{
    "type": "json_object",
  },
  "messages": []map[string]string{
    {
      "role": "system",
      "content": "You classify support tickets. Return JSON with fields: priority, category, reason.",
    },
    {
      "role": "user",
      "content": "Ticket: Checkout fails with 500 after payment authorization.",
    },
  },
})

req, _ := http.NewRequest(
  "POST",
  "https://api.openai.com/v1/chat/completions",
  bytes.NewReader(body),
)
req.Header.Set("Authorization", "Bearer "+os.Getenv("OPENAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")

res, err := http.DefaultClient.Do(req)
if err != nil {
  panic(err)
}
defer res.Body.Close()

raw, _ := io.ReadAll(res.Body)
fmt.Println(string(raw))
}

The key is not the specific wording. The key is that your system message defines output shape and your app validates the result before trusting it.

Context assembly: what to include and what to drop

Longer context is not automatically better context. Every extra token has cost, latency, and attention competition.

A useful assembly pipeline is:

1. Start with stable system rules.
2. Add only the user turn needed for this task.
3. Inject small, relevant retrieval snippets (not whole documents).
4. Include tool results only when they matter for the current answer.
5. Trim old history aggressively.

That token budget check should be explicit in code. Reserve output tokens first, then fit input tokens into the remaining budget. If you do the opposite, you get finish_reason: "length" and clipped answers.

Patterns that usually work

Three patterns consistently improve reliability:

1) Constraint-first system prompts

Lead with non-negotiables and format contract before style preferences.

You are an incident triage assistant.
Rules:
1) Use only provided evidence.
2) If evidence is missing, say "insufficient_evidence".
3) Output JSON with keys: severity, hypothesis, next_action.

2) Delimited input blocks

Wrap raw inputs in clear delimiters to reduce accidental instruction mixing.

Analyze this payload:
<ticket>
...
</ticket>

3) Few-shot examples only when behavior is ambiguous

Examples are powerful but expensive in tokens. Use one or two targeted examples for tricky formatting or classification boundaries, not a giant library of demonstrations.

Comparison: short prompts vs structured prompts

Most production systems land in the middle: structured contract plus minimal examples.

Failure modes you can debug at the prompt layer

Before swapping models, check these:

• Output is invalid JSON: contract too vague or no schema/validation in the app.
• Model ignores a rule: rule buried in user text instead of system scope.
• Responses drift over long chats: stale assistant turns polluting context.
• Hallucinated details: retrieval missing or irrelevant snippets dominating.
• Sudden truncation: no output-token reservation or poor history trimming.

Prompt quality is mostly a systems problem. Better assembly and validation beat endlessly rewriting one paragraph of instructions.

Conclusion

Prompts are a context assembly pipeline, not a sentence-writing contest. Stable system constraints, tight input selection, and explicit output contracts make behavior more predictable with the same underlying model.

Next up is Embeddings & Retrieval — how to fetch the right facts before generation, instead of stuffing entire documents into the context window.