The Core Idea: Every model call in an agent contains instructions, context and the latest input. Good agents keep instructions clear, keep context small and relevant, and demand structured outputs so code can read the model's decisions.
- Beginner: System Instructions
The system instructions are the standing rules of the agent. They are sent at the start of every model call and tell the model who it is, what it may do, and what it must never do.
| Part | Example |
|---|---|
| Role | You are a support agent for an online shop. |
| Goal | Resolve the customer's order question using the tools. |
| Rules | Never reveal internal notes. Ask before issuing a refund. |
| Tools guidance | Use lookup_order for any question about shipping. |
| Output format | Reply with JSON only: a tool call or an answer. |
Tip: Be specific and positive. Always confirm the order ID before looking it up works better than do not be careless.
- Beginner: Agent Prompts
A model call in an agent is built from several pieces that your code assembles every turn:
Remember: The model has no memory of its own between calls. Whatever is not in this assembled prompt, the model does not know.
- Medium: Context Management
The context window is limited and every token costs money and time. Agents add tool results to the history on every step, so context grows quickly. Good context management keeps what matters:
| Technique | What it does |
|---|---|
| Keep the system prompt and newest message | Never drop the essentials |
| Trim the oldest history first | Cheapest way to stay under budget |
| Summarize old steps | Keeps the gist in fewer tokens |
| Shorten tool results | Return only needed fields, not whole pages |
| Pin key facts | Facts survive even when history is trimmed |
Rule of thumb: More context is not better context. Give the model exactly what it needs for the next decision.
- Medium: Structured Outputs
Your loop must read the model's decision with code, so the reply has to follow a fixed format, usually JSON. For an agent there are two kinds of reply:
| Reply type | Shape | What your code does |
|---|---|---|
| Tool call | A tool name plus an arguments object | Validate, run the tool, continue the loop |
| Final answer | An answer field | Stop the loop and return it |
Providers offer JSON mode and schema-constrained output to make the format reliable. Even then, validate every reply: parse it, check the required keys, and handle failures by retrying or asking the model to fix its reply.
Valid shape is not correct content. A reply can have perfect JSON and still ask for the wrong tool. Check the values too.
- Advanced: Tool-Call Responses
After your code runs a tool, the result goes back to the model as a tool message linked to the call. Its design affects how well the agent continues:
| Good practice | Why |
|---|---|
| Include the tool name and a clear status | The model knows which call this answers |
| Return compact JSON, not a whole web page | Saves tokens and avoids confusing the model |
| Put errors in the result, not exceptions | The model can recover |
| Mark untrusted text clearly | Helps defend against injected instructions |
| Truncate and say so | Tell the model when data was cut off |
- Visual Explanation: What Goes Into the Prompt and How Replies Are Read
The figure shows the four pieces that make up every model call, then the reply shapes your code must handle.
Click to see why history is trimmed oldest firstCompare ▼
Part 1 shows which parts are always kept and which are trimmed. Part 2 shows the three outcomes when your code reads a reply. Open the bar to see why trimming starts with the oldest messages.
- Python Implementation: Building Context and Parsing Replies
This script assembles a prompt within a token budget (keeping the system prompt, pinned facts and newest message), then parses four model replies and builds the tool message sent back to the model.
import json def count_tokens(text): return round(len(text.split()) / 0.75) # rough rule: 1 token is about 0.75 words SYSTEM = ( "You are a support agent for a shop. Use tools for order data. " "Never reveal internal notes. Reply with JSON only.") def build_context(system, facts, history, user_message, budget): # Always keep: system instructions, key facts and the newest message. Trim old history first. fixed = [("system", system), ("system", "Facts: " + "; ".join(facts)), ("user", user_message)] used = sum(count_tokens(text) for _, text in fixed) kept = [] for role, text in reversed(history): cost = count_tokens(text) if used + cost > budget: break kept.insert(0, (role, text)) used += cost return fixed[:2] + kept + fixed[2:], used history = [("user", "hello " * 80), ("assistant", "hi there " * 60), ("user", "my order is late " * 10), ("assistant", "let me check " * 10)]facts = ["customer name is Ana", "order id is 4411"]full = sum(count_tokens(t) for _, t in history) + count_tokens(SYSTEM)context, used = build_context(SYSTEM, facts, history, "Where is my order?", budget=150)print("History alone would use", full, "tokens | trimmed context uses", used, "| messages kept:", len(context)) def parse_agent_reply(text): try: data = json.loads(text) except json.JSONDecodeError: return None, "not valid JSON" if "tool" in data and isinstance(data.get("arguments"), dict): return ("tool_call", data), "ok" if "answer" in data: return ("answer", data), "ok" return None, "needs either tool+arguments or answer" replies = [ '{"tool": "lookup_order", "arguments": {"order_id": 4411}}', '{"answer": "Your order ships tomorrow."}', 'Sure! I will look it up.', '{"tool": "lookup_order"}',]for r in replies: parsed, status = parse_agent_reply(r) print(status.ljust(38), "|", r[:50]) tool_result_message = {"role": "tool", "name": "lookup_order", "content": json.dumps({"status": "shipped", "eta_days": 1})}print("Message sent back to the model:", tool_result_message)Trimming cuts the prompt from 388 tokens of history to a short context that still contains everything essential. Only two of the four replies are usable: the others are not JSON or are missing the arguments, and your loop would retry them.
Knowledge Check
Key Points
The Big Picture
Clear rules + only the context that matters + a strict reply format = a predictable agent
With instructions, tools and context under control, you are ready to build your first complete agent.
Up Next: Time to put it all together. In Building Your First AI Agent you will write a complete agent loop that takes a goal, uses tools and finishes multi-step tasks.