%pip install -q "aisuite[openai,anthropic]"W1 Lab — First Model Calls and Prompting Fundamentals
Goal. By the end of this lab you can call a language model from code, explain what a single call does and does not do, and control the model’s behavior through the two prompt slots every application uses: the system prompt and the user prompt. The final task makes that control precise enough to check by machine — answers that come back as clean JSON every time.
Why this is the starting point: the lecture defined an agent as code that acts on model output. Building one therefore needs two abilities before anything else — making the call, and making its output usable by code. This lab practices exactly those two.
The path: set up the tools used all semester (Section 1) → take one call apart (Section 2) → see why “just ask for JSON” is not enough (Section 3) → the core task — a system prompt that forces clean JSON — with two short exercises (Section 4) → cost and completion check (Section 5).
Runtime: Google Colab, top-to-bottom, ~80 minutes. Cells marked ✍️ contain fill-ins.
Sources: lab format adapted from DeepLearning.AI, Agentic AI (Andrew Ng); prompting exercises draw on Anthropic’s Prompt Engineering Interactive Tutorial.
1. Setup
The aim of this section is one working model call from this notebook. That takes three pieces: the client library (1.1), your API key (1.2), and two helper functions the rest of the lab reuses (1.3). Section 1.4 confirms everything works with a test call.
1.1 Installation
aisuite exposes multiple providers (OpenAI, Anthropic) behind one interface, so lab code stays identical whichever provider your key belongs to.
Do: run the cell below; it installs the client into this Colab runtime (about 30 seconds, once per session).
1.2 API key and model
An API key = the secret string that identifies your account to the provider and bills usage to it (issuing steps: the API Setup guide on the course site). Paste it between the quotes below. The key is yours; do not share the notebook with the key still inside.
Do: replace PASTE-YOUR-KEY-HERE with your key and run the cell. Nothing prints; the key now lives in this session’s environment.
import os
os.environ["OPENAI_API_KEY"] = "PASTE-YOUR-KEY-HERE"
MODEL = "openai:gpt-4o-mini" # Anthropic accounts: MODEL = "anthropic:claude-haiku-4-5" and set ANTHROPIC_API_KEY instead1.3 Client and helpers
Two helpers wrap the client for the whole lab: chat sends a full message list and returns the reply text, ask wraps the common single-question case. Both count calls and tokens into module-level totals, read again in Section 5.
Do: run the cell unchanged. Every later cell calls these two helpers.
import aisuite
client = aisuite.Client()
n_calls = 0
n_prompt_tokens = 0
n_completion_tokens = 0
def chat(messages, temperature=0.0, **kwargs):
"""Message list -> assistant reply text. Extra kwargs pass through to the provider."""
global n_calls, n_prompt_tokens, n_completion_tokens
response = client.chat.completions.create(
model=MODEL, messages=messages, temperature=temperature, **kwargs)
n_calls += 1
usage = getattr(response, "usage", None)
if usage is not None:
n_prompt_tokens += usage.prompt_tokens
n_completion_tokens += usage.completion_tokens
return response.choices[0].message.content
def ask(prompt, system=None, temperature=0.0, **kwargs):
"""Single question (optional system instruction) -> reply text."""
messages = ([{"role": "system", "content": system}] if system else [])
messages.append({"role": "user", "content": prompt})
return chat(messages, temperature=temperature, **kwargs)1.4 Verification
Do: run the cell and confirm the output is exactly ready.
print(ask("Reply with exactly: ready"))If the output is ready, key and billing work and every later cell will run. Any error here is a setup problem, not a code problem — recheck the API Setup guide before continuing.
2. Anatomy of a Model Call
Calls work — so what is a call, exactly? This section takes one apart: what goes in (messages with roles, 2.1), what is not kept (state, 2.2), and what makes the answer vary or stay fixed (temperature, 2.3). These three facts are the ground rules for every prompt written in the rest of the course.
2.1 Messages and roles
The first experiment asks the same question twice — once with a standing instruction, once without — to see where behavior control lives in a call. A message = one turn of the exchange: a dict with a role and a content string. system carries standing instructions, user carries the request, assistant carries the model’s previous replies.
Do: run the cell and compare the two answers: what the system instruction changed (length, register) and what stayed identical (the question).
QUESTION = "What is an AI agent?"
plain = ask(QUESTION)
instructed = ask(QUESTION, system="Answer in one sentence, for a graduate ML audience.")
print("PLAIN:\n", plain)
print("\nINSTRUCTED:\n", instructed)The system instruction changed the length and register of the answer while the question stayed identical. Instructions and request travel in separate messages, which is what lets an application fix behavior once and vary only the user input. The task in Section 4 turns this observation into practice.
2.2 Statelessness
Next question: does the model remember the previous cell’s exchange? The experiment tells one call a fact, then asks the next call to repeat it. An API call is stateless = the model sees only the message list inside that one call, and the provider carries no conversation state from one call to the next.
Do: run the cell and read the second answer.
first = ask("My research topic is maritime logistics optimization. Acknowledge in five words.")
second = ask("What is my research topic?")
print("FIRST :", first)
print("SECOND:", second)The second call cannot answer, because the first call’s content never reached it. Conversation is an application-side construct: the client resends the accumulated history with every call.
conversation = [
{"role": "user", "content": "My research topic is maritime logistics optimization. "
"Acknowledge in five words."},
]
conversation.append({"role": "assistant", "content": chat(conversation)})
conversation.append({"role": "user", "content": "What is my research topic?"})
print(chat(conversation))With the history resent, the model answers. Everything a model appears to remember is text that some code placed into the message list.
Exercise — one more turn ✍️
Add a third user turn to conversation (any follow-up about the topic), resend the full history with chat(conversation), and print the reply. The pattern is the two append lines above.
### FILL IN (START) ###
# conversation.append({"role": "user", "content": "..."})
# print(chat(conversation))
### FILL IN (END) ###2.3 Temperature
The last property of a call: does the same input give the same output? Temperature = the sampling parameter that scales the spread of the next-token distribution — 0 collapses sampling toward the most probable token; higher values let lower-probability tokens through more often. The experiment repeats one open question twice at each setting.
Do: run the cell and compare within each pair — the two t=0.0 outputs against each other, then the two t=1.0 outputs against each other.
PROBE = "Name one promising research direction for LLM agents, in one sentence."
print("t=0.0 :", ask(PROBE, temperature=0.0))
print("t=0.0 :", ask(PROBE, temperature=0.0))
print("t=1.0 :", ask(PROBE, temperature=1.0))
print("t=1.0 :", ask(PROBE, temperature=1.0))The two t=0.0 runs are (near-)identical; the t=1.0 runs differ. Deterministic settings suit checking and grading; sampled diversity is itself useful and W2 builds a technique on it (self-consistency).
3. Format Control
Everything so far produced text for a human to read. An agent needs more: code must act on the output, and code can act only on what it can parse. This section tries the obvious approach — simply asking for JSON — and shows why it is not enough. Structured output = model output constrained to a machine-parseable format (here JSON). Models often wrap JSON in Markdown code fences, so the parser below strips them before json.loads.
Do: run the cell, more than once if you like. Either outcome is the observation: a parsed dict means the loose request happened to work this time; PARSE FAILED means it did not — nothing in the request guarantees either.
import json
import re
def parse_json_output(text):
"""Model output -> parsed JSON object; raises ValueError if unparseable."""
stripped = re.sub(r"^```(?:json)?\s*|\s*```$", "", text.strip())
try:
return json.loads(stripped)
except json.JSONDecodeError as exc:
raise ValueError(f"unparseable model output: {text[:120]!r}") from exc
attempt = ask("Give the title and year of the Chain-of-Thought paper as JSON.")
print(attempt)
try:
print(parse_json_output(attempt))
except ValueError as exc:
print("PARSE FAILED:", exc)Whether this parses depends on how strictly the model interpreted “as JSON” — a loose request sometimes yields prose around the object. The fix is never post-hoc string surgery alone; it is an instruction that pins the format exactly, which Section 4 makes you write.
4. Task — Answers as JSON ✍️ (core)
The final task: one system prompt, stated precisely enough that its effect passes a machine check. The goal: make the model answer any research question as clean JSON.
An example first. The cell below solves the same problem for a smaller schema — a sentiment classifier that must return {"label": ..., "confidence": ...}. Read the prompt before running it: it names every key, states the closed vocabulary for label, and forbids everything else.
Do: run the cell and read the raw reply next to its parsed form — clean JSON, exactly two keys.
EXAMPLE_SYSTEM = (
"You are a sentiment classifier. For every user text, return ONLY a raw JSON "
"object — no prose, no Markdown code fences — with exactly these keys and no "
"others:\n"
' "label": exactly one of "positive", "neutral", "negative";\n'
' "confidence": a number between 0 and 1.\n'
"Never add keys, comments, or explanations outside the JSON object."
)
EXAMPLE_USER = "Classify the sentiment of this text.\n\nText: {text}"
reply = ask(EXAMPLE_USER.format(text="The lab finally makes sense to me."),
system=EXAMPLE_SYSTEM)
print("raw reply:", reply)
print("parsed: ", parse_json_output(reply))4.1 Writing JSON_SYSTEM ✍️
Write JSON_SYSTEM so that the answer to any question comes back as raw JSON with exactly these keys:
| key | content |
|---|---|
topic |
short label for the subject area |
answer |
one-sentence direct answer |
difficulty |
one of "intro", "intermediate", "advanced" |
Follow the pattern of EXAMPLE_SYSTEM above: name every key and what belongs in it, state the closed vocabulary for difficulty, and forbid everything else — no prose around the object, no code fences, no extra keys.
Do: fill in JSON_SYSTEM, run the check cell, and iterate until both questions read PASS. Typical first failures: code fences around the JSON, or extra keys the model volunteers — both disappear once the instruction forbids them explicitly.
### FILL IN (START) ###
JSON_SYSTEM = (
""
)
### FILL IN (END) ###QUESTIONS = [
"How can I use an LLM to answer questions over our lab's PDF reports?",
"Why does asking the model to think step by step improve accuracy?",
]
REQUIRED_KEYS = {"topic", "answer", "difficulty"}
n_passed = 0
for question in QUESTIONS:
output = ask(question, system=JSON_SYSTEM)
try:
ok = set(parse_json_output(output)) == REQUIRED_KEYS
except ValueError:
ok = False
n_passed += ok
print(f"{'PASS' if ok else 'FAIL'} {question[:55]}...")
if not ok:
print(" output began:", repr(output[:90]))
print("ALL PASS" if n_passed == len(QUESTIONS) else "KEEP ITERATING")4.2 Exercise — temperature vs. format ✍️
Prediction, written down before running: at t=1.0, does the answer still parse as valid JSON with the three keys on every run?
Do: run the cell and compare with your prediction.
for run in range(3):
output = ask(QUESTIONS[0], system=JSON_SYSTEM, temperature=1.0)
try:
ok = set(parse_json_output(output)) == REQUIRED_KEYS
except ValueError:
ok = False
print(f"run {run}: {'PASS' if ok else 'FAIL'}")4.3 Exercise — provider-enforced JSON mode
OpenAI can enforce JSON at the API level: the parameter response_format={"type": "json_object"} constrains decoding so the reply always parses. It guarantees parseable JSON, not your three keys — those still come from JSON_SYSTEM — and it is OpenAI-specific (an anthropic: model rejects it). Two footnotes from the API docs: the word “JSON” must appear somewhere in the messages (a 400 error otherwise — JSON_SYSTEM already satisfies this), and a reply cut off at the token limit can still fail to parse.
Prediction: with the parameter added, can the check still fail — and on what?
Do: run the cell.
output = ask(QUESTIONS[0], system=JSON_SYSTEM,
response_format={"type": "json_object"})
print(output)
print(parse_json_output(output))5. Cost and Completion
Every call in this notebook was metered. The cell prices the whole session at gpt-4o-mini list prices.
PRICE_PER_M_PROMPT = 0.15 # USD per 1M input tokens, gpt-4o-mini
PRICE_PER_M_COMPLETION = 0.60 # USD per 1M output tokens, gpt-4o-mini
cost = (n_prompt_tokens * PRICE_PER_M_PROMPT
+ n_completion_tokens * PRICE_PER_M_COMPLETION) / 1_000_000
print(f"calls: {n_calls} prompt tokens: {n_prompt_tokens} "
f"completion tokens: {n_completion_tokens}")
print(f"estimated session cost: ${cost:.4f}")A full lab session costs on the order of a cent. Call count and token totals, not dollars, become the binding constraint once loops multiply calls (deck W12: inference economics).
Completion check
All rows must read PASS before submission; grading checks these structural facts, never prose quality.
completion = {
"verification call returned": n_calls > 0,
"JSON_SYSTEM written (>= 40 chars)": len(JSON_SYSTEM.strip()) >= 40,
"both JSON questions PASS": n_passed == len(QUESTIONS),
}
for item, ok in completion.items():
print(f"{'PASS' if ok else 'FAIL':4} {item}")
print("\nLAB COMPLETE" if all(completion.values()) else "\nNOT COMPLETE YET")W2 works on the content of the prompt rather than its format: chain-of-thought, worked-example exemplars, and sampling-based self-consistency, measured on a small code-graded evalset with the same ask/chat helpers. Reference answers for this lab: labs/checkpoints/week01/solution.py, published after the session.