{ "cells": [ { "cell_type": "markdown", "id": "37a593b1", "metadata": {}, "source": [ "# W3 Lab — From a Model Request to a Grounded Answer\n", "\n", "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ralbu85/stml_2026/blob/main/lectures/week03/W3_lab_tools.ipynb)\n", "\n", "**Goal.** Explain how a model requests information, Python retrieves it, and the model uses the returned result to request another tool or answer.\n", "\n", "A **tool** is an operation outside the model that it can request by name and arguments. Generating a request does not execute that operation.\n", "\n", "**Path:** why tools are needed → execute a request by hand → automate the same cycle → extend weather capabilities → use a search result in a forecast request → explain the roles and check the answer.\n", "\n", "| Minutes | Section | Question to resolve |\n", "|---|---|---|\n", "| 0–10 | 1. Setup and motivation | Did the model read a real clock? |\n", "| 10–25 | 2. One cycle by hand | Who requests, executes, and returns the result? |\n", "| 25–35 | 3. The same cycle with aisuite | Which steps does the library carry out? |\n", "| 35–55 | 4. Weather capabilities | One city, two cities, and tomorrow: what information is needed? |\n", "| 55–70 | 5. Search → forecast | Where does the next tool argument come from? |\n", "| 70–75 | 6. Explain the full cycle | What does the model generate, and what does Python execute? |\n", "\n", "The core takes approximately 75 minutes; the system-instruction comparison beside §4.4 is optional and adds about 5 minutes. Skip it to keep the core route to 75 minutes. This lab is not collected. The notebook runs as provided; the short writing activity asks you to improve a tool description and explain it. Setup uses your existing API key. Repeated history-display cells are folded; run them to see the actual requests, results, and answers. They use the same plain loop introduced in §3.\n", "\n", "**Lecture connection.** The lecture explains why tools are needed, how tool descriptions condition model generation, how tools are defined and executed, how results return to the model, and how Toolformer learns from augmented text. The lab makes the request–execution–result cycle observable through concrete functions. Lab section numbers organize these activities and do not reproduce lecture chapter numbers.\n", "\n", "*Sources:* DeepLearning.AI, *Agentic AI*, Module 3 supplies the functions-to-tools progression. Search, weather comparisons, and forecast extension are course adaptations. [Lecture chapter](https://stml.tailcce2b.ts.net/lectures/week03/notes.html).\n" ] }, { "cell_type": "markdown", "id": "47d0dc43", "metadata": {}, "source": [ "## 1. Setup and why tools are needed\n", "\n", "### 1.1 Installation\n", "\n", "Use Python 3.10 or later. In this workspace select **Python 3.12 (stml labs/.venv)** as the notebook kernel; the default Python 3.8 kernel cannot install this package. In Colab, use its Python runtime.\n", "\n", "Run installation, paste the API key in Section 1.2, then run the `ready` check. `ddgs` supplies the DuckDuckGo search in Section 5.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "4ccce77f", "metadata": {}, "outputs": [], "source": [ "%pip install -q \"aisuite[openai,anthropic]==0.1.14\" ddgs\n" ] }, { "cell_type": "markdown", "id": "595462a3", "metadata": {}, "source": [ "### 1.2 API key and model\n", "\n", "Paste your key into the quoted value on the right of `os.environ[\"OPENAI_API_KEY\"] = ...`. Keep `OPENAI_API_KEY` unchanged: it is the environment-variable name read by the client. Run this cell, then Section 1.3; the next output should be `ready`.\n", "\n", "For Anthropic, use `os.environ[\"ANTHROPIC_API_KEY\"] = \"YOUR-KEY\"` and `MODEL = \"anthropic:claude-haiku-4-5\"`.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "24210f44", "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "# Paste your OpenAI key on the right; keep OPENAI_API_KEY unchanged.\n", "os.environ[\"OPENAI_API_KEY\"] = \"PASTE-YOUR-KEY-HERE\"\n", "\n", "MODEL = \"openai:gpt-4o-mini\"\n", "# For Anthropic, set ANTHROPIC_API_KEY and use MODEL = \"anthropic:claude-haiku-4-5\".\n" ] }, { "cell_type": "markdown", "id": "3b806369", "metadata": {}, "source": [ "### 1.3 Client and a first call\n", "\n", "*Do:* run the cell; the output must be `ready`.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "fab6a84e", "metadata": {}, "outputs": [], "source": [ "import aisuite\n", "\n", "client = aisuite.Client()\n", "\n", "messages = [{\"role\": \"user\", \"content\": \"Reply with exactly: ready\"}]\n", "response = client.chat.completions.create(model=MODEL, messages=messages, temperature=0.0)\n", "response.choices[0].message.content # choices[0] = the first reply; .message.content = its text\n" ] }, { "cell_type": "markdown", "id": "2a37f46d", "metadata": {}, "source": [ "### 1.4 Why a model needs a clock tool\n", "\n", "No tool or timestamp is supplied in this request. A generated time therefore does not establish that the runtime clock was read.\n", "\n", "Predict whether the model will decline or produce a time, then run the cell and record the observed response.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "c83a21a8", "metadata": {}, "outputs": [], "source": [ "messages = [{\"role\": \"user\", \"content\": \"What time is it?\"}]\n", "response = client.chat.completions.create(model=MODEL, messages=messages, temperature=0.0)\n", "print(response.choices[0].message.content)\n" ] }, { "cell_type": "markdown", "id": "67ebdac5", "metadata": {}, "source": [ "## 2. A tool round trip through a system prompt\n", "\n", "**Purpose:** follow one complete tool call using ordinary Python syntax: the model writes a function-call expression, Python executes it, and the model receives the return value.\n", "\n", "`aisuite` sends ordinary model API requests in this section, without `tools=` or `max_turns=`. The system prompt describes the function and tells the model how to write the call.\n", "\n", "### 2.1 Define the function that Python will execute\n", "\n", "`get_current_time(timezone_name)` reads the computer's clock and displays the time in the supplied time zone. For example, `get_current_time(\"Asia/Seoul\")` returns the current time in Seoul. `ZoneInfo` is a Python standard-library class that interprets time-zone names; it is not another tool.\n", "\n", "The next cell defines and directly tests this function. Defining it makes it available in this Python notebook. The following system prompt will describe it to the model.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "626800a1", "metadata": {}, "outputs": [], "source": [ "from datetime import datetime\n", "from zoneinfo import ZoneInfo # Python's standard-library time-zone database interface\n", "\n", "def get_current_time(timezone_name: str = \"\"):\n", " \"\"\"Read the runtime clock and return the time as HH:MM:SS in a given time zone.\n", "\n", " Args:\n", " timezone_name: IANA time-zone name, such as \"Asia/Seoul\" or \"America/New_York\".\n", " An empty string uses the runtime computer's local time zone.\n", " \"\"\"\n", " tz = ZoneInfo(timezone_name) if timezone_name else None\n", " return datetime.now(tz).strftime(\"%H:%M:%S\")\n", "\n", "# Direct Python test: the argument controls which time zone the clock displays.\n", "print(\"Direct clock reading in Seoul:\", get_current_time(\"Asia/Seoul\"))\n" ] }, { "cell_type": "markdown", "id": "d1c897fb", "metadata": {}, "source": [ "### 2.2 Ask the model to write a Python function call\n", "\n", "The system prompt describes `get_current_time(timezone_name)` and its string argument. It asks the model to respond with one Python function-call expression.\n", "\n", "For “What time is it in Seoul?”, an expected reply is:\n", "\n", "```python\n", "get_current_time(\"Asia/Seoul\")\n", "```\n", "\n", "This reply is stored as a string in `call_text`. The model writes the function name, parentheses, and argument; Python has not executed that text yet. The API request below supplies only the system and user messages.\n", "\n", "`messages` starts with the system instruction and user question. After the response, `messages.append(...)` adds the actual model reply to that same list. It now contains three messages.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "28d1f7fd", "metadata": {}, "outputs": [], "source": [ "# Describe the existing Python function and ask for a call in ordinary Python syntax.\n", "TOOL_LIST_SYSTEM = \"\"\"Available tools:\n", "- get_current_time(timezone_name: str): reads the runtime clock and returns HH:MM:SS\n", " in the requested time zone. The argument is an IANA time-zone name.\n", " An empty string uses the runtime computer's local time zone.\n", "When the user asks for the current time, output exactly one Python call to\n", "get_current_time with the appropriate string argument. For example, for Tokyo:\n", "get_current_time(\"Asia/Tokyo\")\n", "Output the call only, without Markdown fences, JSON, or explanation.\n", "Do not invent a clock reading or a tool result.\n", "After receiving a message starting with \"Tool result:\", use that result to answer\n", "in ordinary language instead of requesting the same call again.\n", "For questions that do not need this tool, answer normally.\"\"\"\n", "\n", "messages = [\n", " {\"role\": \"system\", \"content\": TOOL_LIST_SYSTEM},\n", " {\"role\": \"user\", \"content\": \"What time is it in Seoul?\"},\n", "]\n", "\n", "# No tools= or max_turns=: the model returns the Python call as ordinary response text.\n", "response = client.chat.completions.create(model=MODEL, messages=messages, temperature=0.0)\n", "call_text = response.choices[0].message.content\n", "\n", "# Append the actual model reply to the same conversation.\n", "messages.append({\"role\": \"assistant\", \"content\": call_text})\n", "print(\"Model output (not executed):\", call_text)\n" ] }, { "cell_type": "markdown", "id": "w3-read-request-text", "metadata": {}, "source": [ "### 2.3 Inspect the Python call written by the model\n", "\n", "`call_text` comes directly from the response in §2.2. Its type is `str`, even when its content looks like Python code.\n", "\n", "For example, `get_current_time(\"Asia/Seoul\")` is still just text while stored in `call_text`. `print(call_text)` displays that text; it does not call the clock. Check that the actual reply is the requested clock call with the intended time-zone argument before moving to the execution cell. Prompt instructions do not guarantee the output format.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-read-request", "metadata": {}, "outputs": [], "source": [ "# This is the actual LLM output from cell 13; do not replace it with an example.\n", "result = None # discard any result left from an earlier attempt\n", "print(\"Python type:\", type(call_text))\n", "print(\"Call written by the LLM:\", call_text)\n", "# Printing a string does not execute the function written inside it.\n" ] }, { "cell_type": "markdown", "id": "w3-execute-request-text", "metadata": {}, "source": [ "### 2.4 Execute the model's Python call with eval\n", "\n", "`eval` is a Python built-in function. It evaluates a Python expression supplied as a string and returns its value.\n", "\n", "If `call_text` contains `get_current_time(\"Asia/Seoul\")`, then `eval(call_text)` calls the `get_current_time` function defined in cell 11 with the argument `\"Asia/Seoul\"`. Both the function name and argument are taken from the model's text. The returned clock reading is stored in `result`.\n", "\n", "`eval` runs the expression in the notebook's existing Python environment. This teaching example executes the clock-call text inspected above; `eval` itself is not a sandbox or a tool-call validator.\n", "\n", "After execution, append the returned value to `messages` as `Tool result: ...`. This program-created `user` message follows the plain-text convention in our system prompt. It is the fourth message; the function result has now joined the earlier question and model request.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-execute-request", "metadata": {}, "outputs": [], "source": [ "# eval is built into Python; get_current_time was defined in cell 11.\n", "# Execute the function call exactly as written in the LLM's response.\n", "result = eval(call_text)\n", "print(\"Executed Python expression:\", call_text)\n", "print(\"Returned clock value:\", result)\n" ] }, { "cell_type": "markdown", "id": "w3-return-result-text", "metadata": {}, "source": [ "### 2.5 Confirm the answer\n", "\n", "*Do:* run the cell. The model answers from the tool result.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-return-result", "metadata": {}, "outputs": [], "source": [ "# Print current message\n", "print(f'current messages:\\n {messages}')\n", "\n", "# Append the actual return value using the plain-text convention in the system prompt.\n", "messages.append({\"role\": \"user\", \"content\": f\"Tool result: {result}\"})\n", "print(f'appended messages:\\n {messages}')\n", "\n", "# Check the result \n", "response = client.chat.completions.create(model=MODEL, messages=messages, temperature=0.0)\n", "print(response.choices[0].message.content)\n" ] }, { "cell_type": "markdown", "id": "w3-auto-description-text", "metadata": {}, "source": [ "## 3. Automate the same cycle with aisuite\n", "\n", "The manual example needed two model requests and `eval` between them. Here we pass the already-defined function as `tools=[get_current_time]`. With `max_turns=2`, aisuite handles the model's call request, executes the function with its arguments, returns the result, and asks the model for the answer.\n", "\n", "The function's name, docstring, and parameter annotation provide its description. We do not write another function or a system prompt for this comparison. The model uses the API's structured call format.\n", "\n", "First prepare the same question about Seoul. Then run the model call and inspect the complete history. The turn limit bounds the loop; it does not force a tool request or guarantee success.\n", "\n", "\n", "**Count model calls, including the final answer.** A clock lookup normally needs\n", "two calls: (1) request the clock, then aisuite executes it; (2) read that result\n", "and answer. `max_turns=2` is an upper bound, not a request to always call twice.\n", "With `max_turns=1`, the clock can execute, but a second model call is still needed\n", "to turn its result into an answer. Week 4 compares continuing one turn at a time\n", "with letting aisuite run the whole loop.\n", "\n", "| Example | Limit | Expected model calls |\n", "|---|---|---|\n", "| Clock or one weather lookup | 2 | Tool request → answer |\n", "| Two cities | 3 | 2 if requested together; 3 if requested in separate responses |\n", "| Search, then forecast | 3 | Search request → forecast request → answer |\n", "\n", "One model response can request several tools. Two HTTP requests inside\n", "`get_weather` are also not two model calls. Check the actual history; a limit\n", "does not guarantee that the model completes the task within it.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-auto-description", "metadata": {}, "outputs": [], "source": [ "messages = [{\"role\": \"user\", \"content\": \"What time is it in Seoul?\"}]\n" ] }, { "cell_type": "markdown", "id": "w3-auto-run-text", "metadata": {}, "source": [ "Pass the function defined in cell 11 through `tools`. The following call performs the round trip that we previously wrote out ourselves.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-auto-run", "metadata": {}, "outputs": [], "source": [ "response = client.chat.completions.create(\n", " model=MODEL, messages=messages,\n", " tools=[get_current_time], # supply the function, including its argument description\n", " max_turns=2, # automate request, execution with arguments, and result return\n", " temperature=0.0)\n", "\n", "# Check the result\n", "if getattr(response.choices[0].message, \"tool_calls\", None):\n", " print(\"Turn limit reached before a final answer. Inspect the tool results below.\")\n", "else:\n", " print(response.choices[0].message.content)\n" ] }, { "cell_type": "markdown", "id": "w3-auto-trace-text", "metadata": {}, "source": [ "aisuite records the round trip as messages with two roles: `assistant` is the model's turn, either a tool request or an answer, and `tool` is the result the program returned for that request. A successful clock round trip therefore has three messages: an `assistant` request, a `tool` result, an `assistant` answer. This replaces the `Tool result:` user message and the `eval` step of Section 2.\n", "\n", "The loop below prints the actual messages in that order, with each request's function name and arguments.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-auto-trace", "metadata": { "cellView": "form", "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "import json\n", "\n", "# Check the raw json structure\n", "trace_json = json.dumps(response.choices[0].intermediate_messages, default=vars)\n", "clock_trace = json.loads(trace_json)\n", "print(clock_trace)\n", "\n", "# Check the intermediate states\n", "for number, step in enumerate(clock_trace, start=1):\n", " print(f\"\\n[{number}] {step['role']} {step.get('name', '')}\")\n", " for call in step.get(\"tool_calls\") or []: \n", " print(\" Function:\", call[\"function\"][\"name\"])\n", " print(\" Arguments:\", json.loads(call[\"function\"][\"arguments\"]))\n", " content = step.get(\"content\") or \"\"\n", " if step[\"role\"] == \"tool\":\n", " content = json.loads(content)\n", " print(content)\n" ] }, { "cell_type": "markdown", "id": "w3-weather-section", "metadata": {}, "source": [ "## 4. Extend weather capabilities\n", "\n", "**Key idea.** A tool determines what information execution can retrieve. We first reuse one tool for two cities, then add a new tool for a different kind of information.\n", "\n", "The examples progress from one city's current temperature → two cities' current temperatures → tomorrow's forecast. Each uses the request → execution → result → answer cycle from §3.\n", "\n", "### 4.1 Define and describe the current-temperature tool\n", "\n", "`get_weather(city)` returns **one city's current temperature in Celsius**. Its argument is the city's name in English. The function first obtains coordinates, then requests temperature data. `requests.get(...).json()` retrieves a service response and reads its data. [Open-Meteo documentation](https://open-meteo.com/en/docs).\n", "\n", "Read the docstring's purpose, argument, and returned information before running the definition. These are the descriptions that aisuite will supply to the model; the Python body performs the two service requests.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "8d46dffa", "metadata": {}, "outputs": [], "source": [ "import requests\n", "\n", "def get_weather(city: str):\n", " \"\"\"Get a city's current temperature in Celsius from Open-Meteo.\n", "\n", " Args:\n", " city: City name in English, for example \"Seoul\" or \"Paris\".\n", " \"\"\"\n", " geo = requests.get(\"https://geocoding-api.open-meteo.com/v1/search\",\n", " params={\"name\": city, \"count\": 1}, timeout=10).json()\n", " place = geo[\"results\"][0]\n", " weather = requests.get(\"https://api.open-meteo.com/v1/forecast\",\n", " params={\"latitude\": place[\"latitude\"], \"longitude\": place[\"longitude\"],\n", " \"current\": \"temperature_2m\"}, timeout=10).json()\n", " return f\"{place['name']}: {weather['current']['temperature_2m']} C\"\n", "\n", "# Test\n", "get_weather(\"Rome\")\n" ] }, { "cell_type": "markdown", "id": "w3-weather-single-text", "metadata": {}, "source": [ "### 4.2 Retrieve one city's temperature\n", "\n", "Prompt: **“What is the current temperature in Seoul in Celsius?”**\n", "\n", "Predict the request `get_weather(city=\"Seoul\")`. Run the request and the folded display cell. Compare the returned temperature with the final answer; the generated city name is an argument, while the temperature comes from execution.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-weather-single", "metadata": {}, "outputs": [], "source": [ "messages = [{\"role\": \"user\", \"content\": \"What is the current temperature in Seoul in Celsius?\"}]\n", "response = client.chat.completions.create(\n", " model=MODEL, messages=messages, tools=[get_weather], max_turns=2, temperature=0.0,\n", ")\n", "# Check the result\n", "if getattr(response.choices[0].message, \"tool_calls\", None):\n", " print(\"Turn limit reached before a final answer. Inspect the tool results below.\")\n", "else:\n", " print(response.choices[0].message.content)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-weather-single-trace", "metadata": { "cellView": "form", "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#@title Show the request, result, and answer - run as-is { display-mode: \"form\" }\n", "trace_json = json.dumps(response.choices[0].intermediate_messages, default=vars)\n", "current_weather_trace = json.loads(trace_json)\n", "\n", "# Check the intermediate steps\n", "for number, step in enumerate(current_weather_trace, start=1):\n", " print(f\"\\n[{number}] {step['role']} {step.get('name', '')}\")\n", " for call in step.get(\"tool_calls\") or []:\n", " print(\" Function:\", call[\"function\"][\"name\"])\n", " print(\" Arguments:\", json.loads(call[\"function\"][\"arguments\"]))\n", " content = step.get(\"content\") or \"\"\n", " if step[\"role\"] == \"tool\":\n", " content = json.loads(content)\n", " print(content)\n" ] }, { "cell_type": "markdown", "id": "w3-weather-compare-text", "metadata": {}, "source": [ "### 4.3 Compare two cities with the same function\n", "\n", "Prompt: **“Which is colder right now, Seoul or Paris? Compare their current temperatures in Celsius.”**\n", "\n", "The tool still returns one city's temperature per call. The model must request Seoul and Paris, receive both results, then compare them. Both city arguments come from the question, so neither lookup needs the other lookup's result.\n", "\n", "**Short writing activity ✍️:** explain `city` in the function's docstring with a concrete mapping, such as `City name in English, for example \"Seoul\" or \"Paris\".` If you edit it, rerun the function-definition cell before the next request. A description change supplies new model input; it does not train the model or change the function's operations.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-weather-compare", "metadata": {}, "outputs": [], "source": [ "WEATHER_COMPARISON_QUESTION = \"Which is colder right now, Seoul or Paris? Compare their current temperatures in Celsius.\"\n", "messages = [{\"role\": \"user\", \"content\": WEATHER_COMPARISON_QUESTION}]\n", "response = client.chat.completions.create(\n", " model=MODEL, messages=messages, tools=[get_weather], max_turns=3, temperature=0.0,\n", ")\n", "\n", "# Check the result\n", "if getattr(response.choices[0].message, \"tool_calls\", None):\n", " print(\"Turn limit reached before a final answer. Inspect the tool results below.\")\n", "else:\n", " print(response.choices[0].message.content)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-weather-compare-trace", "metadata": { "cellView": "form", "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#@title Show the request, result, and answer - run as-is { display-mode: \"form\" }\n", "trace_json = json.dumps(response.choices[0].intermediate_messages, default=vars)\n", "comparison_trace = json.loads(trace_json)\n", "\n", "# Check the intermediate results\n", "for number, step in enumerate(comparison_trace, start=1):\n", " print(f\"\\n[{number}] {step['role']} {step.get('name', '')}\")\n", " for call in step.get(\"tool_calls\") or []:\n", " print(\" Function:\", call[\"function\"][\"name\"])\n", " print(\" Arguments:\", json.loads(call[\"function\"][\"arguments\"]))\n", " content = step.get(\"content\") or \"\"\n", " if step[\"role\"] == \"tool\":\n", " content = json.loads(content)\n", " print(content)\n" ] }, { "cell_type": "markdown", "id": "w3-forecast-missing-text", "metadata": {}, "source": [ "### 4.4 Identify the missing capability\n", "\n", "Prompt: **“For Seoul tomorrow, give the forecast date, minimum and maximum temperature in Celsius, and maximum precipitation probability in percent.”**\n", "\n", "Only `get_weather` is available. Its current temperature cannot supply tomorrow's requested fields. Read whether the model explains that limitation or produces unsupported values. The target is to identify missing information; no particular failure is required.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-forecast-missing", "metadata": {}, "outputs": [], "source": [ "FORECAST_QUESTION = (\n", " \"For Seoul tomorrow, give the forecast date, minimum and maximum temperature \"\n", " \"in Celsius, and maximum precipitation probability in percent.\"\n", ")\n", "messages = [{\"role\": \"user\", \"content\": FORECAST_QUESTION}]\n", "response = client.chat.completions.create(\n", " model=MODEL, messages=messages, tools=[get_weather], max_turns=2, temperature=0.0,\n", ")\n", "\n", "# Check the result\n", "if getattr(response.choices[0].message, \"tool_calls\", None):\n", " print(\"Turn limit reached before a final answer. Inspect the tool results below.\")\n", "else:\n", " print(response.choices[0].message.content)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-forecast-missing-trace", "metadata": { "cellView": "form", "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#@title Show the request, result, and answer - run as-is { display-mode: \"form\" }\n", "trace_json = json.dumps(response.choices[0].intermediate_messages, default=vars)\n", "current_only_trace = json.loads(trace_json)\n", "\n", "# Check the interemdiate results\n", "for number, step in enumerate(current_only_trace, start=1):\n", " print(f\"\\n[{number}] {step['role']} {step.get('name', '')}\")\n", " for call in step.get(\"tool_calls\") or []:\n", " print(\" Function:\", call[\"function\"][\"name\"])\n", " print(\" Arguments:\", json.loads(call[\"function\"][\"arguments\"]))\n", " content = step.get(\"content\") or \"\"\n", " if step[\"role\"] == \"tool\":\n", " content = json.loads(content)\n", " print(content)\n" ] }, { "cell_type": "markdown", "id": "w3-optional-policy-text", "metadata": {}, "source": [ "### Optional comparison after 4.4 — an instruction cannot supply a missing capability\n", "\n", "**Question.** Can a clearer instruction make the current-temperature tool retrieve tomorrow's forecast?\n", "\n", "Repeat the same question with **only `get_weather`**, adding a system instruction to acknowledge missing information. Compare the answer with §4.4: does it distinguish current temperature from tomorrow's forecast? Either run may already acknowledge the limitation.\n", "\n", "**Conclusion.** An instruction guides how the model uses information and explains a limitation. Obtaining tomorrow's forecast requires a tool that retrieves forecast data.\n", "\n", "This comparison adds about 5 minutes. Run the two cells as provided, or skip directly to §4.5; later sections do not depend on these cells.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-optional-policy", "metadata": {}, "outputs": [], "source": [ "WEATHER_SYSTEM = \"\"\"Use returned weather data for the requested city, date, and fields.\n", "Do not treat current temperature as tomorrow's forecast.\n", "If the available tools cannot retrieve the requested information, state what is missing.\"\"\"\n", "messages = [\n", " {\"role\": \"system\", \"content\": WEATHER_SYSTEM},\n", " {\"role\": \"user\", \"content\": FORECAST_QUESTION},\n", "]\n", "response = client.chat.completions.create(\n", " model=MODEL, messages=messages, tools=[get_weather], max_turns=2, temperature=0.0,\n", ")\n", "# Check the result\n", "if getattr(response.choices[0].message, \"tool_calls\", None):\n", " print(\"Turn limit reached before a final answer. Inspect the tool results below.\")\n", "else:\n", " print(response.choices[0].message.content)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-optional-policy-trace", "metadata": { "cellView": "form", "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#@title Show the request, result, and answer - run as-is { display-mode: \"form\" }\n", "trace_json = json.dumps(response.choices[0].intermediate_messages, default=vars)\n", "policy_weather_trace = json.loads(trace_json)\n", "for number, step in enumerate(policy_weather_trace, start=1):\n", " print(f\"\\n[{number}] {step['role']} {step.get('name', '')}\")\n", " for call in step.get(\"tool_calls\") or []:\n", " print(\" Function:\", call[\"function\"][\"name\"])\n", " print(\" Arguments:\", json.loads(call[\"function\"][\"arguments\"]))\n", " content = step.get(\"content\") or \"\"\n", " if step[\"role\"] == \"tool\":\n", " content = json.loads(content)\n", " print(content)\n" ] }, { "cell_type": "markdown", "id": "w3-forecast-definition-text", "metadata": {}, "source": [ "### 4.5 Supply the missing capability with a forecast tool\n", "\n", "**Keep the question; change what the program can retrieve.** `get_forecast(city)` returns tomorrow's local date, minimum/maximum temperature in Celsius, and maximum precipitation probability. The earlier `get_weather(city)` returns current temperature.\n", "\n", "Run the provided function below, then repeat the **same question** with both tools available. Read which function the model requests and which returned values it uses in the answer.\n", "\n", "The function's service code is folded. It reuses the city lookup and reads the second daily entry: today is index `0`, tomorrow is index `1` in the city's local time. Maximum precipitation probability means the maximum of the day's hourly probabilities. These are forecast values. [Open-Meteo documentation](https://open-meteo.com/en/docs).\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-forecast-tool", "metadata": { "cellView": "form", "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#@title Forecast tool - provided implementation; run as-is { display-mode: \"form\" }\n", "def get_forecast(city: str):\n", " \"\"\"Get tomorrow's forecast for a city: date, minimum/maximum temperature in\n", " Celsius, and maximum precipitation probability in percent.\n", " Tomorrow is determined in the city's local time zone. Use this for tomorrow's\n", " forecast, not for the current temperature.\n", "\n", " Args:\n", " city: Short city or town name in English, for example \"Seoul\" or \"Paris\".\n", " \"\"\"\n", " # Resolve the city to coordinates, as in get_weather from Section 4.\n", " geo = requests.get(\n", " \"https://geocoding-api.open-meteo.com/v1/search\",\n", " params={\"name\": city, \"count\": 1}, timeout=10,\n", " ).json()\n", " place = geo[\"results\"][0]\n", "\n", " # Daily arrays contain today at index 0 and tomorrow at index 1.\n", " weather = requests.get(\n", " \"https://api.open-meteo.com/v1/forecast\",\n", " params={\n", " \"latitude\": place[\"latitude\"], \"longitude\": place[\"longitude\"],\n", " \"daily\": \"temperature_2m_min,temperature_2m_max,precipitation_probability_max\",\n", " \"temperature_unit\": \"celsius\", \"timezone\": \"auto\", \"forecast_days\": 2,\n", " }, timeout=10,\n", " ).json()\n", " daily = weather[\"daily\"]\n", " return {\n", " \"city\": place[\"name\"],\n", " \"region\": place[\"admin1\"],\n", " \"country\": place[\"country\"],\n", " \"date\": daily[\"time\"][1],\n", " \"timezone\": weather[\"timezone\"],\n", " \"minimum_temperature_c\": daily[\"temperature_2m_min\"][1],\n", " \"maximum_temperature_c\": daily[\"temperature_2m_max\"][1],\n", " \"maximum_precipitation_probability_percent\": daily[\"precipitation_probability_max\"][1],\n", " }\n", "\n", "get_forecast('Paris')\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-forecast-run", "metadata": {}, "outputs": [], "source": [ "messages = [{\"role\": \"user\", \"content\": FORECAST_QUESTION}]\n", "\n", "# Check the question\n", "print(FORECAST_QUESTION)\n", "\n", "response = client.chat.completions.create(\n", " model=MODEL, messages=messages, tools=[get_weather, get_forecast],\n", " max_turns=2, temperature=0.0,\n", ")\n", "# Check the result\n", "if getattr(response.choices[0].message, \"tool_calls\", None):\n", " print(\"Turn limit reached before a final answer. Inspect the tool results below.\")\n", "else:\n", " print(response.choices[0].message.content)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-forecast-trace", "metadata": { "cellView": "form", "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#@title Show the request, result, and answer - run as-is { display-mode: \"form\" }\n", "trace_json = json.dumps(response.choices[0].intermediate_messages, default=vars)\n", "forecast_trace = json.loads(trace_json)\n", "for number, step in enumerate(forecast_trace, start=1):\n", " print(f\"\\n[{number}] {step['role']} {step.get('name', '')}\")\n", " for call in step.get(\"tool_calls\") or []:\n", " print(\" Function:\", call[\"function\"][\"name\"])\n", " print(\" Arguments:\", json.loads(call[\"function\"][\"arguments\"]))\n", " content = step.get(\"content\") or \"\"\n", " if step[\"role\"] == \"tool\":\n", " content = json.loads(content)\n", " print(content)\n" ] }, { "cell_type": "markdown", "id": "w3-weather-conclusion", "metadata": {}, "source": [ "**Section 4 conclusion.** Two calls to `get_weather` supply two current temperatures. Adding `get_forecast` supplies a different kind of information: tomorrow's forecast. Instructions guide the model's use of these capabilities; the functions determine what execution can retrieve.\n", "\n", "So far, the question has supplied the city name. **Next, we use a place name and obtain the forecast's city argument from a search result.**\n" ] }, { "cell_type": "markdown", "id": "w3-search-section", "metadata": {}, "source": [ "## 5. Use one tool's result to construct the next request\n", "\n", "**Key idea.** A tool result becomes input to the next model call. The model can use it to generate another tool request before answering.\n", "\n", "In §4, “Compare Seoul and Paris” supplied both city arguments. Each lookup could be requested without reading the other city's result.\n", "\n", "Now consider: **“I plan to visit Gyeongbokgung Palace tomorrow. Find its municipality and give the regional forecast.”** The question names the destination. In this exercise, we use search to obtain the municipality needed by `get_forecast(city)`.\n", "\n", "The dependency is: **search for the destination → read its municipality → request that municipality's forecast → answer.** The forecast is for the municipality, rather than a measurement at the palace.\n", "\n", "### 5.1 Add the tool that retrieves the missing location\n", "\n", "`web_search(query)` takes search keywords and returns titles, source links, and short text excerpts. `get_forecast(city)` is the same function used in §4. Run the provided search definition below; its request and return are the relevant interface. [DDGS documentation](https://github.com/deedy5/ddgs#1-text).\n" ] }, { "cell_type": "code", "execution_count": null, "id": "4663496b", "metadata": { "cellView": "form", "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#@title Search tool - provided implementation; run as-is { display-mode: \"form\" }\n", "from ddgs import DDGS\n", "\n", "def web_search(query: str):\n", " \"\"\"Search DuckDuckGo for current information or source links.\n", "\n", " Args:\n", " query: Search keywords, for example \"Gyeongbokgung Palace address\".\n", " \"\"\"\n", " results = DDGS(timeout=10).text(query, backend=\"duckduckgo\", max_results=3)\n", " summaries = []\n", " for item in results:\n", " summaries.append(f\"{item['title']}\\nURL: {item['href']}\\n{item['body']}\")\n", " return \"\\n\\n\".join(summaries)\n", "\n", "web_search(\"What is Sparta?\")\n" ] }, { "cell_type": "markdown", "id": "w3-search-forecast-text", "metadata": {}, "source": [ "### 5.2 Ask about a destination\n", "\n", "The prompt gives a destination rather than a `city` argument. Predict whether the\n", "model searches for the location or uses a city it already knows. We supply both\n", "tools; the model chooses which to request.\n", "\n", "If it searches first, check whether a later forecast argument uses the returned\n", "municipality. If it requests the forecast directly, record that actual route;\n", "it does not demonstrate a search-result dependency. To inspect that dependency,\n", "try the commented prompt that explicitly asks for a source before the forecast.\n", "`max_turns=3` allows search request → forecast request → final answer.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-visit-question", "metadata": {}, "outputs": [], "source": [ "# VISIT_QUESTION = \"\"\"I plan to visit Gyeongbokgung Palace tomorrow.\n", "# First search for its municipality and cite a source that supports that location.\n", "# After reading the search result, get tomorrow's forecast for that municipality.\n", "# Use the municipality's short English name for the forecast request.\n", "# Report the municipality, forecast date, minimum and maximum temperature in Celsius,\n", "# and maximum precipitation probability in percent.\"\"\"\n", "\n", "VISIT_QUESTION = \"\"\"I plan to visit Gyeongbokgung Palace tomorrow.\n", "What would be the weather tommorow?\"\"\"\n", "\n", "messages = [{\"role\": \"user\", \"content\": VISIT_QUESTION}]\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-search-forecast", "metadata": {}, "outputs": [], "source": [ "# The model generates tool names and arguments from the prompt and returned results.\n", "# aisuite executes the requested Python functions and sends their results back.\n", "response = client.chat.completions.create(\n", " model=MODEL,\n", " messages=messages,\n", " tools=[web_search, get_forecast],\n", " max_turns=3,\n", " temperature=0.0,\n", ")\n", "\n", "# Check the result\n", "if getattr(response.choices[0].message, \"tool_calls\", None):\n", " print(\"Turn limit reached before a final answer. Inspect the tool results below.\")\n", "else:\n", " print(response.choices[0].message.content)\n" ] }, { "cell_type": "markdown", "id": "w3-search-read-sequence", "metadata": {}, "source": [ "### 5.3 Follow the actual request, result, and next request\n", "\n", "Run the display cell and read the messages in order. `assistant` contains what the model generated; `tool` contains a Python function's returned result.\n", "\n", "Look for **the municipality in the search result and the same municipality in the later forecast request**. For example, a result locating the palace in Seoul can support the next request `get_forecast(city=\"Seoul\")`. This is an illustrative mapping; the output below is the actual run.\n", "\n", "The library adds each tool result to the conversation before calling the model again. Consequently, the forecast request can use information that was absent from the initial question.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "w3-search-forecast-trace", "metadata": { "cellView": "form", "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#@title Read the actual sequence - run as-is { display-mode: \"form\" }\n", "trace_json = json.dumps(response.choices[0].intermediate_messages, default=vars)\n", "search_forecast_trace = json.loads(trace_json)\n", "\n", "for number, step in enumerate(search_forecast_trace, start=1):\n", " print(f\"\\n[{number}] {step['role']} {step.get('name', '')}\")\n", " # A model request contains the function name and its generated arguments.\n", " for call in step.get(\"tool_calls\") or []:\n", " print(\" Function:\", call[\"function\"][\"name\"])\n", " for argument, value in json.loads(call[\"function\"][\"arguments\"]).items():\n", " print(f\" {argument}: {value}\")\n", " # Returned text and forecast fields are shown without the API envelope.\n", " content = step.get(\"content\") or \"\"\n", " if step[\"role\"] == \"tool\":\n", " content = json.loads(content)\n", " if isinstance(content, dict):\n", " for field, value in content.items():\n", " print(f\" {field}: {value}\")\n", " else:\n", " print(content)\n" ] }, { "cell_type": "markdown", "id": "w3-search-conclusion", "metadata": {}, "source": [ "**Section 5 conclusion.** In the city comparison, both arguments came from the question. Here, a later argument comes from an earlier tool result. Receiving that result and calling the model again makes this dependent sequence possible.\n", "\n", "The model generates the search request, reads the returned location, and generates the forecast request. Python executes each requested function. After the forecast returns, the model has the information needed to answer.\n" ] }, { "cell_type": "markdown", "id": "w3-final-review", "metadata": {}, "source": [ "## 6. Explain who requests, who executes, and what supports the answer\n", "\n", "**Key idea.** Tool use connects model generation to program execution through a conversation that accumulates requests and results.\n", "\n", "### 6.1 The same cycle supports both examples\n", "\n", "1. **The model receives the task, tool descriptions, and conversation so far.** It generates a function name and arguments when requesting a tool.\n", "2. **The Python program executes the requested function.** The function returns information obtained during execution.\n", "3. **The program adds the result to the conversation and calls the model again.** The model can request another tool or produce the final answer.\n", "\n", "In the clock example, the returned time supported the final answer. In the visit example, the returned location supported another request, and the returned forecast then supported the final answer. The same cycle handles both.\n" ] }, { "cell_type": "markdown", "id": "w3-answer-evidence", "metadata": {}, "source": [ "### 6.2 A completed tool sequence still needs an accurate answer\n", "\n", "**Successful execution does not by itself establish answer correctness.** For the visit example, follow three connections in the actual output:\n", "\n", "- **Location → argument:** the searched municipality matches the forecast argument and the returned region/country.\n", "- **Result → answer:** the forecast date, temperatures, and probability in the answer match the returned values.\n", "- **Source → claim:** the cited search result supports the stated location. Weather values come from the forecast tool.\n", "\n", "**Explain the whole example in your own words:** Where did the forecast's city argument come from, who executed the forecast function, and which returned information supports the final answer?\n", "\n", "The lab is not collected and has no automatic score. The homework is `W3_hw_new_tool.ipynb`, due before Week 4.\n" ] }, { "cell_type": "markdown", "id": "w3-week4-transition", "metadata": {}, "source": [ "**Next week — the same loop, two ways.** Week 4 uses a research task to\n", "compare repeated `max_turns=1` calls with one aisuite call using `max_turns=N`.\n", "Both execute tools and pass observations to the model. We will inspect who\n", "advances the loop, actual model/tool counts, and how evidence supports the report,\n", "then connect the observed action–observation cycle to ReAct.\n" ] } ], "metadata": { "colab": { "provenance": [ { "file_id": "https://github.com/ralbu85/stml_2026/blob/main/lectures/week03/W3_lab_tools.ipynb" } ] }, "kernelspec": { "display_name": "Python 3.12 (stml labs/.venv)", "language": "python", "name": "stml" }, "language_info": { "name": "python", "version": "3.12.12" } }, "nbformat": 4, "nbformat_minor": 5 }