{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c00",
   "metadata": {},
   "source": [
    "# W5 Homework — Reflection on a Rendered Chart\n",
    "\n",
    "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ralbu85/stml_2026/blob/main/lectures/week05/W5_hw_chart_reflection.ipynb)\n",
    "\n",
    "**Goal.** Run the reflection loop on code whose result is a picture. A coder object writes matplotlib code; the code runs and saves a chart; a critic object looks at the chart image and returns feedback with revised code; a judge object scores both versions.\n",
    "\n",
    "This is the lab's loop with one difference: the problems to find are visible only in the executed result, so the critic receives the rendered chart itself as an image. `Agent.ask` gets one more argument for this, `image=`.\n",
    "\n",
    "| Section | What happens |\n",
    "|---|---|\n",
    "| 1. Setup | the course classes; `ask(..., image=...)` is new |\n",
    "| 2. Data | the coffee sales log |\n",
    "| 3. Reflection, one step per cell | coder → run → critic (✍️ `REFLECTION_PROMPT`) → run |\n",
    "| 4. The pipeline as one function | `run_workflow` |\n",
    "| 5. The score | a judge scores V1 and V2; target V2 ≥ 4; two exercises |\n",
    "| 6. Completion check | every row PASS, then submit |\n",
    "\n",
    "**Cell labels.** `# ✍️ TYPE` = write the code between the `FILL IN` lines yourself; `# ✍️ WRITE` = answer in words; `# ▶ RUN · example` = read, then run; `# ▶ RUN · setup` / `check` = just run.\n",
    "\n",
    "*Runtime:* Google Colab, top to bottom, about 60 minutes. Submit on the LMS (Section 6). **Due date: see the LMS.**\n",
    "\n",
    "*Sources:* adapted from DeepLearning.AI, *Agentic AI* (Andrew Ng), Module 2 (Reflection), ungraded labs 1–2. The workflow and step order follow the source; the code uses the course `Agent` class.\n",
    "\n",
    "A rendered image supplies evidence about appearance. It does not verify the data, and it does not guarantee that a revision improves the chart."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c01",
   "metadata": {},
   "source": [
    "## 1. Setup\n",
    "\n",
    "*Do:* run the four cells; paste your API key into the second. The last cell defines `show_messages`, `show_image`, and the `Agent` class from the lab, with one addition: `ask(question, image=\"chart_v1.png\")` puts the PNG file into the user message next to the text, so the model sees the picture. `Agent(system, json_mode=True)` and `temperature=0` work as in the lab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c02",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · setup: just run\n",
    "%pip install -q \"openai==1.109.1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c03",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · setup: just run\n",
    "import os\n",
    "\n",
    "OPENAI_API_KEY = \"PASTE-YOUR-KEY-HERE\"   # paste your key between the quotes\n",
    "if OPENAI_API_KEY != \"PASTE-YOUR-KEY-HERE\":\n",
    "    os.environ[\"OPENAI_API_KEY\"] = OPENAI_API_KEY   # the client reads the key from here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c04",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · setup: just run\n",
    "import base64\n",
    "import json\n",
    "from openai import OpenAI\n",
    "from IPython.display import Image, display\n",
    "\n",
    "client = OpenAI()\n",
    "MODEL = \"gpt-4.1\"\n",
    "messages = [{\"role\": \"user\", \"content\": \"Reply with exactly: ready\"}]\n",
    "response = client.chat.completions.create(model=MODEL, messages=messages)\n",
    "reply = response.choices[0].message.model_dump(exclude_none=True)\n",
    "messages.append(reply)\n",
    "reply[\"content\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c05",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · setup: just run\n",
    "# The course helpers and class, as in the practice notebook (v2) and the lab (json_mode, temperature=0), with one addition for this homework: image.\n",
    "import inspect\n",
    "\n",
    "PYTHON_TO_JSON = {str: \"string\", int: \"integer\", float: \"number\", bool: \"boolean\"}\n",
    "\n",
    "def tool_from_function(function):\n",
    "    \"\"\"Build a tool schema from a function's name, docstring, and type annotations.\"\"\"\n",
    "    doc = inspect.getdoc(function)\n",
    "    arg_help = {}\n",
    "    for line in doc.split(\"\\n\"):\n",
    "        line = line.strip()\n",
    "        if \":\" in line and not line.endswith(\":\"):\n",
    "            name, text = line.split(\":\", 1)\n",
    "            arg_help[name.strip()] = text.strip()\n",
    "    properties = {}\n",
    "    for name, parameter in inspect.signature(function).parameters.items():\n",
    "        properties[name] = {\"type\": PYTHON_TO_JSON[parameter.annotation], \"description\": arg_help[name]}\n",
    "    return {\"type\": \"function\",\n",
    "            \"function\": {\"name\": function.__name__, \"description\": doc.split(\"\\n\")[0],\n",
    "                         \"parameters\": {\"type\": \"object\", \"properties\": properties, \"required\": list(properties)}}}\n",
    "\n",
    "\n",
    "def show_messages(messages):\n",
    "    \"\"\"Print each message: position, role, and content.\"\"\"\n",
    "    for i in range(len(messages)):\n",
    "        m = messages[i]\n",
    "        if \"tool_calls\" in m:                          # a request: print each requested function\n",
    "            for request in m[\"tool_calls\"]:\n",
    "                print(i, m[\"role\"], request[\"function\"])\n",
    "        elif type(m[\"content\"]) == list:               # NEW in this homework: text and an image\n",
    "            print(i, m[\"role\"], m[\"content\"][0][\"text\"], \"[image]\")\n",
    "        else:\n",
    "            print(i, m[\"role\"], m[\"content\"])\n",
    "    print()\n",
    "\n",
    "\n",
    "def show_image(path):\n",
    "    \"\"\"Display a saved PNG file.\"\"\"\n",
    "    display(Image(filename=path))\n",
    "\n",
    "\n",
    "class Agent:\n",
    "    \"\"\"A model with its own message list and, optionally, tools.\"\"\"\n",
    "\n",
    "    def __init__(self, system, tool_functions=(), json_mode=False):\n",
    "        self.messages = [{\"role\": \"system\", \"content\": system}]              # SYSTEM message, once\n",
    "        self.tools = []                                                      # TOOL SCHEMAS\n",
    "        self.functions = {}                                                  # name -> Python function\n",
    "        for function in tool_functions:                                      # () = no tools\n",
    "            self.tools.append(tool_from_function(function))\n",
    "            self.functions[function.__name__] = function\n",
    "        self.response_format = {\"type\": \"text\"}\n",
    "        if json_mode:                                                        # from the lab: every reply is valid JSON\n",
    "            self.response_format = {\"type\": \"json_object\"}\n",
    "\n",
    "    def call_model(self):\n",
    "        \"\"\"Call the model with the list; keep the reply and return it.\"\"\"\n",
    "        if len(self.tools) > 0:\n",
    "            response = client.chat.completions.create(model=MODEL, messages=self.messages, tools=self.tools,\n",
    "                                                      response_format=self.response_format, temperature=0)\n",
    "        else:                                                                # no tools: the API rejects an empty list\n",
    "            response = client.chat.completions.create(model=MODEL, messages=self.messages,\n",
    "                                                      response_format=self.response_format, temperature=0)\n",
    "        reply = response.choices[0].message.model_dump(exclude_none=True)\n",
    "        self.messages.append(reply)                                          # ASSISTANT message\n",
    "        return reply\n",
    "\n",
    "    def run_tools(self, reply):\n",
    "        \"\"\"Run each request in the reply and add each result as a tool message.\"\"\"\n",
    "        for request in reply[\"tool_calls\"]:\n",
    "            name = request[\"function\"][\"name\"]\n",
    "            try:\n",
    "                args = json.loads(request[\"function\"][\"arguments\"])\n",
    "                result = self.functions[name](**args)\n",
    "            except Exception as error:                                       # unknown name or wrong arguments\n",
    "                result = \"error: \" + str(error)                              # the error becomes the result\n",
    "            self.messages.append({\"role\": \"tool\", \"tool_call_id\": request[\"id\"], \"content\": json.dumps(result)})   # TOOL message\n",
    "\n",
    "    def ask(self, question, image=None, max_steps=6):\n",
    "        \"\"\"Add the question (and an image, if given), then loop until the model answers in text.\"\"\"\n",
    "        if image is None:\n",
    "            content = question\n",
    "        else:                                                                # NEW in this homework: attach a PNG file\n",
    "            with open(image, \"rb\") as f:\n",
    "                image_text = base64.b64encode(f.read()).decode()             # the file's bytes as text\n",
    "            content = [\n",
    "                {\"type\": \"text\", \"text\": question},\n",
    "                {\"type\": \"image_url\", \"image_url\": {\"url\": \"data:image/png;base64,\" + image_text}},\n",
    "            ]\n",
    "        self.messages.append({\"role\": \"user\", \"content\": content})           # USER message\n",
    "        for step in range(max_steps):\n",
    "            reply = self.call_model()\n",
    "            if \"tool_calls\" in reply:                                        # requests: run the tools\n",
    "                self.run_tools(reply)\n",
    "            else:                                                            # a text answer: done\n",
    "                return reply[\"content\"]\n",
    "        print(\"Stopped: no text answer after\", max_steps, \"steps\")\n",
    "        return None                                                          # the step limit ended the loop\n",
    "\n",
    "    def show(self):\n",
    "        \"\"\"Print this object's messages.\"\"\"\n",
    "        show_messages(self.messages)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c06",
   "metadata": {},
   "source": [
    "## 2. Data\n",
    "\n",
    "The dataset is a coffee vending machine's sales log, one row per purchase: `date`, `time`, payment fields, `price`, `coffee_name`. The cell downloads it from the course repository and adds `year`, `quarter`, and `month` columns. If the download fails, upload `coffee_sales.csv` through the Colab file pane.\n",
    "\n",
    "The model never sees these rows. It sees only the column list (Section 3) and writes code that runs on them.\n",
    "\n",
    "*Do:* run the cell and look at the five sample rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c07",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · setup: just run\n",
    "import urllib.request\n",
    "import pandas as pd\n",
    "\n",
    "CSV_PATH = \"coffee_sales.csv\"\n",
    "if not os.path.exists(CSV_PATH):\n",
    "    urllib.request.urlretrieve(\"https://raw.githubusercontent.com/ralbu85/stml_2026/main/labs/data/coffee_sales.csv\", CSV_PATH)\n",
    "\n",
    "\n",
    "def load_and_prepare_data(csv_path):\n",
    "    \"\"\"CSV path -> DataFrame with year, quarter, and month columns added.\"\"\"\n",
    "    frame = pd.read_csv(csv_path)\n",
    "    frame[\"date\"] = pd.to_datetime(frame[\"date\"], errors=\"coerce\")\n",
    "    frame[\"quarter\"] = frame[\"date\"].dt.quarter\n",
    "    frame[\"month\"] = frame[\"date\"].dt.month\n",
    "    frame[\"year\"] = frame[\"date\"].dt.year\n",
    "    return frame\n",
    "\n",
    "\n",
    "df = load_and_prepare_data(CSV_PATH)\n",
    "display(df.sample(n=5))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c08",
   "metadata": {},
   "source": [
    "## 3. Reflection, One Step per Cell\n",
    "\n",
    "### 3.1 The coder writes V1\n",
    "\n",
    "The coder's system message holds the rules for the code (use `df`, matplotlib, save to the named file, close the figure), the chart requirements, and the column list. The reply comes in JSON mode: the code arrives as the text in the `\"code\"` field, and `json.loads` hands it over.\n",
    "\n",
    "*Do:* run the two cells. The list shows the request as a USER message and the JSON reply as an ASSISTANT message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c09",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "SCHEMA_TEXT = \"\"\"Columns available in the DataFrame df:\n",
    "- date (datetime)\n",
    "- time (HH:MM)\n",
    "- cash_type (card or cash)\n",
    "- card (string)\n",
    "- price (number)\n",
    "- coffee_name (string)\n",
    "- quarter (1-4)\n",
    "- month (1-12)\n",
    "- year (YYYY)\"\"\"\n",
    "\n",
    "CODER_SYSTEM = f\"\"\"You are a data visualization expert. You write matplotlib code for a pandas DataFrame named df.\n",
    "Return JSON with one field, code: the Python code.\n",
    "\n",
    "Rules for the code:\n",
    "1. Assume the DataFrame is already loaded as df. Do not read any file.\n",
    "2. Use matplotlib for plotting (no seaborn).\n",
    "3. Save the figure to the file name given in the request, with dpi=100.\n",
    "4. Do not call plt.show(). Close the figure with plt.close().\n",
    "5. Include all necessary import statements.\n",
    "\n",
    "Chart requirements:\n",
    "- Add a clear, descriptive title.\n",
    "- Label both axes.\n",
    "- Add a legend when more than one series is plotted.\n",
    "- Aggregate the data exactly as the request asks before plotting.\n",
    "\n",
    "{SCHEMA_TEXT}\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c10",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "## Unit 1 - the coder writes the V1 code\n",
    "INSTRUCTION = \"Create a plot comparing Q1 coffee sales in 2024 and 2025 using the data in coffee_sales.csv.\"\n",
    "\n",
    "coder = Agent(CODER_SYSTEM, json_mode=True)\n",
    "answer_v1 = coder.ask(f\"\"\"{INSTRUCTION}\n",
    "Save the figure as chart_v1.png.\"\"\")\n",
    "coder.show()\n",
    "code_v1 = json.loads(answer_v1)[\"code\"]                    # JSON text -> dictionary -> the code text\n",
    "print(code_v1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c11",
   "metadata": {},
   "source": [
    "### 3.2 Run V1\n",
    "\n",
    "`exec(text, {\"df\": df})` runs a text as Python code, with our DataFrame available under the name `df`. This runs model-written code without a sandbox; it is acceptable here only because Colab gives you a disposable machine (Ch. 14 returns to this risk).\n",
    "\n",
    "*Do:* run the cell; the V1 chart appears."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c12",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "## Unit 2 - run the code; it saves chart_v1.png\n",
    "exec(code_v1, {\"df\": df})\n",
    "show_image(\"chart_v1.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c13",
   "metadata": {},
   "source": [
    "A first draft may miss a title, use the wrong data slice, or have a visual defect. The review step checks the result against the request.\n",
    "\n",
    "### 3.3 ✍️ The critic looks at the chart\n",
    "\n",
    "The critic receives three things in one user message: the request, the V1 code, and the rendered chart as an image. Its reply format is fixed, JSON with `\"feedback\"` and `\"code\"`, because the next step runs the revised code.\n",
    "\n",
    "Write `REFLECTION_PROMPT`, the first part of the critic's system message: what the critic checks in the chart, and that the revised code fixes the problems it finds.\n",
    "\n",
    "*Do:* write `REFLECTION_PROMPT`, then run the next two cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c14",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ✍️ TYPE · exercise: write the code between the FILL IN lines yourself\n",
    "### FILL IN (START) ###\n",
    "REFLECTION_PROMPT = \"\"\n",
    "### FILL IN (END) ###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c15",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "CRITIC_SYSTEM = f\"\"\"{REFLECTION_PROMPT}\n",
    "\n",
    "You receive a chart request, the code that drew the chart, and the rendered chart as an image.\n",
    "Return JSON with two fields: feedback (the problems you found) and code (the revised Python code).\n",
    "\n",
    "Rules for the revised code:\n",
    "- Assume df already exists. Do not read any file.\n",
    "- Use pandas and matplotlib only (no seaborn).\n",
    "- Save the figure to the file name given in the request, with dpi=100.\n",
    "- Do not call plt.show(). Close the figure with plt.close().\n",
    "- Include all necessary import statements.\n",
    "\n",
    "{SCHEMA_TEXT}\"\"\"\n",
    "\n",
    "\n",
    "def review_request(instruction, code, out_path):\n",
    "    \"\"\"The user message for the critic: the request, the code, and where to save the revision.\"\"\"\n",
    "    return f\"\"\"Chart request: {instruction}\n",
    "\n",
    "Code that drew the chart:\n",
    "{code}\n",
    "\n",
    "Save the revised figure as {out_path}.\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c16",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "## Unit 3 - the critic reviews V1; the chart travels as an image in the USER message\n",
    "critic = Agent(CRITIC_SYSTEM, json_mode=True)\n",
    "answer_v2 = critic.ask(review_request(INSTRUCTION, code_v1, \"chart_v2.png\"), image=\"chart_v1.png\")\n",
    "critic.show()\n",
    "review_v2 = json.loads(answer_v2)\n",
    "print(\"FEEDBACK:\", review_v2[\"feedback\"])\n",
    "code_v2 = review_v2[\"code\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c17",
   "metadata": {},
   "source": [
    "### 3.4 Run V2 and compare\n",
    "\n",
    "*Do:* run the cell and compare the two charts with the request and the feedback."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c18",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "## Unit 4 - run the revised code; it saves chart_v2.png\n",
    "exec(code_v2, {\"df\": df})\n",
    "print(\"V1:\")\n",
    "show_image(\"chart_v1.png\")\n",
    "print(\"V2:\")\n",
    "show_image(\"chart_v2.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c19",
   "metadata": {},
   "source": [
    "If V2 introduces a defect, read the feedback and the revised code before naming a cause."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c20",
   "metadata": {},
   "source": [
    "## 4. The Pipeline as One Function\n",
    "\n",
    "The four units compose into one function, as in the source lab's `run_workflow`: a new coder writes V1, the code runs, a new critic reviews the chart, the revision runs. The function returns every intermediate result, because intermediate results are what you inspect when a pipeline goes wrong.\n",
    "\n",
    "*Do:* run the cell; it defines `run_workflow`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c21",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "def run_workflow(instruction, name):\n",
    "    \"\"\"Chart request -> V1 and V2 code, the feedback, and both chart files (name_v1.png, name_v2.png).\"\"\"\n",
    "    chart_v1 = name + \"_v1.png\"\n",
    "    chart_v2 = name + \"_v2.png\"\n",
    "\n",
    "    coder = Agent(CODER_SYSTEM, json_mode=True)\n",
    "    answer = coder.ask(f\"\"\"{instruction}\n",
    "Save the figure as {chart_v1}.\"\"\")\n",
    "    code_v1 = json.loads(answer)[\"code\"]\n",
    "    exec(code_v1, {\"df\": df})\n",
    "\n",
    "    critic = Agent(CRITIC_SYSTEM, json_mode=True)\n",
    "    review = json.loads(critic.ask(review_request(instruction, code_v1, chart_v2), image=chart_v1))\n",
    "    exec(review[\"code\"], {\"df\": df})\n",
    "\n",
    "    return {\"code_v1\": code_v1, \"chart_v1\": chart_v1, \"feedback\": review[\"feedback\"],\n",
    "            \"code_v2\": review[\"code\"], \"chart_v2\": chart_v2}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c22",
   "metadata": {},
   "source": [
    "### 4.1 A full run\n",
    "\n",
    "The run uses the name `drink_sales`, so its files do not overwrite the Section 3 charts.\n",
    "\n",
    "*Do:* run the cell. *Try:* after Section 5, run it again with your own request; any aggregation the columns support works (sales by hour of day, by coffee type, by payment method)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c23",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "result = run_workflow(INSTRUCTION, \"drink_sales\")\n",
    "print(\"FEEDBACK:\", result[\"feedback\"])\n",
    "show_image(result[\"chart_v1\"])\n",
    "show_image(result[\"chart_v2\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c24",
   "metadata": {},
   "source": [
    "## 5. The Score ✍️ (core)\n",
    "\n",
    "The claim \"V2 is better than V1\" becomes a measurement. The judge is one more `Agent` object, made with `json_mode=True`: the rubric is its system message, and each version arrives as a user message with the request, the code, and the chart image. Its reply is JSON with one field per criterion, so code adds up the score:\n",
    "\n",
    "| Criterion | 1 point when the chart... |\n",
    "|---|---|\n",
    "| `data_slice` | shows exactly the data the request asks for |\n",
    "| `title` | has a descriptive title |\n",
    "| `axis_labels` | labels both axes |\n",
    "| `legend_readability` | has a legend when several series are plotted, and readable labels |\n",
    "| `matches_instruction` | is the chart type and comparison the request describes |\n",
    "\n",
    "Quality target: **V2 total ≥ 4**. Record the score change V2 − V1 as well; a tie or a drop stays visible.\n",
    "\n",
    "The judge is also a model: compare its scores with the two charts yourself before you trust them. On a miss, sharpen `REFLECTION_PROMPT` in 3.3 and rerun from there.\n",
    "\n",
    "*Do:* run the three cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c25",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "JUDGE_SYSTEM = \"\"\"You grade a chart against a chart request. You receive the request, the code, and the chart as an image.\n",
    "Score each criterion 1 if the chart meets it, else 0:\n",
    "- data_slice: shows exactly the data the request asks for\n",
    "- title: has a descriptive title\n",
    "- axis_labels: labels both axes\n",
    "- legend_readability: has a legend when several series are plotted, and readable labels\n",
    "- matches_instruction: is the chart type and comparison the request describes\n",
    "Return JSON with exactly these five fields, each 0 or 1.\"\"\"\n",
    "\n",
    "CRITERIA = [\"data_slice\", \"title\", \"axis_labels\", \"legend_readability\", \"matches_instruction\"]\n",
    "TARGET_SCORE = 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c26",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "## Unit 1 - a judge grades V1: the request and the code as text, the chart as an image\n",
    "judge_v1 = Agent(JUDGE_SYSTEM, json_mode=True)\n",
    "answer = judge_v1.ask(f\"\"\"Chart request: {INSTRUCTION}\n",
    "\n",
    "Code:\n",
    "{result[\"code_v1\"]}\"\"\", image=result[\"chart_v1\"])\n",
    "judge_v1.show()\n",
    "score_v1 = json.loads(answer)                              # JSON text -> {criterion: 0 or 1}\n",
    "print(score_v1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c27",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "## Unit 2 - the same steps as a function; a new judge for every version\n",
    "def judge_chart(code, chart):\n",
    "    \"\"\"Plotting code and its chart file -> {criterion: 0 or 1}.\"\"\"\n",
    "    judge = Agent(JUDGE_SYSTEM, json_mode=True)\n",
    "    answer = judge.ask(f\"\"\"Chart request: {INSTRUCTION}\n",
    "\n",
    "Code:\n",
    "{code}\"\"\", image=chart)\n",
    "    return json.loads(answer)\n",
    "\n",
    "\n",
    "def total_score(scores):\n",
    "    \"\"\"Add up the points of the five criteria.\"\"\"\n",
    "    total = 0\n",
    "    for name in CRITERIA:\n",
    "        total = total + scores[name]\n",
    "    return total\n",
    "\n",
    "\n",
    "score_v2 = judge_chart(result[\"code_v2\"], result[\"chart_v2\"])\n",
    "total_v1 = total_score(score_v1)\n",
    "total_v2 = total_score(score_v2)\n",
    "print(\"V1:\", score_v1, \"->\", total_v1)\n",
    "print(\"V2:\", score_v2, \"->\", total_v2)\n",
    "print(\"target met:\", total_v2 >= TARGET_SCORE)\n",
    "print(\"score change (V2 - V1):\", total_v2 - total_v1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c28",
   "metadata": {},
   "source": [
    "The per-criterion scores show what the judge decided. Check them against the two charts and the request.\n",
    "\n",
    "### Exercise — a second reflection round ✍️\n",
    "\n",
    "A third version (V3) costs two more calls: one to a new critic, one to a new judge. Write your prediction first: does V3 score higher than V2 by enough to justify the extra calls, or has the score stopped rising?\n",
    "\n",
    "*Do:* write your prediction, then run the two cells and compare."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c29",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ✍️ WRITE · your answer in words (not checked)\n",
    "prediction_v3 = \"\"\"\n",
    "\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c30",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "## Unit 1 - a new critic reviews the V2 chart\n",
    "critic_3 = Agent(CRITIC_SYSTEM, json_mode=True)\n",
    "answer_v3 = critic_3.ask(review_request(INSTRUCTION, result[\"code_v2\"], \"chart_v3.png\"), image=result[\"chart_v2\"])\n",
    "critic_3.show()\n",
    "code_v3 = json.loads(answer_v3)[\"code\"]\n",
    "exec(code_v3, {\"df\": df})\n",
    "show_image(\"chart_v3.png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c31",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "## Unit 2 - score V3 (the second extra call)\n",
    "score_v3 = judge_chart(code_v3, \"chart_v3.png\")\n",
    "print(\"V1 ->\", total_v1, \"| V2 ->\", total_v2, \"| V3 ->\", total_score(score_v3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c32",
   "metadata": {},
   "source": [
    "Further rounds can raise, keep, or lower the measured quality. Compare the charts and the extra calls, and stop at a fixed budget or a checked target.\n",
    "\n",
    "### Exercise — code only vs. the rendered chart ✍️\n",
    "\n",
    "A critic that sees only the code must imagine the chart. Here a new critic gets the same system message and the same request for V1, but no image. The only change in the code is the missing `image=` argument. Its revision is run and scored like V2.\n",
    "\n",
    "Write your prediction first: which critique finds more real defects, and which revision scores higher?\n",
    "\n",
    "*Do:* write your prediction, then run the two cells and compare."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c33",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ✍️ WRITE · your answer in words (not checked)\n",
    "prediction_image = \"\"\"\n",
    "\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c34",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "## Unit 1 - the same request, no image\n",
    "critic_text = Agent(CRITIC_SYSTEM, json_mode=True)\n",
    "answer_text = critic_text.ask(review_request(INSTRUCTION, result[\"code_v1\"], \"chart_v2_codeonly.png\"))   # no image\n",
    "critic_text.show()\n",
    "review_text = json.loads(answer_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c35",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · example: read each line, then run\n",
    "## Unit 2 - run the code-only revision, compare the two critiques, and score it\n",
    "exec(review_text[\"code\"], {\"df\": df})\n",
    "show_image(\"chart_v2_codeonly.png\")\n",
    "print(\"image-based feedback:\", result[\"feedback\"])\n",
    "print()\n",
    "print(\"code-only feedback:\", review_text[\"feedback\"])\n",
    "score_text = judge_chart(review_text[\"code\"], \"chart_v2_codeonly.png\")\n",
    "print()\n",
    "print(\"code-only V2 ->\", total_score(score_text), \"| image-based V2 ->\", total_v2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c36",
   "metadata": {},
   "source": [
    "A code-only reviewer can guess some layout problems, but it cannot inspect the rendered result. The image adds evidence; compare the critiques without assuming one must win. Neither view checks that the underlying data is correct."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c37",
   "metadata": {},
   "source": [
    "## 6. Completion Check\n",
    "\n",
    "Every row must read `PASS` before you submit. The checks are structural; the Section 5 score measures quality.\n",
    "\n",
    "Submit: run the notebook top to bottom, then **File → Download → Download .ipynb**, and upload the file to the LMS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c38",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ▶ RUN · check: just run\n",
    "checks = {\n",
    "    \"REFLECTION_PROMPT written\": len(REFLECTION_PROMPT) > 20,\n",
    "    \"workflow produced both charts\": os.path.exists(result[\"chart_v1\"]) and os.path.exists(result[\"chart_v2\"]),\n",
    "    \"judge returned all criteria\": len(score_v2) == len(CRITERIA),\n",
    "    \"quality target met (V2 >= 4)\": total_v2 >= TARGET_SCORE,\n",
    "}\n",
    "\n",
    "passed = 0\n",
    "for name in checks:\n",
    "    if checks[name]:\n",
    "        print(\"PASS\", name)\n",
    "        passed = passed + 1\n",
    "    else:\n",
    "        print(\"FAIL\", name)\n",
    "if passed == len(checks):\n",
    "    print(\"HOMEWORK COMPLETE\")\n",
    "else:\n",
    "    print(\"NOT COMPLETE YET\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c39",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "Reference fill-in: `labs/checkpoints/week05/solution.py`, published after the homework deadline.\n",
    "\n",
    "Next week splits the work across several agents (Ch. 6). The generate → run → critique → regenerate cycle returns there with the critic as a separate agent."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}