Lesson 11 — tools¶

What you will build¶

A way for the model to ask the application to do something, with three boundaries around it: an allowlist that decides what exists, a schema that decides what arguments are acceptable, and a round limit that decides how long the conversation may go on. The model never runs code; it asks, and the application decides.

What you need¶

Nothing new.

examples/learn_ai/ai11_tools.py

"""Lesson 11 — tools: an allowlist, and arguments checked against a schema.

Run it with:

    python -m learn_ai.ai11_tools
"""

from __future__ import annotations

import asyncio
from collections.abc import Mapping

from xyberos.ai import (
    AIFacade,
    ChatRequest,
    FakeModelProvider,
    Message,
    ModelReply,
    Tool,
    ToolCall,
    ToolEvent,
    ToolSet,
    ToolSpec,
)
from xyberos.errors import AIToolError

TICKETS = {"T-100": "printer is on fire", "T-101": "badge reader is broken"}

LOOKUP_SCHEMA = {
    "type": "object",
    "properties": {"ticket": {"type": "string"}},
    "required": ["ticket"],
    "additionalProperties": False,
}

COUNT_SCHEMA = {
    "type": "object",
    "properties": {"state": {"type": "string", "enum": ["open", "closed"]}},
    "required": ["state"],
    "additionalProperties": False,
}


def lookup_ticket() -> Tool:
    """A read-only tool: it looks a ticket up and can change nothing."""

    async def handler(arguments: Mapping[str, object], context: object) -> dict[str, object]:
        ticket = str(arguments["ticket"])
        return {"ticket": ticket, "title": TICKETS.get(ticket, "no such ticket")}

    return Tool(
        spec=ToolSpec(
            name="lookup_ticket",
            description="Look up one ticket's title by id",
            parameters=LOOKUP_SCHEMA,
        ),
        handler=handler,
    )


def count_tickets() -> Tool:
    """A second read-only tool, with an argument restricted to an enum."""

    async def handler(arguments: Mapping[str, object], context: object) -> dict[str, object]:
        return {"state": arguments["state"], "count": len(TICKETS)}

    return Tool(
        spec=ToolSpec(
            name="count_tickets",
            description="Count tickets in one state",
            parameters=COUNT_SCHEMA,
        ),
        handler=handler,
    )


def watch(events: list[ToolEvent]) -> object:
    """Return the sink that records what the tool layer did."""

    def sink(event: ToolEvent) -> None:
        events.append(event)

    return sink


async def main() -> None:
    """One tool call the loop performs, one call the allowlist refuses, one round limit."""
    events: list[ToolEvent] = []
    tools = ToolSet.of(lookup_ticket(), count_tickets(), on_event=watch(events))  # type: ignore[arg-type]
    print("allowed:", tools.names)
    print("declared:", [spec.name for spec in tools.specs])

    provider = FakeModelProvider(
        [
            ModelReply.tools(ToolCall(name="lookup_ticket", arguments={"ticket": "T-100"})),
            ModelReply.ok("Ticket T-100 is 'printer is on fire'."),
        ]
    )
    response = await AIFacade(provider).chat_with_tools(
        ChatRequest(messages=(Message.user("What is ticket T-100 about?"),)),
        tools=tools,
        max_rounds=3,
    )
    print("answer:  ", response.message.content, f"(rounds: {response.rounds})")
    for event in events:
        print("  event:", event.type.value, event.tool, event.argument_keys)

    try:
        await tools.invoke(ToolCall(name="refund_customer", arguments={"order": "O-9"}))
    except AIToolError as error:
        print("not allowed:", error)

    bad_arguments = await tools.invoke(ToolCall(name="lookup_ticket", arguments={"id": "T-100"}))
    print("bad args:   ", bad_arguments.ok, bad_arguments.error_type, bad_arguments.error)

    looping = FakeModelProvider(
        [ModelReply.tools(ToolCall(name="count_tickets", arguments={"state": "open"}))] * 4
    )
    try:
        await AIFacade(looping).chat_with_tools(
            ChatRequest(messages=(Message.user("Count them."),)), tools=tools, max_rounds=2
        )
    except AIToolError as error:
        print("round limit:", type(error).__name__, error)


if __name__ == "__main__":
    asyncio.run(main())

Both tools are read-only. That is not a coincidence: a tool that writes is a tool the model can use to change the world, and lesson 17's rule — a model may recommend, application code decides — starts with which tools exist at all.

How to run it¶

PowerShellPOSIX

$env:PYTHONPATH = "src;examples"
python -m learn_ai.ai11_tools

PYTHONPATH=src:examples python -m learn_ai.ai11_tools

What it prints¶

allowed: ('lookup_ticket', 'count_tickets')
declared: ['lookup_ticket', 'count_tickets']
answer:   Ticket T-100 is 'printer is on fire'. (rounds: 2)
  event: requested lookup_ticket ('ticket',)
  event: succeeded lookup_ticket ('ticket',)
not allowed: the model asked for tool 'refund_customer', which is not allowed for this call; allowed tools: lookup_ticket, count_tickets
bad args:    False AISchemaError arguments of tool 'lookup_ticket' at $ is missing the required property 'ticket'
round limit: AIToolError the model asked for 'count_tickets' after 2 round(s), which is the configured limit; raise max_rounds or move the orchestration into a workflow

The last three lines are three different refusals, and the difference between them is the lesson:

Refusal	What happens	Why
A tool that is not in the allowlist	`AIToolError` is raised, naming what is allowed	the request is not serviceable at all, and the caller should not have to guess
Arguments that do not match the schema	a failed `ToolResult` is returned, and the loop feeds the error back to the model	the model can correct itself, and one bad call should not end the conversation
More rounds than `max_rounds`	`AIToolError` is raised	the limit is a budget, not a suggestion: an unbounded tool loop is a denial-of-service you wrote

The ToolEvent lines are the audit trail: requested then succeeded, with the argument keys and no values. That is what a log should contain by default — enough to see what happened, nothing that leaks what a customer typed.

Common mistakes¶

Building the tool from the model's arguments. Arguments are schema-checked input, not a command: resolve the ticket id, look it up, and let the handler refuse what the schema cannot express.
Letting a tool take a principal from its arguments. The identity comes from the verified principal (traditional path, lesson 8), never from a tool argument — otherwise the model can choose whose data it reads.
Registering a write tool because it is convenient. A refund, an email, a delete: each one is an action that needs approval, which is the hybrid path's subject.
Ignoring the errors the loop feeds back. A ToolResult with ok=False is information the model can use; swallowing it produces a confident answer built on nothing.
Logging argument values. They are user data; log keys, tool names, outcomes, and durations.
Setting max_rounds high "to be safe". Safety is the limit, not its absence.

Next¶

Lesson 12 uses structured output for routing: an intent, a handler, and the rules that do not need a model at all.