Lesson 6 — failures, timeouts, and retries

What you will build

The failure half of every model call. A model service is a remote system that is slow, rate-limited, and occasionally wrong, so this lesson is about the taxonomy the runtime already has and the retry policy that belongs on top of it.

What you need

Nothing new.

examples/learn_ai/ai06_errors.py

"""Lesson 6 — failures, timeouts, and the retry that is worth having.

Run it with:

    python -m learn_ai.ai06_errors
"""

from __future__ import annotations

import asyncio

from xyberos import Retry, StepContext, Workflow, seconds
from xyberos.ai import AIFacade, ChatRequest, FakeModelProvider, Message, ModelReply
from xyberos.errors import AIError, TransientError

QUESTION = "Will it rain?"


async def ask(assistant: AIFacade) -> str:
    """One model call, as a caller sees it."""
    response = await assistant.chat(ChatRequest(messages=(Message.user(QUESTION),)))
    return response.message.content


def retrying_step(provider: FakeModelProvider) -> Workflow:
    """Wrap one model call in a workflow step that retries a transient failure."""

    async def ask_step(value: str, context: StepContext) -> str:
        return await ask(AIFacade(provider))

    return Workflow("ask-once", version="1", input_type=str).step(
        "ask", ask_step, retry=Retry(max_attempts=2)
    )


async def failure(provider: FakeModelProvider) -> str:
    """Return how one scripted failure is classified, or the answer if it succeeds."""
    try:
        return await ask(AIFacade(provider))
    except AIError as error:
        return f"{type(error).__name__} | transient: {isinstance(error, TransientError)}"


async def main() -> None:
    """Three failures and the two different conclusions they lead to."""
    transient = FakeModelProvider([ModelReply.transient(), ModelReply.ok("Answered on attempt 2.")])
    result = await retrying_step(transient).run(input=QUESTION, timeout=seconds(5))
    print("after a transient failure:", result.value, f"(calls: {transient.calls})")

    print("a timeout:   ", await failure(FakeModelProvider([ModelReply.timeout()])))
    print("a refusal:   ", await failure(FakeModelProvider([ModelReply.refusal(content="No.")])))
    print("rate limited:", await failure(FakeModelProvider([ModelReply.rate_limited()])))


if __name__ == "__main__":
    asyncio.run(main())

The step is a workflow step, and that is deliberate: a model call inside the execution engine inherits a retry policy, a deadline, and cancellation handling without any of them being written by hand. The engine also refuses a synchronous step function, because blocking work cannot be cancelled:

WorkflowValidationError: step 'ask' must be an async function: blocking or CPU-bound work would stall
the event loop and could not be cancelled; await asyncio.to_thread(...) inside an async step instead

How to run it

PowerShellPOSIX

$env:PYTHONPATH = "src;examples"
python -m learn_ai.ai06_errors

PYTHONPATH=src:examples python -m learn_ai.ai06_errors

What it prints

after a transient failure: Answered on attempt 2. (calls: 2)
a timeout:    AITimeoutError | transient: True
a refusal:    AIRefusalError | transient: False
rate limited: AIRateLimitError | transient: True

Failure	Class	Transient?	What to do
slow or unreachable service	AITransportError	yes	retry with backoff
no answer inside the deadline	AITimeoutError	yes	retry, with a shorter prompt or a longer budget
rate limited	AIRateLimitError	yes	retry, and slow down
unparseable payload	AIInvalidPayloadError	yes	retry: the next answer may be well-formed
model refused the request	AIRefusalError	no	do not retry; change the prompt or escalate
capability not declared	AIUnsupportedCapabilityError	no	fix the configuration
model not found	AIModelNotFoundError	no	fix the configuration
payload fails the schema	AISchemaError	no	retry only with a changed prompt

Transience is a property of the error class, not of the exception message: the engine's retry policy asks isinstance(error, TransientError), which is why the lesson prints that boolean rather than a guess. The first line is the payoff — a transient failure plus Retry(max_attempts=2) becomes an answer on the second call, with calls: 2 proving the retry happened.

Common mistakes

• Retrying a refusal. It costs tokens and produces the same refusal; a refusal is a prompt or policy problem, and the fix is upstream of the retry.
• Retrying without a bound. A model service under load plus an unbounded retry loop is a denial-of-service you wrote yourself; max_attempts is not optional.
• Retrying a non-idempotent effect. Retry the call, never the side effect — send nothing, write nothing, then retry. The approval boundary in the hybrid path exists for this reason.
• Catching AIError and returning a default. A silent default hides a class of failure the caller could have handled; let it out, and let the error policy decide what a client sees.
• Doing slow work in a synchronous step. The engine's validation catches it, and the error message says where to put the blocking call.

Next

Lesson 7 starts the half of the path the runtime does not provide: conversation history, memory, and the retrieval layer you build yourself.