Lesson 8 — memory¶
What you will build¶
Memory, which in this path means three ordinary things: a table, a query, and a decision about which rows belong in the next request. There is no framework memory layer, no embedding store, and no automatic recall — and the code below is what a memory looks like when you need one.
What you need¶
The database extra, because memory that survives the process is a database:
examples/learn_ai/ai08_memory.py
"""Lesson 8 — memory is a table and a query, not a framework feature.
Run it with:
python -m learn_ai.ai08_memory
"""
from __future__ import annotations
import asyncio
from typing import Annotated
from sqlalchemy import Column, Integer, MetaData, String, Table, insert, select
from xyberos import App, Inject
from xyberos.ai import AIFacade, ChatRequest, FakeModelProvider, Message, ModelReply
from xyberos.database import DatabaseProvider, temporary_database
SYSTEM_PROMPT = "You are a concise support assistant."
SUBJECT = "acme"
metadata = MetaData()
memories = Table(
"memories",
metadata,
Column("id", Integer, primary_key=True, autoincrement=True),
Column("subject", String(120), nullable=False),
Column("kind", String(40), nullable=False),
Column("text", String(500), nullable=False),
)
class MemoryStore:
"""Rows the application owns, recalled on purpose and put into the prompt by hand.
There is no framework memory layer, and this is what one looks like when you need it: a table,
an insert, a query, and a decision about which rows belong in the next request.
"""
def __init__(self, database: Annotated[DatabaseProvider, Inject()]) -> None:
self._database = database
async def create_schema(self) -> None:
"""Create the table, which a real application would do with a migration."""
async with self._database.transaction() as connection:
await connection.run_sync(metadata.create_all)
async def remember(self, subject: str, kind: str, text: str) -> None:
"""Store one durable fact about a subject."""
async with self._database.transaction() as connection:
await connection.execute(insert(memories).values(subject=subject, kind=kind, text=text))
async def recall(self, subject: str, *, limit: int = 5) -> list[dict[str, object]]:
"""Return what is remembered about a subject, oldest first."""
statement = (
select(memories)
.where(memories.c.subject == subject)
.order_by(memories.c.id)
.limit(limit)
)
async with self._database.transaction() as connection:
rows = (await connection.execute(statement)).mappings().all()
return [dict(row) for row in rows]
def as_system_message(rows: list[dict[str, object]]) -> Message:
"""Turn recalled rows into the one system message the model will see."""
remembered = "; ".join(f"{row['kind']}={row['text']}" for row in rows)
return Message.system(f"What you remember about {SUBJECT}: {remembered}")
async def main() -> None:
"""Remember two things, recall them, and show the prompt they produced."""
provider = FakeModelProvider([ModelReply.ok("A refund takes three business days.")])
app = App()
async with temporary_database() as database:
app.provide(DatabaseProvider, instance=database)
app.provide(MemoryStore)
app.provide(AIFacade, instance=AIFacade(provider), eager=True)
async with app.lifespan():
memory = await app.resolve(MemoryStore)
assistant = await app.resolve(AIFacade)
await memory.create_schema()
await memory.remember(SUBJECT, "preference", "prefers email over phone")
await memory.remember(SUBJECT, "fact", "is on the enterprise plan")
recalled = await memory.recall(SUBJECT)
print("remembered:", [(row["kind"], row["text"]) for row in recalled])
request = ChatRequest(
messages=(as_system_message(recalled), Message.user("How long does a refund take?"))
)
response = await assistant.chat(request)
print("the model saw:", provider.requests[-1].messages[0].content)
print("answer:", response.message.content)
if __name__ == "__main__":
asyncio.run(main())
MemoryStore is a service with two methods, and as_system_message is where the interesting decision
lives: it chooses a representation. The rows are facts; the prompt is a sentence. Turning one into the
other is application policy, and it is the part that changes when the model changes.
The recall is a SELECT ... WHERE subject = ? ORDER BY id LIMIT 5. Lesson 10 replaces that WHERE with
a similarity query over embed; the shape of the store, the insert, and the prompt-building step do not
change.
How to run it¶
What it prints¶
remembered: [('preference', 'prefers email over phone'), ('fact', 'is on the enterprise plan')]
the model saw: What you remember about acme: preference=prefers email over phone; fact=is on the enterprise plan
answer: A refund takes three business days.
| Line | What it establishes |
|---|---|
remembered: [...] |
two rows, in the order they were written, read back with one query |
the model saw: What you remember about acme: ... |
the system message as_system_message built — recall becomes context here and nowhere else |
answer: |
the provider's reply, which is unaffected by any of it |
What memory is not¶
| Not this | Because |
|---|---|
| A bigger conversation window | a window is bounded by cost, and it forgets; a memory row is chosen |
| A vector index | similarity is lesson 10, and it is a query strategy over rows like these |
| A framework feature | anything that decided what to remember for you would be making a product decision |
| Always on | recall costs a query and a prompt; a fact nobody needs is tokens and a distraction |
Common mistakes¶
- Remembering everything. A memory store that grows without a retention rule turns into a slower, more expensive prompt; the useful half of memory is what you choose to forget.
- Recalling without a subject.
WHERE subject = ?is the whole scoping mechanism — the same lesson as the traditional path's tenant, and the reason a memory key belongs to a principal rather than to a request body. - Putting recalled text in the user turn. A memory is an instruction by the application; putting it in a user message lets a stored sentence argue with the system prompt.
- Storing model output as fact. A summary of a conversation is a summary; write it as one, with its source, or the next caller will treat a guess as data.
- Skipping the retention index.
ORDER BY id LIMIT 5is fine for two rows and wrong for two million; the table needs the index its query needs.
Next¶
Lesson 9 loads documents: text you did not write, split into passages, stored so lesson 10 can retrieve them.