Conversation Context and Memory

Keep message history, respect the context window, and summarize when short-term memory gets too long.

The model only sees the messages list you send. If you start a new list every turn, it has amnesia. If you never trim, you blow the context window and the bill.

Short-term memory is the list

Hold messages on the agent object. Each user turn appends. Each run continues the same list. That is conversation context.

class Agent:
    def __init__(self):
        self.messages = [
            {"role": "system", "content": "You scout a local project. Be brief."}
        ]

    def ask(self, user_text: str) -> str:
        self.messages.append({"role": "user", "content": user_text})
        answer = self.run_loop()
        return answer

The window is finite

Every tool result and every assistant paragraph stays until you drop it. A 4k-character file read, five times, is how hobby agents die. Cap tool payloads (you already sliced [:4000]). Cap the list.

Summarize, then drop

When token estimate or message count crosses a line, ask the model once to summarize the older turns into a short system or assistant note, then keep only: system, the summary, and the last few turns (including in-flight tool calls).

def maybe_compact(self) -> None:
    if len(self.messages) < 24:
        return
    older = self.messages[1:-6]
    summary = think([
        {"role": "system", "content": "Summarize this agent trace in 8 bullets. Keep paths and decisions."},
        {"role": "user", "content": json.dumps(older)[:12000]},
    ]).content
    self.messages = [self.messages[0], {"role": "system", "content": "Prior work:\n" + (summary or "")}] + self.messages[-6:]

Two asks in one process should refer to the first. After you force 30 dummy turns, the list should shrink and the agent should still know the project name. Next: remember across processes.