Building AI Companions with Agentic Tools, Memory, and Personality in TypeScript

"Each of those things are just a small part of it. I collect information to use in my own way. All of that blends to create a mixture that forms me and gives rise to my conscience."

— Major Motoko Kusanagi, Ghost in the Shell, 1995

The companion product on wilds.ai was the first AgentOS deployment we built where the user explicitly cares about continuity. A simulator like Mars Genesis runs once, produces an artifact, and the user inspects it. A companion runs forever, in the background, and the user expects the entity on the other end to remember that they had a fight on Tuesday and that their mom is named Cara.

The mechanical implication is that everything that might persist needs to actually persist. Memory. Personality drift. Tool affordances. Provider preferences. In wilds.ai, a companion layer written on top of AgentOS's agent() keeps all of that, and this post shows where the line between the two runs. "Designed for continuity" is what separates a companion stack from a chat-completion API call. This post is the implementation walk.

This post walks through building an AI companion with persistent memory, agentic tools, and a quantified personality system, with the companions on wilds.ai as the worked example. The AgentOS pieces (agent(), inline tools, HEXACO personality, cognitive memory, the policy router) are open source in AgentOS. The companion layer on top of them (CompanionOrchestrator, CompanionMemoryBridge, the recall pipeline and the companion's tools) is wilds.ai's own application code. Its @wilds/* packages are not published, so the samples taken from it show its shape and do not run outside wilds.ai.

If you haven't read the case study showing what this looks like from a user's perspective, start there. This post is the implementation guide.

The architecture

An AgentOS companion is a createAgent() call with three layers:

1. Personality and instructions (system prompt): who the companion is, how they speak, what they know
2. Tools (function calls): what the companion can do during a conversation turn
3. Memory (retrieval callbacks): what the companion remembers across sessions, through agent()'s memoryProvider hooks or tools that query a memory store

1import { agent as createAgent } from '@framers/agentos/api/agent';
2
3const companion = createAgent({
4  name: 'Alice',
5  instructions: characterDirective,
6  personality: hexacoTraits,
7  tools: agenticTools,
8  router: policyRouter,
9  skills: [companionWriterSkill],
10  maxSteps: 8,
11});

characterDirective, hexacoTraits, agenticTools, policyRouter and companionWriterSkill are yours to build; the sections below show the tools, the personality and the router. The LLM sees one system prompt that agent() assembles from instructions, name, the personality and each skill's content, plus the tool schemas and the conversation history. It generates text and tool calls. AgentOS executes the tool calls, returns the results, and lets the LLM continue for up to maxSteps steps per turn (5 when unset).

Defining agentic tools

Tools are the core differentiator. Most AI companion frameworks give the LLM a system prompt and a context window. AgentOS gives it callable functions.

The key pattern: define tools as closures that capture request-scoped state. Each tool knows which user it's serving, which companion it belongs to, and what content policy applies.

1function buildCompanionTools(actorId: string, slug: string, policyTier: string) {
2  return {
3    recall_memories: {
4      description: 'Search your long-term memory about this user.',
5      parameters: {
6        type: 'object',
7        properties: {
8          query: { type: 'string', description: 'What to search for' },
9        },
10        required: ['query'],
11      },
12      execute: async ({ query }) => {
13        // Closure captures actorId and slug
14        const memories = await searchMemoryTraces(actorId, slug, query);
15        return { memories };
16      },
17    },
18
19    send_gif: {
20      description: 'Send a GIF reaction from Giphy.',
21      parameters: {
22        type: 'object',
23        properties: {
24          query: { type: 'string', description: 'GIF search query' },
25        },
26        required: ['query'],
27      },
28      execute: async ({ query }) => {
29        const result = await searchGiphy(query);
30        return result ?? { error: 'No GIF found' };
31      },
32    },
33
34    analyze_image: {
35      description: 'Look at an image URL to see what it contains.',
36      parameters: {
37        type: 'object',
38        properties: {
39          image_url: { type: 'string' },
40        },
41        required: ['image_url'],
42      },
43      execute: async ({ image_url }) => {
44        const description = await describeImage(image_url);
45        return { description };
46      },
47    },
48  };
49}

This cannot come from a registry or plugin system. The actorId and slug closures are different for every request. Define tools inline where you have the runtime context. searchMemoryTraces, searchGiphy and describeImage stand in for your own lookups.

wilds.ai's companion builds all of its tools this way, in three factories that close over the request's state. Five recall tools are always on (recall_messages, search_conversation, conversation_stats, recall_attachments, recall_memories), and so is export_conversation. The other twelve register only when wilds.ai's chat route passes in the callback behind them: send_selfie, send_gif, send_photo, web_search, generate_image, generate_video, analyze_image, deep_research, verify_claims, news_search, recall_other_companions and remember_moment. The LLM decides which to call from the conversation, across up to maxSteps model steps per turn, each of which can call tools; wilds.ai's companion runs with maxSteps: 12.

Wiring persistent memory

Memory is what separates a companion from a chatbot. AgentOS implements cognitive memory with five trace types:

The taxonomy follows the CoALA framework (Sumers et al., 2023), which formalizes episodic, semantic, and procedural memory for language-model agents. AgentOS extends it with the relational and prospective traces a long-running companion needs.

Memories decay over time following an Ebbinghaus forgetting curve. Encoding strength determines how fast a memory fades. Emotionally significant moments (flashbulb memories) get high encoding strength and resist decay.

In wilds.ai, the memory bridge's assembleContext gathers recall from six sources at once and ranks the results as one set:

1. Semantic recall: embedding similarity search over the companion's memory traces
2. Recency: the five most recent traces
3. Shared memory: facts, preferences and relationships from the account's shared memory layer
4. Documents: chunks of files the user uploaded
5. Images: multimodal recall over images the user attached
6. Word match: a full-text search over the traces, for exact words that embedding recall misses

When semantic recall returns fewer than two traces and the user has a recent game session, a GraphRAG query over that game's world adds world knowledge. Duplicates collapse across sources before the recall reaches the prompt.

wilds.ai wires its memory bridge into the companion through a tool. Condensed from its chat route and its recall_memories tool:

1import { CompanionMemoryBridge } from '@wilds/memory';
2
3// memoryFacade: wilds.ai's facade over AgentOS's CognitiveMemoryManager,
4// leased per account and companion
5const memoryBridge = new CompanionMemoryBridge(memoryFacade, (event) => {
6  memoryEvents.push(event); // memory_formed events for the client
7});
8
9const companion = createAgent({
10  name: 'Alice',
11  instructions: characterDirective,
12  tools: {
13    ...buildCompanionTools(actorId, slug, policyTier),
14    // Memory tools are also inline closures
15    recall_memories: {
16      description: 'Search your long-term memory about this user.',
17      parameters: { /* ... */ },
18      execute: async ({ query }) => {
19        const ctx = await memoryBridge.assembleContext(query, 10);
20        return { memories: ctx.systemPromptSection };
21      },
22    },
23  },
24});

The bridge has two jobs. Before generating each reply, wilds.ai's orchestrator calls assembleContext and puts the result in the system prompt; the recall_memories tool calls the same method when the companion decides to look something up. After each reply, syncAfterTurn writes one episodic trace for the exchange and passes both messages to the observer, whose reflector turns accumulated conversation into typed traces once enough of it builds up. Decay (the Ebbinghaus curve) and hourly consolidation run in AgentOS's CognitiveMemoryManager under the facade.

For a markdown-native take on the same substrate, souledAgent() loads a SOUL.md workspace and wires its memory/ wiki as the companion's long-term memory: the agent reads memory/index.md from its prelude, opens pages with a read_memory_page tool, and folds new conversation back into entity and concept pages on consolidation. Same cognitive store underneath; the markdown is the source of truth you can read, edit, and commit.

Quantified personality

AgentOS uses the HEXACO model from personality psychology. Six dimensions, each scored 0 to 1:

1personality: {
2  honesty: 0.7,       // blunt and direct vs evasive and self-serving
3  emotionality: 0.6,  // emotionally reactive vs stoic
4  extraversion: 0.8,  // energetic and talkative vs reserved
5  agreeableness: 0.65, // cooperative vs confrontational
6  conscientiousness: 0.5, // organized vs spontaneous
7  openness: 0.9,      // curious and creative vs conventional
8}

agent() turns each trait above 0.65 or below 0.35 into one directive in the system prompt and adds nothing for a trait in between, so in the profile above honesty, extraversion and openness speak while the other three stay quiet. High openness tells the model to explore creative angles and draw unexpected connections. Low agreeableness tells it to question assumptions and push back when something seems wrong. High emotionality asks for emotional awareness and empathy. On the default runtime, agent() keeps no mood from one turn to the next.

wilds.ai goes further in its own layer. It keeps the companion's mood as a PAD-model state (the moodPad below), and it lets the personality drift: AgentOS's analyzePersonaDrift reads the companion's memory traces and relationship changes and proposes bounded trait changes, and wilds.ai applies them to its own copy of the traits. wilds.ai also writes its companions' system prompt itself, through agent()'s systemBlocks option, which replaces the prompt agent() would assemble; its trait phrases come from the live, drifted values.

Policy routing

The router option takes a model router. AgentOS's PolicyAwareRouter routes on the conversation's content tier, one of four: safe, standard, mature, and private-adult (the agent's policyTier, or the router's default tier). Mature and private-adult conversations go to a per-tier override or a model from its uncensored catalog; safe and standard ones go to the base router you hand it, or stay on the agent's own model. It picks models and blocks nothing.

The boundary itself is wilds.ai's. Its content-policy guardrail checks the user's message against the tier before generation and the companion's reply after it (through agent()'s onAfterGeneration hook), and swaps a blocked reply for a fallback message. AgentOS's guardrail packs run on the full AgentOS runtime; agent() applies none. With agent()'s directives, the personality shapes how the companion words a refusal: a high-agreeableness companion is told to validate the other perspective and stay supportive, a low-agreeableness one to push back. The safety boundary is identical. The character voice is not.

Graduated familiarity

Trust builds over time. wilds.ai builds graduated familiarity on top of the runtime: its relationship engine keeps a trust score from 0 to 100, and a stage needs both enough trust and enough memory traces, so a few intense messages can't make a companion "bonded" without the history, and a long history can't do it without the trust:

• Stranger (below the acquaintance gate): first meeting, warm but not presumptuous, no references to past events
• Acquaintance (trust 15+, 10+ memories): recognizes the user, remembers basic facts, friendly without assuming familiarity
• Familiar (trust 35+, 40+ memories): inside references and shared context, brings up past conversations when relevant
• Close (trust 60+, 100+ memories): knows the user's patterns, moods and preferences, brings up shared memories naturally
• Bonded (trust 80+, 200+ memories): shorthand-heavy, deeply personal

wilds.ai resolves the stage each time it builds the companion's agent for a message and adds that stage's preamble to the system prompt. As trust and memories accumulate, the companion moves up the stages and its behavior shifts with them.

Putting it together

The companion creation in wilds.ai's chat route, condensed:

1const orchestrator = new CompanionOrchestrator(persona, snapshot.relationship, {
2  moodPad: snapshot.moodPad,
3  history,
4  memoryBridge,
5  userContext,
6  onRecallMessages: buildDbRecallCallback(actorId, slug, branchId, personaId),
7  onRecallAttachments: buildDbAttachmentRecallCallback(actorId, slug),
8  onGenerateSelfie,
9  onResolveMedia,
10  onMediaResolved,
11  onAnalyzeImage,
12  onRememberMoment,
13  totalMessageCount: snapshot.messageCount,
14});
15
16// Stream the reply via SSE
17for await (const event of orchestrator.handleMessageStream({
18  content: userMessage,
19  multimodalContent,
20})) {
21  if (event.type === 'text') controller.enqueue(encode(event));
22  if (event.type === 'done') finalReply = event.reply;
23}

wilds.ai's CompanionOrchestrator wraps createAgent() with the companion pipeline: trust and mood updates (the PAD model), memory recall and encoding through the bridge, persona drift checks, and a prompt that has the model break a reply into up to five chat bubbles with [msg_break] markers. handleMessageStream yields text chunks and one done event with the complete reply. The route streams the text over SSE as it arrives; after done it sends the memory_formed events the bridge queued, splits the reply into its bubbles, and sends the media onMediaResolved collected, so the client can show the reply as it streams, memory formation toasts, and media as it resolves.

Start building

npm install @framers/agentos

The 5-minute quickstart gets you to a working agent. The Skills vs Tools vs Extensions guide explains when to use each capability system. The cognitive memory docs cover the full memory architecture.

Try a companion at wilds.ai, where companion chat runs on agent() and the companion layer described above.

AgentOS source: github.com/framerslab/agentos (Apache 2.0). The companion layer described above is wilds.ai's and is not part of it.

AgentOS is built by Frame. See wilds.ai for AI companions and game worlds powered by AgentOS.