Carrier and Beacon: The Dual-Architecture Future of LLM Context

Context windows are getting larger (1M, 2M, 10M tokens), but filling them is slow and expensive. And RAG (Retrieval Augmented Generation) is often dumb—it fetches keywords but misses the narrative. We need a new architecture. We call it Carrier and Beacon.

The Carrier is a massive, slow-moving context vessel. It holds the entire state of the world, the project history, the user's preferences. It doesn't run on every keystroke. It runs periodically, summarizing and consolidating state. Think of it as the 'Long Term Memory' lobe.

The Beacon is a lightweight, fast model (like a distilled 7B or a specialized router). It handles the immediate user interaction. It knows where to look in the Carrier, but it doesn't carry the weight itself.

class BeaconAgent:
    def handle_request(self, user_input):
        # Fast: classify intent
        intent = self.classify(user_input)
        
        # Ping the Carrier for relevant context summary
        context_pointer = self.carrier.get_reference(intent)
        
        # Generate response using pointer + active memory
        return self.llm.generate(user_input, context=context_pointer)

class CarrierSystem:
    def background_process(self):
        # Slow: digest logs, update knowledge graph
        while True:
            logs = self.ingest_recent_activity()
            self.knowledge_graph.update(logs)
            self.optimize_indices()

Standard RAG fetches snippets. The Carrier architecture fetches synthesized understanding. It's the difference between looking up a word in a dictionary and asking a professor who knows the whole book. RAG is stateless retrieval; Carrier is stateful cognition.

Current LLMs are lobotomized after every session. They forget everything. Carrier architecture gives them persistence. It transforms the AI from a 'tool' you pick up into a 'partner' that lives alongside you.