Systems that hold up.
The AI system patterns teams actually search for — explained at the depth needed to build them for production, not demos. Architecture, failure modes, and the evaluation that keeps them honest.
LLM Evaluation Systems (Evals)
How to build production LLM evaluation systems: golden datasets, rubric-based LLM judges, regression gates in CI, online monitoring, and the eval-driven
RL Environments for Agent Training
How to design reinforcement learning environments for training AI agents: task distributions, reward design, verifiers, sandboxed tool execution
Agent Memory Systems
AI agent memory architecture: working, session, episodic, and long-term memory for LLMs, with write rules, forgetting, privacy controls, and production
Agentic RAG Systems
Agentic RAG systems: query planners, retrieval tools, multi-hop RAG loops, iteration limits, a groundedness critic, and when agentic retrieval is wrong to
Computer-Use Agent Systems
Computer-use agent systems: screenshot and DOM perception, action loops, sandboxed browsers, confirmation gates, recovery, evals, and when APIs beat UI
Function-Calling and Tool-Use Systems
Function calling architecture for LLM tool-use systems: schema registry, execution sandbox, idempotency, write confirmations, traces, and key failure
GraphRAG Systems
GraphRAG system design: entity and relation extraction, community summaries, query-time graph traversal, hybrid fallback, failure modes, evals, and when
Human-in-the-Loop AI Systems
Human-in-the-loop AI systems: shadow mode, HITL agent approval, escalation queues, learning from traces, and when people must still stay on the production
Hybrid Retrieval and Re-ranking Systems
Hybrid search for RAG: dense embeddings plus BM25, reciprocal rank fusion, cross-encoder re-ranking, metadata filters, latency budgets, failure modes, and
LLM Guardrail Systems
LLM guardrails architecture as an AI safety layer: pre and post filters, a policy engine, tool allowlists, latency, and fail-closed versus fail-open
LLM Inference and Serving Systems
LLM serving architecture: continuous batching, paged KV cache, quantization, prefix caching, autoscaling, failure modes, evals, and when not to self-host.
LLM Observability and Tracing
LLM tracing and AI observability in production: span models, cost tracking, PII redaction, quality overlays, and prompt monitoring alerts for production
MCP Tool Gateway Systems
Enterprise MCP gateway architecture: server registry, delegated auth, tool allow-lists, sandboxed egress, audit, failure modes, evals — not a protocol
Model Routing and Fallback Systems
Model routing with an LLM router: cheap versus strong pins, canaries, eval-based routes, version pins, and when a fallback model should take over a
PEFT and Fine-Tuning Pipelines
Enterprise PEFT pipeline: datasets, LoRA/QLoRA training, forgetting checks, adapter registry, merge vs multi-adapter serve, failure modes, and when RAG is
Permissioned Retrieval Systems
ACL-aware RAG architecture: query-time permission filtering, IdP groups, no existence leak, audit, failure modes, evals, and why index-time ACL is not
Prompt Injection Defense Systems
Prompt injection defense architecture: trust boundaries, dual-channel prompts, tool policy, detectors, and indirect prompt injection evals used in
RAG Architecture
Enterprise RAG architecture: ingest, hybrid index, retrieve, grounded generation, evals, SLOs, and when retrieval-augmented generation is the wrong design.
Realtime Voice AI Systems
Realtime voice agent architecture: telephony, streaming STT/TTS, turn-taking, barge-in, an 800ms first-audio budget, failure modes, evals, and when not to
AI Red-Teaming Systems
AI red-teaming architecture: attack generators, target harnesses, severity scoring, CI regression, disclosure and triage, failure modes, and when a suite
Structured Generation Systems
Structured generation for LLMs: constrained decoding, JSON schema, repair loops, validators, and binding model output to tool arguments for production
Synthetic Data Generation Systems
Synthetic data pipeline architecture: generators, critics, diversity checks, contamination controls, collapse detection, evals, and when not to train on
System pages are generated from a structured data model — each new pattern ships with its page, TechArticle schema, and FAQ schema automatically.
In short
An AI system is the machinery around the model — retrieval, tools, orchestration, evaluation, and rollout control. These write-ups cover how each part is built and how you tell whether it is working.
- Guides
- 23 systems, in depth
- Depth
- 138 sections total
- Written by
- The team that ships them
Which system fits your problem
All three show up in production together, but they solve different failures. Start with the one matching the symptom you actually have.
| Level | Read | Core components | Reach for it when | |
|---|---|---|---|---|
| Multi-Agent Orchestration Systems | Advanced | 17 min | Planner agent, Specialist workers, Router | Use multiple agents when a task spans distinct skills, tool sets, or permission boundaries — not because the task is merely long. |
| LLM Evaluation Systems (Evals) | Intermediate | 15 min | Golden dataset, Rubric judges, CI regression gate | Teams without evals discover regressions from angry users; teams with evals discover them in CI before deploy. |
| RL Environments for Agent Training | Advanced | 16 min | Task generator, Sandboxed executor, Verifier/reward | The verifier is the product: agents trained with RL will exploit any gap between "what the reward checks" and "what you actually want". |
| Agent Memory Systems | Advanced | 16 min | Working context, Session store, Episodic log | Memory is not "the context window." Split working, session, episodic, and semantic stores; each has a different lifetime and trust level. |
| Agentic RAG Systems | Advanced | 16 min | Query planner, Retrieval tools, Iteration budget | Agentic RAG is for multi-hop and underspecified questions; single-shot RAG remains the right design for lookup queries with one target document. |
| Computer-Use Agent Systems | Advanced | 17 min | Perception (DOM/screenshot), Action planner, Sandboxed executor | Prefer an API or documented protocol over computer use; UI agents exist for the leftover surfaces. |
| Function-Calling and Tool-Use Systems | Advanced | 16 min | Schema registry, Execution sandbox, Idempotency layer | Tool schemas are an API you own: version them, validate arguments server-side, and never trust the model to honor a description-only constraint. |
| GraphRAG Systems | Advanced | 17 min | Entity/relation extractor, Graph store, Community summarizer | Build a graph when questions are relational or corpus-wide (“how do these vendors connect?”); keep chunk RAG when questions are “find the paragraph that says X.” |
| Human-in-the-Loop AI Systems | Intermediate | 15 min | Shadow mode, Approval gates, Escalation queue | Put humans on irreversibility, not on every token — approvals on sends, writes, and payments; autonomy on reads and drafts. |
| Hybrid Retrieval and Re-ranking Systems | Intermediate | 16 min | Dense retriever, Sparse BM25 retriever, Rank fusion (RRF) | Vector-only RAG fails on SKUs, error codes, policy numbers, and rare proper names; BM25 is what catches those tokens. |
| LLM Guardrail Systems | Intermediate | 15 min | Input classifiers, Output filters, Policy engine | Guardrails are a policy engine plus filters, not a single safety prompt; rules that live only in natural language are not enforceable. |
| LLM Inference and Serving Systems | Advanced | 16 min | Continuous batcher, Paged KV cache, Quantization layer | Continuous batching, not static batches, is what makes interactive serving efficient: sequences join and leave the batch as they finish. |
| LLM Observability and Tracing | Intermediate | 16 min | Trace model, Span collector, PII redactor | The trace is the unit of work: one user request, nested spans for retrieval, tools, model, and filters, with a stable trace ID. |
| MCP Tool Gateway Systems | Advanced | 16 min | Server registry, Delegated auth, Policy / allow-list | Agents connect to the gateway, not to a zoo of MCP servers; the gateway is what authenticates, authorizes, and logs. |
| Model Routing and Fallback Systems | Intermediate | 15 min | Request classifier, Model catalog, Canary lane | Route by task difficulty and risk, not by a single default frontier model; extraction and classification rarely need the expensive tier. |
| PEFT and Fine-Tuning Pipelines | Advanced | 16 min | Dataset registry, PEFT trainer (LoRA/QLoRA), Eval harness | Fine-tune for behavior, format, and domain language; use RAG for facts that change. Most “train on our docs” requests are retrieval problems. |
| Permissioned Retrieval Systems | Advanced | 16 min | IdP token resolver, Document ACL index, Query-time filter | Filter at query time against live IdP groups and document ACLs; baking permissions into the index as the only control goes stale the hour access changes. |
| Prompt Injection Defense Systems | Advanced | 15 min | Trust boundaries, Dual-channel prompts, Tool policy engine | The core rule is a trust boundary: retrieved pages, emails, and tickets are untrusted data; only signed system prompts and tool results you issued are trusted. |
| RAG Architecture | Intermediate | 16 min | Ingest pipeline, Hybrid index, Retriever | Retrieval quality dominates model choice: the generator cannot recover from the wrong passages, a stale index, or an empty result set. |
| Realtime Voice AI Systems | Advanced | 16 min | Telephony media plane, Streaming STT, Turn-taking / VAD | Time-to-first-audio around 800ms (and turns under ~1.2s) is the product constraint; quality work after that is wasted if the call already feels laggy. |
| AI Red-Teaming Systems | Advanced | 16 min | Attack generator, Target harness, Severity classifier | Red team the assembled system (prompt, tools, retrieval, guardrails), not the base model in a vacuum; most production failures are integration failures. |
| Structured Generation Systems | Intermediate | 16 min | Schema compiler, Constrained decoder, Repair loop | Constrained decoding (grammar or schema-guided sampling) makes invalid tokens impossible; prompting for JSON does not. |
| Synthetic Data Generation Systems | Intermediate | 15 min | Seed and schema, Generator models, Filter / critic | Start from real seeds (traces, tickets, documents, expert templates); unconstrained generation produces fluent garbage with hidden duplicates. |
Questions teams ask before they build
What is an AI system, as opposed to a model?
A model produces a single output. An AI system is everything around it that makes that output dependable in production: retrieval, tool access, orchestration between agents, evaluation, guardrails, and rollout control. Teams almost never fail because the model is weak — they fail because the system around it is missing.
How soon can AI systems work start?
Typically within a week or two of a scope being agreed. The first delivery is deliberately a small batch so you can check the output against your expectations before volume ramps.
What do you need from our team?
One process owner who knows the workflow, one engineer with access to the systems involved, and a weekly 45-minute review. No standing committee, and no requirement for an ML specialist on your side.
Who owns the output and the data?
You do. Datasets, labels, weights, evaluation suites and runbooks are yours and are handed over at the end. Your data trains your models only, with zero-retention provider settings by default.
Can you scale volume up quickly if we need it?
Yes, and the quality bar holds because the rubric and gold set are already agreed by that point. Ramping is a staffing question, not a re-scoping one, so it usually takes days rather than a new engagement.