https://agensphere.com/journal Engineering notes, teardowns and decision records from Agensphere: how we build AI systems that hold up in production, layer by layer. en Tue, 06 Oct 2026 09:00:00 GMT https://agensphere.com/journal/the-owned-intelligence-layer https://agensphere.com/journal/the-owned-intelligence-layer Tue, 06 Oct 2026 09:00:00 GMT NOTE An enterprise plan gives every employee a powerful model. It does not route work to the right model, know your business, enforce your data rules, record what happened or stay yours when the market moves. That needs an enterprise AI platform you own, and it is what Agensphere builds. https://agensphere.com/journal/ai-audit-trail https://agensphere.com/journal/ai-audit-trail Tue, 06 Oct 2026 09:00:00 GMT NOTE A customer disputes a decision, a regulator asks how it was made, or an agent did something unexpected overnight. Without an AI audit trail showing which model, prompt, context and tools produced the outcome, you cannot defend it, fix it or learn from it. https://agensphere.com/journal/why-ai-pilots-never-reach-production https://agensphere.com/journal/why-ai-pilots-never-reach-production Tue, 06 Oct 2026 09:00:00 GMT NOTE The demo impressed everyone, the pilot ran for three months, and then nothing shipped. AI pilots stall on the way to production for predictable reasons: no measure of success, no owner, no integration with real systems, and no answer for risk. Each one can be designed out from day one. https://agensphere.com/journal/avoid-llm-vendor-lock-in https://agensphere.com/journal/avoid-llm-vendor-lock-in Tue, 06 Oct 2026 09:00:00 GMT DECISION Models are deprecated, repriced and overtaken every few months. When provider-specific calls are scattered through your code, every change is a migration: that is LLM vendor lock-in. One thin seam, with your own prompts and evals behind it, turns switching models into a configuration change. https://agensphere.com/journal/shadow-ai-every-team-buys-its-own-model https://agensphere.com/journal/shadow-ai-every-team-buys-its-own-model Tue, 06 Oct 2026 09:00:00 GMT NOTE Shadow AI is what happens when marketing has one AI subscription, support another and engineering three. Each team solved its own problem, and the company now pays several times for the same capability with no shared context, no shared controls and no view of the total. https://agensphere.com/journal/llm-data-compliance-where-your-data-goes https://agensphere.com/journal/llm-data-compliance-where-your-data-goes Tue, 06 Oct 2026 09:00:00 GMT NOTE Every prompt is a data transfer. Customer records, contracts and employee details leave your systems the moment they are pasted into a model. LLM data privacy starts with a gateway that sees, redacts, routes and records every call. https://agensphere.com/journal/unowned-intelligence https://agensphere.com/journal/unowned-intelligence Tue, 06 Oct 2026 09:00:00 GMT NOTE If your prompts, evaluation data, usage logs and AI workflows live inside a vendor's interface, you are renting a capability you think you are building. AI ownership means the models can be rented while the intelligence around them stays yours. https://agensphere.com/journal/out-of-context-enterprise-ai https://agensphere.com/journal/out-of-context-enterprise-ai Tue, 06 Oct 2026 09:00:00 GMT NOTE A general model knows the internet, not your pricing rules, your customers or last week's policy change. Without a deliberate context layer, enterprise AI gives fluent, generic answers that are wrong for your company. https://agensphere.com/journal/enterprise-ai-paying-frontier-prices-for-routine-work https://agensphere.com/journal/enterprise-ai-paying-frontier-prices-for-routine-work Tue, 06 Oct 2026 09:00:00 GMT NOTE Most enterprise AI traffic is classification, extraction and short summaries. Sending all of it to the most expensive model is the most common way companies overpay for AI. LLM routing, matching each task to the cheapest model that passes your eval, fixes it. https://agensphere.com/journal/agent-memory-planning-and-stopping https://agensphere.com/journal/agent-memory-planning-and-stopping Tue, 06 Oct 2026 09:00:00 GMT GUIDE A basic agent remembers only its message list, plans one step at a time and stops at an arbitrary limit. Real agents manage their memory, plan when the task is long, and stop on purpose. Here is how each part works, and where to go from here. https://agensphere.com/journal/how-ai-agents-work https://agensphere.com/journal/how-ai-agents-work Tue, 06 Oct 2026 09:00:00 GMT GUIDE An AI agent is a model that can call tools, running in a loop until a task is done. That is the whole idea. This walkthrough builds one from scratch, then shows when a fixed workflow is the better choice. https://agensphere.com/journal/how-tool-calling-works https://agensphere.com/journal/how-tool-calling-works Tue, 06 Oct 2026 09:00:00 GMT GUIDE Tool calling, also called function calling, lets a model request an action by returning a structured call that your code executes. Understanding that division of labour is the key to understanding agents, and to keeping them safe. https://agensphere.com/journal/how-rag-works-step-by-step https://agensphere.com/journal/how-rag-works-step-by-step Tue, 06 Oct 2026 09:00:00 GMT GUIDE Retrieval-augmented generation (RAG) answers questions from your own documents by finding the relevant passages first and handing them to the model. Two pipelines, one for indexing and one for answering, and every quality problem lives in one of their steps. https://agensphere.com/journal/why-llms-hallucinate https://agensphere.com/journal/why-llms-hallucinate Tue, 06 Oct 2026 09:00:00 GMT GUIDE LLMs hallucinate because a model generates the most plausible continuation, not the true one. When the facts are missing or blurry in its weights, plausible and true come apart. Grounding, verification and permission to say “I don't know” reduce it far more than prompts asking for accuracy. https://agensphere.com/journal/sampling-temperature-top-p-explained https://agensphere.com/journal/sampling-temperature-top-p-explained Tue, 06 Oct 2026 09:00:00 GMT GUIDE A model does not output an answer. It outputs a probability for every possible next token, and a sampling step picks one. LLM temperature and top-p control that pick, and choosing them well is part of engineering a reliable system. https://agensphere.com/journal/how-llms-are-trained https://agensphere.com/journal/how-llms-are-trained Tue, 06 Oct 2026 09:00:00 GMT GUIDE LLMs are trained in three stages: they learn language and facts by predicting trillions of next tokens, learn to follow instructions from curated examples, and learn which answers people prefer from comparisons. Each stage explains a different behaviour you see in production. https://agensphere.com/journal/attention-and-the-transformer-explained https://agensphere.com/journal/attention-and-the-transformer-explained Tue, 06 Oct 2026 09:00:00 GMT GUIDE Attention is how each token decides which earlier tokens matter to it. Stack attention with a small feed-forward network, repeat it dozens of times, and you have the transformer that powers every modern LLM. https://agensphere.com/journal/tokens-and-embeddings-how-text-becomes-numbers https://agensphere.com/journal/tokens-and-embeddings-how-text-becomes-numbers Tue, 06 Oct 2026 09:00:00 GMT GUIDE A model never sees letters. It sees token IDs, and then vectors. Knowing what a token is in an LLM explains pricing, context limits and odd failures with spelling and numbers; embeddings explain how a model represents meaning. https://agensphere.com/journal/how-a-large-language-model-works https://agensphere.com/journal/how-a-large-language-model-works Tue, 06 Oct 2026 09:00:00 GMT GUIDE A large language model does one thing: given some text, it predicts the next token. Everything else, from chat to code to agents, is built on repeating that step. This walkthrough shows how large language models work by following one request from your keyboard to the reply. https://agensphere.com/journal/put-the-model-inside-the-workflow https://agensphere.com/journal/put-the-model-inside-the-workflow Tue, 06 Oct 2026 09:00:00 GMT NOTE Most AI features fail on adoption, not accuracy. Good AI product design puts intelligence at the exact step where a decision is made, instead of in a chat panel beside the real work. https://agensphere.com/journal/cut-llm-cost-in-the-right-order https://agensphere.com/journal/cut-llm-cost-in-the-right-order Tue, 06 Oct 2026 09:00:00 GMT NOTE Most attempts to reduce LLM cost start with prompt golf and end with a quality regression. Measure first, then take the structural wins: prompt caching, model routing and batch processing, before rewriting a single prompt. https://agensphere.com/journal/human-approval-is-a-durable-wait https://agensphere.com/journal/human-approval-is-a-durable-wait Tue, 06 Oct 2026 09:00:00 GMT DECISION Human-in-the-loop approval should not hold a thread, a socket or a model session while a manager decides. Persist the run, send the request, and resume from the log when the answer arrives, hours or days later. https://agensphere.com/journal/mcp-turns-tools-into-a-protocol https://agensphere.com/journal/mcp-turns-tools-into-a-protocol Tue, 06 Oct 2026 09:00:00 GMT NOTE The Model Context Protocol standardises how models discover and call tools, read resources and use prompt templates. That removes integration glue, and moves the hard questions to permissions, versioning and trust. https://agensphere.com/journal/prompt-injection-is-an-input-trust-problem https://agensphere.com/journal/prompt-injection-is-an-input-trust-problem Tue, 06 Oct 2026 09:00:00 GMT NOTE No system prompt reliably stops prompt injection: a model can follow instructions hidden in an email or a web page. Limit what untrusted text can cause instead: separate data from instructions, scope tools to the user, and gate every risky action. https://agensphere.com/journal/validate-structured-outputs-at-the-boundary https://agensphere.com/journal/validate-structured-outputs-at-the-boundary Tue, 06 Oct 2026 09:00:00 GMT DECISION JSON mode guarantees the shape of LLM structured outputs, not whether they are true. Validate the schema and the business rules where model output enters your system, and feed the exact error back for one bounded retry. https://agensphere.com/journal/hybrid-search-beats-pure-vector-search https://agensphere.com/journal/hybrid-search-beats-pure-vector-search Tue, 06 Oct 2026 09:00:00 GMT NOTE Embeddings are good at meaning and bad at exact strings: order numbers, error codes and product names need keyword search. Hybrid search runs both, merges the results with reciprocal rank fusion, and re-ranks the top of the list. https://agensphere.com/journal/chunk-by-structure-not-token-count https://agensphere.com/journal/chunk-by-structure-not-token-count Tue, 06 Oct 2026 09:00:00 GMT NOTE Fixed 512-token chunks split tables mid-row and separate claims from the context that makes them true. A RAG chunking strategy that follows headings, sections and lists is the cheapest retrieval improvement most systems never make. https://agensphere.com/journal/a-context-window-is-not-memory https://agensphere.com/journal/a-context-window-is-not-memory Tue, 06 Oct 2026 09:00:00 GMT NOTE A bigger context window lets a model read more on each request. It does not give an LLM memory. Treating the two as the same thing is how agents get slow, expensive and forgetful at once. https://agensphere.com/journal/trace-and-cost-every-model-call https://agensphere.com/journal/trace-and-cost-every-model-call Tue, 06 Oct 2026 09:00:00 GMT DECISION If a request gets slow or expensive and you cannot say which step did it, you cannot fix it. LLM observability starts with one span per model and tool call, with tokens and cost attached: cheap to add early, painful to add late. https://agensphere.com/journal/ship-a-prompt-change-with-an-eval https://agensphere.com/journal/ship-a-prompt-change-with-an-eval Tue, 06 Oct 2026 09:00:00 GMT NOTE A prompt is code that runs on someone else's computer and changes behaviour without a compiler error. LLM evaluation gives it a test set, a score and a review gate, so model upgrades stop being surprises. https://agensphere.com/journal/filter-by-permission-before-you-rank https://agensphere.com/journal/filter-by-permission-before-you-rank Tue, 06 Oct 2026 09:00:00 GMT NOTE Retrieval that ranks first and checks access later is a RAG security hole: it leaks data through snippets, scores and empty answers. Put the permission filter inside the search query and the whole class of bug goes away. https://agensphere.com/journal/every-agent-tool-needs-an-effect-class https://agensphere.com/journal/every-agent-tool-needs-an-effect-class Tue, 06 Oct 2026 09:00:00 GMT DECISION If you cannot say what happens when a tool call runs twice, the tool is not ready to give to a model. Four questions, asked before an AI agent tool ships, decide how the runtime is allowed to retry it. https://agensphere.com/journal/keel-make-the-run-deterministic https://agensphere.com/journal/keel-make-the-run-deterministic Tue, 06 Oct 2026 09:00:00 GMT TEARDOWN A refund agent crashed after Stripe moved the money. KEEL is the durable execution runtime we are building so the answer to “how many refunds?” is always one, and so any run can be replayed or forked onto another model.