How to Calculate LLM API Costs: Getting the Token Math Right
When building LLM applications, the most frequently asked question is, "How much will this feature cost per day?" The answer hinges on one variable: how many to
SmallFireDragon Lab
AI Science
Making complex AI concepts understandable for humans
How to Break Down LLM Costs: Understanding Model Cascading
When many teams discuss cost reduction, their immediate reaction is often to switch to smaller models or simply cut budgets. Both approaches are rather blunt. A
1 Token ≈ How Many Characters: The Invisible Unit in LLM Billing
If you’ve ever calculated your AI API bills, you’ll notice that the unit of measurement is neither "characters" nor "words," but rather tokens. The first questi
It Read Tens of Thousands of Words, So Why Did It Still Miss Key Information: Lost in the Middle
You’ve likely encountered this scenario: You feed a 30-page requirements document into a large language model (LLM), asking it to summarize the content and list
Regression Testing for AI Systems: Engineering Practices for LLM Evaluation
In traditional software systems, you modify a function, run the unit tests, and if everything passes (all green), you commit. This approach fails in LLM applica
Depart Only When Requests Are Uneven: Continuous Batching in Inference Services
Last week, a reader asked: "Why do others handle three times the request volume on the same GPU? The model, quantization, and hardware are identical." The gap o
Why Is the First Token of Large Models Always the Slowest?
Next time you send a long document to a large language model (LLM), pay attention to an experience that isn’t rare but is rarely explained: it takes several sec
Structured Output Is Not About "Making the Model Obedient," It’s Constraint Engineering During Decoder Inference
Many production teams face the same problem: the model’s responses are fluent, but the returned JSON occasionally misses a bracket, has an extra comma, or conta
Why the Same Prompt Succeeded This Time but Failed Last Time: Non-Determinism in LLM Inference and Engineering Countermeasures
In production environments, the most tormenting bugs are often not error messages, but "intermittent reliability": the same prompt sent five times yields two co