Day 06: Why creativity settings are product decisions
How sampling settings shape the user experience, not just the writing style.
Temperature gets described as a creativity dial. Accurate enough, and it hides the thing that matters in a product: temperature decides how much variation your users experience.
That makes it a product decision, not a tuning preference, and it belongs to the task, not to the system.
One global setting is the bug
Most codebases set temperature once, in a config, and every call inherits it. Then the same value governs a marketing-copy generator, a customer-facing answer, and a classifier whose output feeds a database write.
Those three want completely different things. The copy generator benefits from variety; identical suggestions every time are useless. The classifier needs the same input to produce the same label, every time, or your data is quietly non-deterministic. The customer answer sits in between and probably leans consistent, because two users asking the same question and getting materially different answers is a support problem.
A single number cannot serve all three, and the one that gets chosen is usually whatever made the demo feel impressive.
Where randomness bites silently
The dangerous case is not creative output that varies. It is randomness leaking into a step something downstream depends on.
A field that is usually "approved" and occasionally "Approved." A list that is normally five items and sometimes four. A JSON response that once in fifty runs arrives with a sentence in front of it. Each is a parsing failure, a data inconsistency, or an audit problem, and each will be intermittent, which makes it expensive to diagnose.
Low temperature does not guarantee determinism, incidentally. It narrows the distribution; it does not eliminate variation. Where output shape genuinely must hold, constrain the shape as well as the sampling.
Test the spread, not the sample
Run the same input ten times at your chosen setting and look at how far the outputs move apart.
That spread is what users actually experience. One good sample tells you the best case; the spread tells you the product. And it is a fast enough check that there is no excuse for shipping a setting nobody measured.
Closing thought
Creativity settings are product decisions because variance is part of the user experience. Choose per task, write down which tasks tolerate variety and which demand repeatability, and give the defaults an owner, before someone tunes them by feel at 5pm.
Day 6 of 60 Days of Production AI Systems.
Which of your features needs consistency more than creativity, even if the demo looks less impressive for it?
Previous: Day 05: Why vague prompts become vague systems
Next: Day 07: Why confident AI answers still need evidence
Day 6 of 60 · AI Systems Basics (chapter 1 of 10)
