Added by: @AIML-engineer
Date added: 2026-03-27
Technologies: LLM APIs (OpenAI, Anthropic, open-source via vLLM/llama.cpp), sampling parameters (temperature, top-p, top-k), prompt engineering tools, A/B testing frameworks
Description
Temperature is often described as a "creativity dial" for LLMs — crank it up for wild ideas, turn it down for factual accuracy. But recent research shows this mental model is wrong. A 2024 study found that temperature's influence on creativity is "far more nuanced and weak" than commonly claimed, with only weak positive correlation to novelty and moderate negative correlation to coherence. This session demystifies what temperature actually does, busts the myths that lead to misconfigured production systems, and gives attendees a practical framework for choosing the right setting for their use case — whether that's a customer support bot, a code generator, or a brainstorming assistant. We'll cover the math (without getting lost in it), run live demos showing how the same prompt behaves at different temperatures, and walk away with a decision matrix attendees can apply immediately.
Technologies Involved
- LLM APIs — OpenAI, Anthropic, open-source models via vLLM/llama.cpp
- Sampling parameters — temperature, top-p, top-k, frequency/presence penalties
- Softmax function and probability distributions
- A/B testing frameworks for prompt evaluation
- Prompt engineering tools — PromptLayer, promptfoo
Benefit to the Audience
Attendees will leave with a practical decision matrix matching temperature ranges to common enterprise use cases, understand why temperature=0 doesn't guarantee determinism, and know how to systematically A/B test temperature settings for their specific applications. They'll also gain research-backed insights: temperature shows weak correlation with novelty and moderate negative correlation with coherence, meaning higher temperature often trades coherence for perceived creativity rather than delivering genuine creative value.
Outline
- Opening hook: "Same prompt, three different personalities" — Live demo submitting an identical prompt at temperatures 0.1, 0.7, and 1.5. Audience predicts what will change (length? tone? accuracy?). Reveal results and pose the central question: what's actually happening under the hood?
- Under the hood: the mathematics of temperature — Visual walkthrough of how temperature scales logits before softmax, flattening or sharpening the probability curve. Show a concrete example: "The cat sat on the..." with candidate next tokens and how their probabilities shift at different temperatures. Key insight: temperature redistributes probability mass — it doesn't "unlock" hidden model knowledge.
- The temperature spectrum: from robot to raconteur — Walk through three zones: low (0.0–0.3) for factual extraction and code; medium (0.4–0.7) for balanced conversation and summarization; high (0.8–1.5+) for brainstorming and creative writing. For each zone, show example outputs for the same prompt and discuss appropriate use cases and risks.
- Myth busters: what temperature isn't — Three myths, research-backed.
- Myth 1: "Temperature = creativity" — cite the 2024 ICCC study showing weak correlation with novelty, moderate negative correlation with coherence.
- Myth 2: "Temperature = 0 means deterministic" — explain why GPU floating-point precision and provider-side variation still introduce randomness.
- Myth 3: "Higher temperature = better for all creative tasks" — show the novelty-coherence trade-off with examples.
- Hands-on lab: find your optimal temperature — Teams receive different enterprise scenarios (customer support bot, knowledge base query, marketing copy generator, code documentation). Each team tests multiple temperatures on the same prompts, scores outputs against defined metrics, and reports what worked. Facilitate discussion on why optimal settings differ by use case.
- Beyond temperature: the full control panel — Brief introduction to complementary parameters: top-p (nucleus sampling), top-k, frequency/presence penalties. Explain the golden rule: adjust temperature or top-p, not both simultaneously. Provide recommended combinations for common tasks.
- Decision framework and takeaways — Present a one-page reference card mapping use cases to temperature ranges. Reiterate key research insights. Close with the main takeaway: temperature is a randomness control, not a creativity dial — test systematically for your specific application.
Q&A — Open floor.
Full Proposal
The deeper idea behind this session is that most practitioners are misconfiguring LLMs because they've absorbed a simplified mental model: "high temperature = creative, low temperature = precise." This seems intuitive but leads to real problems — customer support bots that hallucinate policies at temperature 0.8, or brainstorming tools that produce incoherent nonsense at temperature 1.5.
What makes this session valuable for AI Weekly is the combination of recent research with immediately applicable frameworks. The 2024 ICCC paper "Is Temperature the Creativity Parameter of Large Language Models?" rigorously tested the creativity assumption and found it wanting — temperature shows weak positive correlation with novelty and moderate negative correlation with coherence. A 2025 study "Exploring the Impact of Temperature on Large Language Models: Hot or Cold?" systematically evaluated temperature 0–2 across six capabilities, finding optimal temperatures vary by task and model size. And a 2026 Nature paper on divergent creativity showed temperature increases raised DAT scores but also highlighted the trade-offs. These findings directly contradict the "creativity dial" intuition.
The live demo is the session's anchor — watching the same prompt produce materially different outputs at different temperatures makes the concept tangible. The audience prediction game at the start engages people immediately. The hands-on lab with enterprise scenarios ensures everyone leaves having done the work, not just heard about it.
One thing to be deliberate about: temperature interacts with model size, prompt structure, and other sampling parameters. There's no universal "right" setting. The session frames temperature selection as an optimization problem requiring systematic testing, not a rule to memorize. This keeps expectations honest and positions attendees to actually improve their systems rather than cargo-culting magic numbers.
The session should close with everyone able to look at their own LLM applications and make informed decisions about temperature — or, more importantly, set up the A/B tests to find what actually works for their specific use case.
Added by: @AIML-engineer
Date added: 2026-03-27
Technologies: LLM APIs (OpenAI, Anthropic, open-source via vLLM/llama.cpp), sampling parameters (temperature, top-p, top-k), prompt engineering tools, A/B testing frameworks
Description
Temperature is often described as a "creativity dial" for LLMs — crank it up for wild ideas, turn it down for factual accuracy. But recent research shows this mental model is wrong. A 2024 study found that temperature's influence on creativity is "far more nuanced and weak" than commonly claimed, with only weak positive correlation to novelty and moderate negative correlation to coherence. This session demystifies what temperature actually does, busts the myths that lead to misconfigured production systems, and gives attendees a practical framework for choosing the right setting for their use case — whether that's a customer support bot, a code generator, or a brainstorming assistant. We'll cover the math (without getting lost in it), run live demos showing how the same prompt behaves at different temperatures, and walk away with a decision matrix attendees can apply immediately.
Technologies Involved
Benefit to the Audience
Attendees will leave with a practical decision matrix matching temperature ranges to common enterprise use cases, understand why temperature=0 doesn't guarantee determinism, and know how to systematically A/B test temperature settings for their specific applications. They'll also gain research-backed insights: temperature shows weak correlation with novelty and moderate negative correlation with coherence, meaning higher temperature often trades coherence for perceived creativity rather than delivering genuine creative value.
Outline
Q&A — Open floor.
Full Proposal
The deeper idea behind this session is that most practitioners are misconfiguring LLMs because they've absorbed a simplified mental model: "high temperature = creative, low temperature = precise." This seems intuitive but leads to real problems — customer support bots that hallucinate policies at temperature 0.8, or brainstorming tools that produce incoherent nonsense at temperature 1.5.
What makes this session valuable for AI Weekly is the combination of recent research with immediately applicable frameworks. The 2024 ICCC paper "Is Temperature the Creativity Parameter of Large Language Models?" rigorously tested the creativity assumption and found it wanting — temperature shows weak positive correlation with novelty and moderate negative correlation with coherence. A 2025 study "Exploring the Impact of Temperature on Large Language Models: Hot or Cold?" systematically evaluated temperature 0–2 across six capabilities, finding optimal temperatures vary by task and model size. And a 2026 Nature paper on divergent creativity showed temperature increases raised DAT scores but also highlighted the trade-offs. These findings directly contradict the "creativity dial" intuition.
The live demo is the session's anchor — watching the same prompt produce materially different outputs at different temperatures makes the concept tangible. The audience prediction game at the start engages people immediately. The hands-on lab with enterprise scenarios ensures everyone leaves having done the work, not just heard about it.
One thing to be deliberate about: temperature interacts with model size, prompt structure, and other sampling parameters. There's no universal "right" setting. The session frames temperature selection as an optimization problem requiring systematic testing, not a rule to memorize. This keeps expectations honest and positions attendees to actually improve their systems rather than cargo-culting magic numbers.
The session should close with everyone able to look at their own LLM applications and make informed decisions about temperature — or, more importantly, set up the A/B tests to find what actually works for their specific use case.