Mastering Temperature and Top_p in ChatGPT API

In AI text generation models like ChatGPT, temperature and top_p are two parameters that control randomness and diversity in responses. They affect how tokens (words or phrases) are selected from the model's probability distribution. Here’s the difference:

1. Temperature (𝛕)

• Controls randomness by scaling the probability distribution.


• A higher temperature (e.g., 1.0 - 1.5) makes responses more diverse and creative, but also more random and unpredictable.


• A lower temperature (e.g., 0.1 - 0.3) makes responses more focused, deterministic, and repetitive.


• 
  Example:
  
  
  
  
  • 
    High Temperature (1.2) → "The sky is a canvas of endless hues, shifting from pink to deep indigo."
  
  
  • 
    Low Temperature (0.2) → "The sky is blue."
  
  
  
  
  
  

2. Top-P (Nucleus Sampling)

• Controls randomness by restricting token selection to a subset of the most likely tokens.


• Instead of considering all possible words, top_p ensures only the most probable ones (whose cumulative probability adds up to p) are considered.


• 
  Top-p = 1.0 → No restriction (like greedy sampling).


• 
  Lower top-p (e.g., 0.3 - 0.5) → Limits the choice to fewer words, making output more focused and deterministic.


• 
  Example:
  
  
  
  
  • 
    High Top-P (0.9 - 1.0) → "The concert was an electrifying spectacle, filled with vibrant energy."
  
  
  • 
    Low Top-P (0.3) → "The concert was enjoyable."
  
  
  
  
  
  

When to Use What?

• 
  For highly creative outputs → Use higher temperature (e.g., 0.8 - 1.2) and/or higher top_p.


• 
  For factual, precise responses → Use lower temperature (e.g., 0.2 - 0.5) and/or lower top_p.

💡 Tip: You can tweak either temperature or top_p, but not both aggressively at the same time, as they both influence randomness.

Here's a TypeScript example using OpenAI's API to demonstrate the difference between temperature and top_p in text generation.

Setup:

1. Install openai package:
   

   npm install openai dotenv

1. Create a .env file and add your API key:
   

   OPENAI_API_KEY=your_api_key_here

1. Use the following TypeScript code:

TypeScript Code: Temperature vs. Top-P

import { OpenAI } from 'openai';

import dotenv from "dotenv";



dotenv.config();



const openai = new OpenAI (

  {

    apiKey: process.env.OPENAI_API_KEY,

  }

);



async function generateText(

  prompt: string,

  temperature: number = 1.0,

  top_p: number = 1.0

): Promise<void> {

  try {

    const response = await this.openai.chat.completions.create({

      model: "gpt-4",

      messages: [{ role: "user", content: prompt }],

      temperature: temperature, // Controls randomness

      top_p: top_p, // Controls diversity

      max_tokens: 100,

    });



    console.log(`\n🔥 Temperature: ${temperature}, 🎯 Top-P: ${top_p}`);

    console.log(response.data.choices[0].message?.content);

  } catch (error) {

    console.error("Error:", error);

  }

}



const prompt = "Describe a futuristic city in 2050.";



// High temperature → More creative, unpredictable

generateText(prompt, 1.2, 1.0);



// Low temperature → More deterministic, factual

generateText(prompt, 0.2, 1.0);



// High top_p → More diverse words considered

generateText(prompt, 1.0, 0.9);



// Low top_p → Very focused choices, likely repetitive

generateText(prompt, 1.0, 0.3);

Expected Results:

High Temperature (1.2)

• "The city of 2050 is a dazzling wonderland of neon-lit floating districts, sky gardens that recycle air, and AI-powered robots running entire industries."

Low Temperature (0.2)

• "In 2050, cities have more electric vehicles, AI automation, and smart infrastructure for sustainability."

High Top-P (0.9)

• "By 2050, cities are vibrant with self-repairing roads, quantum-powered networks, and holographic entertainment hubs."

Low Top-P (0.3)

• "Cities in 2050 are automated, green, and efficient."

Key Takeaways:

• 
  Higher Temperature → More creativity, but less consistency.


• 
  Lower Temperature → More deterministic and factual.


• 
  Higher Top-P → More diverse wording.


• 
  Lower Top-P → More predictable and repetitive responses.