Explains interacting with a deployed generative AI model, prompt types, generation parameter tuning, playground to API mapping, and secure programmatic usage examples.
In the previous lesson we deployed a generative AI model in Microsoft Foundry, opened it in the interactive environment, and reviewed generated code. In this article we’ll take the next step: show how to interact programmatically with that model, explain how prompts influence responses, and describe how generation parameters (temperature, top P, max tokens, stop sequences) shape the output.What you’ll learn
Revisit the interactive environment and the two kinds of prompts you can provide.
Map the interactive Playground fields to API/SDK call parameters.
Understand key generation parameters and how to tune them.
See examples that include system instructions and parameter settings in code.
Interactive environment — recapThe interactive environment is a sandbox for exploring and evaluating a deployed generative model before embedding it into a production application. The left pane is where you configure the model’s instructions, tools, and generation settings; the right pane provides a chat-like interface for trying prompts and reading responses.Two prompt types matter:
System prompt: sets the model’s role and persistent behavior (for example, “You are an assistant that helps people find information about the history of computing.”).
User prompt: the one-off request or question (for example, “What were Alan Turing’s contributions to AI?”).
Together the system and user prompts establish context and the specific task for each call. Keep system instructions stable when you need consistent agent behavior, and let user prompts vary per request.
Design tip: when building an application, decide which behaviors must remain consistent (put those in the system prompt) and which should be supplied per request via user prompts. This separation makes prompts easier to manage and more predictable in production.Controlling generation with parametersPrompts define what the model should do; parameters control how it produces the output. The most commonly used generation parameters are summarized below.
Parameter
Purpose
Typical use
Temperature
Controls randomness and diversity. Lower = more deterministic; higher = more creative.
temperature: 0.0–0.3 for support scripts, 0.6–1.0 for creative tasks.
Top P
Nucleus sampling — limits token selection to top cumulative probability. Alternative to temperature.
Use instead of or alongside temperature to shape diversity.
Max output tokens
Limits the length of responses (number of tokens).
Prevents runaway responses; use to cap cost and output size.
Stop sequences
Tells the model to stop when a specific sequence appears.
Useful for structured outputs or when embedding answers into prompts.
Think of prompts as the “what” and parameters as the “how.” Tune both to match your application’s needs.
Why temperature matters — quick exampleUsing the same system and user prompts but changing only the temperature produces noticeably different replies.User input:
“The package arrived two days late, but the product quality was excellent.”At temperature 0.2 (more predictable):
We're sorry your parcel arrived two days late.We're glad the keyboard met your expectations.
At temperature 0.9 (more varied/creative):
Two days late – not our finest hour! Delighted the keyboard won you over, though.
Higher temperature increases variation; lower temperature produces consistent, script-like responses suitable for customer support. Choose the range that matches your desired tone and reliability.Programmatic usage: Playground → API/SDK mappingAfter you’ve validated behavior in the interactive environment, copy the generated example code to your application and replace secrets with secure credentials. The Playground fields map to API/SDK parameters as shown:
Playground field
API/SDK field (example)
System prompt
instructions or system_message
User prompt
input
Temperature
temperature
Max output tokens
max_output_tokens
Stop sequences
stop
Basic example (API key authentication, Python)
Replace base_url and api_key with your Foundry endpoint and API key.
Do not hard-code secrets in production — use environment variables or a secrets store.
from openai import OpenAIclient = OpenAI( base_url="https://your-foundry-endpoint", api_key="YOUR_API_KEY")response = client.responses.create( model="your-model-deployment-name", input="What is the Turing Test?")print(response.output[0])
Never commit API keys in code. Use environment variables, a Key Vault, or managed identity (Azure AD) for production deployments.
Including system instructions and generation parameters in codeTo keep agent behavior consistent across calls, include a system instruction and set generation parameters such as temperature and max_output_tokens. Below is a concise pattern that mirrors the Playground call to the Responses API.
from openai import OpenAIimport osendpoint = os.environ.get("FOUNDRY_ENDPOINT", "https://your-foundry-endpoint")deployment_name = "gpt-4.1-mini"client = OpenAI( base_url=endpoint, api_key=os.environ.get("FOUNDRY_API_KEY") # Use env var or secret store in production)system_message = "You are a creative copywriter for quirky brands. Reply with one tagline only, no quotes."response = client.responses.create( model=deployment_name, instructions=system_message, input="What is the capital of France?", temperature=0.7, max_output_tokens=40)# SDKs may expose different accessors; adjust if neededprint(response.output[0])
Parameter tuning examples:
temperature = 0.0 -> “Paris.”
temperature = 0.5 -> “Paris — the city of light and culture.”
temperature = 0.9 -> “Paris — where croissants meet couture and every street hums a story.”
Deploying base models in Foundry and calling themWhen you click “Deploy a base model” in Foundry, you’re presented with a catalog of models (for example: gpt-4.1-mini). Prefer current, supported models for production; older/legacy models may be marked end-of-life.Once deployed, open the model in the interactive environment to:
Configure system instructions.
Enable or disable tools (web search, code interpreter).
Attach knowledge sources (file uploads, knowledge bases, MCP).
Adding tools such as web search enables real‑time information retrieval; disabling them constrains the model to its internal knowledge.
Example: tools in the interactive environment (web search)If web search is disabled, asking “What’s the current temperature in London?” will prompt the model to indicate it lacks real-time data and suggest checking a weather site or app. If web search is enabled, the model can query the internet and return a sourced, up-to-date answer (for example, citing an external site such as https://www.accuweather.com/).
The interactive environment shows generated code in multiple languages (Python, C#, etc.). Copy the example that matches your stack and replace secrets with secure credentials.Final consolidated Python example (API key pattern)
from openai import OpenAIimport osendpoint = os.environ.get("FOUNDRY_ENDPOINT", "https://your-foundry-endpoint")deployment_name = "gpt-4.1-mini"client = OpenAI( base_url=endpoint, api_key=os.environ.get("FOUNDRY_API_KEY") # use env var or secrets store in production)system_message = "You are a creative copywriter for quirky brands. Reply with one tagline only, no quotes."response = client.responses.create( model=deployment_name, instructions=system_message, input="What is the capital of France?", temperature=0.7, max_output_tokens=40)print(response.output[0])
If you must use Azure AD (managed identity / service principal) instead of API keys, follow the secure credential pattern recommended by your environment and SDK. For example, obtain a token using DefaultAzureCredential and supply it to your client per the provider’s documentation. See Microsoft’s Azure identity docs for guidance: https://learn.microsoft.com/azure/active-directory/.Recap
Use the interactive environment to prototype system + user prompts and tune parameters.
Parameters (temperature, top P, max tokens, stop sequences) shape how text is generated; tune them to match the task (predictability vs. creativity).
Copy working example code from the Playground, and replace inline secrets with secure secrets management before using in production.
Test in the interactive environment, then integrate the same calls via the SDK or REST API in your application.
When designing prompts for applications, separate stable agent behavior (system instructions) from per-request content (user prompts). Tune generation parameters to match the application’s use case (predictable responses for support; higher creativity for marketing or brainstorming).
