MS Foundry setup¶

This runbook sets up the language model the agent uses: a chat model deployed in Microsoft Foundry (formerly Azure AI Foundry), which the agent calls over HTTPS. Follow the steps in order. Each step explains what you are doing and why, and gives both the portal path and the az CLI commands.

New to Foundry? Read foundry-concepts.md first. It explains resources, deployments, quota, cost and access, and whether to use a shared or a dedicated resource.

Quick path: scripts/setup-foundry.sh does Steps 1, 2 and 4a in one go. It creates the resource group, the Foundry resource and the model deployment, gives you the data-plane role, and prints the .env values. It is safe to re-run: existing parts are skipped, and a missing resource group is created. Networking (Step 5) stays manual. A suggested layout is one resource for non-prod clusters and one for prod (layout):

# ./scripts/setup-foundry.sh <subscription> <resource-group> <foundry-name> <location>
./scripts/setup-foundry.sh "<subscription>" my-foundry-rg my-foundry westeurope
# optional: [DEPLOYMENT_NAME] [MODEL_NAME] [MODEL_VERSION] [SKU_NAME] [CAPACITY] [UPGRADE_OPTION]
#           defaults: k8s-servant gpt-5.4-mini 2026-03-17 DataZoneStandard 50 OnceCurrentVersionExpired

The steps below explain what the script does and how to do each part by hand.

Using an existing shared resource (one your organisation already runs): skip Step 1, create the agent's own deployment in Step 2 (e.g. k8s-servant), and continue from Step 3. Agree any network change (Step 5) with the resource's owners.

Facts and commands were checked against Microsoft Learn and az 2.90 on 2026-10-09. Links to the sources are at the end of each section and in Reference.

Contents¶

1. What you are setting up and why
2. Prerequisites
3. Step 1 – Create the Foundry resource
4. Step 2 – Deploy a chat model
5. Step 3 – Find the endpoint and deployment name
6. Step 4 – Give access
7. Step 5 – Networking
8. Step 6 – Smoke test
9. Cost and limits
10. Content safety and data handling
11. Troubleshooting
12. Reference

1. What you are setting up and why¶

 k8s-servant ──HTTPS + Entra ID token──▶ Foundry resource ──▶ model deployment "gpt-5-4-mini"
 (laptop: az login,                    https://<name>.openai.azure.com/openai/v1/
  cluster: Workload Identity)           (OpenAI-compatible "v1" API)

Why this API. For GPT and other OpenAI-style models the agent uses the standard openai Python SDK with the Responses API (LLM_PROVIDER=foundry), which, unlike Chat Completions, can carry a reasoning model's hidden steps between tool calls (model-providers.md). Claude on Foundry uses the Messages API and partner models with only Chat Completions use openai-compatible (recipes in the same page). Changing the model within a provider later only changes LLM_MODEL.

Why Entra ID. Locally the agent uses your az login session; in the cluster it uses Workload Identity. This avoids long-lived API keys that would have to be stored and rotated.

2. Prerequisites¶

• Azure CLI 2.60 or newer (az version), logged in with az login, on the right subscription (az account set --subscription "<name or id>").
• Permissions on the target resource group:
• Contributor, to create the resource and deployments;
• User Access Administrator or Owner, to assign roles in Step 4.
• A region with the model you want.
• For EU data residency choose an EU region, for example Sweden Central or West Europe. gpt-5.4-mini is offered there as GlobalStandard and DataZoneStandard.
• Check before you start, using the commands in Step 2.
• Quota. New subscriptions have a default tokens-per-minute (TPM) quota per model and region. Raising it is a request in the portal (Foundry → Quota).

Set variables used in the commands below:

RG=my-foundry-rg              # resource group (create it with: az group create -n $RG -l $LOCATION)
LOCATION=westeurope           # or swedencentral
FOUNDRY=my-foundry            # resource name; also used as the custom subdomain, must be globally unique
DEPLOYMENT=gpt-5-4-mini        # name you give the deployment; the agent uses it as LLM_MODEL

3. Step 1: Create the Foundry resource¶

Portal: ai.azure.com, then Create new → Foundry resource. Pick the subscription, resource group, region and name. Alternatively, in the Azure portal go to Create a resource → Foundry.

CLI:

az cognitiveservices account create \
  --name "$FOUNDRY" \
  --resource-group "$RG" \
  --location "$LOCATION" \
  --kind AIServices \
  --sku S0 \
  --custom-domain "$FOUNDRY" \
  --assign-identity \
  --yes

Optional: a project, if you want to use the Foundry portal's playground with a project scope.

az cognitiveservices account project create --name "$FOUNDRY" --resource-group "$RG" \
  --project-name k8s-servant --location "$LOCATION"

Sources: Quickstart: create Foundry resources, Create projects.

4. Step 2: Deploy a chat model¶

Choose a model¶

The agent needs a chat model that is good at tool calling (function calling) and has a large context window: tool outputs such as pod lists and logs are big.

Check what your resource can deploy, and which versions exist:

az cognitiveservices account list-models --name "$FOUNDRY" --resource-group "$RG" \
  --query "[?contains(name,'gpt-5')].{name:name, version:version, skus:skus[].name}" -o table

Choose the deployment type (where prompts are processed)¶

Data at rest stays in the resource's geography for every type. Only processing differs.

Choose the capacity¶

--sku-capacity is in units of 1,000 tokens per minute (TPM). Rough sizing:

• One ask uses about 5–15 model calls. Each call resends the system prompt and the conversation so far, about 10k–40k tokens at the end of a long investigation, so one question totals about 100k–300k tokens over 1–3 minutes.
• An hourly health check is similar. Measured on a test cluster (~600 Deployments, ~1,250 pods): 150k–225k tokens per check, about 10 model calls in 2–3 minutes, with peaks of about 90k tokens in one minute. The log line model call: N prompt + M completion tokens shows the usage of every call.
• With 50 (= 50k TPM) deep checks hit HTTP 429 (the SDK retries, which slows a check down and can still fail). Use 150–200 if the subscription's quota allows it (Foundry portal → Quota). Capacity only limits throughput; you pay for tokens used, not for capacity. Change an existing deployment in the Foundry portal: Models + endpoints → the deployment → Edit → Tokens per Minute Rate Limit.
• Reasoning effort. gpt-5.x models reason only when the request asks for it. The agent sends reasoning_effort = LLM_REASONING_EFFORT (default medium). Without it gpt-5.4-mini answered after 3 shallow tool calls and missed firing alerts.

Deploy¶

Portal: deployments are managed in the Foundry portal, not the Azure portal; the Azure portal's "Model deployments" entry only redirects there. Open ai.azure.com → select the Foundry resource → Models + endpoints → Deploy model → Deploy base model → pick the model, deployment type and capacity, and set the deployment name.

CLI:

az cognitiveservices account deployment create \
  --name "$FOUNDRY" \
  --resource-group "$RG" \
  --deployment-name "$DEPLOYMENT" \
  --model-name gpt-5.4-mini \
  --model-version 2026-03-17 \
  --model-format OpenAI \
  --sku-name DataZoneStandard \
  --sku-capacity 50

A new deployment can take a few minutes before requests to it succeed. Until then you get HTTP 404 DeploymentNotFound.

Version upgrade policy. A deployment pins a model version, but its upgrade policy decides what happens when Microsoft releases a newer one:

With OnceCurrentVersionExpired, upgrading is your decision: deploy the new version on non-prod, test, then change prod (see foundry-concepts.md). Before the retirement date, plan the move.

The CLI cannot set the policy (az cognitiveservices account deployment has no option for it), so the script sets it on the ARM resource, also on an existing deployment:

az resource update \
  --ids "$(az cognitiveservices account show -n "$FOUNDRY" -g "$RG" --query id -o tsv)/deployments/$DEPLOYMENT" \
  --set properties.versionUpgradeOption=OnceCurrentVersionExpired \
  --api-version 2025-06-01

Portal: Foundry portal → Models + endpoints → the deployment → Edit → Version update policy. Check the result with az cognitiveservices account deployment show -n "$FOUNDRY" -g "$RG" --deployment-name "$DEPLOYMENT" --query properties.versionUpgradeOption.

Sources: Deploy models, Deployment types, Quota, Models, Retirements, Upgrade policy.

5. Step 3: Find the endpoint and deployment name¶

Portal: Azure portal → the Foundry resource → Overview (or Resource Management → Keys and Endpoint). The Foundry portal shows the same values on the resource's overview page.

CLI:

az cognitiveservices account show --name "$FOUNDRY" --resource-group "$RG" --query properties.endpoint -o tsv
az cognitiveservices account deployment list --name "$FOUNDRY" --resource-group "$RG" --query "[].name" -o tsv

Set the agent's variables (in .env locally, or Helm values in the cluster):

LLM_PROVIDER=foundry
LLM_ENDPOINT=https://<name>.openai.azure.com/
LLM_MODEL=gpt-5-4-mini
LLM_AUTH=entra

• Use the <name>.openai.azure.com hostname. The v1 API is documented for https://<name>.openai.azure.com/openai/v1/ and https://<name>.services.ai.azure.com/openai/v1/. If properties.endpoint prints a *.cognitiveservices.azure.com URL, build the openai.azure.com one from your custom subdomain.
• You do not need to add /openai/v1/. The agent appends it (see normalize_endpoint in k8s_servant/llm/providers.py). The endpoint must start with https://.
• LLM_MODEL is the deployment name from Step 2, not the model name.

6. Step 4: Give access¶

The agent authenticates with Entra ID. Whoever calls the model needs a data-plane role on the resource. Owner and Contributor are not enough: they manage the resource but cannot call models.

Which role. It depends on the model (Microsoft Learn): - GPT and other OpenAI models: Cognitive Services OpenAI User on the Foundry resource. It is the least-privileged role that can call them, and it grants access to OpenAI models only. - Claude and other non-OpenAI models (DeepSeek, Llama, Grok, partner models): Cognitive Services User. Cognitive Services OpenAI User is not enough for them. scripts/setup-workload-identity.sh assigns it when run with MODEL_ROLE="Cognitive Services User". - Foundry User (formerly Azure AI User) also works for both but is broader, granting all Foundry data actions.

Role assignments take up to about 5 minutes to apply.

FOUNDRY_ID=$(az cognitiveservices account show --name "$FOUNDRY" --resource-group "$RG" --query id -o tsv)

Portal: role assignments are made in the Azure portal: the Foundry resource → Access control (IAM) → Add → Add role assignment → role Cognitive Services OpenAI User (or Cognitive Services User for non-OpenAI models) → select the user or managed identity → Review + assign. As Owner of the resource you can assign it to yourself.

a) You, for local development¶

az role assignment create \
  --assignee "$(az ad signed-in-user show --query id -o tsv)" \
  --role "Cognitive Services OpenAI User" \
  --scope "$FOUNDRY_ID"
az login   # if not logged in; the agent picks up this session through DefaultAzureCredential

b) The agent in the cluster¶

The agent pod authenticates with AKS Workload Identity: a user-assigned managed identity with a federated credential for the agent's ServiceAccount. scripts/setup-workload-identity.sh creates the identity and assigns it this role (set MODEL_ROLE="Cognitive Services User" for Claude and other non-OpenAI models; deployment.md). Done by hand, the assignment is:

az role assignment create \
  --assignee-object-id "<managed identity principalId>" \
  --assignee-principal-type ServicePrincipal \
  --role "Cognitive Services OpenAI User" \
  --scope "$FOUNDRY_ID"

c) API key: local experiments only¶

az cognitiveservices account keys list --name "$FOUNDRY" --resource-group "$RG"
# then: LLM_AUTH=api-key and put <key1> in the file LLM_API_KEY_FILE points to

Keys are discouraged: - they are long-lived shared secrets; - anyone holding one can use the model; - they need rotation.

Recommended: use LLM_AUTH=entra (the agent then refuses to start if a key is also configured) and set disableLocalAuth: true on the resource, so keys cannot be used at all.

Reading keys needs Cognitive Services Contributor; the OpenAI User role cannot see them. Once everyone uses Entra ID, disable key authentication altogether:

az resource update --ids "$FOUNDRY_ID" --set properties.disableLocalAuth=true

Microsoft documents this setting through PowerShell, Bicep/ARM (disableLocalAuth: true) and Azure Policy. The az resource update form above sets the same ARM property. It can take a while to apply; afterwards an old key gets HTTP 401.

Cluster in another tenant than Foundry. If the pod's identity has no role on the Foundry resource (for example because the resource lives in another tenant or subscription that you do not control), use a Foundry API key kept in Key Vault: Workload Identity still reads the key from Key Vault, and LLM_AUTH: api-key uses it. Deliver the key as a Secret or mounted file as in model-providers.md (llmApiKey.existingSecret or llmApiKey.file). In that case do not disable local auth on that resource.

Token scope. The agent requests tokens for https://ai.azure.com/.default, the current scope in Microsoft's documentation. It can be changed with LLM_TOKEN_SCOPE.

Sources: Configure Entra ID, Foundry RBAC, Disable local auth.

7. Step 5: Networking¶

From a laptop. If the resource allows public network access (the default), nothing else is needed.

From the cluster. If the cluster has open outbound internet access, nothing else is needed. If it is private or its egress goes through a firewall, the agent pod can only reach the Foundry resource if one of these is in place:

Option A, the commands.

az network private-endpoint create \
  --name "pe-$FOUNDRY" \
  --resource-group "<network rg>" \
  --vnet-name "<vnet>" --subnet "<subnet>" \
  --private-connection-resource-id "$FOUNDRY_ID" \
  --group-id account \
  --connection-name "pe-$FOUNDRY"

• Private DNS zones, one per hostname you use, each linked to the VNets whose DNS the cluster uses (the hub DNS):
• privatelink.openai.azure.com: required, because the agent uses <name>.openai.azure.com.
• privatelink.services.ai.azure.com and privatelink.cognitiveservices.azure.com: add them if you use those hostnames, for example from the portal.
• Check the result. From inside the cluster, <name>.openai.azure.com must resolve to the private IP of the endpoint.
• Close public access once private access works: in the portal, Networking → Public network access: Disabled.
• Ownership. Private DNS zones and firewall rules often belong to a network team. Request them with the resource ID and the hostname.

Testing from inside the cluster. Once the agent is deployed (deployment.md), run kubectl -n k8s-servant exec deploy/k8s-servant -c agent -- k8s-servant doctor. Its model line proves DNS, network, authentication and deployment in one go.

Sources: Private link for Foundry, Private endpoint DNS, Virtual networks.

8. Step 6: Smoke test¶

With curl (independent of the agent)¶

TOKEN=$(az account get-access-token --scope https://ai.azure.com/.default --query accessToken -o tsv)
curl -sS -X POST "https://$FOUNDRY.openai.azure.com/openai/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d "{\"model\": \"$DEPLOYMENT\", \"messages\": [{\"role\": \"user\", \"content\": \"Reply with OK\"}]}"

• "model" is the deployment name.
• With a key, replace the Authorization header with -H "api-key: $KEY".

With the agent¶

$ set -a; source .env; set +a
$ k8s-servant doctor
[OK  ] config           endpoint https://my-foundry.openai.azure.com/, deployment gpt-5-4-mini
...
[OK  ] model            answered 'OK' in 1.4s

The model check sends one tiny request. If it fails, the line says why; see Troubleshooting.

9. Cost and limits¶

• Billing. You pay per token, with input and output priced separately (per 1M tokens; prices on the Azure OpenAI pricing page). For reasoning models, internal reasoning tokens are billed as output.
• Capacity (TPM) limits throughput only. An idle deployment costs nothing.
• Typical usage. One ask is about 100k–300k tokens. An hourly health check is about 24 such runs a day (fewer when unchanged checks are skipped). Multiply by the per-token price of your model for a daily estimate; mini models keep this small.
• Budget alert. Azure portal → Cost Management → Budgets → create one on the resource group, filtered to the Foundry resource, with an email alert at for example 80%. A budget alerts but does not stop spending.

Sources: Manage costs.

10. Content safety and data handling¶

• Content filter. Every deployment has a default guardrail:
• hate, sexual, violence and self-harm are filtered at Medium on prompts and completions;
• Prompt Shields (jailbreak) and protected-material checks are on.

A filtered request returns HTTP 400 with content_filter. Normal cluster data does not trigger it. - Your data. - Prompts and completions are not used to train models and are not shared with OpenAI. The models keep no state between requests. - Abuse monitoring may store flagged content for review in the resource's geography (for an EU resource, reviewers are in the EEA). - What the agent sends. The system prompt (rules, environment description, skill catalog) and the outputs of its tools: object names, statuses, events and log lines. Secrets are never sent: - the kubectl tool refuses them; - the MCP server is configured to deny them; - in the cluster, RBAC forbids them; - the MCP config toolset, which would expose the kubeconfig, is disabled.

Remember that application logs can contain sensitive data if applications log it.

Sources: Default safety policies, Data privacy, Abuse monitoring.

11. Troubleshooting¶

k8s-servant turns these into one-line advice (model_error_text in k8s_servant/llm/errors.py).

Source: Troubleshoot errors.

12. Reference¶

• Quickstart: create Foundry resources
• v1 API and API version lifecycle
• Deploy models · Deployment types · Quota
• Models sold by Azure · Region availability · Retirement schedule
• Configure Entra ID · Foundry RBAC · Disable local auth
• Private link · Private endpoint DNS
• Default safety policies · Data privacy
• Manage costs · Troubleshoot errors