How can I update the capacity of a finetuned GPT model on Azure using Python?

I want to update the capacity of a finetuned GPT model on Azure. How can I do so in Python?

The following code used to work a few months ago (it used to take a few seconds to update the capacity) but now it does not update the capacity anymore. No idea why. It requires a token generated via az account get-access-token:

import json
import requests

new_capacity = 3 # Change this number to your desired capacity. 3 means 3000 tokens/minute.

# Authentication and resource identification
token = "YOUR_BEARER_TOKEN"  # Replace with your actual token
subscription = ''
resource_group = ""
resource_name = ""
model_deployment_name = ""

# API parameters and headers
update_params = {'api-version': "2023-05-01"}
update_headers = {'Authorization': 'Bearer {}'.format(token), 'Content-Type': 'application/json'}

# First, get the current deployment to preserve its configuration
request_url = f'https://management.azure.com/subscriptions/{subscription}/resourceGroups/{resource_group}/providers/Microsoft.CognitiveServices/accounts/{resource_name}/deployments/{model_deployment_name}'
r = requests.get(request_url, params=update_params, headers=update_headers)

if r.status_code != 200:
    print(f"Failed to get current deployment: {r.status_code}")
    print(r.reason)
    if hasattr(r, 'json'):
        print(r.json())
    exit(1)

# Get the current deployment configuration
current_deployment = r.json()

# Update only the capacity in the configuration
update_data = {
    "sku": {
        "name": current_deployment["sku"]["name"],
        "capacity": new_capacity  
    },
    "properties": current_deployment["properties"]
}

update_data = json.dumps(update_data)

print('Updating deployment capacity...')

# Use PUT to update the deployment
r = requests.put(request_url, params=update_params, headers=update_headers, data=update_data)

print(f"Status code: {r.status_code}")
print(f"Reason: {r.reason}")
if hasattr(r, 'json'):
    print(r.json())

What's wrong with it?

It gets a 200 response but it silently fails to update the capacity:

C:\Users\dernoncourt\anaconda3\envs\test\python.exe change_deployed_model_capacity.py 
Updating deployment capacity...
Status code: 200
Reason: OK
{'id': '/subscriptions/[ID]/resourceGroups/Franck/providers/Microsoft.CognitiveServices/accounts/[ID]/deployments/[deployment name]', 'type': 'Microsoft.CognitiveServices/accounts/deployments', 'name': '[deployment name]', 'sku': {'name': 'Standard', 'capacity': 10}, 'properties': {'model': {'format': 'OpenAI', 'name': '[deployment name]', 'version': '1'}, 'versionUpgradeOption': 'NoAutoUpgrade', 'capabilities': {'chatCompletion': 'true', 'area': 'US', 'responses': 'true', 'assistants': 'true'}, 'provisioningState': 'Updating', 'rateLimits': [{'key': 'request', 'renewalPeriod': 60, 'count': 10}, {'key': 'token', 'renewalPeriod': 60, 'count': 10000}]}, 'systemData': {'createdBy': '[email protected]', 'createdByType': 'User', 'createdAt': '2025-10-02T05:49:58.0685436Z', 'lastModifiedBy': '[email protected]', 'lastModifiedByType': 'User', 'lastModifiedAt': '2025-10-02T09:53:16.8763005Z'}, 'etag': '"[ID]"'}

Process finished with exit code 0

posted 4 months ago

CC BY-SA 4.0

4mo ago

Franck Dernoncourt‭

143 reputation 24 10 14 33