Error Listing/Downloading models (HTTP 429 Error)

[samuel100]

Summary

Earlier versions of Foundry Local always called the Foundry model catalog at the eastus endpoint. As usage has grown this single region has started to throttle requests, surfacing as catalog/list/download failures for users. You will see something like the following error message:

foundry_local_sdk.exception.FoundryLocalException: Failed to get model list: Error: System.Net.Http.HttpRequestException: Response status code does not indicate success: 429 (Received too many requests in a short amount of time. Retry again after 1 seconds.).
   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() + 0xf8
   at Microsoft.Neutron.AzureFoundry.AzureFoundryApiService.<FetchModelPageFromApiAsync>d__13.MoveNext() + 0x214
--- End of stack trace from previous location ---
   at Microsoft.Neutron.AzureFoundry.AzureFoundryApiService.<FetchModelsAsyncImpl>d__18.MoveNext() + 0x1d0
--- End of stack trace from previous location ---
   at Microsoft.Neutron.AzureFoundry.AzureFoundryApiService.<FetchAllLocalModelsAsyncImpl>d__16.MoveNext() + 0x5c
--- End of stack trace from previous location ---
   at Microsoft.Neutron.AzureFoundry.AzureFoundryApiService.<FetchAllLocalModelsAsync>d__14.MoveNext() + 0x5c
--- End of stack trace from previous location ---
   at Microsoft.Neutron.AzureFoundry.AzureFoundryService.<ListCatalogModelsAsync>d__20.MoveNext() + 0xb04
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.AzureModelCatalog.<GetLatestCatalogAndCachedModelInfo>d__29.MoveNext() + 0x184
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.AzureModelCatalog.<FetchModelInfoAsync>d__25.MoveNext() + 0x64
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.BaseModelCatalog`1.<GetCachedInfoAsync>d__20.MoveNext() + 0x1ac
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.BaseModelCatalog`1.<GetModelsAsync>d__8.MoveNext() + 0x68
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.NativeInterop.<>c__DisplayClass10_1.<<ExecuteCommandManaged>b__0>d.MoveNext() + 0x74
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.NativeInterop.<ExecuteWithTracker>d__9.MoveNext() + 0xa8

Fix

Foundry Local 1.2.0 introduces automatic region detection. Traffic is now spread across regions rather than be concentrated on eastus.

Recommended action: upgrade to Foundry Local 1.2.0 or later.

Still seeing errors after upgrading?

You can pin a specific region in the Configuration using:

config = Configuration(
        app_name="appname",
        additional_settings={"ModelRegistryRegion": "westus"},
        log_level="Debug"
    )

• Valid values: any Azure region slug (lowercase letters, digits, hyphens), e.g. westus2, westeurope, southeastasia.
• Default is "auto" (region detection as described above).
• Setting an explicit region skips detection and sends all catalog requests directly to that region.

Valid Azure regions

Here are a list of valid Azure regions to try:

"eastus", "australiaeast", "australiasoutheast", "brazilsouth",
    "canadacentral", "canadaeast", "centralindia", "centralus",
    "eastasia", "eastus2", "francecentral", "germanywestcentral",
    "japaneast", "japanwest", "jioindiawest", "koreacentral",
    "northcentralus", "northeurope", "norwayeast", "southafricanorth",
    "southcentralus", "southeastasia", "swedencentral", "switzerlandnorth",
    "uaenorth", "uksouth", "ukwest", "westcentralus", "westeurope",
    "westus", "westus2", "westus3", "qatarcentral", "polandcentral",
    "southindia", "switzerlandwest", "italynorth", "spaincentral",
    "israelcentral", "taiwannorth", "malaysiawest"

If problems persist after trying a different region, please reply with:

• Foundry Local version
• The ModelRegistryRegion value you tried
• A snippet of the log including the Created AzureModelCatalog line and any FetchModelPageFromApiAsync / catalog error lines

[anch-msft]

Commenting here because @bmehta001 said in #752 to comment here if the instructions here didn't help.

It appears the issue is that the version of Foundry Local that you get from winget cli is 0.8.119. There isn't a newer Microsoft.FoundryLocal package available. Is there any plan to publish the v1.2.0 release packages to winget?

[anch-msft]

Searching some more, it looks like there's multiple topics on winget cli version being out of date and it looks like it's on the roadmap for some time this month.

Note that CLI users can no longer browse the model catalog because of this. Existing users might be fine, but new users that have freshly downloaded Foundry Local CLI are stuck.