Skip to content

fix: Enhance Cosmos DB firewall handling in post deployment script and address code quality issues#271

Merged
Avijit-Microsoft merged 31 commits into
mainfrom
dev
Jun 29, 2026
Merged

fix: Enhance Cosmos DB firewall handling in post deployment script and address code quality issues#271
Avijit-Microsoft merged 31 commits into
mainfrom
dev

Conversation

@HarshB-Microsoft

Copy link
Copy Markdown
Contributor

Purpose

This pull request enhances the deployment workflows for both Linux and Windows environments by introducing support for fallback deployment regions and improved quota handling. It also makes the GPT model capacity configuration more flexible and robust, and refines how environment variables and region selection are managed throughout the deployment process. The most significant changes are grouped below.

Deployment Region Fallback and Retry Logic

  • Added support for specifying fallback regions (AZURE_NONAI_LOCATIONS) in both Linux and Windows deployment workflows. If deployment in the primary region fails due to insufficient resources, the workflow will automatically attempt deployment in the next available region from the fallback list. This is implemented with ordered retry logic and error handling for quota-related failures. [1] [2]
  • Improved quota check and region selection logic in job-deploy.yml to allow automatic fallback to valid regions when the user-selected region lacks sufficient quota, instead of failing the deployment immediately. The workflow now continues with the first region that meets quota requirements and updates the GitHub summary accordingly. [1] [2] [3] [4]

Environment Variable and Input Handling

  • Introduced new workflow inputs and environment variables for GPT model capacity (AZURE_ENV_GPT_MODEL_CAPACITY) and fallback regions (AZURE_NONAI_LOCATIONS), with sensible defaults and propagation through all relevant workflow steps. [1] [2] [3] [4] [5] [6] [7] [8]
  • Made GPT capacity environment variables configurable via repository variables, allowing for easier adjustments without code changes.

Quota Check Enhancements

  • Refined the quota check step to handle both manual and automatic triggers, with improved fallback logic for region lists, and clearer error messages when no suitable regions are configured. [1] [2]

These changes collectively improve the reliability and flexibility of the deployment workflows, especially in scenarios where regional capacity constraints may otherwise cause failures.


References:
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17]

Does this introduce a breaking change?

  • Yes
  • No

How to Test

  • Get the code
git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install
  • Test the code

What to Check

Verify that the following are valid

  • ...

Other Information

Ashwal-Microsoft and others added 30 commits June 11, 2026 01:17
- Return base agent name instead of version-scoped ID in create_or_update_prompt_agent
- Use global regex replace for comma removal in price parsing (textParsers.ts)
- Fix logger name from 'agent_framework_foundry' to 'agent_framework.foundry'
- Add defensive parsing for E2E_WARMUP_MS env var
- Replace empty except with logged debug message in test_byocc.py

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Auto-detected public IP (via ipify/ipinfo) can differ from the IP Cosmos
actually sees behind a corporate proxy, causing a Forbidden firewall error.

- Retry the Cosmos upload on Forbidden: extract the real egress IP from the
  Cosmos error message, add it to the firewall, and retry (up to 3 attempts).
- Support COSMOS_FIREWALL_IP override (single IP/CIDR or comma-separated list).
- Capture original ipRules up front so restore stays accurate after recovery.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…pendency sequence

- Added aiServicesPollScript deployment script to wait for AI Services to reach terminal provisioning state
- Ensures AI Services is fully provisioned before private endpoint attempts connection modifications
- Fixes private endpoint provisioning race condition (RequestConflict error)
- Private endpoint module now depends on polling script for proper resource sequencing
- Uses system-assigned identity for deployment script
- Removed unnecessary explicit dependencies from private endpoint module
- Abort if existing ipRules can't be read, so restore never wipes rules
- Set current_ip in COSMOS_FIREWALL_IP override path for actionable warnings
- Use fixed-string grep (-Fqx) when checking for the blocked IP
- Stringify az error (Out-String) in PowerShell throw message

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
fix: Cosmos DB firewall block on data upload behind proxy/VPN
Add explanatory comments to 5 intentional empty-except blocks (py/empty-except) and bind awaited result in voice_live.py (py/ineffectual-statement). No behavior change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown

Coverage

Coverage Report •
FileStmtsMissCoverMissing
routers
   voice_live.py3674089%31–34, 155–156, 162–163, 174, 177, 214–217, 238–240, 308–311, 328, 330, 376–378, 381, 384, 386–388, 405–406, 415–416, 617, 646–647, 722, 731
TOTAL240025889% 

Tests Skipped Failures Errors Time
402 0 💤 0 ❌ 0 🔥 9.433s ⏱️

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request improves deployment reliability by adding region fallback / retry behavior, strengthening quota validation, and making Cosmos DB firewall handling more resilient during post-deploy data uploads.

Changes:

  • Added quota-validation enhancements and richer region-selection logic, including exposing available GPT capacity to downstream jobs.
  • Improved post-deployment Cosmos DB firewall enable/restore behavior and added “retry on Forbidden” recovery for Cosmos uploads.
  • Updated deployment workflows (Linux/Windows) to retry deployments across fallback regions when capacity-related failures occur.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/e2e-test/tests/test_byocc.py Adds clarification comments around click fallback behavior in an E2E test.
tests/e2e-test/pages/webUserPage.py Adds comments clarifying expected Playwright timeout fall-through behavior.
src/App/package-lock.json Dependency lockfile updates (tooling/runtime packages).
src/api/app/routers/voice_live.py Minor change to bind awaited close result (lint/quality adjustment).
infra/scripts/data_scripts/run_upload_data_scripts.sh Safer Cosmos firewall rule capture/restore and retry-on-Forbidden recovery for Cosmos uploads.
infra/scripts/data_scripts/run_upload_data_scripts.ps1 Windows equivalent of safer firewall capture/restore and retry-on-Forbidden recovery.
infra/scripts/checkquota.sh Adds supported-region filtering, subscription-level quota gating, Search availability checks, and exports available GPT capacity.
infra/scripts/agent_scripts/run_create_agents_scripts.sh Trims extracted deployment output values and scopes role assignment to Foundry subscription.
infra/scripts/agent_scripts/run_create_agents_scripts.ps1 Same trimming + Foundry subscription scoping; removes duplicated resource-id parsing.
infra/scripts/agent_scripts/requirements.txt Updates aiohttp dependency version.
infra/main.json Regenerated ARM template output reflecting Bicep changes (incl. dependsOn wiring).
infra/main.bicep Ensures private endpoint depends on DNS zone modules to avoid ordering issues.
.github/workflows/job-send-notification.yml Makes GPT_MIN_CAPACITY configurable via repo variables.
.github/workflows/job-deploy.yml Adds quota/region fallback behavior, exposes GPT capacity output, and adds subscription-wide Search service count check.
.github/workflows/job-deploy-windows.yml Adds region retry loop for azd deployment and propagates new capacity/fallback inputs.
.github/workflows/job-deploy-linux.yml Adds region retry loop for azd deployment and propagates new capacity/fallback inputs.
Files not reviewed (1)
  • src/App/package-lock.json: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread infra/scripts/checkquota.sh
Comment thread infra/scripts/checkquota.sh
Comment thread .github/workflows/job-deploy.yml
fix: merging main to dev
@Avijit-Microsoft Avijit-Microsoft merged commit db8ace4 into main Jun 29, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants