APIOps Toolkit for Azure APIM v7.0.1

Fixed

• Publisher now correctly detects relationships between policies and backends (#847).

Full Changelog: v7.0.0...v7.0.1

APIOps Toolkit for Azure APIM v7.0.0

Breaking changes

• Extractor configuration section names now match the publisher's.

  Before

  apiNames:
  - api1
  - api2
  productNames:
  - product1
  - product2

  After

  apis:
  - api1
  - api2
  products:
  - product1
  - product2

  See configuration.extractor.yaml in the repo root for a correct sample.

• SOAP API specifications are no longer extracted. The WSDL exported by APIM cannot be reimported by APIM (#387).

• Tag relationships are now stored under the tag instead of the linked API or product.

  Before

   apis/{apiName}/tags/{tagName}/apiTagInformation.json
   products/{productName}/tags/{tagName}/productTagInformation.json
  

  After

   tags/{tagName}/apis/{apiName}/tagApiInformation.json
   tags/{tagName}/products/{productName}/tagProductInformation.json
  

Note on configuration key casing: during the v7 alpha cycle we briefly experimented with space-separated (named values) and snake_case (named_values) keys. Based on community feedback (#813, #814, #817, #819, #823), we reverted to camelCase (e.g. namedValues).

Added

• Workspace support

  The extractor and publisher now support the following workspace-scoped resources: APIs, operations, policies, diagnostics, schemas, releases, backends, groups, loggers, named values, policy fragments, products, subscriptions, tags, and version sets. Note that creating workspaces through the tool is NOT supported. You can only manage pre-existing workspaces.

• Nested configuration for child resources

  We now support nested levels of configuration for the extractor and publisher. This allows the configuration of child resources (e.g. api operations, workspace api diagnostics, etc).

  Sample extractor configuration
  Only operation1 and operation2 will be extracted in api1. All operations in api2 will be extracted.

  apis:
  - api1:
      operations:
      - operation1
      - operation2
  - api2
  ...

  Sample publisher configuration
  The display name of diagnostic3 in api2 in workspace1 will be overriden with my display name.

  workspaces:
  - workspace1:
      apis:
      - api2:
          diagnostics:
          - diagnostic3:
              properties:
                displayName: my display name
  ...

• Empty configuration in extractor

  Previously, if we wanted to skip extracting all resources of a type, we had to put a random placeholder value (e.g. ignore). We now support the more intuitive [].

  Before

  apis: ignore # Workaround to skip extracting all APIs.

  After

  apis: [] # All APIs will be skipped.

• Better ID references across APIM instances

  Many APIM resources reference others through IDs. For example, here is a diagnostic that references a logger through the loggerId property:

   {
    "properties": {
      ...
      "loggerId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.ApiManagement/service/apimService1/loggers/azuremonitor"
      ...
    }
  }

  If we publish this diagnostic as-is to another APIM instance, the publisher will fail. The logger reference is specific to APIM instance apimService1 in resource group rg1 and subscription 00000000-0000-0000-0000-000000000000. To work around this, we have to edit the JSON or override the publisher configuration to reference the new APIM instance.

  Going forward, the extractor will use relative references where possible. In the above scenario, it will write loggerId: "/loggers/azuremonitor". This relative reference will work when publishing across instances.

• Dry-run mode for the publisher (#754)

  The publisher can now run in dry-run mode. It logs what it would do without making changes to APIM. Dry-run is disabled by default, but you can control this by:

  • Setting the DRY_RUN configuration value to true (or false to disable).
  • Passing --dry-run to the publisher executable (the bare flag enables dry run).
  • Passing --dry-run true|false to the publisher executable explicitly.

• Pre-publish validation

  Before making any changes to APIM, the publisher now validates the resource set to catch these problems earlier:

  • Cyclical relationships (e.g. subscriptionA depending on groupB which depends on apiC with depends on subscriptionA). These are always flagged as errors and will throw an exception.
  • Missing predecessors (e.g. a policy that uses a named value that isn't in source). Behavior is controlled by the STRICT_VALIDATION configuration variable:
    • STRICT_VALIDATION=true: the publisher throws an exception.
    • Otherwise: the publisher logs a warning and continues. This supports scenarios where different teams own different resource types.

• Configuration logging at startup

  The extractor and publisher now log configuration values. This should make it a lot easier to troubleshoot configuration issues. Sensitive values are redacted, with only the first 3 characters shown.

  Extractor version is 7.0.0.
  Configuration summary
    apimServiceName ........................... (not set)
    apiSpecificationFormat .................... (not set)
    API_MANAGEMENT_SERVICE_NAME ............... api****
    API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH . C:\Users\me\AppData\Local\Temp\apiops
    API_SPECIFICATION_FORMAT .................. (not set)
    APPLICATION_INSIGHTS_CONNECTION_STRING .... (set to whitespace)
    ARM_API_VERSION ........................... (not set)
    AZURE_BEARER_TOKEN ........................ eyJ****
    AZURE_CLOUD_ENVIRONMENT ................... (not set)
    AZURE_RESOURCE_GROUP_NAME ................. wor****
    AZURE_SUBSCRIPTION_ID ..................... 105****
    COMMIT_ID ................................. (not set)
    CONFIGURATION_YAML_PATH ................... C:\Users\me\Local\Temp\apiops\configuration.extractor.yaml
    DRY-RUN ................................... (not set)
    DRY_RUN ................................... true
    OTEL_EXPORTER_OTLP_ENDPOINT ............... https://ote****
    STRICT_VALIDATION ......................... (not set)
  

• Extractor and publisher executable versions now match the GitHub release version automatically, so it's clear which build you're running (#811).

Fixed

• Improved publishing of API specifications across several scenarios (#818).
• Worked around an APIM issue where importing certain API specifications wiped the display name and description.
• Publisher now puts named values one at a time to avoid race conditions with policies.
• Fixed issue with backend pools that depend on individual backends (#797).
• Fixed empty serviceUrl values causing deployment failures (#760).
• Made tag APIs, tag products, and product APIs more reliable on both extract and publish.
• Made HTTP retry logic more robust against transient APIM errors.

New Contributors

• @daviian made their first contribution in #640
• @james-woodbridge made their first contribution in #687
• @Jeroen-VdB made their first contribution in #699
• @smiddler made their first contribution in #691
• @ollief87 made their first contribution in #676
• @felipementel made their first contribution in #658
• @cedrox made their first contribution in #716
• @UmairSyed made their first contribution in #729
• @LockTar made their first contribution in #745
• @Chad-Dumas-MSFT made their first contribution in #781
• @sergiopadure made their first contribution in #782
• @wagnerb3 made their first contribution in #780

Full Changelog: v6.0.2...v7.0.0

APIOps Toolkit for Azure APIM v7.0.0-rc.1.0.0

Added

• Added integration tests consistent with v7 code changes.

Fixed

• Made tag APIs, tag products, and product APIs more reliable when extracting/publishing.
• Updated publisher to take these dependencies into account when publishing policies:
  • Policy fragments (#825 )
  • Named values
  • Configuration overrides
• Updated publisher to take deleted composite resources into account when creating dependencies.
• Publisher now puts named values one at a time. This addresses a race issue with policy references.
• Worked around an issue where APIM returns the wrong API specification in these scenarios:
  • Getting multiple APIs within the same version set in parallel.
  • Getting multiple revisions of the same API in parallel.
• Worked around an issue where APIM wipes the display name and description when importing certain API specifications.
• Disabled extracting SOAP APIs specifications. The specifications exported by APIM cannot be reimported by APIM (#387 ).
• Fixed service URL issues with workspace APIs.
• Fixed an issue where potentially sensitive data could be leak when the log level is Debug. We only want to allow sensitive logging when the log level is Trace.

Full Changelog: v7.0.0-beta.1.0.0...v7.0.0-rc.1.0.0

APIOps Toolkit for Azure APIM v7.0.0-beta.1.0.0

Breaking changes

• Reverted back to camel case for configuration key names (namedValues in v6 -> named values in v7.0.0-alpha.2 -> named_values in v7.0.0-alpha.3 -> namedValues now). Worked around the main reason for the change, and having it as snake case broke too many users for minimal gains (#813, #814, #817, #819, #823).

Full Changelog: v7.0.0-alpha.3.0.1...v7.0.0-beta.1.0.0

APIOps Toolkit for Azure APIM v7.0.0-alpha.3.0.1

Added

• The extractor and publisher now log configuration values. This should make it a lot easier to troubleshoot configuration issues. Sensitive values are redacted, with only the first 3 characters shown.

Extractor version is 7.0.0.
Configuration summary
  apimServiceName ........................... (not set)
  apiSpecificationFormat .................... (not set)
  API_MANAGEMENT_SERVICE_NAME ............... api****
  API_MANAGEMENT_SERVICE_OUTPUT_FOLDER_PATH . C:\Users\me\AppData\Local\Temp\apiops
  API_SPECIFICATION_FORMAT .................. (not set)
  APPLICATION_INSIGHTS_CONNECTION_STRING .... (set to whitespace)
  ARM_API_VERSION ........................... (not set)
  AZURE_BEARER_TOKEN ........................ eyJ****
  AZURE_CLOUD_ENVIRONMENT ................... (not set)
  AZURE_RESOURCE_GROUP_NAME ................. wor****
  AZURE_SUBSCRIPTION_ID ..................... 105****
  COMMIT_ID ................................. (not set)
  CONFIGURATION_YAML_PATH ................... C:\Users\me\Local\Temp\apiops\configuration.extractor.yaml
  DRY-RUN ................................... (not set)
  DRY_RUN ................................... true
  OTEL_EXPORTER_OTLP_ENDPOINT ............... https://ote****
  STRICT_VALIDATION ......................... (not set)

Fixed

• Publisher now correctly puts API specifications (#818 )

Full Changelog: v7.0.0-alpha.3.0.0...v7.0.0-alpha.3.0.1

APIOps Toolkit for Azure APIM v7.0.0-alpha.3.0.0

Added

• Extractor and publisher versions now automatically match the release version (#811 ).
• Updated extractor and publisher configuration samples to show v7 changes (new keys and support for nested resources).

Fixed

• Publisher now ensures that an API (and its specification) is put before putting its child API operation policies.
• Publisher now ensures that named values are put before resources that refer to them.

Breaking changes

• Configuration key names are now snake_cased (from namedValues in v6 to named values in v7.0.0-alpha.2 to named_values now).

Full Changelog: v7.0.0-alpha.2.0.0...v7.0.0-alpha.3.0.0

APIOps Toolkit for Azure APIM v7.0.0-alpha.2.0.0

Added

• Added unit tests for the extractor and publisher.
• We now support a dry-run option for the publisher (#754). It will process files normally but not make any changes to APIM. This is useful for validating configurations and previewing changes before executing them. This is disabled by default, but you can control this by:
  • Setting the DRY_RUN configuration value to true | false
  • Passing --dry-run to the publisher executable. This naked flag enables dry run mode.
  • Passing --dry-run true|false to the publisher executable.
• The publisher now attempts to validate resource state before making changes to APIM. This reduces the chances of failing mid-run because of an issue that could have been caught earlier. Specifically, it checks that:
  • Resources don't have cyclical relationships (e.g. subscription A depends on group B which depends on API C which depends on subscription A). This shouldn't happen if teams rely on the extractor, but code-first deployments or editing the extracted JSON incorrectly can cause this.
  • Resources aren't missing predecessors. For example, let's say we put policy A that uses named value B. The publisher will check whether named value B is also in source control. If the predecessor is missing:
    • If the configuration STRICT_VALIDATION is set to true, then the publisher will throw an error.
    • Otherwise, the publisher will log a warning saying that the named value was not found in source. This allows for scenarios where different teams manage different resources (maybe another team is responsible for managing named values).

Fixed

• Fixed a bug when publishing workspace API revisions

Full Changelog: v7.0.0-alpha.1.0.4...v7.0.0-alpha.2.0.0

APIOps Toolkit for Azure APIM v7.0.0-alpha.1.0.4

Full Changelog: v7.0.0-alpha.1.0.3...v7.0.0-alpha.1.0.4

APIOps Toolkit for Azure APIM v7.0.0-alpha.1.0.3

Fixes #802

APIOps Toolkit for Azure APIM v7.0.0-alpha.1.0.2

Fix for #760