azure / azure-spring-apps-reference-architecture Goto Github PK

Request you to expand on UPN - user principal or service principal and how to create that in the CLI README

For cleaning up, should developers call destroy twice - first destroy app gateway and then everything else? We should document that

Add a parameter for Internal Application Gateway with WAF. Default can be external.

We should explain where to get the app FQDN and self-signed certificate (create KV and get self-signed should be okay?)

I used this to reference how to build Azure Spring Cloud through Terraform. It was very helpful. I would however suggest some variable name changes. The variables imply it should be a route id which doesn't exist instead of the route table id. which does exist.

sc_default_apps_route ---> sc_default_apps_route_table_id
sc_default_runtime_route ---> sc_default_runtime_route_table_id

https://github.com/Azure/azure-spring-cloud-reference-architecture/blob/8d735cce1a774d928739c09d42af014062369e9f/terraform/modules/azure_spring_cloud/variables.tf#L18
https://github.com/Azure/azure-spring-cloud-reference-architecture/blob/8d735cce1a774d928739c09d42af014062369e9f/terraform/modules/azure_spring_cloud/variables.tf#L18

Request you to check-in CLI script for private app gateway.

Tags are used by enterprise orgs for managing permissions to create and manage resources and manage costs. Without tags, customers may not be able to provision these resources

On a new non-Microsoft tenant, attempting to deploy produces the error

Failed to create Azure Spring Cloud service instance spring-3lybqjv6zyd4s in VNet due to customer error: Please verify resource provider Microsoft.ContainerService has been registered successfully..

When registering the Azure Spring Cloud provider, we need a step to register the ContainerService provider as well:

az provider register --namespace 'Microsoft.ContainerService'

Version used: Terraform version: 1.1.7

I am trying to deploy Spring Cloud using the terraform config files provided in this repo. I have run it previously with an older terraform version (1.1.3) and haven't gotten the following errors.

-modules/azure_firewall/main.tf:
The sku_name and sku_tier are now required firewall attributes. They need to be specified in the azure_firewall_instance resource.
The "AZFW_Hub" sku_name does not support IP Configurations.

-modules/my_sql/main.tf
The minimal TLS version for the my_sql_server resource has to be specified with a valid parameter.

-modules/key_vault/main.tf
The permission naming case has to updated from lower case from ["get, "regeneratekey", "listsas"] to ["Get", "RegenerateKey", "ListSAS"]

-modules/azure_spring_cloud/main.tf
The "instrumentation_key" argument is deprecated and should be removed.

Typically, in all our quickstarts and tutorials, because these documents create Azure resources that are expensive, we supply instructions for how to delete them. We should do that sam here

Examples:
https://docs.microsoft.com/en-us/azure/spring-cloud/spring-cloud-quickstart?tabs=Azure-CLI&pivots=programming-language-java#clean-up-resources

https://github.com/microsoft/azure-spring-cloud-training/tree/master/99-conclusion

There is new functionality for Azure Spring Apps - Spring Cloud gateway starting with terraform version 3.39.
Request is to upgrade the TF plans to this version or later.

Re README instructions at:

The firewall extension gets auto-installed. Hence, it looks like we can remove that as a pre-requisite

bash-3.2$ az extension add --name firewall
No matching extensions for 'firewall'. Use --debug for more information.
bash-3.2$ az network firewall --help
CommandNotFoundError: 'firewall' is misspelled or not recognized by the system.
Try this: 'az extension add --name <anextension>'
Still stuck? Run 'az network --help' to view all commands or go to 'https://aka.ms/cli_ref' to learn more
bash-3.2$ az network firewall list
The command requires the extension azure-firewall. Do you want to install it now? The command will continue to run after the extension is installed. (Y/n): y
Run 'az config set extension.use_dynamic_install=yes_without_prompt' to allow installing extensions without prompt.
The installed extension 'azure-firewall' is in preview.

Suggest that we use two resource groups to house hub and spoke resources. Typically, the hub and spoke are owned by two different groups within enterprise organization and two different cost centers

#53 is addressed for Terraform and CLI but not yet for ARM Template. Request you to apply the same hyperlinks to remind users about VM and MySQL administrator name and password

Reladed #46

We should explain how developers can specify tags when they start the deployment

Typically, in all our quickstarts and tutorials, because these documents create Azure resources that are expensive, we supply instructions for how to delete them. We should do that sam here

Examples:
https://docs.microsoft.com/en-us/azure/spring-cloud/spring-cloud-quickstart?tabs=Azure-CLI&pivots=programming-language-java#clean-up-resources

https://github.com/microsoft/azure-spring-cloud-training/tree/master/99-conclusion

The ACME Fitness app has additional endpoints that need to be whitelisted on the Azure Firewall Rules
https://jcenter.bintray.com

Right now, the default is azureuser and is in the variables file. Given that the script accepts admin name for MySQL, let's follow the same approach

The object ID is different per tenant.

Typically, in all our quickstarts and tutorials, because these documents create Azure resources that are expensive, we supply instructions for how to delete them. We should do that sam here

Examples:
https://docs.microsoft.com/en-us/azure/spring-cloud/spring-cloud-quickstart?tabs=Azure-CLI&pivots=programming-language-java#clean-up-resources

https://github.com/microsoft/azure-spring-cloud-training/tree/master/99-conclusion

The following query will share the general list of available regions but not scoped to regions where Azure Spring Cloud is supported

The following query will supply the list of regions that support Azure Spring Cloud
https://azure.microsoft.com/global-infrastructure/services/?products=spring-cloud&regions=all

There are open compliance tasks that need to be reviewed for your azure-spring-cloud-reference-architecture repo.

Action required: 1 compliance task

To bring this repository to the standard required for 2021, we require administrators of this and all Microsoft GitHub repositories to complete a small set of tasks within the next 60 days. This is critical work to ensure the compliance and security of your Azure GitHub organization.

Please take a few minutes to complete the task at: https://repos.opensource.microsoft.com/orgs/Azure/repos/azure-spring-cloud-reference-architecture/compliance

• The GitHub AE (GitHub inside Microsoft) migration survey has not been completed for this private repository

You can close this work item once you have completed the compliance tasks, or it will automatically close within a day of taking action.

If you no longer need this repository, it might be quickest to delete the repo, too.

GitHub inside Microsoft program information

More information about GitHub inside Microsoft and the new GitHub AE product can be found at https://aka.ms/gim or by contacting [email protected]

FYI: current admins at Microsoft include @selvasingh

Related #46
We should specify how tags are supplied by developers when they invoke quickstart

Hello,

I am deploying the 03 module and I was getting this error.
2023-04-16T10:31:20.2842118Z �[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mProvider configuration not present�[0m 2023-04-16T10:31:20.2847863Z �[31m│�[0m �[0m 2023-04-16T10:31:20.2850796Z �[31m│�[0m �[0m�[0mTo work with azurerm_public_ip.azure_bastion (orphan) its original provider 2023-04-16T10:31:20.2855544Z �[31m│�[0m �[0mconfiguration at provider["registry.terraform.io/hashicorp/azurerm"].hub is 2023-04-16T10:31:20.2856068Z �[31m│�[0m �[0mrequired, but it has been removed. This occurs when a provider 2023-04-16T10:31:20.2858758Z �[31m│�[0m �[0mconfiguration is removed while objects created by that provider still exist 2023-04-16T10:31:20.2859307Z �[31m│�[0m �[0min the state. Re-add the provider configuration to destroy 2023-04-16T10:31:20.2859774Z �[31m│�[0m �[0mazurerm_public_ip.azure_bastion (orphan), after which you can remove the 2023-04-16T10:31:20.2860166Z �[31m│�[0m �[0mprovider configuration again. 2023-04-16T10:31:20.2860491Z �[31m╵�[0m�[0m 2023-04-16T10:31:20.2860734Z �[31m╷�[0m�[0m 2023-04-16T10:31:20.2861102Z �[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mProvider configuration not present�[0m 2023-04-16T10:31:20.2861418Z �[31m│�[0m �[0m 2023-04-16T10:31:20.2861835Z �[31m│�[0m �[0m�[0mTo work with azurerm_bastion_host.azure_bastion_instance (orphan) its 2023-04-16T10:31:20.2862266Z �[31m│�[0m �[0moriginal provider configuration at 2023-04-16T10:31:20.2862713Z �[31m│�[0m �[0mprovider["registry.terraform.io/hashicorp/azurerm"].hub is required, but it 2023-04-16T10:31:20.2863193Z �[31m│�[0m �[0mhas been removed. This occurs when a provider configuration is removed 2023-04-16T10:31:20.2863712Z �[31m│�[0m �[0mwhile objects created by that provider still exist in the state. Re-add the 2023-04-16T10:31:20.2864091Z �[31m│�[0m �[0mprovider configuration to destroy 2023-04-16T10:31:20.2864528Z �[31m│�[0m �[0mazurerm_bastion_host.azure_bastion_instance (orphan), after which you can 2023-04-16T10:31:20.2865007Z �[31m│�[0m �[0mremove the provider configuration again. 2023-04-16T10:31:20.2865321Z �[31m╵�[0m�[0m 2023-04-16T10:31:20.2865557Z �[31m╷�[0m�[0m 2023-04-16T10:31:20.2865935Z �[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mProvider configuration not present�[0m 2023-04-16T10:31:20.2866267Z �[31m│�[0m �[0m 2023-04-16T10:31:20.2866575Z �[31m│�[0m �[0m�[0mTo work with 2023-04-16T10:31:20.2866993Z �[31m│�[0m �[0mazurerm_subnet_network_security_group_association.bastion_nsg_assoc 2023-04-16T10:31:20.2867445Z �[31m│�[0m �[0m(orphan) its original provider configuration at 2023-04-16T10:31:20.2867918Z �[31m│�[0m �[0mprovider["registry.terraform.io/hashicorp/azurerm"].hub is required, but it 2023-04-16T10:31:20.2868418Z �[31m│�[0m �[0mhas been removed. This occurs when a provider configuration is removed 2023-04-16T10:31:20.2868907Z �[31m│�[0m �[0mwhile objects created by that provider still exist in the state. Re-add the 2023-04-16T10:31:20.2869337Z �[31m│�[0m �[0mprovider configuration to destroy 2023-04-16T10:31:20.2869793Z �[31m│�[0m �[0mazurerm_subnet_network_security_group_association.bastion_nsg_assoc 2023-04-16T10:31:20.2870306Z �[31m│�[0m �[0m(orphan), after which you can remove the provider configuration again. 2023-04-16T10:31:20.2870660Z �[31m╵�[0m�[0m 2023-04-16T10:31:20.2870889Z �[31m╷�[0m�[0m 2023-04-16T10:31:20.2871280Z �[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mProvider configuration not present�[0m 2023-04-16T10:31:20.2871615Z �[31m│�[0m �[0m 2023-04-16T10:31:20.2872041Z �[31m│�[0m �[0m�[0mTo work with azurerm_subnet.azure_bastion (orphan) its original provider 2023-04-16T10:31:20.2872570Z �[31m│�[0m �[0mconfiguration at provider["registry.terraform.io/hashicorp/azurerm"].hub is 2023-04-16T10:31:20.2873063Z �[31m│�[0m �[0mrequired, but it has been removed. This occurs when a provider 2023-04-16T10:31:20.2873581Z �[31m│�[0m �[0mconfiguration is removed while objects created by that provider still exist 2023-04-16T10:31:20.2874067Z �[31m│�[0m �[0min the state. Re-add the provider configuration to destroy 2023-04-16T10:31:20.2875734Z �[31m│�[0m �[0mazurerm_subnet.azure_bastion (orphan), after which you can remove the 2023-04-16T10:31:20.2876150Z �[31m│�[0m �[0mprovider configuration again. 2023-04-16T10:31:20.2876434Z �[31m╵�[0m�[0m 2023-04-16T10:31:20.2876659Z �[31m╷�[0m�[0m 2023-04-16T10:31:20.2877045Z �[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mProvider configuration not present�[0m 2023-04-16T10:31:20.2877381Z �[31m│�[0m �[0m 2023-04-16T10:31:20.2877805Z �[31m│�[0m �[0m�[0mTo work with azurerm_resource_group.hub_rg (orphan) its original provider 2023-04-16T10:31:20.2878317Z �[31m│�[0m �[0mconfiguration at provider["registry.terraform.io/hashicorp/azurerm"].hub is 2023-04-16T10:31:20.2878801Z �[31m│�[0m �[0mrequired, but it has been removed. This occurs when a provider 2023-04-16T10:31:20.2879334Z �[31m│�[0m �[0mconfiguration is removed while objects created by that provider still exist 2023-04-16T10:31:20.2879815Z �[31m│�[0m �[0min the state. Re-add the provider configuration to destroy 2023-04-16T10:31:20.2880301Z �[31m│�[0m �[0mazurerm_resource_group.hub_rg (orphan), after which you can remove the 2023-04-16T10:31:20.2880743Z �[31m│�[0m �[0mprovider configuration again. 2023-04-16T10:31:20.2881039Z �[31m╵�[0m�[0m 2023-04-16T10:31:20.2881290Z �[31m╷�[0m�[0m 2023-04-16T10:31:20.2881677Z �[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mProvider configuration not present�[0m 2023-04-16T10:31:20.2881999Z �[31m│�[0m �[0m 2023-04-16T10:31:20.2882419Z �[31m│�[0m �[0m�[0mTo work with azurerm_network_security_group.bastion_nsg (orphan) its 2023-04-16T10:31:20.2882841Z �[31m│�[0m �[0moriginal provider configuration at 2023-04-16T10:31:20.2883293Z �[31m│�[0m �[0mprovider["registry.terraform.io/hashicorp/azurerm"].hub is required, but it 2023-04-16T10:31:20.2883814Z �[31m│�[0m �[0mhas been removed. This occurs when a provider configuration is removed 2023-04-16T10:31:20.2884330Z �[31m│�[0m �[0mwhile objects created by that provider still exist in the state. Re-add the 2023-04-16T10:31:20.2884772Z �[31m│�[0m �[0mprovider configuration to destroy 2023-04-16T10:31:20.2885250Z �[31m│�[0m �[0mazurerm_network_security_group.bastion_nsg (orphan), after which you can 2023-04-16T10:31:20.2885673Z �[31m│�[0m �[0mremove the provider configuration again. 2023-04-16T10:31:20.2885980Z �[31m╵�[0m�[0m 2023-04-16T10:31:20.2886239Z �[31m╷�[0m�[0m 2023-04-16T10:31:20.2886602Z �[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mProvider configuration not present�[0m 2023-04-16T10:31:20.2886934Z �[31m│�[0m �[0m 2023-04-16T10:31:20.2887342Z �[31m│�[0m �[0m�[0mTo work with azurerm_virtual_network.hub_vnet (orphan) its original 2023-04-16T10:31:20.2887757Z �[31m│�[0m �[0mprovider configuration at 2023-04-16T10:31:20.2888219Z �[31m│�[0m �[0mprovider["registry.terraform.io/hashicorp/azurerm"].hub is required, but it 2023-04-16T10:31:20.2888718Z �[31m│�[0m �[0mhas been removed. This occurs when a provider configuration is removed 2023-04-16T10:31:20.2889175Z �[31m│�[0m �[0mwhile objects created by that provider still exist in the state. Re-add the 2023-04-16T10:31:20.2889682Z �[31m│�[0m �[0mprovider configuration to destroy azurerm_virtual_network.hub_vnet 2023-04-16T10:31:20.2890150Z �[31m│�[0m �[0m(orphan), after which you can remove the provider configuration again. 2023-04-16T10:31:20.2890471Z �[31m╵�[0m�[0m 2023-04-16T10:31:20.3056017Z ##[warning]Can't find loc string for key: TerraformPlanFailed 2023-04-16T10:31:20.3063081Z ##[error]Error: TerraformPlanFailed 1 2023-04-16T10:31:20.3128187Z ##[section]Finishing: Terraform : azurerm

It cleared when I added an extra provider alias 'hub', which is required by the 02 module.
Was this a needed step.

Jumpbox is provisioned with 3.8.2 version of maven. Petclinic scripts point to 3.6.2 which results in below error when running Powershell script C:\ProgramData\chocolatey\lib\maven\apache-maven-3.6.3\bin\mvn' is not recognized as the name of a cmdlet

I am unable to access the spring cloud app I deployed using this terraform config. The test endpoint is showing 'connect failed' is there a way to fix this.

The error message is as below:

Metric category 'Performance (Java)' is not supported.. [Code: "BadRequest"]

The root cause is:

L681 ~ L738 of the file https://github.com/Azure/azure-spring-cloud-reference-architecture/blob/main/CLI/deploy-azurespringcloud-internal.sh

Should be changed as below:

    --metrics '[
        {
            "category": "AllMetrics",
            "enabled": true,
            "retentionPolicy": {
                "enabled": false,
                "days": 0
                }
        }
    ]'

Shall we document the syntax restrictions by referencing relevant published documents on docs.microsoft.com? Particularly for:

VM admin name
VM admin password
MySQL admin name
MySQL admin password

Here is a sample output of Azure CLI script run:

{- Finished ..
  "dnsName": "bst-792b9eda-e643-4ac5-8fb6-22f6aed909a2.bastion.azure.com",
  "etag": "W/\"a2854bdc-e636-424a-9be6-da89b2ec9898\"",
  "id": "/subscriptions/1c638cf4-608f-4ee6-b680-c329e824c3a8/resourceGroups/sc-corp-rg/providers/Microsoft.Network/bastionHosts/corp-bastion-svc",
  "ipConfigurations": [
    {
      "etag": "W/\"a2854bdc-e636-424a-9be6-da89b2ec9898\"",
      "id": "/subscriptions/1c638cf4-608f-4ee6-b680-c329e824c3a8/resourceGroups/sc-corp-rg/providers/Microsoft.Network/bastionHosts/corp-bastion-svc/bastionHostIpConfigurations/bastion_ip_config",
      "name": "bastion_ip_config",
      "privateIpAllocationMethod": "Dynamic",
      "provisioningState": "Succeeded",
      "publicIpAddress": {
        "id": "/subscriptions/1c638cf4-608f-4ee6-b680-c329e824c3a8/resourceGroups/sc-corp-rg/providers/Microsoft.Network/publicIPAddresses/azure-bastion-ip",
        "resourceGroup": "sc-corp-rg"
      },
      "resourceGroup": "sc-corp-rg",
      "subnet": {
        "id": "/subscriptions/1c638cf4-608f-4ee6-b680-c329e824c3a8/resourceGroups/sc-corp-rg/providers/Microsoft.Network/virtualNetworks/vnet-hub/subnets/AzureBastionSubnet",
        "resourceGroup": "sc-corp-rg"
      },
      "type": "Microsoft.Network/bastionHosts/bastionHostIpConfigurations"
    }
  ],
  "location": "eastus2",
  "name": "corp-bastion-svc",
  "provisioningState": "Succeeded",
  "resourceGroup": "sc-corp-rg",
  "tags": null,
  "type": "Microsoft.Network/bastionHosts"
}
The password length must be between 12 and 123. Password must have the 3 of the following: 1 lower case character, 1 upper case character, 1 number and 1 special character.

Firewall extension is auto installed

bash-3.2$ az extension add --name firewall
No matching extensions for 'firewall'. Use --debug for more information.
bash-3.2$ az network firewall --help
CommandNotFoundError: 'firewall' is misspelled or not recognized by the system.
Try this: 'az extension add --name <anextension>'
Still stuck? Run 'az network --help' to view all commands or go to 'https://aka.ms/cli_ref' to learn more
bash-3.2$ az network firewall list
The command requires the extension azure-firewall. Do you want to install it now? The command will continue to run after the extension is installed. (Y/n): y
Run 'az config set extension.use_dynamic_install=yes_without_prompt' to allow installing extensions without prompt.
The installed extension 'azure-firewall' is in preview.

az login step when running the powershell leads to the need for signing in using browser. The default experience with IE is painful. After a lot of clicks to allow it to open the link it finally results in the below error

1. when I run
   terraform plan -out my.plan --var-file ../parameters.tfvars
   from https://github.com/Azure/azure-spring-apps-landing-zone-accelerator/blob/main/Scenarios/ASA-Secure-Baseline/Terraform/07-LZ-AppGateway.md
   failed with:
   

https://github.com/Azure/azure-spring-cloud-reference-architecture/search?q=jumphost
https://github.com/Azure/azure-spring-cloud-reference-architecture/search?q=jump_box