Giter VIP home page Giter VIP logo

deployment-environments's Introduction

Azure Deployment Environments Community

The goal of this repository is to provide sample infrastructure-as-code(IaC) templates that could be used to get started with the Azure Deployment Environments service. This repository will also be used to provide documentation on capabilities that are in private preview(gated) and engage with customers who are experimenting with the capabilities in private preview.

Azure Deployment Environments(ADE) empowers development teams to quickly and easily spin-up app infrastructure with project-based templates that establish consistency and best practices while maximizing security, compliance, and cost efficiency. This on-demand access to secure environments accelerates the different stages of the software development lifecycle in a compliant and cost-efficient manner.

An Environment is a collection of Azure resources on which your application is deployed. For example, to deploy a web application, you might create an environment consisting of an App Service, Key Vault, Cosmos DB and a Storage account. An environment could consist of both Azure PaaS and IaaS resources such as AKS Cluster, App Service, VMs, databases, etc.

Environments folder consists of sample templates that you can use to quickly get started with the service.

Documentation folder details out capabilities that are currently in private preview and instructions on how to try them out.

Note - ADE provides support for any IaC framework through its extensibility model, that allows customers to define and bring in their own container image. Learn more on how to leverage ADE's extensibility model and deploy using your favorite IaC templates.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

deployment-environments's People

Contributors

colbylwilliams avatar j-rewerts avatar jacksonkays avatar jfolberth avatar microsoft-github-operations[bot] avatar microsoftopensource avatar sagarlankala avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

deployment-environments's Issues

[Developer Portal] How many parameters can an environment have?

Hi Team,

When creating environment with parameter inputed from developer portal, the provisioning task finish with 4 parameters. When I add 1 more (5 params) I always get error message and not know what the rootcause is.
image

I think it can be the limitation of allowing only 4 parameters. there is no error if I use Azure CLI.

Thanks

Catalog sync results and troubleshooting logs

We found catalog sync a little difficult to troubleshoot, we believe the experience could be improved in different ways, every bullet may be it's own issue maybe we can split this.

  • When the catalog sync succeeds using the CLI, the command does not show any information about it, it just returns blank. Using the portal, we get a notification.
  • When the catalog does not sync, either in the CLI or Portal, there is some try and error to figure it out the issue.
    • There are not logs in the logs workspace (Azure Monitor) to be queried.
    • At the end, one of the issues was that we cannot provide a default value and required set to true in any of the manifest parameters. The error was not clear ("Incorrect file type") #13
    • When listing the catalog items, would be useful to see/query the manifest version and/or the git hash, so we know if the latest sync was successful or not.
    • Adding a catalog item list to the portal (UX). #14

Unable to create an environment (create-command returns "environment not found"-error)

I was trying to set up a deployment environment following the quickstart instructions. Everything works fine except the final step where I'm supposed to create the actual environment via the az devcenter dev environment create-command.
Tools are updated (installed today).
As per the documentation, I have ran the following command (I replaced the variables {catalog} and {catalog-item}, of course):

az devcenter dev environment create `
    --dev-center-name devcenter `
    --project-name my-dev-project `
    --environment-name my-first-env `
    --environment-type Dev `
    --catalog-name {catalog} `
    --catalog-item-name {catalog-item}

The command fails with the error message

(EnvironmentNotFound) The environment resource was not found.
Code: EnvironmentNotFound
Message: The environment resource was not found.

Indeed, the environment my-first-env cannot be found. It doesn't exist, and I want to create it.

deploying environment using main.bicep due to InvalidContentLink

Hi, i successfully used the sample web app from this repo (https://github.com/Azure/deployment-environments/tree/main/Environments/WebApp) to deploy using the following yaml:

name: WebApp-1
version: 1.0.0
summary: Azure Open AI Chatbot Web App Environment
description: Deploys an Azure Open AI Chatbot Web App without a data store
runner: ARM
templatePath: azuredeploy.json

From the documentation I understood that one could also point to a bicep file to deploy the environment so i changed the yaml to:

name: WebApp-1
version: 1.0.0
summary: Azure Open AI Chatbot Web App Environment
description: Deploys an Azure Open AI Chatbot Web App without a data store
runner: ARM
templatePath: main.bicep

I tried deploying the environment with and without having the associated azuredeploy.json file in the same folder.

My question are:
Is build-arm.py used at some step of the process or not?
Would it suffice to only have a bicep template and yaml manifest file in the environment folder or does it require the azuredeploy.json ARM template next to it?

When pointing to the bicep template i get this error:

InvalidArmTemplate
The template file provided failed deployment validation (Code: InvalidContentLink,  Message: Unable to download deployment content from **'https://lyd087dnw8ner705.blob.core.windows.net**/templates/project-genai.simpleaichatbot.133426215507879710/02-AzureAIDemo-A-SimpleChatbot/Environment/WebChatbot1/main.bicep?skoid=0efc828b-fa51-4535-913d-a277c19e5e05&sktid=33e01921-4d64-4f8c-a055-5bdaffd5e33d&skt=2023-10-24T11%3A30%3A51Z&ske=2023-10-25T11%3A45%3A51Z&sks=b&skv=2021-08-06&sv=2021-08-06&st=2023-10-24T11%3A30%3A51Z&se=2023-10-25T11%3A45%3A51Z&sr=c&sp=rl&sig=*Sanitized Azure Storage Account Shared Access Signature*'. The tracking Id is 'e282487d-db60-456d-9804-30525bbe8a53'. Please see https://aka.ms/arm-deploy-resources for usage details.).

Thanks in advance !!!

Not getting option to create Deployment environments on devportal

Hi,
I have been using Deployment environments for various demos but suddenly the option to create new environments disappeared from my dev portal. I can still access the environments created in the past but no option to create anything new. Checked all the setting and everything seems to be fine. I can still create devboxes from the portal but no environments. What can be the issue?

Unable to list the Catalog Items available to any projects

Apologies if I'm creating this ticket in the wrong place.

As per the documentation I have setup an ADE environment with a catalog that is syncing correctly over both GIT and AzureDevOps.
https://learn.microsoft.com/en-us/azure/deployment-environments/quickstart-create-access-environments

However, I'm unable to List the catalog items for any project using the Azure CLI. The tooling is up to date.

As per the documentation, I have ran the following command:

#List the Catalog Items available to a specific project.
az devcenter dev catalog-item list --dev-center "{devCenterName}" --project-name "{projectName}" -o table

This simply returns "[]". Is there anything I may need to investigate?


However, the below command works as intended and returns the correct number of environments. 
# list the type of environments you can create in a specific project.
az devcenter dev environment-type list --dev-center "{devCenterName}" --project-name "{projectName}" -o table

Environments not showing up in devportal.microsoft.com after succesful deployment

Hi, since yesterday whenever I deploy an Environment it will finish successful however when i refresh the devportal the newly created environment is gone all of the sudden. I made sure that the logged on user has Contributor permissions in the environment resource group as well as Deployment Environment User and DevCenter Dev Box User.

What does work:

  • access to the resource group of the environment by going to the Azure Portal directly
  • access to devbox that i deployed with the devportal
  • visibility of devbox in devportal (i.e. i can see the tile for the dev box)

What's broken:

  • visibility of the deployment environments in the devportal that I created is not working, I've tested this with both a test user account as well as an account with owner permissions. Neither is working.

Last week it did work and nothing changed in the configuration of the dev center.

Feature request | break the glass ability to destroy a DevCenter and all dependent resources

It's cumbersome, nearly impossible to destroy DevCenters that depend on Projects that depend on Environments. Sometimes these environments depend on catalog items that no longer exists or have changed which makes it hard to delete ending up in a lot of trash in the dev portal.

Would be great to have some sort of garbage collector / purge functionality that cleans up idle / orphan / broken environments (like a soft deletion of keys in AKV). This might not be desired once thins are up & running but would definitely speed up the exploration and piloting phase. An optional feature.

Would be great to have the ability to delete a DevCenter and all dependent resources in a forced manner. "Danger Zone" type of functionality.

Dev center reports deployment as failed when actually succeeded

This is a tricky report because it only happened once, and we could not replicate it anymore.

We created an environment, for a catalog item that took close to 4 minutes to complete. The devcenter project reported that the deployment failed, although the deployment was still in progress in the resource group. The deployment completed successfully but the dev center project did not update the status of the environment as successful.

Our best guess was regarding some timeout on the devcenter, however in subsequent environment creations this error did not occurred anymore.

The catalog item is based on this resource group: https://github.com/Azure/aks-baseline-automation/tree/main/IaC/bicep/rg-hub

Enable Auto Expiration

Where can I find documentation on enabling auto expiration and setting the expiration date?
Does this feature currently exists?
What is the timeline for adding this feature?

May 24 2022 this article mentions expirations and has a screenshot:
https://techcommunity.microsoft.com/t5/azure-developer-community-blog/introducing-azure-deployment-environments/bc-p/3920365#M1066

4 Months ago Scott Hanselman mentions setting expirations in his video here:
https://www.youtube.com/watch?v=_rRiVELgdf4
I can even have an expiration date to make sure that I don't accidentally keep a bunch of Azure resources around. I don't want to waste any money. They'll get deleted automatically when I'm done. With them we'll pick from a template. Notice here we've got different templates. I'm going to pick podcast A ...

3 months ago at Microsoft Build 2023:
https://www.youtube.com/watch?v=yTC40g4oyck
We're also working on adding an auto expiry functionality, which will help the developer to define an auto expiry, and that will automatically delete the environment based on the configured date. In the second demo, we saw an automated operation where there was a streamlined deployment of ...

Ability to View Quotas

Please provide a way for customers to:

  1. To verify their current quota limits
  • Runtime limit per month per region per Azure subscription
  • Runtime limit per deployment
  • vStorage limit per environment
  1. To verify the remaining runtime minutes remaining for the month

Not having these numbers is exacerbated by having to enter a support ticket for quota increases. The support ticket system asks for Additional Quota. This is very error prone because will the support person add the value you enter to the current quota or will they enter the value you filled out as the new quota. There is no way to know what the quota limit is before or after a change is made.

How dev infra team test the catalog template before releasing to main branch

I understand developers can self service environment with Azure deployment environment but how dev infra team can test the catalog template before releasing to main branch so that developers can use that. Do we need to use some testing framework for IAC separately against some sand box environment. also can anyone help me to decide best practice to keep code base for infra and app in case someone using Azure deployment environment. Please help

Removing Access to User

A user that has created deployment environments appears to be the only one who can delete those environments.

When access was removed from a user their environments in Project\Environments were no longer displayed.

When the environments still exists but are not displayed in Project\Environments the option to delete the Project\EnvironmentType pairing becomes enabled. Attempting to delete the pairing results in an error. This is also true of the Project itself. The delete button at the top of the Overview page is available but the delete will fail and a banner will appear on the Overview page stating that the resource is in a failed state.

Restoring the users access to try to resolve these issues does not appear to work.

Pass Environment Type to Manifest/ARM Template

I cannot find any documentation about how to pass the environment type (DEV/QA/PROD) to the manifest and/or ARM template.

Is this happening automatically?
Is there a reserved parameter name for this?

If this is possible please point me to the documentation. If it is not please make this a feature request.

Creating an environment from the dev portal doesn't use the manifest default values

Hi Team,

As the parameters' default values of the ARM template I use in catalog are defined for a production environment, I set different default values in the manifest.yaml file.
However, when I create an environment from the dev portal it is using the default values of the ARM template instead of the ones defined in the manifest.

It is quite counterintuitive as it is not the values displayed in the dev portal during creation that are actually used.

Is it the expected behaviour? Is it possible to improve that behaviour as it would be very useful to be able to use different manifest files with the same ARM template for different environment type?

Access to Environment Templates Cannot be Restricted to Certain Users

We need the ability to have different sets of environment templates for each group in our organization Developers, QA, and Admins.

There does not appear to be a way to have split what is in the Catalog in a Dev Center between different users or environment types.

This led to us experimenting with multiple DevCenters which it appears is not well supported at this point.

Feature request | Ability to show environment deployment outputs in the Portal

As a developer, I would like to see some metadata about my successful deployment in the Dev Portal (i.e. a web app URL or API endpoint).

When orchestrating your deployment from CI/CD pipelines, we could workaround this limitation by querying the Azure resources and using something like GitHub Actions summary (https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/).

In Dev Portal led deployments we can't get any outputs in the UI. Would be great if we could define outputs in the azure.yml file which match ARM or Terraform outputs and are displayed in a Details tab for each environment. Similar to the Dev Box More info tab:

image

Consider different output types such as links, plain text, sensitive values, etc.

Thanks.

ADE with extensibility and custom container image

Hi!

Im trying to use the custom container image here to run Terraform, and to begin with I have tried the ARM/Bicep image defined here --> https://github.com/Azure/deployment-environments/blob/main/Runner-Images/ARM-Bicep/Dockerfile I wanted to try and run the container image locally just to make sure it is working as intended. But when I try and run the image locally, I get this error message

docker run nameofimage

_Initializing runner
Unrecognized command or argument 'Initializing runner'.

Description:
Logs providing context to the operation that is being run.

Usage:
ade log [options]

Options:
--content The content to log.
--type The level of logging for the content. Allowed values are 'log', 'verbose', and 'error'. [default: log]
--operation The operation which is creating the log. []
-?, -h, --help Show help and usage information

Uploading environment state to
Unrecognized command or argument 'Uploading environment state to '_

Should it have worked to run the container locally just so I can see use that way to debug that the terraform actions are running as intended?

Is there any guidance to use terraform IaC Codes with AzureDeployment environment ?

We were trying for the GitOps flow for our developers with ADE, so that we can make use of our existing terraform codes with GitOps method using AzureDevops Repos as catalogs repo. We followe the MS documents and which is not stating anything about how to use or configure terraform IaC codes with the AzureDeployment Environment.

Not all projects from same DevCenter are selectable in the Dev Portal

Hi,

I have deployed a DevCenter configured to use ADE using this Azure Developer CLI template:

https://github.com/lopezleandro03/azd-devcenter

This IaC provisions 3 projects with 3 environments each and set the same project members across all projects. However, in the Dev Portal I can only select the first project. The expected behavior would be to see a dropdown with all my projects.

image
image

Same RBAC roles applied to all projects. This seems to be a bug.

Any guidance? Thanks.

Deployment Environments in Multiple Dev Centers

I could not figure out anyway to restrict access to the environment templates that exist in the catalog. For example, QA team members should be able to create test environments without being able to create other types of environments.

Therefore, I created multiple Dev Centers. I granted all Dev and QA team members access to one DC1 and only Admins access to DC2. I have access to both DC1 and DC2.

When we create an environment in either DevCenter the environment is created like normal. If we log out of devportal and log back in that environment is no longer displayed. This was repeatable multiple times and they did still exist (they could be viewed in the DevCenter project).

For now I am going to remove access to one of the DevCenters and make all templates available in the remaining one.

DeploymentCanceledByTimeout

Is this a per object timeout or a timeout for creating the environment?
How do I increase this timeout?
Where is there documentation about this timeout?
Is there a way to execute the ARM templates asynchronously?

Error details
DeploymentCanceledByTimeout
The timeout for the deployment was reached, and the deployment was canceled.

As the only step in this deployment I am creating a VM from an Image Template.

These times are very consistent over multiple tests.
0:40 the resources group has been created and is visible in the portal
1:15 VM is visible in the portal and it's state is Creating
11:15 timeout occurs in devportal
11:45 VM is Running

location: eastus2
osDiskType: Premium_LRS
virtualMachineSize: Standard_B4ms
OS: Win11

Environments of a project are not displayed in the Azure Portal

The environments of a project are not displayed in the portal. There's just one of them displayed (the last one).
However, when using the az cli command, all the environments are listed.

Screenshot from 2023-01-25 15-26-54

az devcenter dev environment list --dev-center FidalgoDevCenter --project-name neubergerberman-Fixed-Income-POC
Command group 'devcenter' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
[
  {
    "catalogItemName": "KeyVault",
    "catalogName": "ATR-Catalog",
    "description": null,
    "environmentType": "Development",
    "name": "fixed-income-poc-kv-dev-unittest",
    "owner": "355996a9-f9e2-4d81-8e21-*******",
    "parameters": {
      "env": "Development"
    },
    "provisioningState": "Failed",
    "resourceGroupId": "/subscriptions/09fc08dd-a989-4402-****-*****/resourceGroups/neubergerberman-fixed-income-poc-fixed-income-poc-kv-dev-unittest",
    "scheduledTasks": {},
    "tags": {}
  },
  {
    "catalogItemName": "KeyVault",
    "catalogName": "ATR-Catalog",
    "description": null,
    "environmentType": "Development",
    "name": "fixed-income-poc-kv-dev",
    "owner": "355996a9-f9e2-4d81-8e21-*******",
    "parameters": {
      "env": "Development"
    },
    "provisioningState": "Succeeded",
    "resourceGroupId": "/subscriptions/09fc08dd-a989-4402-*****-********/resourceGroups/neubergerberman-fixed-income-poc-fixed-income-poc-kv-dev",
    "scheduledTasks": {},
    "tags": {}
  },
  {
    "catalogItemName": "FunctionAppDotNet",
    "catalogName": "ATR-Catalog",
    "description": null,
    "environmentType": "Development",
    "name": "fixed-income-poc-new-dev",
    "owner": "355996a9-f9e2-4d81-8e21-*******",
    "parameters": {
      "env": "Development"
    },
    "provisioningState": "Succeeded",
    "resourceGroupId": "/subscriptions/09fc08dd-a989-4402-*****-******/resourceGroups/neubergerberman-fixed-income-poc-fixed-income-poc-new-dev",
    "scheduledTasks": {},
    "tags": {}
  },

...

When the manifest.json is not correct it shows "Incorrect file type" instead an actual error.

When sync a catalog with an incorrect manifest.json is not correct it shows "Incorrect file type" instead an actual error. We found out the issue was that we cannot have a default with value and required: true in the parameters, there was a lot of trial/error on catalog sync to figure it out.

Example param:


  - id: location
    name: 'Location'
    description: 'Region location'
    type: string
    default: 'eastus'
    required: true

manifest.yaml differs from MS documentation

https://learn.microsoft.com/en-gb/azure/deployment-environments/configure-catalog-item#add-a-new-catalog-item

Indicates that the manifest.yaml should have the structure:

name: WebApp
    version: 1.0.0
    description: Deploys an Azure Web App without a data store
    engine:
      type: ARM
      templatePath: azuredeploy.json

However the manifests in this sample project follow the structure:

name: FunctionApp
version: 1.0.0
summary: Azure Function App Environment
description: Deploys an Azure Function App, Storage Account, and Application Insights
runner: ARM
templatePath: azuredeploy.json

Can open a PR if required, just need confirmation which is correct

Portal | Environments provisioning state refresh

When an environment is created the provisioning state in the Environments page does not get refreshed, we need to refresh the page manually. An intermediate solution would be to just add a refresh button.

Deployment monthly usage has exceeded the limit error when deploying environment

When running az devcenter dev environment create command within CICD Github Actions pipeline we started getting the next error:
Screenshot 2023-05-26 at 7 42 40 PM
We don't see any other quotas, that can be found in the portal, exceeded their limit in our subscription.

Could you please advice is there are any quotas on ADE deployments? If so, how it can be found and how a new quota request can be created? As per message we thought that with beginning of the month this error may gone, but we still get it.

Please let me know if more details are needed.

AzureDeployment Environment Dev center setup failing with key vault Integration

I was trying to create a Azuredeployment environment and Dev center with in it. When I am trying to create a catalog within the Devcenter using the keyvault integration (with private endpoint enabled), the catalog creation is failing.

We couldn't connect to ade-zzzzzzz. The DevCenter is misconfigured. Inform DevCenter owners that the key vault could not be found, and is required for this operation.

We already created a system managed identity for the devcenter and add thed the system managed identity to full permission for the secret in the keyvault. But still the catalog creation is failing. So a private keyvault is not supported with AzureDeployment environment?

Increase deployments verbosity and feedback

Many provisioning errors result on zero feedback to the developer making it very hard to troubleshoot.

When the error happens during Terraform plan or Terraform apply, the whole output is displayed, which is great. However, if the error happens before the processor starts deploying, I get a "None".

image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.