Giter VIP home page Giter VIP logo

asa-vtd's Introduction

Demo setup: Realize Integrated Analytical Solutions with Azure Synapse Analytics

Contents

Requirements

  1. An Azure Account with the ability to create an Azure Synapse Workspace

  2. Make sure the following resource providers are registered for your Azure Subscription.

    • Microsoft.Sql
    • Microsoft.Synapse
    • Microsoft.StreamAnalytics
    • Microsoft.EventHub

    See further documentation for more information on registering resource providers on the Azure Portal.

  3. A Power BI Pro or Premium account to host Power BI reports, dashboards, and configuration of streaming datasets.

Environment setup instructions

Azure Setup

Task 1: Create a resource group in Azure

  1. Log into the Azure Portal using your Azure credentials.

  2. On the Azure Portal home screen, select the + Create a resource tile.

    A portion of the Azure Portal home screen is displayed with the + Create a resource tile highlighted.

  3. In the Search the Marketplace text box, type Resource group and press the Enter key.

    On the new resource screen Resource group is entered as a search term.

  4. Select the Create button on the Resource group overview page.

  5. On the Create a resource group screen, select your desired Subscription and Region. For Resource group, enter synapse-in-a-day-demos, then select the Review + Create button.

    The Create a resource group form is displayed populated with Synapse-MCW as the resource group name.

  6. Select the Create button once validation has passed.

Task 2: Create Azure Synapse Analytics workspace

  1. Deploy the workspace through the following Azure ARM template (press the button below):

  2. On the Custom deployment form fill in the fields described below.

    • Subscription: Select your desired subscription for the deployment.
    • Resource group: Select the resource group you previously created.
    • Unique Suffix: This unique suffix will be used naming resources that will created as part of your deployment. Make sure you follow correct Azure Resource naming conventions.
    • SQL Administrator Login Password: Provide a strong password for the SQLPool that will be created as part of your deployment. Visit here to read about password rules in place. Your password will be needed during the next steps. Make sure you have your password noted and secured.
    • Location: The datacenter where your Azure Synapse environment will be created.

    Important: The location field under 'Settings' will list the Azure regions where Azure Synapse Analytics (Preview) is available as of September 2020. This will help you find a region where the service is available without being limited to where the resource group is defined.

  3. Check the I agree to the terms and conditions stated above, then select the Purchase button. The provisioning of your deployment resources will take approximately 13 minutes. Wait until provisioning successfully completes before continuing. You will need the resources in place before running the scripts below.

    Note: You may experience a deployment step failing in regards to Role Assignment. This error may safely be ignored.

Before starting

Steps & Timing

The entire script will take a little over an hour to complete. Major steps include:

  • Configure Synapse resources
  • Download all data sets and files into the data lake (~15 mins)
  • Execute the setup and execute the SQL pipeline (~30 mins)
  • Execute the Cosmos DB pipeline (~25 mins)

Task 1: Pre-requisites

Task 1: Download artifacts and install PowerShell modules

  1. Open a PowerShell Window as an administrator, run the following command to download the artifacts

    mkdir c:\labfiles
    
    cd c:\labfiles
    
    git clone https://github.com/ctesta-oneillmsft/asa-vtd.git synapse-in-a-day-deployment
  • Install Azure PowerShell cmdlet

    Open Windows PowerShell as an Administrator on your desktop and execute the following:

    if (Get-Module -Name AzureRM -ListAvailable) {
        Write-Warning -Message 'Az module not installed. Having both the AzureRM and Az modules installed at the same time is not supported.'
        Uninstall-AzureRm -ea SilentlyContinue
        Install-Module -Name Az -AllowClobber -Scope CurrentUser
    } else {
        Install-Module -Name Az -AllowClobber -Scope CurrentUser
    }
  • Install Az.CosmosDB cmdlet

    Install-Module -Name Az.CosmosDB -AllowClobber
  • Install sqlserver module

    Install-Module -Name SqlServer -AllowClobber
  • Install Azure CLI

    Invoke-WebRequest -Uri https://aka.ms/installazurecliwindows -OutFile .\AzureCLI.msi; Start-Process msiexec.exe -Wait -ArgumentList '/I AzureCLI.msi /quiet'; rm .\AzureCLI.msi
  • Close the Windows PowerShell window so you can import the newly installed Az.CosmosDB cmdlet.

Task 3: Execute setup scripts

  • Open Windows PowerShell as an Administrator and execute the following:

    Import-Module Az.CosmosDB
  • Change directories to the root of this repo within your local file system.

    cd c:\labfiles\synapse-in-a-day-deployment\artifacts\environment-setup\automation\
  • Run Set-ExecutionPolicy Unrestricted.

  • Execute Connect-AzAccount and sign in to your Microsoft user account when prompted.

  • Execute az login and sign in to your Microsoft user account when prompted.

  • Execute .\01-environment-setup.ps1

  1. You will be prompted to setup your Azure PowerShell and Azure CLI context.

  2. You may be prompted to enter the name of your desired Azure Subscription. You can copy and paste the value from the list to select one. Note: Be certain to include the number next to the subscription name when you copy it.

    Select the resource group you selected during Task 2.2. This will make sure automation runs against the correct environment you provisioned in Azure.

    The Azure Cloud Shell window is displayed with a selection of resource groups the user owns.

    During the execution of the automation script you may be prompted to approve installations from PS-Gallery. Please approve to proceed with the automation.

    The Azure Cloud Shell window is displayed with a sample of the output from the preceding command.

    NOTE This script will take up to 75 minutes to complete.

  3. Sign in into the Power BI Portal using your Azure credentials.

  4. From the hamburger menu select Workspaces to access the list of workspaces available to you. Select your workspace.

    The workspaces button from the hamburger menu is selected to list workspaces available.

  5. Select the Settings icon from the top right bar, and select Settings again to navigate to the settings page.

    The settings button on the Power BI portal clicked and the Settings selection on the context menu selected.

  6. Select datasets tab to access the list of datasets available. Then select 2-Billion Rows Demo dataset to access its settings. From the settings page open Data source credentials and select Edit credentials.

    The datasets tab is selected. From the list of datasets 2-Billion Rows Demo is selected. Edit credentials will be selected next.

  7. Select Microsoft Account for the Authentication method and select Sign In to complete the process.

    From the list of authentication methods Microsoft Account is picked. The sign in button is selected.

asa-vtd's People

Contributors

ctesta-oneillmsft avatar givenscj avatar joelhulen avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.