Giter VIP home page Giter VIP logo

sql-action's Introduction

Azure SQL Deploy

This repository contains the sql-action GitHub Action for deploying changes to Azure SQL Database, Azure SQL Managed Instance, or SQL Server in a dacpac, SQL scripts, or an SDK-style SQL project. With the Azure SQL Action for GitHub, you can automate your workflow to deploy updates to Azure SQL or SQL Server.

Get started today with a free Azure account!

Looking to develop with SQL for free, locally and offline, before deploying with GitHub? Check out the Azure SQL local emulator and SQL Server Developer Edition!

🚀 Usage

The definition of this GitHub Action is in action.yml. Learn more in the user guide.

- uses: azure/[email protected]
  with:
    # required, connection string incl the database and user authentication information
    connection-string:

    # required, path to either a .sql, .dacpac, or .sqlproj file
    path:

    # optional when using a .sql script, required otherwise
    # sqlpackage action on the .dacpac or .sqlproj file, supported options are: Publish, Script, DeployReport, DriftReport
    action:

    # optional additional sqlpackage or go-sqlcmd arguments
    arguments:

    # optional additional dotnet build options when building a database project file
    build-arguments:

🎨 Samples

Build and deploy a SQL project

Note: The database project must use the Microsoft.Build.Sql SDK.

# .github/workflows/sql-deploy.yml
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: azure/[email protected]
      with:        
        connection-string: ${{ secrets.AZURE_SQL_CONNECTION_STRING }}
        path: './Database.sqlproj'
        action: 'publish'
        build-arguments: '-c Release'                 # Optional build options passed to dotnet build
        arguments: '/p:DropObjectsNotInSource=true'   # Optional properties and parameters for SqlPackage Publish

Deploy SQL scripts to an Azure SQL Database with a temporary firewall rule

# .github/workflows/sql-deploy.yml
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: azure/login@v1                            # Azure login required to add a temporary firewall rule
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}
    - uses: azure/[email protected]
      with:        
        connection-string: ${{ secrets.AZURE_SQL_CONNECTION_STRING }}
        path: './sqlscripts/*.sql'

Deploy a DACPAC to an Azure SQL database with Allow Azure Services access enabled

# .github/workflows/sql-deploy.yml
on: [push]

jobs:
  build:
    runs-on: windows-latest
    steps:
    - uses: actions/checkout@v3
    - uses: azure/[email protected]
      with:
        connection-string: ${{ secrets.AZURE_SQL_CONNECTION_STRING }}
        path: './Database.dacpac'
        action: 'publish'
        arguments: '/p:DropObjectsNotInSource=true'   # Optional properties parameters for SqlPackage Publish

📓 User Guide

Authentication and Connection String

The v1.x version of sql-action supports SQL authentication only in the connection string. Starting in v2, AAD Password, AAD Service Principal, and AAD Default authentications are also supported. Read more about implementing sql-action with different authentication methods in the connection guide.

Arguments

sql-action supports passing arguments to SqlPackage, go-sqlcmd, and dotnet build.

  • SqlPackage: SqlPackage publish properties are passed to the SqlPackage utility from the arguments property. More information on these properties is available in the SqlPackage publish documentation. SqlPackage parameters that do not impact the source or target setting are also valid, including /Profile: for a publish profile, /DeployReportPath: for a deployment report, and /Variables: to set SQLCMD variable values.
  • go-sqlcmd: go-sqlcmd parameters are passed to the go-sqlcmd utility from the arguments property. This enables SQLCMD variables -v to be passed to scripts as seen in the sqlcmd documentation.
  • dotnet build: dotnet build options are passed to the SQL project build step from the build-arguments property. More information on options is available in the dotnet build documentation.

Environments

sql-action is supported on both Windows and Linux environments. The default images include the prerequisites:

  • sqlpackage (for sqlproj or dacpac deployment)
  • dotnet (for sqlproj build)

Firewall Rules/Access

Note

This Firewall Rules section of the document is specific to Azure SQL Database. For Azure SQL Managed Instance and SQL Server it is recommended to review the connection guide.

If you can use the option Allow Azure Services and resources to access this server on Azure SQL Database, you are all set and you don't need to to anything else to allow GitHub Action to connect to your Azure SQL Database.

If you cannot use the aforementioned option on Azure SQL Database, the action can automatically add and remove a SQL server firewall rule specific to the GitHub Action runner's IP address. Without the firewall rule, the runner cannot communicate with Azure SQL Database. Read more about this in the connection guide.

Azure Credentials for Login (quickstart)

To enable the action to automatically add/remove a firewall rule, add an Azure/login step before the sql-action step. Also, the service principal used in the Azure login action needs to have elevated permissions, i.e. membership in SQL Security Manager RBAC role, or a similarly high permission in the database to create the firewall rule. Read more about this and other authentication methods in the connection guide.

Paste the output of the below az cli command as the value of secret variable, for example AZURE_CREDENTIALS.

az ad sp create-for-rbac --role contributor --sdk-auth --name "sqldeployserviceprincipal" \
  --scopes /subscriptions/{subscription-id}/resourceGroups/{resource-group}

Replace {subscription-id}, {resource-group} with the subscription ID and resource group of the Azure SQL server

The command should output a JSON object similar to this:

{
  "clientId": "<GUID>",
  "clientSecret": "<GUID>",
  "subscriptionId": "<GUID>",
  "tenantId": "<GUID>",
  // ...
} 

Secrets

All the above examples use {{secrets.AZURE_SQL}} syntax for sensitive information, where content such as connection strings are stored in GitHub secrets. To create secrets in GitHub, navigate within your repository to Settings and then Secrets. Be careful to check the connection string which you copy from Azure SQL as the connection string has this Password={your_password} and you will need to supply the correct password for your connection string.

📦 End-to-End Examples

Create Azure SQL Database + SQL Projects

  1. Follow the tutorial Azure SQL Quickstart to create a single database
  2. Copy the below template and paste the contents in .github/workflows/ in your project repository as sql-workflow.yml.
# .github/workflows/sql-workflow.yml
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: azure/[email protected]
      with:        
        connection-string: ${{ secrets.AZURE_SQL_CONNECTION_STRING }}
        path: './Database.sqlproj'
        action: 'publish'
  1. Place the connection string from the Azure Portal in GitHub secrets as AZURE_SQL_CONNECTION_STRING. Connection string format is: Server=<server.database.windows.net>;User ID=<user>;Password=<password>;Initial Catalog=<database>.
  2. Copy the below SQL project template and paste the content in your project repository as Database.sqlproj.
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build">
  <Sdk Name="Microsoft.Build.Sql" Version="0.1.3-preview" />
  <PropertyGroup>
    <Name>reactions</Name>
    <DSP>Microsoft.Data.Tools.Schema.Sql.SqlAzureV12DatabaseSchemaProvider</DSP>
    <ModelCollation>1033, CI</ModelCollation>
  </PropertyGroup>
</Project>
  1. Place any additional SQL object definitions in the project folder or in subfolders. An example table to get you started is:
CREATE TABLE [dbo].[Product](
	[ProductID] [int] IDENTITY(1,1) PRIMARY KEY NOT NULL,
	[Name] [nvarchar](100) NOT NULL,
	[ProductNumber] [nvarchar](25) NOT NULL,
	[Color] [nvarchar](15) NULL,
	[StandardCost] [money] NOT NULL,
	[ListPrice] [money] NOT NULL,
	[Size] [nvarchar](5) NULL,
	[Weight] [decimal](8, 2) NULL,
	[ProductCategoryID] [int] NULL,
	[ProductModelID] [int] NULL,
	[ModifiedDate] [datetime] NOT NULL
)
  1. Commit and push your project to GitHub repository, you should see a new GitHub Action initiated in Actions tab.
  2. For further use of SQL projects in VS Code and Azure Data Studio, check out http://aka.ms/azuredatastudio-sqlprojects for more information.

Create Azure SQL Database + Deploy Existing Schema (dacpac)

  1. Create a dacpac from an existing SQL database with either SSMS, Azure Data Studio or SqlPackage CLI. Place the dacpac file at the root of your repository.
  2. Follow the tutorial Azure SQL Quickstart to create a single database
  3. Copy the below template and paste the contents in .github/workflows/ in your project repository as sql-workflow.yml, changing the dacpac file name as appropriate.
# .github/workflows/sql-workflow.yml
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: azure/[email protected]
      with:        
        connection-string: ${{ secrets.AZURE_SQL_CONNECTION_STRING }}
        path: './PreviousDatabase.dacpac'
        action: 'publish'
  1. Place the connection string from the Azure Portal in GitHub secrets as AZURE_SQL_CONNECTION_STRING. Connection string format is: Server=<server.database.windows.net>;User ID=<user>;Password=<password>;Initial Catalog=<database>.
  2. Commit and push your project to GitHub repository, you should see a new GitHub Action initiated in Actions tab.

✏️ Contributing

For more information on contributing to this project, please see Contributing.

sql-action's People

Contributors

aksm-ms avatar amrutakawade avatar asranja avatar balaga-gayatri avatar caohai avatar chlafreniere avatar dependabot[bot] avatar dzsquared avatar emil-eklund avatar ensignpayton avatar florentato avatar gsuttie avatar joshfieldstad avatar kanika1894 avatar kaverma avatar llali avatar michaelcurrin avatar microsoftopensource avatar msftgits avatar n-usha avatar shueybubbles avatar starblind avatar yorek avatar zainuvk avatar zijchen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sql-action's Issues

detectIPAddress method results in error due to db name not being specified

The SqlUtils.detectIPAddress method that is testing the connection results in the following error...

The server principal "(db name)" is not able to access the database "master" under the current security context...

due to the database name not being specified. Can -d "${connectionString.database} be added to the following line?

await exec.exec(`"${sqlCmdPath}" -S ${serverName} -U "${connectionString.userId}" -P "${connectionString.password}" -Q "select getdate()"`, [], {

Firewall

Any idea how I figure out the firewall part:-

Failed to add firewall rule. Unable to detect client IP Address. Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Named Pipes Provider: Could not open a connection to SQL Server [53]. .

Getting Failed to add firewall rule error during SQL DB deployment using GitHub Hosted Runner

Getting Failed to add firewall rule error during SQL DB deployment using GitHub Hosted Runner although 'Allow Azure Services And resources to access this server" is checked in the SQL server

Error: Failed to add firewall rule. Unable to detect client IP Address. Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login failed for user 'MCSSQLADMIN'..
Error: The process 'C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\SQLCMD.exe' failed with exit code 1

DeployReport support

Looking at the comment here: https://github.com/Azure/sql-action/blob/master/src/AzureSqlAction.ts#L29
It appears only the publish /action is supported at present. What we would like is for the DeployReport to be supported so as to provide a schema compare given the DDL changes to the target compared to the source dacpac.

I think this addition would greatly lead to adoption as right now you are blind to the net changes that are being made to the target until after those changes are published

@v1.1: Error: Cannot find module '@actions/core'

Hi there,

I am getting below error when running with @v1.1:

Error: Cannot find module '@actions/core'

Require stack:
- /actions-runner/_work/_actions/azure/sql-action/v1.1/lib/main.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:797:15)
    at Function.Module._load (internal/modules/cjs/loader.js:690:27)
    at Module.require (internal/modules/cjs/loader.js:852:19)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object.<anonymous> (/actions-runner/_work/_actions/azure/sql-action/v1.1/lib/main.js:34:27)
    at Module._compile (internal/modules/cjs/loader.js:959:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.Module._load (internal/modules/cjs/loader.js:727:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/actions-runner/_work/_actions/azure/sql-action/v1.1/lib/main.js'
  ]
}

It seems like the node_modules directory is missing from the v1.1 release? Can you help to take a look?

Thanks,
Yi

Allow "Database" instead of requiring "Initial Catalog" in connection string

Database and Initial Catalog are interchangeable in a MSSQL connection string: https://docs.microsoft.com/dotnet/api/system.data.sqlclient.sqlconnectionstringbuilder.initialcatalog#remarks

However if only "Database" is supplied to sql-action, following error is thrown:

Error: Missing required keys in connection string. Please ensure that the keys 'User Id', 'Password', 'Initial Catalog' are provided in the connection string.

Failed to add firewall rule - Using self-hosted runner and Deny public access in AzureSQL server

I have an error when deploying a bacpac from a self hosted runner...I have a sql server but denying public access in the sql server firewall options (obviously this disables the option to add any rule by runner ip)

How could I deploy a bacpac from a self hosted runner to a server that only uses private endpoint. I need to automate this action.

"Error: Failed to add firewall rule. Unable to detect client IP Address. Error: There was a error when attempting to execute the process 'sqlcmd'. This may indicate the process failed to start. Error: spawn sqlcmd ENOENT."

sql-action failing to deploy to Azure Gov for Synapse Workspace, but works in Azure Commercial

I am trying to deploy dacpac to Synapse Workspace. This works in Azure Commercial but not in Azure Gov. I am getting this error below.

Error: Unable to get details of SQL server servername.sql.azuresynapse.usgovcloudapi.net. Sql server servername.sql.azuresynapse.usgovcloudapi.net' was not found in the subscription.

  • uses: azure/sql-action@v1
    with:
    server-name: "${{ env.DB_Server_Name }}.sql.azuresynapse.usgovcloudapi.net"
    connection-string: "Server=tcp:${{ env.DB_Server_Name }}.sql.azuresynapse.usgovcloudapi.net,1433;Initial Catalog=${{ env.DB_Name }};Persist Security Info=False;User ID=${{ env.SQL_ADMIN_USER_NAME }};Password=${{ env.SQL_ADMIN_USER_PSWD }};MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;"
    dacpac-package: ${{ env.DACPAC_Path }}
    arguments: "/Variables:DataLakeAccessKey="${{ env.DATA_LAKE_ACCESS_KEY }}" /Variables:DataLakeAccessId="${{ env.DATA_LAKE_ACCESS_ID }}" /Variables:DataLakeEncryptionMasterKey="${{ env.DATA_LAKE_ENCRYPTION_MASTER_KEY }}""

Questions:
Is it possible that some resource providers are not exposed in Azure Gov?
I tried to deploy dacpac package via sqlpackage.exe and that works. Only the issue with this GitHub action.

Unable to authenticate to Azure SQL Server via service principal

Hi,

Not sure if this is the correct issue template to use, but I am currently working with a customer to build out workflows to deploy DACPACs into their Azure SQL instance. We have created a service principal, but we keep bumping into the following error:

Failed to add firewall rule. Unable to detect client IP address.

Currently, our action is configured as follows:

      - name: Azure SQL Deploy
        uses: Azure/[email protected]
        with:
          connection-string: 'Server=${{ env.serverName}};User ID=***;Password=***;Initial Catalog=${{ env.catalog_name }} ;Authentication=ActiveDirectoryServicePrincipal;'
          dacpac-package: MyDacpac.dacpac
          arguments: '/p:IncludeCompositeObjects=true' 

Are service principals currently not supported by this task?

Proposal: v2 of sql-action

The team has been discussing improvements to sql-action that would also require some breaking changes. As we prepare to implement changes we wanted to gather feedback and suggestions by sharing the proposed changes here. Please feel free to drop comments on this thread and if any topic gets detailed we might move it over to "discussions" for better thread support.

Issues mentioned below will be tagged with a v2-candidate tag soon.


1. Azure Active Directory support

We'd like to support AAD auth for username/password, service principal, and managed identity. (#60, #78, #93)

To do this smoothly, connectivity testing and script execution would be done directly through node-mssql instead of invoking sqlcmd within the action. The workflow would need to provide a connection string to the action, but any valid connection string supported by Tedious would work.

This change has a side benefit of reducing confusion around the connection string components (#72, #79) and we'd remove the confusing (and no longer used) server name parameter.

This change would remove the ability to use sqlcmd arguments during sql-file script execution.

2. Login credentials that do not have access to master

We'd like to allow sql-script execution by database logins with less access. (#89) This can be incorporated into the change from sqlcmd to node-mssql.

3. Additional actions

There's been interest voiced to have a subset of functionality from SqlPackage (#55) as well as the ability to build but not deploy a SQL project.


Proposed yml interface

Within reason, we need to make some changes to the action.yml definition to support additional actions and remove the unused server-name parameter.

name: 'Azure SQL Deploy'
description: 'Deploy a database project, dacpac, or a SQL script to Azure SQL database'
inputs:
  connection-string:
    description:  'The connection string, including authentication information, for the Azure SQL Server database.'
    required: true
  path:
    description: 'Path to file or files to use. Wildcard supported in path for .sql files.'
    required: true
  action:
    description: 'Selection for action to perform on SQL scripts, project file, or dacpac. Options: Script, Publish, PublishScript, PublishReport, NoDeploy'
    required: true
  arguments:
    description: 'In case of dacpac or sql project files are selected, additional SqlPackage arguments that will be applied. When script action is selected, additional go-sqlcmd arguments that will be applied.'
    required: false
  build-arguments:
    description: 'In case of sql project file is selected, additional arguments that will be applied to dotnet build when building the database project.'
    required: false
runs:
  using: 'node12'
  main: 'lib/main.js'

Breaking Changes

  1. server-name parameter is proposed to be completely removed
  2. sqlcmd arguments are proposed to no longer be available for script execution

Sound off!

Here's some ideas:

  • Will these changes benefit your SQL deployments?
  • Are we removing anything you use? (sqlcmd arguments)
  • What are we not tackling that you would like to see?

Speed up download time with Webpack

I notice that the node_module folder is included in the tag of the publication and need it at run time, but it's very big. I modified to use Webpack to package the project, In my sel-hosted, the Set Up Job step saves 15 seconds. so I recommend using it and remove node_module from tag.

I submitted an RP #68 and I tested it very well

Firewall Failure

Hi,

I was building a workflow to run a sql script, the workflow is giving me errors related to firewall. (using github hosted runner)
Failed to add firewall rule. Unable to detect client IP Address. Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Named Pipes Provider: Could not open a connection to SQL Server [53]. . Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login timeout expired. Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.. Error: The process 'C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\SQLCMD.exe' failed with exit code 1

we have the Allow access to azure serves on in our database.
MicrosoftTeams-image (1)

As per the documentation, we wouldn't need to login if this is enabled.
Even then, tried logging in with a service principal that has appropriate accesses.
Still, getting similar error.
The DB connection string that I am using
server="host_name.mysql.database.azure.com"; port=3306; initial catalog="db_name"; user id="user@host_name"; password="************"; SslMode=Preferred;
tried setting the sslmode=Required and even removing the sslmode from the connection string.
Would appreciate any help here.

Thanks

Action does not support AccessToken\ServicePrincipal

The current task requires a connection string with a hardcoded user name and password (for internal validation). I would like to request adding the feature to support either AccessToken based connection strings, or directly configuring the task to use service principal credentials to acquire and use the token internally.

Action not accepting multiple sql files or folder (despite being documented)

As per the documentation here:

  sql-file:
    description: 'Path to SQL script file. *.sql or a folder to deploy'

So we should be able to pass a directory name or pass wildcard like *.sql. But when done so, it fails complaining:
(for wildcard)

Error: Muliple files found matching pattern ./abc/xyz/*.sql

and , for folder,

Error: Invalid sql file path provided as input ./abc/xyz/

Is this feature not supported yet? If so, can the documentation please be corrected?

dacpac file

The docs don't mention anything about how to create the dacpac and where to place it etc - for others working through this

sql-action V2 does not appear to be working with on-premises SQL Server

I have mentioned this to Drew Skwiers-Koballa casually in the past but thought better raise it here for visibility.

I seem to have issues using sql-action V2 to deploy to SQL Server on-premises. I am using a self-hosted GitHub Runner that works perfectly well with sql-action v1.3.

One of the types of errors I get is below, but it changes depending on which connection string format I use:

"Error: Failed to add firewall rule. Unable to detect client IP Address. unable to open tcp connection with host 'localhost:1433': dial tcp [::1]:1433: connectex: No connection could be made because the target machine actively refused it."

Maybe there is a workaround for using it with v2 that is not documented. However, for now I workaround it by using v1.3.

Unable to use it with on-prem SQL Server

Hello!

I'm trying to use this action to deploy some SQL Script to a SQL Container which is spin up as a services container at build time so we can validate the build with unit tests on Github Enterprise on a self-hosted GitHub Actions agent.

The SQL container is defined like this:

    services:
      sql:
        image: mcr.microsoft.com/mssql/server:2022-latest
        env:
          ACCEPT_EULA: Y
          SA_PASSWORD: yourStrong(!)Password
        ports:
          - 1433

The actions setup is as follow:

- uses: azure/[email protected]
        with:        
          connection-string: ${{ env.SQL_CS }}
          sql-file: 'src/*.sql'
        env:
          SQL_CS: "Server=localhost,${{ job.services.sql.ports[1433] }};Initial Catalog=MyDB;User Id=sa;Password=yourStrong(!)Password"

The error is shown like this:

image

Am I missing something? The readme says it is supposed to work with SQL Server, not just with Azure SQL. Can someone shed a light on it?

Thanks!

arguments not passing variable to sqlcmd

I have the following action:

uses: azure/sql-action@v2
with:
  connection-string: ${{ secrets.CONNECTION_STRING }}
  path: './test.sql'
  arguments: '-v client_name="testname"'

According to the documentaiton, this should pass the arguments to the SQLCMD utility but it's not working. The line below shows the sqlcmd statement from the logs and there is no -v option.
sqlcmd -S sqldb-***.database.windows.net -d Auth -U *** -i ./test.sql

I also get this error when the script runs.
'client_name' scripting variable not defined.

This is what the documentation says:
go-sqlcmd: go-sqlcmd parameters are passed to the go-sqlcmd utility from the arguments property. This enables SQLCMD variables -v to be passed to scripts as seen in the sqlcmd documentation.

Scripting Variables not being passed to sqlcmd

When arguments are passed to sqlcmd they are not appended to the running command resulting in undefined variables.

    - name: 'deploy sql script'
      uses: azure/sql-action@v2
      with:
        connection-string: ${{ inputs.connection_string }}
        arguments: '-v environment="${{ inputs.environment }}"'
        path: "${{ inputs.script-path }}"

In the logs I see the command is run without the -v switch
sqlcmd -S ********.database.windows.net -d ******** --authentication-method=ActiveDirectoryDefault -i ./SqlScripts/add-users.sql

Resulting in an error showing the variable as undefined.

'environment' scripting variable not defined.

ServerName should not be needed

Given that the connection string already contains the server name, the property ServerName in the YAML file should be removed. Just like the database name is retrieved from the connection string, also the server name can be retrieved from the connection string, therefore asking one param less to the end user

SQL Managed Instance

Does this action support SQL Managed Instance deployment ?

I followed every step to connect to Azure, and deploy my database which is in SQL MI, only to end up in message:

Unable to get details of SQL server . No SQL servers were found in the subscription.

Got a post where people from Microsoft confirms this action is only limited to Azure SQL.

https://learn.microsoft.com/en-us/answers/questions/1155045/github-deploy-actions-for-azure-sql-managed-instan?page=1&orderBy=Helpful&comment=answer-1155167#newest-answer-comment

Any help on SQL Managed Instance deployment will be greatly appreciated.

Failed to add firewall rule. Unable to detect client IP Address. Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login failed for user 'sql_user'

Hi, I am getting this error "Failed to add firewall rule. Unable to detect client IP Address. Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login failed for user 'sql_user'" when executing the task. My code is:

steps:
- name: Checkout
uses: actions/checkout@v2
- name: Login via Azure CLI
uses: azure/login@v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
- name: Execute SQL Script
uses: azure/sql-action@v1
with:
server-name: xxxxxx.database.windows.net
connection-string: Server=xxxxxx.database.windows.net; User ID=sql_user; Password=xxxxxx Initial Catalog=xxxxx
sql-file: './test.sql'
- name: Azure logout
run: |
az logout

The Azure credentials used has Contributor role. I have tested the same credentials using a powershell action and I've been able to add and remove firewall rules to the sql server.
Regarding the connection string, I can log in the database via SSMS using the values in the connection string. Also tested using different server names like tcp:servername, servername,1443 ... etc.

I don't know if the error thrown by the action is the real one because even without adding the login action I get the same error.

Any help is more than welcome :)
Thanks.

Azure Login is still required, despite closed issue.

The documentation has been updated, but the action still requires the information. No updates were made to the code in the repo, just the readme. Can this be addressed?

Originally posted by @jonthenerd in #10 (comment)

If it is supported to not have to supply Azure Login credentials, an example of how to do so (as just omitting it doesn't work) would be much appreciated.

v2 - Alter Database Failures aren't Retried

Note: I'm not convinced that we need to add retry logic here, but let's chat about this. Creating this issue just to ensure that we don't forget about it.

Link to deployment logs where I ran into this issue:
https://github.com/zijchen/sql-action-samples/runs/8026524238?check_suite_focus=true#step:4:1202

Error SQL72014: Core Microsoft SqlClient Data Provider: Msg 42008, Level 16, State 120, Line 5 ODBC error: State: HY000: Error: 0 Message:'[Microsoft][ODBC Driver 17 for SQL Server]Unspecified error occurred on SQL Server. Connection may have been terminated by the server.'.
Error SQL72045: Script execution error.  The executed script:
IF EXISTS (SELECT 1
           FROM   [sys].[databases]
           WHERE  [name] = N'$(DatabaseName)')
    BEGIN
        ALTER DATABASE [$(DatabaseName)]
            SET ALLOW_SNAPSHOT_ISOLATION OFF;
    END


Error SQL72014: Core Microsoft SqlClient Data Provider: Msg 5069, Level 16, State 1, Line 5 ALTER DATABASE statement failed.
Error SQL72045: Script execution error.  The executed script:
IF EXISTS (SELECT 1
           FROM   [sys].[databases]
           WHERE  [name] = N'$(DatabaseName)')
    BEGIN
        ALTER DATABASE [$(DatabaseName)]
            SET ALLOW_SNAPSHOT_ISOLATION OFF;
    END

Allow for AAD username and password login

The underlying SQLCMD tool allows for login to SQL Server by using an AAD username and password combination or even doing integrated login. This is currently not supported by the GitHub sql-action, since the -G switch is not used for AAD login.

I am currently working on a fully automated setup of an Azure SQL Server database fronted by a web app. Code for this can be found at: https://github.com/Azure-Samples/app-service-networking-samples

However I am bumping in the following restrictions for fully automating this flow:

  • I need to use AAD credentials to be able to allow for mi.sql script, since the script executes CREATE USER accountName FROM EXTERNAL PROVIDER; this needs to be done from logged in AAD account
  • I cannot login with AAD credentials through sql-action

I tried using:

     connection-string: ${{ format('Server={0};Initial Catalog={1};User Id={2};Password={3};AUTHENTICATION = ActiveDirectoryPassword', steps.deploy.outputs.sqlServerFullyQualifiedDomainName, steps.deploy.outputs.databaseName, secrets.AAD_USERNAME, secrets.AAD_PASSWORD) }}

However, the AUTHENTICATION = ActiveDirectoryPassword gets ignored.

On execution I get a 'Failed to add firewall rule', from sqlutils.ts file. However, there the login is done without the -G switch for SQLCMD and hence failing for AAD accounts.

Could be that this same issue exists deeper in the code on execution of the actual SQL command, currently I get a failure on FW settings.

Additionally making this integrated AAD login would be nice as well for people wanting to use account of the runner for log in.

SQLAction DriftReport fails to execute

Problem description/replication steps

When sql-action is invoked like below:

      - name: Generate Drift Report
        uses: azure/sql-action@v2
        with:
          connection-string: ${{ secrets.BAR_CONNECTIONSTRING }}
          path: foo\bar.dacpac
          action: 'DriftReport'
          arguments: "/OutputPath:'${{ github.workspace }}/driftreport'"

The below error is given:

'SourceFile' is not a valid argument for the 'DriftReport' action.

To resolve this, I attempted to remove the path parameter from the azure/sql-action invocation. This throws the below error.

Error: Input required and not supplied: path

Microsoft documentation does not contain a /SourceFile: parameter for the sql package driftreport action. https://learn.microsoft.com/en-us/sql/tools/sqlpackage/sqlpackage-deploy-drift-report?view=sql-server-ver16#driftreport-action-parameters

It looks like this could be resolved by modifying this case statement to only add /SourceFile: for Publish, Script, and Deploy report. https://github.com/Azure/sql-action/blob/master/src/AzureSqlAction.ts#:~:text=case%20SqlPackageAction.,break%3B

Support GO statements in v2

GO statements are sqlcmd batch terminators, but are not native T-SQL statements. Since v2 no longer uses sqlcmd, scripts that have GO in them would no longer work.

v2 - Connection string escaping for Authentication option

Connection string currently requires quotation marks around the Authentication option, otherwise it throws error such as

sqlcmd: error: unexpected argument Directory

Example connection string that fails but should work: 'Server={0};Initial Catalog={1};Authentication=Active Directory Service Principal;User Id={2};Password={3}'.

Action does not work when connection string server name contains "tcp:"

This action fails when the connection string contains a server name with "TCP" e.g. Server=tcp: MY_SERVER.

It fails when creating firewall rules due to filtering azure resources based off the Server value (including the text "tcp:". i.e. it looks for an azure resource with the name tcp: MY_SERVER

The error is thrown:
Unable to get details of SQL server tcp: MY_SERVER. Sql server 'tcp: MY_SERVER' was not found in the subscription.

this._resource = sqlServers.filter((sqlResource) => sqlResource.name === serverName)[0];

If no firewall is required, the connection works fine.

This should also be able to work using non-standard ports. i.e. Server=MY_SERVER,MY_PORT

Can I connect to an Azure SQL database using a managed identity?

If I have an Azure App Service with a managed identity, I can give it db_datareader/db_datawriter access to my database like this:

CREATE USER [$appName] FROM EXTERNAL PROVIDER;
ALTER ROLE db_datareader ADD MEMBER [$appName];
ALTER ROLE db_datawriter ADD MEMBER [$appName];

(Where $appName is the name of the managed identity service principal.)

I can then connect to that database from the app service with a connection string like this:

Server=myserver.database.windows.net,1433;Initial Catalog=mydatabase;Authentication=Active Directory Default

Now, I've just learned that I can give my Azure SQL Server its own managed identity, and I'd love to be able to perform database actions (e.g. Entity Framework migrations) using that instead of maintaining an MSSQL user/password as the database owner.

Would it be possible to integrate this action with azure/login in such a way that the connection string can use the Authentication=Active Directory Default trick to connect using the service principal you've already logged in with? Or maybe you can think of an even easier way to pull this off.

This is all in the spirit of not having to store passwords anywhere. This way I wouldn't even need to store my db_owner password as a GitHub secret.

Thanks!

Action cannot detect IP address of the runner

I am running a yaml pipeline with the following job:

image

The login action uses an SPN that has the role SQL Security Manager, as per the documentation.

Expectation: The action would automatically detect the runner IP, add the firewall rule, and remove it afterward.

Error:
image

Build without deployment

There's a use case where a dacpac might be needed to save to pipeline artifacts or do other deployments - can the action be provided a sqlproj file without connection info to output a dacpac in the pipeline?

support for user authentication without access to master db

Hi gang,

I'm trying to use Azure/sql-action in the "deploy" phase of my workflow when deploying an ASP.NET Core app to Azure.

My Azure SQL server definitely has "Allow Azure services and resources to access this server" set to "Yes", but when my workflow runs I get the following error:

Error: Failed to add firewall rule. Unable to detect client IP Address. Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login failed for user '(redacted)'..

My code looks like this:

  • name: Apply EF migration script
    uses: Azure/sql-action@v1
    with:
    server-name: (my server name).database.windows.net
    connection-string: ${{ secrets.CONNECTIONSTRING_PUCKA }}
    sql-file: ./migrate.sql

In the server-name attribute I've tried it with and without the "tcp:" prefix (which is in the connection string) but I get the same error both ways.

Your README says if I have "allow azure services" set to "yes" then I should be fine. Am I doing something wrong?

Why mandate the Azure SPN login?

Hello team, this action is very useful, however the requirement to have a SPN which is used to adjust the firewall rules is a bit restrictive. In many cases, the target logical Azure SQL server already exists and has firewall rules adjusted already, or maybe the customer is running with self-hosted runner within a private VNET etc. In those cases, this step of mandating the Azure login is functionally not needed. Functionally, an ops person should be able to just specify the connection string and theoretically be able to specify that the firewall rules adjustment are not needed. Eager to know your thoughts on this.

feature request : Add support of Xml PublishProfile configuration

Hi,

Could you add the Xml PublishProfile configuration as an input of the Github action ?

For example what we have for Azure Devops :

  - task: SqlAzureDacpacDeployment@1
    displayName: "Execute Azure SQL : DacpacTask"
    inputs:
      azureSubscription: "${{parameters.serviceConnection}}"
      AuthenticationType: connectionString
      ConnectionString: "$(kv-SQL-ConnectionString)"
      DacpacFile: "$(Pipeline.Workspace)/drop/SQL/bin/Release/.Database.dacpac"
      PublishProfile: "$(Pipeline.Workspace)/drop/SQL/Profile.publish.xml" 

The kind of stuff we can configure in the publish profile :

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <IgnoreIdentitySeed>False</IgnoreIdentitySeed>
    <IgnoreColumnCollation>False</IgnoreColumnCollation>
    <IncludeTransactionalScripts>True</IncludeTransactionalScripts>
    <CreateNewDatabase>False</CreateNewDatabase>
    <BackupDatabaseBeforeChanges>False</BackupDatabaseBeforeChanges>
    **<BlockOnPossibleDataLoss>False</BlockOnPossibleDataLoss>**
    <NoAlterStatementsToChangeCLRTypes>False</NoAlterStatementsToChangeCLRTypes>
    <ProfileVersionNumber>1</ProfileVersionNumber>
    <ScriptDatabaseOptions>False</ScriptDatabaseOptions>
    <IncludeCompositeObjects>True</IncludeCompositeObjects>
    <TargetTimeout>4500</TargetTimeout>
    <IgnoreNotForReplication>True</IgnoreNotForReplication>
    <IgnoreFullTextCatalogFilePath>True</IgnoreFullTextCatalogFilePath>
    <VerifyDeployment>True</VerifyDeployment>
    <IgnorePartitionSchemes>True</IgnorePartitionSchemes>
    <DeployDatabaseInSingleUserMode>True</DeployDatabaseInSingleUserMode>
    <!--Drop and Exclude Objects-->
    <DropObjectsNotInSource>False</DropObjectsNotInSource>
    <IgnoreRoleMembership>True</IgnoreRoleMembership>
    <IgnoreUserSettingsObjects>True</IgnoreUserSettingsObjects>
    <IgnoreUserLoginMappings>True</IgnoreUserLoginMappings>
    <DoNotDropUsers>True</DoNotDropUsers>
    <IgnorePermissions>True</IgnorePermissions>
    <DoNotDropLogins>True</DoNotDropLogins>
    <DoNotDropLinkedServers>True</DoNotDropLinkedServers>
  </PropertyGroup>
</Project>

Thank you,
Kind regards.

[Feature Request] skip add firewall rule

Hi,
I have a customer who has very strict access, they don't want to open a firewall every time and they used self-hosted. could you add a parameter like as.

  skipFirewallSetting:
    description: 'Parameters which skip Firewall Setting'
    required: false
    default: 'false'
        let skipFirewallSetting = (core.getInput('skipFirewallSetting') != 'true');
        
        if(skipFirewallSetting){
            const runnerIPAddress = await SqlUtils.detectIPAddress(inputs.serverName, inputs.connectionString);
            if(runnerIPAddress) {
                let azureResourceAuthorizer = await AuthorizerFactory.getAuthorizer();
                let azureSqlResourceManager = await AzureSqlResourceManager.getResourceManager(inputs.serverName, azureResourceAuthorizer);
                firewallManager = new FirewallManager(azureSqlResourceManager);
                await firewallManager.addFirewallRule(runnerIPAddress);
            }
        }

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.