Giter VIP home page Giter VIP logo

azure-sql-db-hyperscale-autoscaler's Introduction

page_type languages products description urlFragment
sample
tsql
csharp
sql
json
azure-sql-database
azure-functions
dotnet
azure
Automatically scale up or down Azure SQL Hyperscale depending on active workload
azure-sql-hyperscale-autoscaler

Azure SQL Hyperscale Autoscaler

License

This is a sample on how autoscaling of Azure SQL DB Hyperscale can be implemented using Azure Functions. The code just uses a simple moving average on the CPU load for the last minute; if the value is outside minimum or maximum boundaries it will initiate a scale-up or scale-down.

A detailed article related to this repository is available here:

Autoscaling Azure SQL Hyperscale

Scaling up or down is pretty fast in Hyperscale so responding to workload spikes can be done pretty quickly.

Deploy

Azure SQL

Azure Function stores autoscaler data right into the monitored database itself, in the dbo.AutoscalerMonitor table. This is useful both to understand how and why the autoscaler took some actions, but also if you want to save historical data to create better autoscaling algorithms. Please use the script ./SQL/create-table to setup the objects in the target database before running the Azure Function. If you plan to use the autoscaler in a production environment, is recommended to use a different database other than the monitored one to store historical autoscaler data.

The provided script also create a sample Numbers table that can be used to execute some load testing to check how the autoscaler works.

Autoscaler data, as an additional sample, is also sent to Application Insight, so autoscler actions can be monitored directly from Azure Portal dashboard.

Azure Function

Deploy the solution to an Azure Function and then add the following application settings:

"AzureSQLConnection": "...",
"HighThreshold": 70,
"LowThreshold": 20,
"vCoreMin": 2,
"vCoreMax": 8,
"RequiredDataPoints": 5
  • AzureSQLConnection: Connection string to Azure SQL Hyperscale to monitor. Make sure the user used to login to the database has the right permission to run ALTER DATABASE command.
  • HighThreshold, LowThreshold: the minium and maximum threshold values after which scaling up or down will be initiated
  • vCoreMax, vCoreMin: the maximum and minimum number of cores you want to use as limits to scale up and down
  • RequiredDataPoints: Number of data points that needs to be gathered before initiating any autoscale action

Test

To the the autoscaler, if you created the Numbers test table, you can run the query ./SQL/load-test.sql to create some workload. It is suggested that you create a new Hyperscale database with 2vCores to run the test. Tool like SQL Query Stress can be used to execute multiple query in parallel. A sample configuration is available in SQL folder: just put the correct connection information and when run it will generate a 80% load on a Gen5 2vCore Hyperscale database. This will be enough to initiate a scale-up action.

Notes

The solution requires Azure Functions 3.0. If you are using Visual Studio 2019 you don't have to do anything special. If you are using Visual Studio code, read here how to make sure you have Azure Function 3.0 installed on your machine: Develop Azure Functions using .NET Core 3.1

Disclaimer

This sample is intended to show how to auto-scale Azure SQL Hyperscale Database, and therefore is not intended to be used in production as is. If you want to use it in production, make sure you correctly understand what is the workload pattern of your database and test if the used moving average can handle it nicely. Unless you have a very predictable and stable workload pattern, it is very likely that a different algorithm other than the simple moving average will be better suited to serve your specific workload. Machine learning can also help here, as it provides solution to the "Demand Forecasting" problem. For example: Auto-train a time-series forecast model, or ARIMA or more in general: Demand Forecasting

How to contribute

All contributions are more than welcome. Please refert to the Code Of Conduct to learn what are the basic rules to follow and then fork the repo and start to submit your PR.

azure-sql-db-hyperscale-autoscaler's People

Contributors

mtrilbybassett avatar yorek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

azure-sql-db-hyperscale-autoscaler's Issues

Code was not updating the RequestedSLO as expected.

We had to change line 200 of the AutoScaler.cs file from ...

var index = availableSlos.IndexOf(currentSLO.ToString());

to...

var index = availableSlos.IndexOf(currentSLO.ToString().ToLower());

Once we did that, the RequestedSLO was showing correctly on the [dbo].[AutoscalerMonitor] table and also the ALTER DATABASE script was executing as expected to scale up/scale down the server.

Neither were happening with the old line 200.

I am also a Microsoft employee, you can reach me at [email protected].

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.