These are open source and community supported tools. Support is provided on a best effort basis by project contributors.
This repo contains the source code for the Snowflake Maven and Gradle plugins, which will help developers publish User-Defined Functions (UDF) and Stored Procedures for Snowflake.
The plugins can create a stage on Snowflake, copy your build artifact and dependency .jar files to the stage, and run the CREATE... DDL to create your UDF or stored procedure in the account.
Interested in contributing? See the Contributing Guide for guidance.
| Tool | Required Version |
|---|---|
| JDK | 11 |
| Maven | 3 |
Put the following Maven coordinates in the <plugins> block of the POM file.
<plugin>
<groupId>com.snowflake</groupId>
<artifactId>snowflake-maven-plugin</artifactId>
<version>0.1.0</version>
</plugin>You can provide your account authentication information using a properties file or individually specifying your account parameters directly in the plugin config:
Create a file, profile.properties, in the root of the project
with information to establish a JDBC connection to your Snowflake account:
# profile.properties
URL=https://MY_ACCOUNT_NAME.snowflakecomputing.com:443
USER=username
PASSWORD=password
# Optional properties:
ROLE=ACCOUNTADMIN
WAREHOUSE=DEMO_WH
DB=MY_DB
SCHEMA=MY_SCHEMAThen specify this file using the <propertiesFile> tag in the auth section:
<plugin>
<groupId>com.snowflake</groupId>
<artifactId>snowflake-maven-plugin</artifactId>
<version>0.1.0</version>
<configuration>
<auth>
<propertiesFile>profile.properties</propertiesFile>
</auth>
</configuration>
</plugin>Alternatively, you can specify your account information directly in the plugin using the url, user, and password fields. The role, db, and schema fields are optional. An example is shown below.
<plugin>
<groupId>com.snowflake</groupId>
<artifactId>snowflake-maven-plugin</artifactId>
<version>0.1.0</version>
<configuration>
<auth>
<url>https://MY_ACCOUNT_NAME.snowflakecomputing.com:443</url>
<user>myUsername</user>
<password>${env.MY_PASSWORD}</password> <!-- Env var injection for secrets -->
<!-- optional auth configuration -->
<role>accountadmin</role>
<db>${env.MY_ORG_DB}</db>
<schema>${env.SCHEMA}</schema>
</auth>
</configuration>
</plugin>If a properties file and auth fields are specified in the plugin, then the values provided in the plugin are given priority.
Specify UDFs and Stored Procedures objects that should be created on Snowflake
by adding a new <function> tag under <functions> or a <procedure> tag under <procedures> for each object. The arguments follow the CREATE FUNCTION
and CREATE PROCEDURE syntax:
<name>is the name of the UDF/stored proc to be assigned on Snowflake<handler>isclassName.methodNamefor the handler method<args>is a list of<arg>which each require a<name>and<type><returns>is the return type<stage>is the name of the internal stage that will be created (if it doesn't exist) and where files will be uploaded. Note: Choose a new stage name or an existing stage where artifact and dependency.jarfiles can be uploaded.
Example plugin configuration on POM:
<plugin>
<groupId>com.snowflake</groupId>
<artifactId>snowflake-maven-plugin</artifactId>
<version>0.1.0</version>
<configuration>
<auth>
<propertiesFile>profile.properties</propertiesFile>
</auth>
<stage>STAGE_NAME</stage>
<functions>
<function>
<name>funcNameOnSnowflake</name>
<handler>PackageName.ClassName.MethodName</handler>
<args>
<arg>
<name>firstArg</name>
<type>integer</type>
</arg>
<arg>
<name>secondArg</name>
<type>string</type>
</arg>
<!-- More arg go here.. -->
</args>
<returns>string</returns>
</function>
<!-- More functions go here.. -->
</functions>
<procedures>
<procedure>
<name>procNameOnSnowflake</name>
<handler>PackageName.ClassName.SomeMethodName</handler>
<args>
<arg>
<name>a</name>
<type>string</type>
</arg>
<!-- More arg go here.. -->
</args>
<returns>string</returns>
</procedure>
<!-- More procedures go here.. -->
</procedures>
</configuration>
</plugin>After configuration, run:
mvn clean package snowflake:deploy
mvn clean package will build the project and mvn snowflake:deploy executes the plugin goal, deploying your objects to Snowflake.
As mentioned in auth fields, your account properties can be read directly from the environment variables of your CI pipeline. This can be helpful for keeping secrets out of source control, and to deploy to different environments (QA, UAT, production) but just changing the env vars in different pipelines.
Auth parameters can optionally be provided as arguments when running the plugin from the CLI. Values from CLI arguments will override any values set in the properties file or the POM:
mvn snowflake:deploy \
-Ddeploy.auth.user="username" \
-Ddeploy.auth.password="password" \
-Ddeploy.auth.url="myaccount.snowflakecomputing.com" \
-Ddeploy.auth.role="myrole" \
-Ddeploy.auth.db="mydb" \
-Ddeploy.auth.schema="myschema"A single function or procedure can also be specified through command line arguments. The command line function/procedure will be created along with any objects defined in the POM. The arguments have the following syntax:
mvn snowflake:deploy \
-Ddeploy.type="{procedure | function}" \
-Ddeploy.name="<name>" \
-Ddeploy.args="[ <arg_name> <arg_data_type> ] [ , ... ]" \
-Ddeploy.handler="<class>.<handler>" \
-Ddeploy.returns="<data_type>"As an example:
mvn clean package snowflake:deploy \
-Ddeploy.type="procedure" \
-Ddeploy.name="mvnStringConcat" \
-Ddeploy.args="a string, b string" \
-Ddeploy.handler="SimpleUdf.stringConcat" \
-Ddeploy.returns="string"| Tool | Required Version |
|---|---|
| JDK | 11 |
The Gradle plugin has not yet been published to Maven Central or the Gradle Plugin Portal. You can install it locally using the instructions below. Alternbatively, you can use this Gradle plugin developed by Snowflake Data Superhero, Stewart Bryson.
Clone this repository and publish it to your local .m2 repository:
git clone https://github.com/your-username/snowflake-maven-gradle-plugins.git
cd snowflake-maven-gradle-plugins/
gradle publishToMavenLocalSpecify the following at the top of settings.gradle:
pluginManagement {
repositories {
mavenLocal() // local Maven .m2 repository
}
}You can provide your account authentication information using a properties file or individually specifying your account parameters directly in the plugin config:
Create a properties file profile.properties in the root of the project
with information to establish a JDBC connection to your Snowflake account:
# profile.properties
URL=https://MY_ACCOUNT_NAME.snowflakecomputing.com:443
USER=username
PASSWORD=password
# Optional properties:
ROLE=ACCOUNTADMIN
WAREHOUSE=DEMO_WH
DB=MY_DB
SCHEMA=MY_SCHEMAIn your buid.gradle, provide configuration for auth to the plugin:
snowflake {
auth {
propertiesFile = "profile.properties"
}
}Alternatively, you can specify your account information directly in the plugin using the url, user, and password fields. The role, db, and schema fields are optional. An example is shown below.
snowflake {
auth {
url = 'https://MY_ACCOUNT_NAME.snowflakecomputing.com:443'
user = 'myUsername'
password = System.getenv('SNOWFLAKEPWD') // Env var injection for secrets
// Optional:
role = 'accountadmin'
db = 'myDB'
schema = System.getenv('SNOWFLAKESCHEMA')
}
}If a properties file and auth fields are specified in the plugin, then the values provided in the plugin are given priority.
Specify UDFs and Stored Procedures that should be published to Snowflake
by creating a new function closure in the functions block or procedure closure under procedures for each.
The arguments follow the CREATE FUNCTION
and CREATE PROCEDURE syntax:
functionNameorprocedureNameis the name to be used on SnowflakehandlerispackageName.className.methodNamefor the handler methodargsis a list of argument strings for the function which are formatted as "[ <arg_name> <arg_data_type> ] [ , ... ]"returnsis the return typestageis the name of the internal stage that will be created (if it doesn't exist) and where files will be uploaded. Note: Choose a new stage name or an existing stage where artifact and dependency.jarfiles can be uploaded.
Example plugin configuration on POM:
plugins {
id 'com.snowflake.snowflake-gradle-plugin'
}
snowflake {
auth {
propertiesFile = './path/to/file'
}
stage = 'STAGE_NAME'
functions {
functionName {
args = ["a string", "b int"]
returns = "string"
handler = "PackageName.ClassName.methodName"
}
// More functions here
}
procedures {
procedureName {
args = ["a string, b string"]
returns = "string"
handler = "PackageName.ClassName.methodName"
}
// More procedures here
}
}After configuration, run the following to publish your functions and procedures:
gradle snowflakeDeployThe snowflakeDeploy task will trigger the jar task if jar is not up to date.
As mentioned in auth fields, your account properties can be read directly from the environment variables of your CI pipeline. This can be helpful for keeping secrets out of source control, and to deploy to different environments (QA, UAT, production) but just changing the env vars in different pipelines.
Auth parameters can optionally be provided as arguments when running the plugin from the CLI. Values from CLI arguments will override any values set in the properties file or gradle build file:
gradle snowflakeDeploy \
--auth-url="myaccount.snowflakecomputing.com" \
--auth-user="username" \
--auth-password="password" \
--auth-role="myrole" \
--auth-db="mydb" \
--auth-schema="myschema"A single function or procedure can also be specified through command line arguments.
The command line function/procedure will be created along with any defined in build.gradle.
The arguments have the following syntax:
gradle snowflakeDeploy \
--deploy-type="{procedure | function}" \
--deploy-name="<name>" \
--deploy-args="[ <arg_name> <arg_data_type> ] [ , ... ]" \
--deploy-handler="<class>.<handler>" \
--deploy-returns="<data_type>"As an example:
gradle snowflakeDeploy \
--deploy-type="procedure" \
--deploy-name="mvnStringConcat" \
--deploy-args="a string, b string" \
--deploy-handler="SimpleUdf.stringConcat" \
--deploy-returns="string"When uploading to stage, the plugin will structure dependency artifacts like a local .m2 cache,
with directories following an artifact's organization name and version.
By default, build artifacts will overwrite upon each publish but existing dependencies files will not be uploaded again unless the version changes.
Special thanks to...
- Stewart Bryson for guidance and providing a reference in his own Gradle plugin for Snowflake
- Jonathan Cui for bootstrapping the project during his internship at Snowflake
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent