Snowflake Build 2023: The most exciting news had nothing to do with AI
Written by
Dan Luszcz, Senior Manager
Published
December 12, 2023
Highlights from Snowflake Build
Snowflake Native Apps were showcased throughout the conference, either as the focal point of presentations or the tool of choice for demonstrating other features. Unfortunately, there were not any exciting reveals on the Native Apps front, but it was good to hear that support in Azure environments should be coming soon. The main attraction was Snowflake’s upcoming AI and LLM tooling. We are excited to start using these new LLM tools, specifically Cortex, and we know our clients are as well. On that note, one of the biggest disappointments of the event was how few of these new tools are available to the public at this time.
Buried in the lineup of large language models, artificial intelligence, and native app sessions was something even more exciting: Snowflake is releasing support for Git integration and declarative database change management.
While not as flashy as a computer that will write Christmas themed jokes for you, this will have a much wider impact on our clients and is a huge differentiator for Snowflake.
The Power of Declarative Schema Management
Database change management has been a challenge in the industry for years. Tooling for source control, collaboration, and automated deployments for applications and cloud infrastructure are robust and best practice at every company. Unfortunately, the same cannot be said for database schema management. Due to a lack of quality tooling, too many companies continue to manage database schema by hand and use little or no source control.
The problem centers around imperative vs declarative schema management tools. Most tools on the market today are imperative, like Flyway or Schemachange, which handle schema management via a series of create/alter scripts which must be executed in sequence. They enable automated deployments and source control, but are cumbersome to use, require extremely strict discipline on the development team, make collaboration challenging, and ultimately bring too many disadvantages for widespread adoption. Up until now, a combination of Terraform and an imperative tool was the only way to perform automated deployments to Snowflake.
Declarative schema management is significantly more powerful. Table definitions are maintained simply as CREATE scripts. When deployed, a schema compare tool identifies the differences between the target database and the table definition, generates the necessary ALTER statements, and updates the target database accordingly. In a declarative repository, users can look at version history on a single file and see how the table has evolved over time, compared to an imperative repository where they would need to identify every create/alter script that impacted the table which can be extremely challenging. It also makes it much easier for developers to collaborate on a schema without stepping on each other’s toes.
The only major player in the declarative space was Microsoft SQL Server, which enabled users to set up a SQL Project and deploy schema changes via a schema compare tool in a completely automated fashion. This works great, but only if you are using SQL Server. This is why it is such a big deal that Snowflake is now releasing their own tooling to support declarative schema management.
Getting Started with Snowflake’s New DevOps Features
Source control and schema management will be accomplished by using three key Snowflake functions:
1. Git Integration, which essentially lets you to setup a Snowflake stage that’s connected to your Git repository
2. EXECUTE IMMEDIATE, which lets you run scripts directly from a stage (see above)
3. CREATE OR ALTER, which is a new DDL command which will perform a schema compare and update the target table to match the new schema in your latest definition
Through the combination of these three new features, you can now maintain your entire database schema in git, organized however you like, and deployed automatically via the Snowpark CLI. The icing on the cake: it’s baked right into Snowflake, so no additional licensing or security concerns to worry about.
Get Started
OneSix is here to help your organization build the strategy, technology, and teams you need to unlock the power of your data for AI.