AWS DMS Snowflake

Replicating Database to Snowflake with AWS DMS •

AWS DMS or the Database Migration Service of the Amazon Web Service is a cloud-based service that optimizes database migration between on-premises databases to the cloud, cloud providers, relational databases, data warehouses, and NoSQL databases. All the benefits of the cloud such as increased speed and database performance can be had with AWS DMS. It is also used by businesses to mobilize data with unlimited scale, concurrency, and performance.

•

AWS DMS Snowflake is capable of delivering seamlessly the Data Cloud on AWS. Here, businesses can combine their data in silos, discover and share governed data, and carry out multiple and diverse workloads.

•

Snowflake delivers a smooth experience between different public clouds, helping organizations, users, and data providers to take their businesses to new heights in the Data Cloud. Further, a continuous inflow of data can be processed and loaded into a database of Snowflake with the services offered by the AWS DMS Snowflake on the AWS Cloud.

How is this done?

First Amazon Kinesis Firehose delivers the data to Amazon Simple Storage Service (S3). Once received, the Amazon Simple Notification Service (Amazon SNS) sends a notification to Snowflake Snowpipe that loads the data into a Snowflake database. The problem here is that the Kinesis Data Firehose cannot be directly connected to Snowpipe.

This AWS DMS Snowflake process is completely automated with users having continuous access to data generated in real-time, avoiding multiple manual COPY commands, and getting the benefit of support for semi-structured data on load.

To enable AWS DMS Snowflake, the Snowflake account must have the required permissions to create storage integrations with data present concurrently in a table in Snowflake.