CODEWITHSUNDEEP
amazon-web-servicessnowflake-cloud-data-platformsnowflake-schema
Atharv Thakur
Atharv Thakur

Reputation: 701

How to make Snowflake as application back end data base for fast search

To save storage cost we are planning to migrate from Aurora/Mysql to Snowflake for one of our use case where we store Audit related information . We Store all Audit info in Aurora to gives us milliseconds latency when we combine this Aurora into Application .

We do have huge amount of Audit info size is 12 TB and has Text column also and it is growing . Now to save cost and keeping future growth in mind we are exploring other option when we can save money and performance also can match . while doing research cam to know about Snowflake and we are doing POC on this but i observe the search on ID on primary key does not give us performance same as Aurora Mysql .

So wanted some expert advice how can we make Snowflake as our application Back end where i can do Insert/Update/Delete and display record directly from Snowflake Database .

Upvotes: 2

Views: 1047

Answers (1)

Felipe Hoffa
Felipe Hoffa

Reputation: 59325

2022 update

Things have changed since my reply below!

  • Check the Snowflake Search Optimization Service:

The search optimization service can significantly improve the performance of certain types of lookup and analytical queries that use an extensive set of predicates for filtering. https://docs.snowflake.com/en/user-guide/search-optimization-service.html

  • Unistore and Hybrid Tables are coming to Snowflake:

Unistore is a new workload that delivers a modern approach to working with transactional and analytical data together in a single platform. https://www.snowflake.com/blog/introducing-unistore/

Don't do this.

I read from the requirements in the question that you are looking for a backend that will:

  • Retrieve rows by id in milliseconds.
  • Be a backend for an app that's constantly performing updates and deletes.

Those are not the strengths of Snowflake, nor what people love it for.

Read more about the strengths of Snowflake and the workloads you would use it for at https://www.snowflake.com/cloud-data-platform/.

Upvotes: 3

Related Questions