CODEWITHSUNDEEP
amazon-s3amazon-redshiftamazon-quicksight
Raviteja Gubba
Raviteja Gubba

Reputation: 1116

is redshift really required when quicksight can query directly from s3 using athena?

We have data dumped into S3 buckets and we are using these data to pull some reports in Quicksight some directly accessing s3 as datasource and for other reports, we used Athena to query S3.

At which point, one need to use Redshift? Is there any advantage of using Redshift over S3+Athena?

Upvotes: 1

Views: 1263

Answers (1)

darcoli
darcoli

Reputation: 441

No you might be perfectly fine with just QuickSight, Athena and S3 - it will be relatively cheaper as well if you keep Redshift out of the equation. Athena is based on PRESTO and is pretty comprehensive in terms of functionality for most SQL reporting needs.

You would need Redshift if you approach or hit the QuickSight's SPICE limits and would still like your reports to be snappy and load quickly. From a data engineering side, if you need to update existing records it is easier to micro batch and update records in RedShift. With athena/s3 you also need to take care of optimising the storage format (use orc/parquet file formats, use partitions, not use small files etc...) - it is not rocket science but some people prefer paying for RedShift and not having to worry about that at all. In the end, RedShift will probably scale better when your data grows very large (into the petabyte scale). However, my suggestion would be to keep using Athena and follow it's best practices and only use RedShift if you anticipate huge growth and want to be sure that you can scale the underlying engine on demand (and, of course, are willing to pay extra for it).

Upvotes: 2

Related Questions