CODEWITHSUNDEEP
ruby-on-railsrubypsql
Hassan-Zahid
Hassan-Zahid

Reputation: 427

Ruby on Rails: Best way to show big data report efficiently

I am implementing a report that shows the analysis of quite a large set of data. The process includes joins and filtering on 4 database tables that themselves have around a million records on average. The complete processing and page rendering was taking around two minutes. I have improved this time to around 1 minute 35 seconds by optimizing my queries and applying indexing to the database.

Considering we cannot further optimize the queries, what are the possible solutions to improve the process speed that are usually used in Ruby on Rails.

Upvotes: 3

Views: 993

Answers (1)

michal.samluk
michal.samluk

Reputation: 352

Maybe you could move generating report to some background job? You have 2 options here: 1. Generate after some action/cron job (e.g. project finished -> generate report) 2. Have a nice button (request report) and then display nice message to user that you are going to email him a link to the report (or even you could display it on frontend after processing is finished).

I once was doing something similar and we were storing full html on S3 after generating report. Caching just the data was not enough - the amount of the data was so big that generating HTML was taking quite some time.

Upvotes: 1

Related Questions