CODEWITHSUNDEEP
mysqlruby-on-railsdatabase
freedom
freedom

Reputation: 751

mysql huge operations

I am currently importing a huge CSV file from my iPhone to a rails server. In this case, the server will parse the data and then start inserting rows of data into the database. The CSV file is fairly large and would take a lot time for the operation to end.

Since I am doing this asynchronously, my iPhone is then able to go to other views and do other stuff.

However, when it requests another query in another table.. this will HANG because the first operation is still trying to insert the CSV's information into the database.

Is there a way to resolve this type of issue?

Upvotes: 1

Views: 214

Answers (1)

enjayem
enjayem

Reputation: 951

As long as the phone doesn't care when the database insert is complete, you might want to try storing the CSV file in a tmp directory on your server and then have a script write from that file to the database. Or simply store it in memory. That way, once the phone has posted the CSV file, it can move on to other things while the script handles the database inserts asynchronously. And yes, @Barmar is right about using an InnoDB engine rather than MyISAM (which may be default in some configurations).

Or, you might want to consider enabling "low-priority updates" which will delay write calls until all pending read calls have finished. See this article about MySQL table locking. (I'm not sure what exactly you say is hanging: the update, or reads while performing the update…)

Regardless, if you are posting the data asynchronously from your phone (i.e., not from the UI thread), it shouldn't be an issue as long as you don't try to use more than the maximum number of concurrent HTTP connections.

Upvotes: 1

Related Questions