CODEWITHSUNDEEP
mysqlsqlfileimport
efx
efx

Reputation: 173

Importing a large sql file with MySQL

I'm looking to do some social network analysis in the future. The data I have is stored in a sql file. I am trying to import it to MySQL running on my PC so I can take a look at some of it. I just want to make sure I've done this right. Please see this imgur link.

http://i.imgur.com/C6n0XyJ.png

I only ask because it's been like that for over one hour, which I did expect, but there hasn't been any visual confirmation that it is actually working. "Importing now..." or something would have been nice, or a percentage counter.

I know it's done something, because in a different command line window I did: 

mysql> show databases;

and 'tweetdata' did indeed appear in the list underneath the default MySQL tables (information_schema, mysql, etc.). I just want to make sure it hasn't just died and I did the right command to import the data!

P.S. Yes, you can all see the root password I made for my local SQL server which I made today and is running only on my PC at home with no data on it at all :)

Update:

Thanks for your input Tadman, glad to hear not giving any indication of progress is just how MySQL is. I had this error message after a while, though: 

ERROR 1049 (42000): Unknown database 'disasters'

Here's a screenshot of what I'm looking at: http://i.imgur.com/GRRgAsm.jpg

But when I checked the folder, the tweetdata folder is 14.3GB in size, so I think the import probably worked? How can the db be unknown if it shows up in response to 

show databases;

?

Upvotes: 0

Views: 942

Answers (2)

MacMladen
MacMladen

Reputation: 121

My suggestion would be to try to divide the sql file into smaller chunks so that machine will be able to handle them or to try to do import inside mysql. If you already done import, you may also look in the database itself, how much got imported.

Remember always to check on import if all tables and all records got imported.

Splitting the SQL file

You cannot just arbitrarily split a file but have to take care of integrity of SQL statements, not to break them and also to have proper heading for the statements. Usually there is declaration, like

--
-- Table structure for table `actions`
--

DROP TABLE IF EXISTS `actions`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `actions` (
 /* ... here goes table declaration

followed by

--
-- Dumping data for table `actions`
--

LOCK TABLES `actions` WRITE;
/*!40000 ALTER TABLE `actions` DISABLE KEYS */;
INSERT INTO `actions` VALUES
/* ... entries in brackets ending with comma

and ending with UNLOCK TABLES but your structure may vary.

The important thing is to break before declaration.

Debugging practice would tell you to try to first split in half and try each import. If one or both fail, keep splitting until they succeed or you find where the problem lies.

Use MySQL

When you import database (or execute query) inside MySQL command line, you are not echoed for each successful record but for the whole table and that can also help you find the breaking point.

Take a look at database

You may also look at database and see which is last table or last record that get imported. That way you may also try to look at SQL and find / try to eliminate what was causing problem. By repeating you should end up with complete and proper input.

Upvotes: 0

tadman
tadman

Reputation: 211740

Unfortunately you don't get much feedback when importing a database dump like that. If it's created in transactional mode, you won't even see any data until it's been fully imported, too.

One way to see if it's doing anything is to look at the database directory where the raw table are stored and see which files are being created and how large they are getting. This is made easier with the file_per_table option.

Upvotes: 1

Related Questions