CODEWITHSUNDEEP
pythonexcelcsvsqlite
Saimu
Saimu

Reputation: 1362

sqlite3 or CSV files

First of all, I'm a total noob at this. I've been working on setting up a small GUI app to play with database, mostly Microsoft Excel files with large numbers of rows. I want to be able to display a portion of it, I want to be able to choose the columns I'm working with through the menu so I can perform different task very efficiently.

I've been looking into the .CSV files. I can create some sort of list or dictionary with it (Not sure) or I could just import the Excel table into a database then do w/e I need to with my GUI. Now my question is, for this type of task I just described, which of the 2 methods would be best suited? (Feel free to tell me if there is a better one)

Upvotes: 1

Views: 182

Answers (1)

miku
miku

Reputation: 188074

It will depend upon the requirements of you application and how you plan to extend or maintain it in the future.

A few points in favour of sqlite:

  • standardized interface, SQL - with CSV you would create some custom logic to select columns or filter rows
  • performance on bigger data sets - it might be difficult to load 10M rows of CSV into memory, whereas handling 10M rows in sqlite won't be a problem
  • sqlite3 is in the python standard library (but then, CSV is too)

That said, also take a look at pandas, which makes working with tabular data that fits in memory a breeze. Plus pandas will happily import data directly from Excel and other sources: http://pandas.pydata.org/pandas-docs/stable/io.html

Upvotes: 3

Related Questions