CODEWITHSUNDEEP
pythonexcelhdf5
Paweł Chomicki
Paweł Chomicki

Reputation: 81

How to write big set of data to xls file?

I have really big database which I want write to xlsx/xls file. I already tried to use xlwt, but it allows to write only 65536 rows (some of my tables have more than 72k rows). I also found openpyxl, but it works too slow, and use huge amount of memory for big spreadsheets. Are there any other possibilities to write excel files?

edit: Following kennym's advice i used Optimised Reader and Writer. It is less memory consuming now, but still time consuming. Exporting takes more than hour now (for really big tables- up to 10^6 rows). Are there any other possibilities? Maybe it is possible to export whole table from HDF5 database file to excel, instead of doing it row after row- like it is now in my code?

Upvotes: 8

Views: 7354

Answers (3)

ben_frankly
ben_frankly

Reputation: 9950

Use the Optimized Reader and Writer of the openpyxl package. The optimized reader and writer run much faster and use far less memory than the standard openpyxl methods.

Upvotes: 3

Sahil kalra
Sahil kalra

Reputation: 9064

Try and use XlsxWriter in Constant Memory mode.

For .xls files I fear there's no memory optimized way. Did you find any ?

Upvotes: 4

Billy
Billy

Reputation: 51

XlsxWriter work for me. I try openpyxl but it error. 22k*400 r*c

Upvotes: 1

Related Questions