CODEWITHSUNDEEP
c++stringtext-files
nauti
nauti

Reputation: 1446

c++ Reading big text file to string (bigger than string::max_size)

I have a huge text file (~5GB) which is the database for my program. During run this database is read completely many times with string functions like string::find(), string::at(), string::substr()...

The problem is that this text file cannot be loaded in one string, because string::max_size is definitely too small.

How would you implement this? I had the idea of loading a part to string->reading->closing->loading another part to same string->reading->closing->... Is there a better/more efficient way?

Upvotes: 2

Views: 684

Answers (2)

Ben Voigt
Ben Voigt

Reputation: 283981

How would you implement this?

With a real database, for instance SQLite. The performance improvement from having indexes is more than going to make up for your time learning another API.

Upvotes: 1

OriginalCliche
OriginalCliche

Reputation: 401

Since this is a database, I'm assuming it'd have many records. That to me implies best idea would be to implement a data class for each records and populate a list/vector/etc depending upon how you plan to use it. I'd also look into persistent cache as the file is big.

And within in your container class of all records, you could implement search etc functions as you see fit. But as suggested for a db of this size, you're probably best of using a database.

Upvotes: 0

Related Questions