CODEWITHSUNDEEP
javadatabase
Amir
Amir

Reputation: 2258

Should I use a text file or Database?

So I'm putting together an RSS parser which will process an RSS feed, filter it, and then download the matched items. Assume that the files being downloaded are legal torrent files.

Now I need to keep a record of the files that I have already downloaded, so they aren't done again.

I've already got it working with SQLite (create database if not exists, insert row if a select statement returns nothing), but the resulting jar file is 2.5MB+ (due to the sqlite libs).

I'm thinking that if I use a text file, I could cut down the jar file to a few hundred kilobytes.

I could keep a list of the names of files downloaded - one per line - and reading the whole file into memory, search if a file exists, etc.

The few questions that occur to me know:

Anyway, what do you guys think? I could use some advice here, as I'm still new to programming and doing this as a hobby thing :)

Upvotes: 7

Views: 1708

Answers (2)

AlexR
AlexR

Reputation: 115368

Theoretically DB (either relational or NoSQL is better. But if the distribution size is critical for you using file system can be preferable.

The only problem here is the performance of data access (either for write or for read). Probably think about the following approach. Do not use one single file. Use directory that contains several files instead. The file name will contain key (or keys) that allow access specific data just like key in map. In this case you will be able to access data relatively easily and fast.

Probably take a look on XStream. They have implementation of Map that is implemented as described above: stores entries on disk, each entry in separate file.

Upvotes: 2

aleroot
aleroot

Reputation: 72636

If you need to keep track only of few informations (like name of the file), you can for sure use a simple text file.

Using a BufferedReader to read you should achieve good performance.

Upvotes: 4

Related Questions