CODEWITHSUNDEEP
sqldatabase
eNsirius
eNsirius

Reputation: 69

Storing very big files in database

Is it a good idea to store big files (about 100 GB) in database?
Currently we think about saving data in a folder using NBT format or using mysql/postgresql database.

Upvotes: 6

Views: 17020

Answers (4)

Anthony Accioly
Anthony Accioly

Reputation: 22481

My personal experience is that the database is not a good place for large blobs (unless they support file system aware storage, such as SQL Server FILESTREAM and Oracle BFILE).

There is a reason for most clouds to provide a separate storage for blobs. The life cycle of big data files is different than your typical day to day data... Different life span, different way to serve contents, different caching strategies, different backup plans, etc, etc.

Take a look at:

I would follow their lead and either come up with a "File system aware" storage system (e.g., storing file system paths in your database) or use a separate storage mechanism.

Every time I had to deal with applications that store blobs in the database (images, pdf and such) I've spend much more time dealing with tablespace, backups and performance problems than I did setting backup / static file serving / caching strategies on file system aware solutions.

Upvotes: 3

Michael Durrant
Michael Durrant

Reputation: 96604

Use the file system.

Use the database to store the file location and name

You can also (Unix bases systems) use utilities like cut, paste and join (example)
to work on files at the file level.

Upvotes: 0

Bohemian
Bohemian

Reputation: 425418

It would be a very bad idea to store big files in a database, even if you can.

Just store the file name in the database - leave the file on disk.

The main reason for not stiring big files on gbe database is database backup times become ridiculous, and there's nothing to be gained, and you lose the ability to store your content on distributed storage.

Also recovery time become huge too, if your database becomes corrupted and needs to be rebuilt from backups.

You can do anything with anything, but that doesn't mean you should.

Upvotes: 2

Andyz Smith
Andyz Smith

Reputation: 708

Databases are intended to sort, filter and perform calculations on large numbers of small pieces of data. If you just want to have a file system, (that has limited support for aggregating the total count of files grouped by upload date , for example) just use a file system.

Upvotes: 3

Related Questions