CODEWITHSUNDEEP
c#filefilesystemsqueue
Dmytro Shevchenko
Dmytro Shevchenko

Reputation: 34611

How to resize a file, "trimming" its beginning?

I am implementing a file-based queue of serialized objects, using C#.

From the standpoint of file system, that would just mean copying file header several bytes further on the disk, and then moving the "beginning of the file" pointer. The question is how to implement this in C#? Is it at all possible?

Upvotes: 2

Views: 355

Answers (2)

NSGaga
NSGaga

Reputation: 14312

Easiest that I can see

1) stream out (like a log, dump it into file),
(note: you'd need some delimiters and a 'consistent format' of your 'file' - based on what your data is)

2) and later stream in (just read file from start, in one go, and process w/o removing anything)

and that'd work fine, FIFO (first in first out).

So, my suggestion - don't try to optimize that by removing, skipping etc. (rather regroup and use more files.

3) If you worry about the scale of things - then just 'partition' that into small enough files, e.g. each 100 or 1,000 records (depends, do some calculations).

You may need to make some sort of 'virtualizer' here, which maps files, keeps track of your 'database' as, if it's over multiple files. The simplest is to just use the file-system and check file times etc. Or add some basic code to improve that.

However, I think you may have problems if you have to ensure 'transactions' - i.e. what if things fail so you need to keep track of where the file left off, retrace etc.

That might be an issue, but you know best if it's really necessary to have that (how critical). You can always work 'per file' and per smaller files. If it fails, rollback and do the file again (or log problems). If it succeeds you can delete file (after success) and go on like that.

This is very 'hand made' approach but should get you going with a simple and not too demanding solution (like you're describing). Or something along those lines.

I should probably add...

You could also save you some trouble and use some portable database for that or something similar. This is was purely based on the idea of hand-coding a simplest solution (and we could probably come up with something smarter, but being late this is what I have :).

Upvotes: 1

David Heffernan
David Heffernan

Reputation: 613302

Files don't work that way. You can trim off the end, but not the beginning. In order to mutate a file to remove content at the beginning you need to re-write the entire file.

I expect you'll want to find some other way to solve your problem. But a linear file is totally inappropriate for representing a FIFO queue.

Upvotes: 1

Related Questions