CODEWITHSUNDEEP
pythonpython-3.xpickle
paul_schaefer
paul_schaefer

Reputation: 441

Efficient way to cyclically update data of a class instance already saved on disk

There is a class Class containing an initial empty list list and a method to add an element to the list. Each time an element was added to the list the whole instance should be saved to disk. So in most cases the already saved data only needs to be extended by the data of the last added element.

Is there an efficient way to achive such a behavior? Deleting the saved data and writing it to disk again seems not to be very efficient. I'm looking for a kind of "update already saved data" like i.e. Git does when doing a commit.

At the moment I save the data using pickle. I prefer to save the whole Class-instance to not loose the class reference.

from pathlib import Path

class Class():
    def __init__(self) -> None:
        self.list: list[object] = []

    def add_element(self, obj: object, path: Path) -> None:
        self.list.append(obj)
        # now update already saved data in directory given by `path`

Upvotes: 0

Views: 26

Answers (1)

Onuralp Arslan
Onuralp Arslan

Reputation: 360

You could try a linked list for this task.

Lets say you have elements A B C F G to be added your class. Your clas should remember first and most recent element. You can add A in a format it pounts to a and next element. Lets call it node

When you add B you update A node and your main object. Then when you add C you only update B node and main object. Then when you add D you only update C node and main object.

So each time you only update last Last node and your object to remember your last node.

Although it still updates your object your object consists of data for only name of 2 objects and you only update the last node so big amonut of data that is stored inside all of your other nodes (other then first and last) are not even reached in this process let alone being deleted and re-writed. So if you have 1 million elements iside your list you only update node for last element to point your freshly added element.

Then you can pickle all the nodes between keeping them same structure.

Upvotes: 0

Related Questions