Temporarily keep a key and value in a dictionary

Question

I have a python dictionary which is added with info from time to time.

Conditions:

1. There should be 3 or less keys in the dictionary at all time. If there is a need to add another key-value pair, the earliest added key in the dictionary at that moment should be deleted and this new one should be added.
2. Once a key-value is added, after 1 day, it should be deleted.

Example:

result = {}

def add_key(key, val):
    test = result.get(key) # Test to see if key already exists
    if test is None:
        if len(result.keys()) < 3:
            result[key] = val # This key and pair need to be deleted from the dictionary after
                            # 24 hours
        else:
            # Need code to remove the earliest added key.

add_key('10', 'object 2')

## Need another code logic to delete the key-value pair after 24 hours

How should I go about this issue?

Answer 1

Here's how to do what I recommended in my comment under you question — namely to do it by implementing a dictionary subclass that does what's needed via metadata it maintains internally about each of the values it contains.

This can be done fairly easily via the abstract base classes for containers in the collections.abc module because it minimizes the number of methods that need to be implemented in the subclass. In this case abc.MutableMapping was used at the base class.

Note that I've add a new method named prune() that show how all entries older than a specified amount could be removed. In addition to that, I've left some extraneous calls to print() in the code to make what it's doing clearer when testing it — which you'll probably want to remove.

from collections import abc
from datetime import datetime, timedelta
from operator import itemgetter
from pprint import pprint
from time import sleep


class LimitedDict(abc.MutableMapping):
    LIMIT = 3

    def __init__(self, *args, **kwargs):
        self._data = dict(*args, **kwargs)

        if len(self) > self.LIMIT:
            raise RuntimeError(f'{type(self).__name__} initialized with more '
                               f'than the limit of {self.LIMIT} items.')

        # Initialize value timestamps.
        now = datetime.now()
        self._timestamps = {key: now for key in self._data.keys()}

    def __getitem__(self, key):
        return self._data[key]

    def __setitem__(self, key, value):
        if key not in self:  # Adding a key?
            if len(self) >= self.LIMIT:   # Will doing so exceed limit?
                # Find key of oldest item and delete it (and its timestamp).
                oldest = min(self._timestamps.items(), key=itemgetter(1))[0]
                print(f'deleting oldest item {oldest=} to make room for {key=}')
                del self[oldest]

        # Add (or update) item and timestamp.
        self._data[key], self._timestamps[key] = value, datetime.now()

    def __delitem__(self, key):
        """Remove item and associated timestamp."""
        del self._data[key]
        del self._timestamps[key]

    def __iter__(self):
        return iter(self._data)

    def __len__(self):
        return len(self._data)

    def prune(self, **kwargs) -> None:
        """ Remove all items older than the specified maximum age.

        Accepts same keyword arguments as datetime.timedelta - currently:
        days, seconds, microseconds, milliseconds, minutes, hours, and weeks.
        """
        max_age = timedelta(**kwargs)
        now = datetime.now()
        self._data = {key: value for key, value in self._data.items()
                        if (now - self._timestamps[key]) <= max_age}
        self._timestamps = {key: self._timestamps[key] for key in self._data.keys()}



if __name__ == '__main__':

    ld = LimitedDict(a=1, b=2)
    pprint(ld._data)
    pprint(ld._timestamps)
    sleep(1)
    ld['c'] = 3  # Add 3rd item.
    print()
    pprint(ld._data)
    pprint(ld._timestamps)
    sleep(1)
    ld['d'] = 4  # Add an item that should cause limit to be exceeded.
    print()
    pprint(ld._data)
    pprint(ld._timestamps)
    ld.prune(seconds=2) # Remove all items more than 2 seconds old.
    print()
    pprint(ld._data)
    pprint(ld._timestamps)

Answer 2

This code is my best effort at solving the first problem. It adds an indicator to the end of the they key and always deletes the earliest key when a new key is added:

global myDict
myDict = {}
globals()["currVal"] = 3
def add_key(key, val):
    if key not in [k[0:-1] for k in myDict.keys()]: # Test to see if key already exists
        if len(myDict.keys()) < 3:
            myDict[str(key) + str(len(myDict.keys())+1)] = val # This key and pair need to be deleted from the dictionary after        
        else:
            deleteKeys = []
            for k in myDict:
                if k.endswith(str(globals()["currVal"]-2)):
                    deleteKeys.append(k) 
            myDict[str(key) + str(globals()["currVal"]+1)] = val
            for ks in deleteKeys:
                del myDict[ks]
            globals()["currVal"] += 1
    return myDict

add_key('100', 'object 1')
print(myDict)
add_key('101', 'object 2')
print(myDict)
add_key('102', 'object 3')
print(myDict)
add_key('103', 'object 4')
print(myDict)
add_key('104', 'object 5')
print(myDict)
add_key('105', 'object 6')
print(myDict)

Output:

{'1001': 'object 1'}
{'1001': 'object 1', '1012': 'object 2'}
{'1001': 'object 1', '1012': 'object 2', '1023': 'object 3'}
{'1012': 'object 2', '1023': 'object 3', '1034': 'object 4'}
{'1023': 'object 3', '1034': 'object 4', '1045': 'object 5'}
{'1034': 'object 4', '1045': 'object 5', '1056': 'object 6'}

Answer 3

Inspired by the SO question Python Datatype for a fixed-length FIFO

---

One way of doing this is by using collections.

I created this short demo using collections.deque

We start by initializing the variable x to a list containing 5 empty dictionaries, and a max length of 5. Then creating the method add_v which uses the method appendleft to have it behave like a FIFO stack with a size limit of 5.

import collections

x = collections.deque([{}]*5, 5)


def add_key(key, value):
    x.appendleft({key: value})

Note: x will be a list containing dictionaries and not a dictionary containing entries.

Answer 4

Here's one idea:

• Subclass dict with your own class type (see this question).
• In each function that deals with key, transform the key into a tuple of (datetime.now(), key).
• At the beginning of each function, first do the 24-hour check and remove any expired items.
• At the end of __setitem__ truncate the container down to 3 elements if necessary.

Answer 5

I know this is only a partial answer to your question, but here is an approach for the first part of the question:

Define a variable holding the last added key on the same scope as the dictionary. Than you can remove said key in your else statement. Don't forget to update the variable each time you add a key.