CODEWITHSUNDEEP
pythonmmap
oraant
oraant

Reputation: 379

contextlib.closing causes mmap bug

In the normal code, everything is fine, I can read/and write correctly.

for i in range(10):
    sleep(1)

    m = mmap.mmap(-1, 1024, access=mmap.ACCESS_WRITE, tagname='share_mmap')
    m.seek(0)
    cnt = m.read_byte()

    if cnt == 0:
        print("Load data to memory")
        m.seek(0)
        m.write(b"FFFFFFFFFFFFFFFFFF")

    else:
        m.seek(0)
        info_str=m.read().translate(None, b'\x00').decode()
        print("The data is in memory: ", info_str)

Result:

> Load data to memory
> The data is in memory:  FFFFFFFFFFFFFFFFFF
> The data is in memory:  FFFFFFFFFFFFFFFFFF

But if I wrap it with contextlib, like all the tutorial's code, then I can't read anymore.

for i in range(10):
    sleep(1)

    # !!!! LOOK AT HERE !!!! I only changed this line.
    with contextlib.closing(mmap.mmap(-1, 1024, access=mmap.ACCESS_WRITE,
                            tagname='share_mmap')) as m:
        m.seek(0)
        cnt = m.read_byte()

        if cnt == 0:
            print("Load data to memory")
            m.seek(0)
            m.write(b"FFFFFFFFFFFFFFFFFF")

        else:
            m.seek(0)
            info_str=m.read().translate(None, b'\x00').decode()
            print("The data is in memory: ", info_str)

Result:

> Load data to memory
> Load data to memory
> Load data to memory

Why? Also, why does everyone like do it in this way — haven't they encountered this bug?

Upvotes: 1

Views: 139

Answers (2)

Joseph Sible-Reinstate Monica
Joseph Sible-Reinstate Monica

Reputation: 48572

You're mapping anonymous memory as opposed to a file, so the only thing that's making you get the same mapping each time is the tag name. If you already have a mapping with a given tag name, then mmap gives you the same mapping again, but if not, you get a fresh mapping. Your first snippet works because the old mappings are still there (since you never closed them), but in your second snippet, the old mapping is gone before the new one gets created, and since it's anonymous memory, that means the data is gone too.

To fix the problem, either do the mapping outside the loop or map file-backed memory instead.

Upvotes: 1

martineau
martineau

Reputation: 123443

Apparently it doesn't work with anonymous memory (fileno -1) since it does with a real file as illustrated below. Not sure if this is a bug, or just an undocumented limitation.

import contextlib
import mmap
from time import sleep
import tempfile


with tempfile.TemporaryFile() as mmapfile:
    for i in range(5):
        sleep(1)

        # Works if you use a real file.
        with contextlib.closing(mmap.mmap(mmapfile.fileno(), 1024, access=mmap.ACCESS_WRITE,
                                tagname='share_mmap')) as m:
            m.seek(0)
            cnt = m.read_byte()

            if cnt == 0:
                print("Load data to memory")
                m.seek(0)
                m.write(b"FFFFFFFFFFFFFFFFFF")

            else:
                m.seek(0)
                info_str=m.read().translate(None, b'\x00').decode()
                print("The data is in memory: ", info_str)

Output:

Load data to memory
The data is in memory:  FFFFFFFFFFFFFFFFFF
The data is in memory:  FFFFFFFFFFFFFFFFFF
The data is in memory:  FFFFFFFFFFFFFFFFFF
The data is in memory:  FFFFFFFFFFFFFFFFFF

Upvotes: 0

Related Questions