CODEWITHSUNDEEP
pythoncfilemmapshared
giupardeb
giupardeb

Reputation: 841

Resolved: C and Python: fill a file empty through mmap

I want create a shared mapped file between C (that create the mapped file and only read from it) and python (that write and fill the mapped file). I have this C code :

#include <sys/types.h>
#include <sys/mman.h>
#include <err.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(void)
{
    char *shared;
    int fd = -1;
    fd = open("hello.txt", O_RDWR, 0);
    if (fd == -1) {
        printf("unable to open");
        return 0;
    }

    shared = (char *)mmap(NULL, 1, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
    system("python /home/giuseppe/Documents/test_mmap/mappalo.py");
    printf("C CODE: %c\n",shared[0]);

}

and this is a python code:

import sys
import os
import mmap

with open( "hello.txt", "wb" ) as fd:
    fd.write("1")
with open( "hello.txt", "r+b" ) as fd:
    mm = mmap.mmap(fd.fileno(), 0, flags=mmap.MAP_SHARED, access=mmap.ACCESS_WRITE, offset=0)
    print("content read from file")
    print(mm.readline())
    mm[0] = "01"
    mm.close()
    fd.close()

when I executed the C code, I get this error:

content read from file
1
Traceback (most recent call last):
  File "/home/giuseppe/Documents/test_mmap/mappalo.py", line 11, in <module>
    mm[0] = "01"
IndexError: mmap assignment must be single-character string
C CODE: 1

How can I write from python code a long string in a mapped file and then read it from C code?

Thank you so much for your time

UPDATE I want to write two strings, so I write this code:

arr = bytes("Hello123;", 'utf-8')
arr1 = bytes("Hello412;", 'utf-8')
size = sys.getsizeof(arr) + sys.getsizeof(arr1)

with buf:
    buf[:size] = struct.pack('ss', arr1, arr)

But I have the following error: 

IndexError: mmap slice assignment is wrong size

UPDATE 2 Thank you to Sam Mason that allow me to resolve. This is the code:

C code:

#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
  int fd = open("hello.txt", O_RDWR | O_CREAT, 0666);
  if (fd == -1) {
      perror("unable to open");
      return 1;
  }

  if (ftruncate(fd, 4096) < 0) {
    perror("unable to set length");
    return 1;
  }

  int *shared = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
  //system("python3.7 mappalo.py");
  printf("C CODE: %s\n", shared);
  return 0;
}

Python code:

import mmap
import struct

with open('hello.txt', 'r+b') as fd:
    buf = mmap.mmap(fd.fileno(), 4096, access=mmap.ACCESS_DEFAULT)

arr = bytes("Hello123;", 'utf-8')
arr1 = bytes("Hello412;", 'utf-8')
size = len(arr) + len(arr1)
types = str(len(arr))+'s' + str(len(arr1))+'s'

with buf:
    buf[:size] = struct.pack(types, arr1, arr)

Upvotes: 0

Views: 488

Answers (1)

Sam Mason
Sam Mason

Reputation: 16184

you're doing various strange things in both your C code and Python code. I'll start with some code that "works":

first, C code:

#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
  int fd = open("hello.txt", O_RDWR | O_CREAT, 0666);
  if (fd == -1) {
      perror("unable to open");
      return 1;
  }

  if (ftruncate(fd, 1024) < 0) {
    perror("unable to set length");
    return 1;
  }

  int *shared = mmap(NULL, 1024, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
  system("python mmap-test.py");
  printf("C CODE: %i %i\n", shared[0], shared[1]);
  return 0;
}

I've trimmed headers a bit, and make sure that the file has some data in it (in this case it'll be lazily filled with zeros by the kernel). I also treat it as an array of ints as a single byte is pointless to transfer. also, you're generally recommended not to cast from void* to another type. I also set the permission mask to something sensible, zero will just break things later.

next Python code:

import mmap
import struct

with open('hello.txt', 'r+b') as fd:
    buf = mmap.mmap(fd.fileno(), 1024, access=mmap.ACCESS_DEFAULT)

with buf:
    buf[:8] = struct.pack('ii', 123, 7901)

I just open it once, map the data in, encode two Python ints to a byte array of two C-ints. the non-nesting of with statements is meant to show that mmap holds the file open on its own, but you might want to nest them in your own code. your two opens in Python were also creating a second file (of the same name, replacing the C file) which was probably confusing things. also you weren't writing the correct sized/typed data into the mmaped space

Upvotes: 2

Related Questions