Pythonic pre-processor for C++

Question

I am frequently finding myself thinking "Why is the C++ pre-processor so weak?"

The conventional counter seems to be "C++ uses template meta-programming instead." However, there are plenty of situations where templates don't hack it.

For example, I'm currently writing a trampoline that takes a C-struct function-pointer table (that belongs to a library I'm interfacing) and maps the functions onto methods of a corresponding C++ class. Every time an update to the library changes the C-struct, it is going to create some maintenance hell. I need to modify the source code in four different places.

And updates aside, there is a huge amount of duplication in my code that just can't be circumnavigated.

What I really need is code to generate code.

So how about using something akin to Python as a preprocessor?

#[
funcpointers = []
for i in members(TheCStruct):
    if i.name.beginswith("tp_"):
        funcpointers.append(i)

def cxxsig(x):
    //ToDo

def cxxname(x):
    //ToDo
#]

So straightaway a preprocessor object is created storing all function pointers.

Later we can use this variable:

class CxxWrapper 
{
    #[
    for i in funcpointers:
        print 'virtual ' + cxxsig(i) + '{ std::cout << "' + cxxname(i) + '"ctor\n"; }'
    #]

etc. So that the above example would generate lines like:

virtual int my_fp(void*, int, float) { std::cout << "my_fp ctor\n"; }

Similar loops would handle constructing the trampolines etc. And when the underlying C-struct changes, everything remains synchronised.

It would be possible to single step through the preprocessor. Debugging at the preprocessor level would be just as easy as debugging Python code.

So my question is: has something like this been attempted? If not, is there a good reason why not? And if so, what was the conclusion?

Answer 1

Probably there is no such widespread tool already because it's not needed in most use cases, and in many other use cases it would make the source code much harder to understand and maintain (planning far ahead in the future).

FYI Such a preprocessor (pcpp) can be surprisingly simple:

#! /usr/bin/python
"""Python 2.x C and C++ preprocessor."""
import re, sys, textwrap
def ml_escape(msg):
  return re.sub(
      r'[^\n-+./\w:;<>]', lambda match: '\\%03o' % ord(match.group(0)), msg)
output = []; oa = output.append
def lit(s):
  if s.endswith('\n'):
    oa('__import__("sys").stdout.write("""%s\\n""")\n' % ml_escape(s[:-1]))
  elif s:
    oa('__import__("sys").stdout.write("""%s""")\n' % ml_escape(s))
def cod(s): oa(textwrap.dedent(s))
f = open(sys.argv[1])
data, i = f.read(), 0
sc = re.compile(r'(?sm)^[ \t]*#\[[ \t]*$(.*?\n)[ \t]*#\][ \t]*$').scanner(data)
for match in iter(sc.search, None):
  j = match.start(0)
  lit(data[i : match.start(0)])
  cod(data[match.start(1) : match.end(1)])
  i = match.end()
lit(data[i : len(data)])
exec compile(''.join(output), f.name, 'exec') in {}

Example input (t.cp):

#include <stdio.h>
#[
import re
def c_escape(msg):
  return re.sub(
      r'[^-+./\w]', lambda match: '\\%03o' % ord(match.group(0)), msg)
def repeat_msg(count, msg):
  for i in xrange(count):
    print 'puts("%s");' % c_escape(msg)
#]
int main() {
  #[
  repeat_msg(5, 'hi"')
  #]
  return 0;
}

Example command:

$ ./pcpp t.cp >t.c
$ gcc -W -Wall -s -O2 t.c
$ ./a.out
hi"
hi"
hi"
hi"
hi"

Example output:

#include <stdio.h>

int main() {
puts("hi\042");
puts("hi\042");
puts("hi\042");
puts("hi\042");
puts("hi\042");

  return 0;
}

The pcpp implementation above is careful to keep newlines, so if you have an error in your Python code, the traceback will contain the filename t.cp and the correct line number. By adding the emission of #line directives, line numbers in C(++) error messages can be fixed as well.

(Please note that c_escape doesn't work as expected with digraphs and trigraphs.)

Answer 2

Consider writing a code generator - it may be specific to this problem or more generic. It doesn't have to be a preprocessor, at least the initial version. You could develop/extend it later.

Code generators are commonly used, including cases like this one. One thing to keep in mind are proper dependencies in the build system so that the generated files are re-generated etc.