Create a portal_user_catalog and have it used (Plone)

Question

I'm creating a fork of my Plone site (which has not been forked for a long time). This site has a special catalog object for user profiles (a special Archetypes-based object type) which is called portal_user_catalog:

$ bin/instance debug
>>> portal = app.Plone
>>> print [d for d in portal.objectMap() if d['meta_type'] == 'Plone Catalog Tool']
[{'meta_type': 'Plone Catalog Tool', 'id': 'portal_catalog'},
 {'meta_type': 'Plone Catalog Tool', 'id': 'portal_user_catalog'}]

This looks reasonable because the user profiles don't have most of the indexes of the "normal" objects, but have a small set of own indexes.

Since I found no way how to create this object from scratch, I exported it from the old site (as portal_user_catalog.zexp) and imported it in the new site. This seemed to work, but I can't add objects to the imported catalog, not even by explicitly calling the catalog_object method. Instead, the user profiles are added to the standard portal_catalog.

Now I found a module in my product which seems to serve the purpose (Products/myproduct/exportimport/catalog.py):

"""Catalog tool setup handlers.

$Id: catalog.py 77004 2007-06-24 08:57:54Z yuppie $
"""

from Products.GenericSetup.utils import exportObjects
from Products.GenericSetup.utils import importObjects

from Products.CMFCore.utils import getToolByName

from zope.component import queryMultiAdapter
from Products.GenericSetup.interfaces import IBody

def importCatalogTool(context):
    """Import catalog tool.
    """
    site = context.getSite()
    obj = getToolByName(site, 'portal_user_catalog')
    parent_path=''

    if obj and not obj():
        importer = queryMultiAdapter((obj, context), IBody)
        path = '%s%s' % (parent_path, obj.getId().replace(' ', '_'))
        __traceback_info__ = path
        print [importer]
        if importer:
            print importer.name
            if importer.name:
                path = '%s%s' % (parent_path, 'usercatalog')
                print path

            filename = '%s%s' % (path, importer.suffix)
            print filename
            body = context.readDataFile(filename)
            if body is not None:
                importer.filename = filename # for error reporting
                importer.body = body

        if getattr(obj, 'objectValues', False):
            for sub in obj.objectValues():
                importObjects(sub, path+'/', context)

def exportCatalogTool(context):
    """Export catalog tool.
    """
    site = context.getSite()
    obj = getToolByName(site, 'portal_user_catalog', None)
    if tool is None:
        logger = context.getLogger('catalog')
        logger.info('Nothing to export.')
        return

    parent_path=''

    exporter = queryMultiAdapter((obj, context), IBody)
    path = '%s%s' % (parent_path, obj.getId().replace(' ', '_'))
    if exporter:
        if exporter.name:
            path = '%s%s' % (parent_path, 'usercatalog')
        filename = '%s%s' % (path, exporter.suffix)
        body = exporter.body
        if body is not None:
            context.writeDataFile(filename, body, exporter.mime_type)

    if getattr(obj, 'objectValues', False):
        for sub in obj.objectValues():
            exportObjects(sub, path+'/', context)

I tried to use it, but I have no idea how it is supposed to be done; I can't call it TTW (should I try to publish the methods?!). I tried it in a debug session:

$ bin/instance debug
>>> portal = app.Plone
>>> from Products.myproduct.exportimport.catalog import exportCatalogTool
>>> exportCatalogTool(portal)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File ".../Products/myproduct/exportimport/catalog.py", line 58, in exportCatalogTool
    site = context.getSite()
AttributeError: getSite

So, if this is the way to go, it looks like I need a "real" context.

Update: To get this context, I tried an External Method:

# -*- coding: utf-8 -*-
from Products.myproduct.exportimport.catalog import exportCatalogTool
from pdb import set_trace

def p(dt, dd):
    print '%-16s%s' % (dt+':', dd)

def main(self):
    """
    Export the portal_user_catalog
    """
    g = globals()
    print '#' * 79
    for a in ('__package__', '__module__'):
        if a in g:
            p(a, g[a])
    p('self', self)
    set_trace()
    exportCatalogTool(self)

However, wenn I called it, I got the same <PloneSite at /Plone> object as the argument to the main function, which didn't have the getSite attribute. Perhaps my site doesn't call such External Methods correctly?

Or would I need to mention this module somehow in my configure.zcml, but how? I searched my directory tree (especially below Products/myproduct/profiles) for exportimport, the module name, and several other strings, but I couldn't find anything; perhaps there has been an integration once but was broken ...

So how do I make this portal_user_catalog work? Thank you!

Update: Another debug session suggests the source of the problem to be some transaction matter:

>>> portal = app.Plone
>>> puc = portal.portal_user_catalog
>>> puc._catalog()
[]
>>> profiles_folder = portal.some_folder_with_profiles
>>> for o in profiles_folder.objectValues():
...     puc.catalog_object(o)
...
>>> puc._catalog()
[<Products.ZCatalog.Catalog.mybrains object at 0x69ff8d8>, ...]

This population of the portal_user_catalog doesn't persist; after termination of the debug session and starting fg, the brains are gone.

Answer 1

It looks like the problem was indeed related with transactions. I had

import transaction
...
class Browser(BrowserView):
    ...
    def processNewUser(self):
        ....
        transaction.commit()

before, but apparently this was not good enough (and/or perhaps not done correctly).

Now I start the transaction explicitly with transaction.begin(), save intermediate results with transaction.savepoint(), abort the transaction explicitly with transaction.abort() in case of errors (try / except), and have exactly one transaction.commit() at the end, in the case of success. Everything seems to work.

Of course, Plone still doesn't take this non-standard catalog into account; when I "clear and rebuild" it, it is empty afterwards. But for my application it works well enough.