How to clean up old interfaces on zc.relation catalog?

Question

I was using plone.directives.form version 1.0 with Plone 4.2.5 and after upgrading to 4.2.6 I started seeing the following traceback and I guess its due to plone.directives.form being upgraded to version 1.1.

How can I avoid this error? The only line of code that is not from default Plone on the traceback is on der.freitag.handlers where it does a transaction.commit() and the content type is just a regular dexterity content type.

1385740390.020.496977141203 http://10.100.0.207:8081/website/front-page/atomkraft/++add++der.freitag.customizablearticlelink
Traceback (innermost last):
 Module ZPublisher.Publish, line 138, in publish
 Module ZPublisher.mapply, line 77, in mapply
 Module ZPublisher.Publish, line 48, in call_object
 Module plone.z3cform.layout, line 70, in __call__
 Module plone.z3cform.layout, line 54, in update
 Module plone.dexterity.browser.add, line 112, in update
 Module plone.z3cform.fieldsets.extensible, line 59, in update
 Module plone.z3cform.patch, line 30, in GroupForm_update
 Module z3c.form.group, line 138, in update
 Module z3c.form.action, line 99, in execute
 Module z3c.form.button, line 315, in __call__
 Module z3c.form.button, line 170, in __call__
 Module plone.dexterity.browser.add, line 99, in handleAdd
 Module z3c.form.form, line 247, in createAndAdd
 Module plone.dexterity.browser.add, line 78, in add
 Module plone.dexterity.utils, line 152, in addContentToContainer
 Module Products.BTreeFolder2.BTreeFolder2, line 455, in _setObject
 Module zope.event, line 31, in notify
 Module zope.component.event, line 24, in dispatch
 Module zope.component._api, line 136, in subscribers
 Module zope.component.registry, line 321, in subscribers
 Module zope.interface.adapter, line 585, in subscribers
 Module zope.component.event, line 32, in objectEventNotify
 Module zope.component._api, line 136, in subscribers
 Module zope.component.registry, line 321, in subscribers
 Module zope.interface.adapter, line 585, in subscribers
 Module der.freitag.handlers, line 126, in set_customizable_article_link_id
 Module transaction._manager, line 89, in commit
 Module transaction._transaction, line 329, in commit
 Module transaction._transaction, line 443, in _commitResources
 Module ZODB.Connection, line 567, in commit
 Module ZODB.Connection, line 623, in _commit
 Module ZODB.Connection, line 658, in _store_objects
 Module ZODB.serialize, line 422, in serialize
 Module ZODB.serialize, line 431, in _dump
PicklingError: Can't pickle <class 'plone.directives.form.schema.Schema'>: attribute lookup plone.directives.form.schema.Schema failed

EDIT: the object that is being created has a relation field (a z3c.relationfield.schema.RelationChoice) and it turns out that zc.relation keeps a list of all interfaces provided by each member of any relation. Thus, after upgrading from plone.directives.form version 1.0 to version 1.1 the interfaces on plone.directives.form can no longer be resolved.

From z3c.relationfield documentation I don't see any option to update relations, so the only solution would be to get all relations and recreate them?

Answer 1

One more option, I developed a package called collective.diversion that is designed to ease the pain of pickling errors when moving a class. Neither of the above scripts worked for me, however using collective.diversion did.

Adding the package to the buildout and including the following ZCML caused the items to be loaded, and they'll be persisted back in the correct place on write, so reindexing the catalogue should be sufficient.

<configure
  xmlns="http://namespaces.zope.org/zope"
  xmlns:diversion="http://namespaces.plone.org/diversion">

  <diversion:class
      old="plone.directives.form.schema.Schema"
      new="plone.supermodel.model.Schema"
      />

</configure>

Answer 2

In case someone runs into this type of problem and cannot bring the old package back, here's another approach:

import transaction

from AccessControl.SecurityManagement import newSecurityManager
from AccessControl.User import system

from Testing.makerequest import makerequest

from zope.component.hooks import setSite
from zope.globalrequest import setRequest

from zc.relation.interfaces import ICatalog
from z3c.relationfield.event import _relations
from z3c.relationfield.event import _setRelation
from zope.component import getUtility

app = makerequest(app)
newSecurityManager(None, system)
portal = app.Plone
setSite(portal)
portal.REQUEST['PARENTS'] = [portal]
portal.REQUEST.setVirtualRoot('/')
setRequest(portal.REQUEST)

THRESHOLD = 100

relations_catalog = getUtility(ICatalog)

paths = ['/'.join(r.from_object.getPhysicalPath())
         for r in relations_catalog.findRelations() if r.from_object]

relations_catalog.clear()

counter = 0
for path in paths:
    obj = app.unrestrictedTraverse(path)
    for name, relation in _relations(obj):
        _setRelation(obj, name, relation)
    counter += 1
    if counter % THRESHOLD == 0:
        transaction.savepoint()
transaction.commit()

Answer 3

Just for reference that's how I fixed it:

While still on plone.directives.form 1.0 update your objects so that they do no longer provide the plone.directives.form.schema.Schema interface.

Then re-create the relations:

from z3c.relationfield import RelationValue
from zc.relation.interfaces import ICatalog
from zope.app.intid.interfaces import IIntIds
from zope.component import getUtility
from zope.event import notify
from zope.lifecycleevent import ObjectModifiedEvent

logger = logging.getLogger(LOGGER)
relations_catalog = getUtility(ICatalog)
intids = getUtility(IIntIds)

relations = [rel for rel in relations_catalog.findRelations()]
len_relations = len(relations)
logger.info('Relations needed to update: {0}'.format(len_relations))

for relation in relations:
    # get the object link and the object linked
    object_with_link = relation.from_object
    object_linked_to = relation.to_object

    # remove the broken relation
    object_with_link.reference = None

    # let the catalog remove the old relation
    notify(ObjectModifiedEvent(object_with_link))

    # create a new relation
    object_linked_to_intid = intids.getId(object_linked_to)
    new_relation = RelationValue(object_linked_to_intid)
    object_with_link.reference = new_relation

    # let the catalog know about this new relation
    notify(ObjectModifiedEvent(object_with_link))

After this, stop the instance, run buildout again to update plone.directives.form to version 1.1 and voilà!

Answer 4

The Schema class is now in plone.supermodel.model, not plone.directives.form.schema.

However, the real problem you should try to fix is that the code is for some reason trying to store a schema in the ZODB. Pickling/unpickling Zope interfaces is not supported.