CODEWITHSUNDEEP
pythonplonezodb
rain2o
rain2o

Reputation: 498

Plone Migrate Blob Data to "bushy" layout IOError Errno 21

I am trying to migrate our blobstorage (using Plone 4.3.2 and ZODB3 3.10.5) from 'lawn' to 'bushy' layout. While running the script, I get the following traceback:

(11719) Blob directory `var/blobstorage-lawn/` has layout marker set. Selected `lawn` layout.
(11719) The `lawn` blob directory layout is deprecated due to scalability issues on some file systems, please consider migrating to the `bushy` layout.
Migrating blob data from `var/blobstorage-lawn/` (lawn) to `var/blobstorage` (bushy)
Traceback (most recent call last):
  File "bin/migrateblobs", line 19, in <module>
    sys.exit(ZODB.scripts.migrateblobs.main())
  File "/var/db/zope/plone43_dev/buildout-cache/eggs/ZODB3-3.10.5-py2.7-linux- x86_64.egg/ZODB/scripts/migrateblobs.py", line 77, in main
    migrate(source, dest, options.layout)
  File "/var/db/zope/plone43_dev/buildout-cache/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/scripts/migrateblobs.py", line 52, in migrate
    link_or_copy(source_file, dest_file)
  File "/var/db/zope/plone43_dev/buildout-cache/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/scripts/migrateblobs.py", line 30, in link_or_copy
    shutil.copy(f1, f2)
  File "/var/db/zope/plone43_dev/Python-2.7/lib/python2.7/shutil.py", line 119, in copy
    copyfile(src, dst)
  File "/var/db/zope/plone43_dev/Python-2.7/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 21] Is a directory: '/var/db/zope/plone43_dev/zeocluster/var/blobstorage-lawn/0x00/0x00'

I don't understand why it is trying to copy a directory. Is this a bug in the product? Or could my blobstorage be corrupt? It is a dev environment, and I have been having some other issues with the blobstorage, which is why I'm trying to migrate to bushy in hopes that it will resolve some issues.

Thoughts or solutions?

Upvotes: 2

Views: 452

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1121276

You appear to have a bushy layout intermixed with your lawn layout.

The lawn layout uses a flat structure; directories are named after the OID, with in each directory just the revisions of the blob files. The bushy layout uses one directory per byte in the OID, leading to a tree of directories.

When moving from lawn to bushy the script takes directories assuming they are valid OIDs, and on each assumes that all it'll find in the directory is revision files.

You, however, already have a bushy layout structure. The script is trying to move the directory 0x00 out of a top-level directory 0x00. That's exactly the kind of directories you'd find in a bushy layout, not a lawn layout. Your structure is corrupted indeed.

It may be that all that is wrong is the marker file; if all you have at the top level are 0xhh 2 digit hex numbered directories, then you have just a bushy layout disguised as a lawn. You can then try changing the .layout file in the var/blobstorage-lawn directory from lawn to bushy and see if your ZODB still works. If not, it probably is beyond repair.

If you have a mix of 0xhh and longer 0xhhhhhhhhh hex directories (the latter only containing files, no directories) then you managed to put both a lawn and a bushy layout into the one blob storage. If the layout is marked as lawn, the bushy part is most likely out of date. You can try and move all directories with just 2 hex digits out to a new blobstorage directory (and add a new .layout file with the content bushy to it), but I am not too confident that anything useful is contained in it.

Upvotes: 5

Related Questions