Migration to Plone 3: Trouble and a Cheat

I’ve been using Plone for a number of years now and always lag a bit behind in my installation version when compared with the latest version. Last time I left it so long that I had to actually install multiple Plone versions in order to get myself to the latest copy. Each time, I would update my content, use the included migration tool, then uninstall and go to the next version. So I was hopeful this time that the migration from my 2.5.1 Plone to version 3.1.7. Needless to say, it is a work in progress and I didn’t make it! But here are some of the things that I came across.

Migration Fails, Part I

First, if you’re going to upgrade or move your Plone, make sure you’ve reviewed the documentation, specifically the Plone Upgrade Guide.  It specifically deals with moving from a 2.5.x site to a 3.x site.  Unfortunately, the normal migration path didn’t work.  Once I’d installed a clean copy of 3.1.7 with the Unified Installer (which worked great), and moved my Data.fs file over, Plone wasn’t accessible.  When I reverted to the installed Data.fs, it came up fine.  I like the Installer’s ability to create your password and instance name as part of the install script.

Migration Fails, Part II

If you can’t get your Data.fs over, and there might be loads of reasons why you can’t – incompatible Products, missing data, etc. – you can always try to export your Plone site.  In my case, I knew there were incompatible products.  For one thing, the Rich Topic product I use for my Frontpage is not available in Plone 3, in part because it looks like the Collection functionality duplicates it.

Step 1:  Create a Plone Site Copy

I did a simple copy and pasted my site back into Plone.  I renamed it so that it was distinguishable from my “live” (production sounds unnecessarily grand!) site.  Then I used the QuickInstaller to uninstall products I thought might cause problems with the migration and rolled back any customizations I thought would (a) be easy to recreate or (b) were no longer current.

Step 2:  Export the Copied Plone

Then I returned to my root, selected my copied Plone site, and exported it.  This dumped a copy out which I could then copy over to the Plone 3 site.  This is my only near-constant complaint about Plone.  Every time you install a new version, the file structure seems to be vastly different.  In this case, there was no import folder created, so I finally figured out that I needed to drop the [plone-site].zexp file into /usr/local/Plone/[instance name]/parts/instance/import.  I returned to my Zope Management Interface (ZMI) and started the import.

Note:  3.1.7 also incorporates a new configuration method, called buildout.  It seems great but I couldn’t get to Plone if I changed it from the default port 8080.  This is not a hassle, since my current 2.5.1 site is on a different port but it conflicted with my Web mail.

Step 3:  Fix Your Import

The import process highlights a bunch of problems with your .zexp file.  This meant going back to Step 1, editing that copied Plone site, exporting it (step 2), then reimporting it.

The first problem I found was some polls I’d created using a product I no longer had installed.  The polls were broken, which was no big deal, but the content references were causing the import function to choke.  So I killed the files and progressed to the next error.

This onec dealt with mime-types.  The three items I had to fix and I found how thanks to Softwerkes:  add new mime-types for text/x-web-markdown, text/x-web-textile, and text/x-web-intelligent.  It looks like these were introduced in earlier versions of Plone so I’m not sure how I have been able to migrate forward without hitting this snag before.  In any event, once I’d added those, the import worked just fine.

Step 4:  Use the Migration Tool

Plone is an amazing content management system, with remarkable features.  This migration tool has to be one of the best, though, for helping you get from version to version.  You can even do a “dry run” to see what is going to happen before you commit.  I ran the migration tool and it bailed out.  I  got an error that said:


list.index(x): x not in list

The migration tool upgraded me through all the “dot” versions from 2.5.1 to 3.0.6 and then collapsed.  This error appears to be pretty common (Google: exceptions.valueerror list.index(x): x not in list ) and it had me stymied.  There was no obvious solution for what it was and there was no way to get to 3.1.7 with that error in the way.

Plone Cheat:  If X Not in List, Stop at 3.0.6

I decided that the migration tool was sending me a sign!  So I returned to Plone.org and downloaded a copy of Plone 3.0.6, since that was the version that the tool indicated was successfully applied prior to aborting.  This is not normally recommended but I wanted to get into 3 and then see if I could work forward.  For example, I know that my main_template is customized and will need to be redone to work in 3.  Perhaps once I get that worked out and eliminate the differences there, this error message can be overcome (or, if it’s a bug, fixed in a later version).

Once I’d removed Plone 3.1.7, I expanded the 3.0.6 files and ran the install.  As I mentioned above, the install process runs differently (fewer options), installs Plone in a different directory from 3.1.7, and overall maintains a file structure much more like 2.5.3.  From that perspective, it was helpful because I was able to reduce my learning curve at the same time.

Step 5:  Rinse, Rinse, Rinse, Repeat

Now I returned to my cleaned up .zexp file (Step 3) and imported it into my 3.0.6 Plone.  This worked cleanly, just as it had with 3.1.7.  Then I turned to the Migration tool and it ran through without any problems.  Now I was in business.

Step 6:  Get Yer Red Hot Products

I returned to the Plone.org site and downloaded the products I needed.  Plone 3 uses a new system (eggs) for distributing Products but I don’t think it’s utilized in 3.0.6, because the installation of these products was very similar to 2.5.3.  I added them back in, with the other default products provided with Plone 3, and then took a look at my site.

It’s Clean Up Time, Time To Put Your Toys Away

Not so good!  As I indicated (and you can tell from this site if you’re looking at it in January 2009), I’d edited my header template, my main_template, and other files to manage my look and feel.  These disabled the Plone 3 template from running.  I did the following:

  • Remove all custom files that were non-images (including CSS) from the Plone Skins/Custom folder and dumped them in a separate location;
  • Went to my Plone site’s properties (when you’re looking at the root of your site, the Properties tab at the top) and removed any portlet information.

Once removed, I reloaded my site and it worked fine.  It didn’t look pretty but it worked and the NuPlone theme/skin is really well done.  My 2.5.1 portlets gave errors, but that’s because I’d removed them from the skins/custom folder.  I started to move all of that content back and slowly my 2.5.3 look-and-feel started to appear.  The only page I cannot return to its home is the main_template.  The site will load without visible errors, but there’s no real content.  So the first item of business will be to get my new main_template to incorporate the features I’d placed in the old one.

The other problem I know I have is that the Collections function returns an unknown error.  It is a repeating error that is as vague as the earlier “list” error (see step 4).  It is:

Macro expansion failed

TypeError:  iteration over non-sequence

When I click to create Collections, and in some other places of the site, I get this error and am stuck.  That’s the second item!  But I’m guessing that it’s something relatively small and quirky that I will get sorted out sooner or later.  Then I can drag the few bits of new content from this site to that, and I’ll be ready to go live.

As always, a Plone migration provides lots of excitement!