PigPog Upgraded to Drupal 4.7

We finally got the upgrade done – you’re now reading this in Drupal 4.7 (unless it’s a year or two later, and you’re reading it in Drupal 5.1 or something). I’ll go through the things this changes first, then I’ll add a few notes on how we went about upgrading, in case it’s useful to anyone…

What This Means for Visitors

The sort version: Very little. To start with, at least, there’s hardly any changes. If you read by RSS it will change things a little, but not a lot.

  • A few layout changes. Most of them were almost complete before the upgrade anyway, to make the upgrade process easier.
  • Ad placement a bit different. We’ve got a lot more options now for this (Drupal 4.7 is based on the PHP Template engine, which means we can do all sorts of things that we couldn’t before. For the moment, we’ve got a tower ad block in the right hand side, and a floating square similar to before, but that can appear on the left or the right. We’ll see how that works out. The idea is to get around ‘ad blindness’ a bit, but we can get rid of it if it’s too annoying.
  • Tagging. Drupal 4.7 supports Free Tagging, so we can add tags to posts – rather than just having the usual ‘Art’, ‘Craft’, etc., tags at the top of an article, we can have any tags we like, and clicking them should take you to any related posts. It will take a while for us to get tags added to most of the old posts, though, so this won’t have much effect straight away.
  • Mass RSS Reposting. At least through Bloglines, and possibly other readers, every page will suddenly seem to have changed, so you’ll see a whole mass of reposts. Sorry about that, but there’s not really anything we can do about it. It may yet happen again as we tweak things, but we’ll try to keep it to a minimum.
  • All RSS Feeds are now full feeds. The only reason we had partial feeds before was because we had no way of creating full feeds, except for the one main site feed. Now we can make all of our feeds full, so we have. We think most people will much prefer it that way.

You can tell I’m stretching for things to mention there, can’t you? Drupal is more of an engine to let you build the site you want. We built PigPog, and upgrading the back end doesn’t really change that. It should make things a bit faster, and more scalable.

What This Means for Us

There’s a few more changes from our point of view. The back end of PigPog is now a bit more AJAX-y, which makes it easier for us to post and manage things. As I mentioned, the PHP Template engine lets us do lots more things with the placement of ads and other content. Before, for example, we had no way to control what was shown anywhere other than in the real content area, depending on the logged in user. We couldn’t show or not show an ad in the header block, for example, depending on if a user was logged in or not. Now, we can do all of these things. So far, we haven’t, but we’ll probably start making little improvements as time goes by.

How We Upgraded

Previous Situation

  • We were running PigPog on a modified version of Drupal with a node.module, in order to show the big ad block only for anonymous users.
  • We relied on a few modules, most importantly MarkSmarty, (I think) NodeList (for the alphabetical subject listings), Image and Image_Filter, and the vital Spam module.

Early Preparations

A bit of thinking about our navigation lead me to the idea of getting rid of the alphabetical listings entirely, and replacing them with statically generated pages that we actually wrote ourselves. It’s more work, but it should be better organised and make more sense. Since this would let us do without a contrib module, I worked on this first,and got it done before even trying anything else.

I put a bit of thought into the modules we were using, and which we could manage without. Trackback, for example, is a really good module, and I liked having it, but we’d not had a genuine trackback get through the spam filters in months, and with the spam attempts coming in at a rate of almost one a minute, it was way too much work to go through the filtered ones in case a legitimate one had been blocked.

So, we were left with a short list of modules we needed. I then checked on the main Drupal site, to see which ones were now listed as 4.7 compatible. Most were. Image was, but Image_Filter wasn’t, which was going to be a bit of a problem. On with the testing.

Staging Environment

The idea of just running the upgrade, and seeing if it worked, didn’t hold a lot of appeal, with the site live. Even with backups, it would take a while to do the upgrade, and a while again to switch back, so I wanted to be as sure as I could about what was going to happen. So, I built a staging environment. Nothing complex, I just used phpMyAdmin to make a copy of the database, then downloaded a copy of the site and re-uploaded it into a subfolder called ‘staging’. I only had to edit the settings.php file (in sites) to make it work there. Then, I created a test post there and made sure it didn’t show up on the real site. Yes, I know it couldn’t possibly, but I like doing this sort of sanity check – sometimes it does exactly what you know it couldn’t do, and then you find out that you screwed up before anyone else does 😉

Once that was in place, I could then try doing the upgrade in that folder. I forgot to disable the non-core modules first, but since I’d deleted the entire folder and replaced it with only the core modules anyway, it didn’t seem to make any difference. I screwed up the database connection details at first, and spent far too long trying to work out what was wrong. I knew I couldn’t have got that wrong, because I pasted it in from the working copy. I still managed to screw it up, though, and the result was the database upgrade hanging for several minutes, before finally coming back with a connection error.

Primary and Secondary Links

The primary and secondary links are done quite differently now, but the upgrade process attempts to bring the old ones across. For mine, it screwed them up a bit, but a couple of minutes work in the Menu section of Administration did the trick there.

AdSense Ads

I managed to get something pretty close to what we’d had before, with the help of a bit of code from Rational Dev – which saved me a nice chunk of time.

At the moment, there’s a medium rectangle, that’s randomly placed on the left or right of a page. It’s a bit smaller than before, but it’s there for logged in users as well as anonymous users. We’ll probably try to change that soon, so we can go back to being nice to our members. We like or members.

Image and Image_Filter

Image_Filter isn’t 4.7 compatible yet, but it appeared to work. I say appeared to, because I hadn’t coped the files folder up to the staging folder. Over a modem, that would have taken a long time, so I decided to just work on the principle that if it didn’t work, we’d find some way around it.

Spam Filter

I got the updated 4.7-compatible version of the spam filter installed, and tried running the update script to modify the database. It failed, and left the spam filter causing errors that stopped the cron job every time.

I got around this by emptying all of the spam- tables in the database (emptying, not dropping), then running the update again. It ran cleanly this time, and everything seems to be working ok again now.

Going Live

Once everything had gone well, I cleared the database and recopied it. I went back through the changes that had now been put back, ran all of the updates again, and decided the time had come to try doing it for real.

To make things easy, I actually did this by moving the staging environment up a step. I don’t have shell access, so I did this by downloading the folders as they were on the server, then renaming all of the Drupal folders and files in the root folder. I updated the local copy of settings.php so it pointed at the right base folder and the real database name. I then copied up all of the folders. Whilst that was happening, I renamed the drupal database, then copied the staging database to the live database name. Then, I copied up the files from the root, ending with index.php and .htaccess, so nothing would try to access Drupal until everything else was in place.

Connecting to the site showed that nothing was working. The front page appeared, but with no stylesheet, so it was a mess, and all of the links were broken. It turned out I had left a trailing slash on the base directory setting (could have been worse: a following Axl would have been really annoying, and wouldn’t have given us an update in twelve years). That corrected, most things worked. The logo still didn’t – I’d included a leading slash in the path for it, hoping to make it go from the root, but it turned out it was trying to access it from a location beginning with ‘//’. The image module didn’t work correctly until I set the file access to be ‘Public’.

Those things corrected, the site was pretty much fine. All now seems to be working ok, and we’re now going through tweaking things to get them just how we like them.

Lessons for Drupal Upgrades

  • There’s a couple of videos you can download from Drupal’s site giving you a quick overview of the new features and the upgrade process. If you’re going to be doing this, get them. Yes, they’re very simple, but the fact that you’ve just watched someone doing it does wonders for your confidence.
  • Backup. Really. Backup. You need to know how you’re going to get back up and running again if it goes wrong. If it goes horribly wrong and you didn’t take backups, people will point and laugh.
  • Consider a staging environment. It’s not that much effort to set up. If it’s just a personal blog, you might not be too worried, but if you get a lot of visitors, you really don’t want your site to be down for a few hours whilst you experiment. Even if you go for doing the upgrade itself live, the fact that you’ve already run it once against your database, and it worked, means you’ll know much better what you’re doing.
  • Avoid relying on contrib modules, if you can. Sometimes, you need them, but the more you have, the more likely it is that upgrades will be held up waiting for compatibility updates to modules. There’s some we need, and some we can manage without. I don’t think I could manage without Markdown support, so MarkSmarty is pretty much vital. Trackbacks are nice, but they’ve been made almost useless by spammers, so that I can manage without.
  • Keep contrib modules in a contrib folder. I started by just dumping any extra modules in the modules folder. Creating a contrib folder under the modules folder makes life easier when you upgrade, and Drupal finds things just as easily in subfolders.
  • Do things by official methods wherever you can. Running a modified version of node.module caused problems, because we couldn’t just upgrade – I’d have to make the same changes to the new version of the module. In the end, we found ways to do without the changes, and the PHP Template should make it much easier to achieve the same things.

It’s been quite a lot of work, but a fair bit of it was because of bad choices I made when I first set Drupal up. It’s all been worthwhile, though – the changes aren’t huge, but they’re all good, and it’s important to be up to date with security patches and bug fixes. This should also leave us in a much easier position for doing the next Drupal upgrade.